A rag enhanced small-scale language model question and answer accuracy improvement method and system

Abstract

The application discloses a RAG enhanced small-scale language model question and answer accuracy improvement method and system. The method comprises the following steps: adjusting the text form of the answer basis in the knowledge base according to the text fragmentation processing mode; obtaining and utilizing the first verification set to optimize the RAG model; taking the question expanded by the natural language technology as the input, utilizing the optimized RAG model to search in the knowledge base related to the bidding scene, obtaining a plurality of initial answer bases; outputting the plurality of initial answer bases to the initial answer basis scoring model respectively, screening and sorting the plurality of initial answer bases according to the scoring results, and completing the deduplication filtering processing; utilizing the natural language technology to respectively perform content simplification and summarization on each processed initial answer basis, and obtaining the final answer basis; and generating the answer by utilizing the RAG model according to the question and the final answer basis. The application can realize the improvement of the RAG enhanced small-scale language model question and answer accuracy.

Description

Technical Field

[0001] This application relates to the technical field of natural language processing and question answering systems, specifically to a method and system for improving the accuracy of question answering in RAG-enhanced small-scale language models. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technology, especially the advancements in natural language processing and knowledge base technologies, the application of small-scale language models based on RAG (Retrieval-Augmented Generation) enhancement in domain-specific question-answering systems has received widespread attention. These systems not only improve information retrieval efficiency but also, to some extent, enhance the way enterprises manage internal knowledge. Particularly in fields such as bidding and procurement, by inputting bidding documents into a knowledge base and combining RAG enhancement with language model technology, automated question-answering functions have been achieved, significantly improving work efficiency and reducing labor costs.

[0003] Currently, the common approach to solving problems in the bidding Q&A field is to use a combination of knowledge base, RAG enhancement, and language model technology. The specific process is as follows: First, the bidding documents are pre-processed and entered into the knowledge base. When someone asks a question, the program automatically matches the first 5-10 relevant pieces of information from the knowledge base as the basis for the answer. This information, along with the original question, is then provided to the language model, which finally generates the final answer and provides it to the user.

[0004] While the aforementioned methods are more intelligent than traditional approaches, they still have several shortcomings, resulting in numerous invalid or even erroneous answers. The main reasons for invalid answers are: First, the RAG enhancement process fails to retrieve sufficiently rich content, leaving the language model lacking the necessary "information" for accurate responses. Second, this technology is typically used in local area network environments, unable to access larger-scale language model resources via the internet, and the maximum number of supported contextual parameters is very limited; simply adding extra parameters cannot improve accuracy. Third, when users ask questions, the question text is often too short and much shorter than the existing text segments in the question-and-answer database, resulting in extremely low absolute similarity. This can easily lead to key details being overlooked because they don't meet the pre-set text similarity threshold, further weakening the quality and credibility of the final answer. Therefore, a method and system are needed to improve the accuracy of question-and-answering using RAG-enhanced small-scale language models. Summary of the Invention

[0005] To improve the accuracy of question answering in RAG-enhanced small-scale language models, this application provides a method and system for improving the accuracy of question answering in RAG-enhanced small-scale language models.

[0006] Firstly, this application provides a method for improving the accuracy of question answering in RAG-enhanced small-scale language models, including: Adjust the text format of the answers in the knowledge base associated with the bidding scenario according to the text segmentation processing method; The first validation set is obtained by acquiring question-answer pairs containing the answer basis stored in the form of text fragments in the bidding scenario, and the RAG model is optimized by using the first validation set. By using natural language processing technology, the content or number of questions input to the RAG model are expanded. The expanded questions are then used as input, and the optimized RAG model is used to search the knowledge base associated with the bidding scenario to obtain multiple initial answer bases. Based on deep learning algorithms, an initial answer basis scoring model is constructed within the RAG model framework. Multiple initial answer basis criteria are output separately to form the initial answer basis scoring model. The multiple initial answer basis criteria are filtered and sorted according to the RAG model token constraints and the output scores, and deduplication filtering is performed to obtain multiple processed initial answer basis criteria. Using natural language processing technology, the content of each processed initial answer basis criteria is simplified and summarized to obtain the final answer basis criteria. Based on the question and the basis for the final answer, the RAG model is used to generate the response.

[0007] By adopting the above scheme, the RAG model is optimized using answer pairs containing text fragments as supporting evidence, thereby improving the accuracy of the retrieval output. Text fragmentation of the knowledge base is used to replace word-by-word keyword retrieval analysis, thereby improving information coverage and ensuring the retention of key answer evidence. Natural language processing technology is used to expand the questions and retrieve the initial answer evidence in the optimized RAG model. The scoring model is used to filter, sort, and deduplicate the initial answer evidence, removing invalid information while adapting to the context length limit of small-scale language models.

[0008] Preferred options also include: Based on adjusting the text format of the answers in the knowledge base associated with the bidding scenario according to the text segmentation processing method, we continue to convert the text into vectors according to the large and small blocks of text to complete the vectorized storage of each text block, and generate a knowledge base that allows the vectors corresponding to the small text blocks to index the vectors corresponding to the large text blocks. The second validation set is obtained by replacing the question-answer pairs in the bidding scenario, which contain the answer basis stored in the form of vectors corresponding to large and small blocks of text, with the answer basis stored in the form of text fragments. The second validation set is then used to optimize the retrieval of the RAG model. During the process of using the optimized RAG model to search the knowledge base associated with the bidding scenario, when the vector corresponding to a small piece of text is retrieved, the vector corresponding to a related large piece of text is indexed, thereby obtaining multiple initial answer bases containing large pieces of text.

[0009] By adopting the above scheme, in the bidding Q&A scenario, the text format of the answer basis in the knowledge base is processed into large and small blocks and converted into vector storage. This creates an efficient knowledge base that can index the vector corresponding to the large block of text with the vector corresponding to the small block of text. This improves the information coverage during the retrieval process, especially when facing short consultation questions, and obtains relevant complete large block of text content, avoiding the omission of effective information due to differences in text length.

[0010] Preferred options also include: The text format of the answers in the knowledge base associated with the bidding scenario is adjusted according to different text segmentation processing methods, and different types of knowledge bases are generated accordingly so that each type of knowledge base stores the answers whose text format is adjusted according to a single text segmentation processing method; the different text segmentation processing methods include: clauses, paragraphs, and chapters; In the bidding scenario, the question-answer pairs containing the answer basis stored in the form of text generated by single text segmentation are obtained as the third validation set. The RAG model is optimized by using the third validation set, and correspondingly, several RAG models optimized by retrieval are obtained. For the expanded target problem, machine learning algorithms are used to evaluate the complexity and importance of the expanded problem information, respectively, and a weighted score result of the problem complexity score and the problem information importance score is obtained. Based on the weighted score result, a preset text segmentation processing method corresponding to the range of the score result value is matched. Based on the pre-defined text segmentation processing method, a matching and optimized RAG model is used to search the knowledge base where the text format of the stored answer basis is the same as the text format generated by the pre-defined text segmentation processing method, and obtain multiple initial answer basis.

[0011] By adopting the above scheme, the text format of the answer basis in the knowledge base is adjusted according to different text segmentation processing methods (clauses, paragraphs, chapters), and a knowledge base with corresponding text formats is generated. In addition, the text segmentation processing method is matched with the evaluation results of the complexity of the target question and the importance of information. The adaptive matching retrieval optimization RAG model is used to search in the knowledge base with specific formats, which effectively improves the accuracy and relevance of multiple initial answer basis.

[0012] Preferred options also include: When adjusting the text format of the answers in the knowledge base associated with the bidding scenario according to the text segmentation processing method, natural language analysis technology is used to analyze the logical coherence and information relevance of multiple text segments, and the logical coherence strength and information relevance strength of multiple text segments with logical coherence and / or information relevance are evaluated respectively; a tagging mechanism is introduced to display and distinguish multiple text segments with different logical coherence strength and information relevance strength. After simplifying and summarizing the content of each initial answer, the simplified initial answer is sorted and reorganized according to the coherence and information relevance marked in the original text content. This ensures that the text fragments of the answer that belong to the context are sorted adjacently, and the text fragments of the answer that have stronger information relevance are sorted closer together, thus obtaining the final answer.

[0013] By adopting the above scheme, after obtaining the initial basis for the answer, the content of the initial basis is simplified and summarized, and sorted and organized according to its logical coherence and information relevance. This makes the final answer basis not only more concise and clear, but also ensures the reasonable arrangement of context and related content, thereby improving the accuracy and rationality of the small-scale language model in the bidding question-and-answer scenario.

[0014] Preferred options also include: The system receives user feedback data on the generated answers in real time. When the user's satisfaction with the feedback is lower than the preset satisfaction level, the system displays the basis of the initial answer and prompts the user to evaluate the satisfaction level of the initial answer basis. The system also retains the user's satisfaction data on the initial answer basis. When the frequency of user satisfaction being lower than the preset satisfaction level within a preset time period is greater than the preset frequency, determine whether the frequency of user satisfaction with the initial answer basis being lower than the preset satisfaction level within the corresponding preset time period is greater than the preset frequency. If so, select to optimize and train the RAG model optimized for retrieval; otherwise, optimize and train the initial answer basis rating model.

[0015] By adopting the above approach, the user feedback mechanism is used to determine whether the root cause of user dissatisfaction with the generated answers lies in problems with the retrieval optimization RAG model or the scoring model. The optimization targets are then adaptively selected based on the problems to improve user satisfaction with the generated answers.

[0016] Preferred options also include: Before adjusting the text format of the answer basis in the knowledge base associated with the bidding scenario according to the text segmentation processing method, the information of the answer basis in the knowledge base of the bidding scenario is expanded, including: supplementing the information by collecting the bidding text content associated with the knowledge graph in the bidding scenario, or supplementing the information by collecting the text content of the multimodal data transformation in the bidding documents in the bidding scenario.

[0017] By adopting the above approach, the text content associated with each bidding document or the text content transformed from multimodal data in the bidding document can be collected using knowledge graphs in the bidding scenario to supplement the answer, thereby enriching the content of the answer basis at the source and improving the quality of the answer.

[0018] Preferred options also include: The text format of the answer basis in the knowledge base associated with the bidding scenario, which is adjusted according to the text segmentation processing method, includes: segmenting the text content of the answer basis in the knowledge base associated with the bidding scenario according to the preset text segmentation processing method, and adding the outline title of the segment to the answer basis before each text format.

[0019] By adopting the above approach and adding the corresponding outline title to each piece of text, the information coverage in the subsequent retrieval process can be effectively improved, helping the language model to more accurately locate relevant knowledge points, thereby improving the accuracy of the answers.

[0020] Secondly, this application provides a RAG-enhanced small-scale language model question-answering accuracy improvement system, comprising: The knowledge base segmentation adjustment module is used to adjust the text format of the answers in the knowledge base associated with the bidding scenario according to the text segmentation processing method. The RAG model retrieval optimization module is used to obtain question-answer pairs containing answer evidence stored in the form of text fragments in the bidding scenario as the first validation set, and to optimize the RAG model retrieval using the first validation set. The RAG model initial detection result acquisition module is used to expand the content or number of questions input to the RAG model using natural language technology. The expanded questions are then used as input, and the optimized RAG model is used to search the knowledge base associated with the bidding scenario to obtain multiple initial answer bases. The RAG model final result acquisition module is used to construct an initial answer basis scoring model based on deep learning algorithms within the RAG model framework. It outputs multiple initial answer basis scores separately, filters and sorts the multiple initial answer basis scores according to the RAG model token restrictions and output scores, and performs deduplication filtering to obtain the processed multiple initial answer basis scores. Using natural language processing technology, it simplifies and summarizes the content of each processed initial answer basis score to obtain the final answer basis. The RAG model-based answer generation module is used to generate answers based on the question and the criteria for the final answer using the RAG model.

[0021] By adopting the above scheme, the text segmentation processing method of the knowledge base associated with the bidding scenario is adjusted to cover more information; the retrieval performance of the RAG model is optimized and the matching accuracy is improved by using question-answer pairs containing answer evidence stored in the form of text segments as a validation set; the expansion of the input question, the screening, sorting and deduplication of multiple initial answer evidences ensure the quality and conciseness of the final selected answer evidence, and together improve the accuracy of small-scale language model question answering in the bidding scenario.

[0022] Thirdly, this application provides a computer-readable storage medium including a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to perform the method described above.

[0023] Fourthly, this application provides a computer device, the computer device including a memory, a processor and a program stored in the memory and executable thereon, the program being executed by the processor to implement the steps of the method described above.

[0024] In summary, this application has the following beneficial effects: 1. By optimizing the text segmentation process, the coverage of key information was improved; an optimization method based on the validation set was introduced to optimize the RAG model retrieval and improve the retrieval hit rate; natural language processing technology was used to expand the questions and simplify and rank the initial answer criteria, which improved the adaptability of the retrieval content, removed redundant and invalid information, and maximized the adaptation to the context parameter constraints of the small-scale language model, thereby enhancing the accuracy of the answers. Optimizations were completed from multiple aspects, including questions and retrieval, to improve the overall accuracy of question answer generation. 2. By using an adaptive text segmentation processing method, combined with large and small block segmentation strategies and vectorized storage, the coverage of key information and retrieval hit rate are improved. 3. Introduce a user feedback mechanism. Based on user satisfaction with the generated answers and the basis of the initial answers, optimize the RAG model or scoring model accordingly, thereby obtaining more accurate answers using a better model. Attached Figure Description

[0025] Figure 1 This is a flowchart of the method for improving the accuracy of question answering in a RAG-enhanced small-scale language model as described in a specific embodiment; Figure 2This is an illustration of the text segmentation process within the knowledge base in the RAG-enhanced small-scale language model question-answering accuracy improvement method described in a specific embodiment. Figure 3 This is a schematic diagram illustrating the process of obtaining the basis for the final answer in the RAG-enhanced small-scale language model question-answering accuracy improvement method described in a specific embodiment; Figure 4 This is a schematic diagram of the retrieval segmentation strategy in the RAG-enhanced small-scale language model question-answering accuracy improvement method described in a specific embodiment; Figure 5 This is a schematic diagram of the RAG-enhanced small-scale language model question-answering accuracy improvement system described in a specific embodiment. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0027] Considering that small-scale language models used in local area network environments struggle to achieve ideal response accuracy, especially due to limitations in context length and knowledge base fragmentation granularity, this application primarily adopts the following scheme, significantly improving the question-answering accuracy and applicability of the RAG-enhanced small-scale language model. A more detailed technical implementation scheme is described below.

[0028] Example 1 like Figure 1 As shown in the embodiment of this application, a method for improving the accuracy of question answering in a RAG-enhanced small-scale language model is disclosed, the steps of which include: S1. Adjust the text format stored in the knowledge base associated with the bidding scenario according to the text segmentation processing method.

[0029] Specifically, RAG-enhanced small-scale language models, hereinafter referred to as "RAG models," are an extension of a language model's ability to answer questions. They utilize extensive data from a knowledge base to enable the model to answer specialized questions in the bidding question-answering domain. Therefore, the knowledge base serves as the foundation for obtaining answers, and to maximize information coverage and ensure information completeness, such as... Figure 2 As shown, the texts used as the basis for answers, such as the text of the bidding documents and the text related to the bidding project stored in the knowledge base associated with the bidding scenario, are segmented. When segmenting the text, it is no longer further subdivided into sentences, but into clauses, paragraphs or chapters, so as to ensure that as much information as possible is covered when searching for fragments related to the question.

[0030] S2. Introduce validation set data to complete the retrieval optimization of the RAG model.

[0031] Specifically, the first validation set is obtained by obtaining answer pairs in the bidding scenario. The answer pairs include the answer basis of typical questions in the bidding scenario and expert-generated answers, and the answer basis is stored in the form of text fragments.

[0032] The RAG model primarily comprises two basic models: a retrieval model and an answer generation model. The retrieval model uses the semantics of the question to retrieve relevant answer criteria from a knowledge base. The answer generation model then uses these retrieved criteria to generate an answer to the question. Therefore, the first validation dataset is traversed, with questions from this dataset output to the RAG model. The retrieval results (answer criteria related to the question) are compared with the reference data (expert-generated answers) in the first validation set to confirm their similarity. Since the first validation set may contain multiple records, the overall decision on whether optimization is needed can be made from two perspectives: First, whether all criteria are matched. If any retrieval result has a similarity greater than 95% with a validation criterion, it is marked as a match, indicating a good retrieval result. Second, the similarity of each criterion is evaluated. The highest similarity value between each reference criterion and the retrieval result, or the average of multiple records, is taken. If the final similarity is greater than the preset retrieval similarity value, the current RAG retrieval result is good, and no retrieval optimization is needed; otherwise, the first validation set data is used as the training set for RAG model retrieval optimization.

[0033] The similarity calculation method involves vectorizing the two texts and calculating a value between 0% and 100% using the cosine algorithm. The higher the value, the greater the similarity. The embedding model used for vectorization is bge-large-zh-v1.5.

[0034] S3. Using the optimized RAG model, retrieve multiple initial answer bases.

[0035] Specifically, to avoid problems where the length difference between the consultation question and the text fragments in the knowledge base is too large, resulting in no matching, a question expansion method is adopted to retrieve more sufficient answer evidence. In this embodiment, natural language processing technology is used to expand the content or quantity of the questions input to the RAG model.

[0036] Using the expanded question as input, the optimized RAG model is used to search the knowledge base associated with the bidding scenario to obtain multiple initial answer bases.

[0037] S4. Filter and select from the multiple initial responses to obtain the final response.

[0038] While considering the need to cover as much information as possible, the size of the RAG model also needs to be taken into account. Within a limited range of context parameters, the effective information needs to be maximized. Therefore, some invalid information needs to be removed to adapt to the context length of the small model.

[0039] like Figure 3 As shown, each initial response is scored, and the scores are used to reorder multiple initial responses. A further filtering process, incorporating model token constraints, selects responses with higher relevance to the question. Specifically, an initial response scoring model is built within the RAG model framework (e.g., between the retrieval model and the response generation model) based on a deep learning algorithm. This model is trained using a training set of historically retrieved initial responses with various ratings. Multiple initial responses are output to the initial response scoring model. The initial responses are then filtered and sorted based on the RAG model token constraints and the output scores, retaining the top-scoring responses. For example, in a bidding inquiry scenario, the top four responses are selected.

[0040] While scoring and filtering, it is also necessary to perform deduplication and filtering to obtain multiple initial answers after processing.

[0041] In addition, to further improve the accuracy of the input-response generation model, each initial response basis after the above processing is either summarized or sorted based on the logic and information relevance of the text fragments in the initial response basis.

[0042] Specifically, natural language processing technology is used to condense and summarize the content of each initial response after processing to obtain the final response.

[0043] After simplifying and summarizing the content of each initial answer, the simplified initial answer is sorted and reorganized according to the coherence and information relevance marked in the original text content. This ensures that the text fragments of the answer that belong to the context are sorted adjacently, and the text fragments of the answer that have stronger information relevance are sorted closer together, thus obtaining the final answer.

[0044] Specifically, the strength of coherence and information relevance marked in the text content on which the initial answer is based includes: when adjusting the text form of the answer in the knowledge base associated with the bidding scenario according to the text segmentation processing method, using natural language analysis technology to analyze the logical coherence and information relevance of multiple text segments, and evaluating the strength of logical coherence and information relevance of multiple text segments with logical coherence and / or information relevance respectively; introducing a marking mechanism to display and distinguish multiple text segments with different strengths of logical coherence and information relevance.

[0045] S5. Based on the question and the basis for the final answer, generate an answer using the RAG model.

[0046] Specifically, the question and the final answer are input into the RAG model, and the final answer is generated using the answer generation model.

[0047] By employing the above methods, the knowledge base is rationally segmented into text, improving information coverage; the retrieval effect is enhanced by introducing natural language expansion technology and validation set optimization; and invalid information interference is reduced through scoring models and content simplification, thereby maximizing the amount of effective information under limited resource conditions.

[0048] Example 2 The method differs from Embodiment 1 above in that it combines a large and small block partitioning strategy with vectorized storage to further improve the coverage and retrieval hit rate of key information. The method also includes: like Figure 4 As shown, based on adjusting the text format of the answer basis in the knowledge base associated with the bidding scenario according to the text segmentation processing method, the vectorization storage of each text segment is completed by converting the text into vectors according to the size of the text segmentation, and a knowledge base is generated that allows the vector corresponding to the small text segment to index the vector corresponding to the large text segment.

[0049] That is, for each text segment, large blocks and small blocks of text are divided, and small blocks of text are generated from the large blocks. For example, large block A is divided into small blocks A_1, A_2, and A_3, and large block B is divided into B_1, B_2, and B_3. Corresponding small block indexes are generated so that the associated large blocks can be found through the small blocks.

[0050] To retrieve the corresponding large and small text blocks from the knowledge base, the text supporting the answers stored in the knowledge base related to the bidding scenario is segmented into large text blocks, which are then further segmented into smaller text blocks, and finally converted into large and small text block vectors for storage. Accordingly, to improve retrieval within the knowledge base in large and small text block format, answer pairs containing the answer support stored as vectors in the bidding scenario are used to replace question-answer pairs containing the answer support stored as text blocks in the bidding scenario as a second validation set. This second validation set is then used to optimize the RAG model's retrieval process.

[0051] In order to improve the coverage of key information, when the RAG model with optimized retrieval is used to search the knowledge base associated with the bidding scenario, when the vector corresponding to a small piece of text is retrieved, the vector corresponding to a large piece of text is indexed, thereby obtaining multiple initial answer bases containing large piece of text information.

[0052] Example 3 The difference from Example 1 lies in that different text segmentation methods are required for different questions based on their complexity and information importance. For example, simple and generally important questions do not require searching for answers with high information coverage to obtain highly accurate answers, while complex or relatively important questions require searching for answers with the highest possible information coverage. An adaptive slicing method is used to meet the needs of different questions. The method also includes: The text format of the answers in the knowledge base associated with the bidding scenario is adjusted according to different text segmentation processing methods, and different types of knowledge bases are generated accordingly. Each type of knowledge base stores the answer basis whose text format is adjusted according to a single text segmentation processing method. The different text segmentation processing methods include: clauses, paragraphs, and chapters. The first knowledge base stores the answer basis whose text format is adjusted according to the clause text segmentation processing method, the second knowledge base stores the answer basis whose text format is adjusted according to the paragraph text segmentation processing method, and the third knowledge base stores the answer basis whose text format is adjusted according to the chapter text segmentation processing method. Of course, different text processing methods can also be mixed and different types of knowledge bases can be generated accordingly, so that each type of knowledge base stores the answer basis whose text format is adjusted according to a mixed text segmentation processing method.

[0053] In addition, to better assist in retrieving the corresponding text segments, the text content of the answer basis in the knowledge base associated with the bidding scenario is segmented according to the preset text segmentation processing method, and the outline title of the segment is added before each text segment of the answer basis.

[0054] In order to adaptively retrieve the content of specific segments, the question-answer pairs containing the answer basis stored in the form of text generated by processing a single text segment in the bidding scenario are obtained as the third validation set. The RAG model is optimized by using the third validation set, and the RAG model with several retrieval optimizations is obtained accordingly. For the expanded target problem, machine learning algorithms are used to evaluate the complexity and importance of the expanded problem information, respectively, to obtain a weighted score of the problem complexity score and the problem information importance score. Based on the weighted score, a preset text segmentation processing method corresponding to the range of the score result value is matched; for example, the preset text segmentation processing method corresponding to the first score result value range is the clause text segmentation processing method, the preset text segmentation processing method corresponding to the second score result value range is the paragraph text segmentation processing method, and the preset text segmentation processing method corresponding to the third score result value range is the chapter text segmentation processing method.

[0055] The matching preset text segmentation processing method corresponds to the matching optimized RAG model. For example, if the current question matches the paragraph text segmentation processing method, the matching optimized RAG model related to the paragraph text segmentation processing method is used to search in the knowledge base where the text format of the stored answer basis is the same as the text format generated by the preset text segmentation processing method, and multiple initial answer basis are obtained. For example, the matching optimized RAG model is used to search in the second knowledge base where the text format of the paragraph text is the same.

[0056] Example 4 The difference from Embodiment 1 above is the introduction of a feedback mechanism to improve the accuracy of question answering. The method further includes: Receive real-time user feedback data on the generated answers, including user satisfaction. When a user's satisfaction with the feedback (e.g., 60 points) is lower than the preset satisfaction level (e.g., 80 points), the initial answer basis is displayed and the user is prompted to evaluate the satisfaction level of the initial answer basis. In this embodiment, the satisfaction level of all initial answer basis is evaluated one by one or as a whole, and the user's satisfaction data of the initial answer basis is retained, such as the average value of the satisfaction level of each evaluation or the overall satisfaction value.

[0057] If the frequency of user satisfaction being lower than the preset satisfaction level within a preset time period is greater than the preset frequency, it indicates that users are generally dissatisfied with the generated answers. The system then checks if the frequency of user satisfaction with the initial answer basis being lower than the preset satisfaction level within the corresponding preset time period is also greater than the preset frequency. If so, it indicates that users are generally dissatisfied with the initial answer basis for the generated answers, requiring further improvement in the search accuracy of the initial answer basis. Therefore, the system is selected to retrain the optimized RAG model. Otherwise, it indicates that users are generally satisfied with the initial answer basis for the generated answers, but are dissatisfied with the final generated answers, suggesting a potential large error in obtaining the final answer basis. Therefore, the system is selected to retrain the initial answer basis scoring model.

[0058] Example 5 The difference from Embodiment 1 above lies in the further optimization of the knowledge base construction process. In addition to static data collection, a dynamic monitoring function has been added, enabling timely updates to the knowledge base content and maintaining its timeliness and sophistication. The method also includes: Before adjusting the text format of the answers in the knowledge base associated with the bidding scenario according to the text segmentation processing method, the information of the answers in the bidding scenario knowledge base is expanded, specifically including: Information can be supplemented by collecting and associating bidding text content with knowledge graphs in bidding scenarios, or by collecting and supplementing text content converted from multimodal data in bidding documents in bidding scenarios, such as text content converted from video and audio.

[0059] like Figure 5 As shown in the embodiments, this application discloses a RAG-enhanced small-scale language model question-answering accuracy improvement system, specifically including: The knowledge base segmentation adjustment module 101 is used to adjust the text format of the answer basis in the knowledge base associated with the bidding scenario according to the text segmentation processing method; RAG model retrieval optimization module 102 is used to obtain question-answer pairs containing answer basis stored in the form of text fragments in the bidding scenario as the first validation set, and to optimize the RAG model retrieval using the first validation set; The RAG model initial detection result acquisition module 103 is used to expand the content or number of questions input to the RAG model using natural language technology. The expanded questions are used as input, and the optimized RAG model is used to search in the knowledge base associated with the bidding scenario to obtain multiple initial answer bases. The RAG model final result acquisition module 104 is used to construct an initial answer basis scoring model based on deep learning algorithms within the RAG model framework. It outputs multiple initial answer basis scores separately, filters and sorts the multiple initial answer basis scores according to the RAG model token restrictions and output scores, and performs deduplication filtering to obtain the processed multiple initial answer basis scores. Using natural language processing technology, it simplifies and summarizes the content of each processed initial answer basis score to obtain the final answer basis. The RAG model-generated answer acquisition module 105 is used to generate an answer based on the question and the basis for the final answer using the RAG model.

[0060] In a specific embodiment, the knowledge base segmentation adjustment module 101 is further used to adjust the text form of the answer basis in the knowledge base associated with the bidding scenario according to the text segmentation processing method, and then continue to convert the vectors generated by the division of large and small blocks of text to complete the vectorized storage of each block of text, thereby generating a knowledge base that allows the vectors corresponding to large blocks of text to be indexed by the vectors corresponding to small blocks of text. The RAG model retrieval optimization module 102 is further used to obtain the answer pair containing the answer basis stored in the form of vector corresponding to large and small blocks of text in the bidding scenario to replace the question-answer pair containing the answer basis stored in the form of text fragments in the bidding scenario as the second verification set, and then use the second verification set to optimize the retrieval of the RAG model. The RAG model initial detection result acquisition module 103 is also used to index the corresponding vector of related large text when the vector corresponding to a small text is retrieved during the retrieval process in the knowledge base associated with the bidding scenario using the optimized RAG model, thereby obtaining multiple initial answer bases containing large text.

[0061] In one specific embodiment, the knowledge base segmentation adjustment module 101 is further configured to adjust the text format of the answer basis in the knowledge base associated with the bidding scenario according to different text segmentation processing methods and generate different types of knowledge bases accordingly, so that each type of knowledge base stores the answer basis whose text format is adjusted according to a single text segmentation processing method; the different text segmentation processing methods include: clauses, paragraphs, and chapters; The RAG model retrieval optimization module 102 is also used to obtain question-answer pairs containing the answer basis stored in the form of text generated by single text segmentation in the bidding scenario as a third verification set, and to use the third verification set to perform retrieval optimization on the RAG model, thereby obtaining a number of RAG models optimized by retrieval. The RAG model initial detection result acquisition module 103 is also used to evaluate the complexity and importance of the expanded question using machine learning algorithms for the expanded target question, obtain a weighted score result value of the question complexity score and the question information importance score, match the preset text segmentation processing method corresponding to the score result value range according to the weighted score result value, match the optimized RAG model according to the matched preset text segmentation processing method, and use the matched optimized RAG model to search in the knowledge base where the text format of the stored answer basis is the same as the text format generated by the preset text segmentation processing method to obtain multiple initial answer basis.

[0062] In a specific embodiment, the RAG model final test result acquisition module 104 is further configured to, when adjusting the text form of the answer basis in the knowledge base associated with the bidding scenario according to the text segmentation processing method, use natural language analysis technology to perform logical coherence and information relevance analysis on multiple text segments, and evaluate the logical coherence strength and information relevance strength of multiple text segments with logical coherence and / or information relevance respectively; introduce a marking mechanism to display and distinguish multiple text segments with different logical coherence strength and information relevance strength; after simplifying and summarizing the content of each initial answer basis after processing, sort and reorganize each initial answer basis after content simplification according to the coherence strength and information relevance strength marked in the text content of the original initial answer basis, so that the answer basis text segments belonging to the context content are sorted adjacently and the answer basis text segments with stronger information relevance are sorted closer together, thus obtaining the final answer basis.

[0063] A specific embodiment further includes: a RAG model-generated answer verification module 106, used to receive user feedback data on the generated answer in real time; when the user's satisfaction with the feedback is lower than a preset satisfaction level, the module displays the initial answer basis and prompts the user to evaluate the satisfaction level of the initial answer basis, and retains the user's satisfaction data for the initial answer basis; when the frequency of user satisfaction being lower than the preset satisfaction level within a preset time period is greater than a preset frequency, the module determines whether the frequency of user satisfaction with the initial answer basis being lower than the preset satisfaction level within the corresponding preset time period is greater than the preset frequency; if so, the module selects to optimize and train the RAG model after retrieval optimization; otherwise, it optimizes and trains the initial answer basis scoring model.

[0064] A specific embodiment further includes: a knowledge base content supplementation module 107, used to expand the information of the answer basis in the knowledge base of the bidding scenario before adjusting the text form of the answer basis in the knowledge base associated with the bidding scenario according to the text segmentation processing method, including: supplementing information by collecting the bidding text content associated with the knowledge graph in the bidding scenario, or supplementing information by collecting the text content of the multimodal data transformation in the bidding documents in the bidding scenario.

[0065] This application also discloses a computer-readable storage medium.

[0066] Specifically, the computer-readable storage medium stores a computer program that can be loaded by a processor and executed, such as the RAG-enhanced small-scale language model question-answering accuracy improvement method described above. The computer-readable storage medium includes, for example, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0067] This application also discloses a computer device.

[0068] Specifically, the computer device includes a memory and a processor, and the memory stores a computer program that can be loaded by the processor and executed to improve the accuracy of the RAG-enhanced small-scale language model question answering method described above.

[0069] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A method for improving the accuracy of question answering in a RAG-enhanced small-scale language model, characterized in that, include: Adjust the text format of the answers in the knowledge base associated with the bidding scenario according to the text segmentation processing method; The first validation set is obtained by acquiring question-answer pairs containing the answer basis stored in the form of text fragments in the bidding scenario, and the RAG model is optimized by using the first validation set. By using natural language processing technology, the content or number of questions input to the RAG model are expanded. The expanded questions are then used as input, and the optimized RAG model is used to search the knowledge base associated with the bidding scenario to obtain multiple initial answer bases. Based on deep learning algorithms, an initial answer basis scoring model is constructed within the RAG model framework. Multiple initial answer basis criteria are output separately into the initial answer basis scoring model. The multiple initial answer basis criteria are filtered and sorted according to the RAG model token constraints and the output scores, and deduplication filtering is performed to obtain the processed multiple initial answer basis criteria. Using natural language processing technology, the content of each initial response is simplified and summarized to obtain the basis for the final response. Based on the question and the criteria for the final answer, the answer is generated using the RAG model; it also includes: Based on adjusting the text format of the answer basis in the knowledge base associated with the bidding scenario according to the text segmentation processing method, we continue to convert the text into vectors according to the large and small blocks of text to complete the vectorized storage of each text block, and generate a knowledge base that allows the vector corresponding to the small text block to index the vector corresponding to the large text block. The second validation set is obtained by replacing the question-answer pairs in the bidding scenario, which contain the answer basis stored in the form of text corresponding to large and small blocks, with the answer basis stored in the form of text fragments. The second validation set is then used to optimize the retrieval of the RAG model. During the process of using the optimized RAG model to search the knowledge base associated with the bidding scenario, when the vector corresponding to a small piece of text is retrieved, the vector corresponding to a related large piece of text is indexed, thereby obtaining multiple initial answer bases containing large pieces of text. It also includes: when adjusting the text format of the answer basis in the knowledge base associated with the bidding scenario according to the text segmentation processing method, using natural language analysis technology to analyze the logical coherence and information relevance of multiple text segments, and evaluating the logical coherence strength and information relevance strength of multiple text segments with logical coherence and / or information relevance respectively; introducing a tagging mechanism to display and distinguish multiple text segments with different logical coherence strength and information relevance strength; After simplifying and summarizing each initial response, the simplified initial responses are then sorted and reorganized according to the strength of coherence and information relevance marked in the original initial response text. This ensures that response text fragments belonging to the same context are ordered adjacently, and response text fragments with stronger information relevance are ordered closer together, thus obtaining the final response; this also includes: The text format of the answers in the knowledge base associated with the bidding scenario is adjusted according to different text segmentation processing methods, and different types of knowledge bases are generated accordingly so that each type of knowledge base stores the answers whose text format is adjusted according to a single text segmentation processing method; the different text segmentation processing methods include: clauses, paragraphs, and chapters; In the bidding scenario, the question-answer pairs containing the answer basis stored in the form of text generated by single text segmentation are obtained as the third validation set. The RAG model is optimized by using the third validation set, and correspondingly, several RAG models optimized by retrieval are obtained. For the expanded target problem, machine learning algorithms are used to evaluate the complexity and importance of the expanded problem information, respectively, and a weighted score result of the problem complexity score and the problem information importance score is obtained. Based on the weighted score result, a preset text segmentation processing method corresponding to the range of the score result value is matched. Based on the pre-defined text segmentation processing method, a matching and optimized RAG model is used to search the knowledge base where the text format of the stored answer basis is the same as the text format generated by the pre-defined text segmentation processing method, and obtain multiple initial answer basis.

2. The method for improving the accuracy of question answering in RAG-enhanced small-scale language models according to claim 1, characterized in that, Also includes: The system receives user feedback data on the generated answers in real time. When the user's satisfaction with the feedback is lower than the preset satisfaction level, the system displays the basis of the initial answer and prompts the user to evaluate the satisfaction level of the initial answer basis. The system also retains the user's satisfaction data on the initial answer basis. When the frequency of user satisfaction being lower than the preset satisfaction level within a preset time period is greater than the preset frequency, determine whether the frequency of user satisfaction with the initial answer basis being lower than the preset satisfaction level within the corresponding preset time period is greater than the preset frequency. If so, select to optimize and train the RAG model optimized for retrieval; otherwise, optimize and train the initial answer basis rating model.

3. The method for improving the accuracy of question answering in RAG-enhanced small-scale language models according to claim 1, characterized in that, Also includes: Before adjusting the text format of the answer basis in the knowledge base associated with the bidding scenario according to the text segmentation processing method, the information of the answer basis in the knowledge base of the bidding scenario is expanded, including: supplementing the information by collecting the bidding text content associated with the knowledge graph in the bidding scenario, or supplementing the information by collecting the text content of the multimodal data transformation in the bidding documents in the bidding scenario.

4. The method for improving the accuracy of question answering in RAG-enhanced small-scale language models according to claim 1, characterized in that, Also includes: The text format of the answer basis in the knowledge base associated with the bidding scenario, which is adjusted according to the text segmentation processing method, includes: segmenting the text content of the answer basis in the knowledge base associated with the bidding scenario according to the preset text segmentation processing method, and adding the outline title of the segment to the answer basis before each text format.

5. A RAG-enhanced small-scale language model question-answering accuracy improvement system, characterized in that, include: The knowledge base segmentation adjustment module is used to adjust the text format of the answer basis in the knowledge base associated with the bidding scenario according to the text segmentation processing method; it is also used to further convert the text format of the answer basis in the knowledge base associated with the bidding scenario according to the text segmentation processing method, and then convert the vectors generated by the divided large and small blocks of text to complete the vectorized storage of each block of text, generating a knowledge base that allows the vectors corresponding to the small blocks of text to index the vectors corresponding to the large blocks of text. The RAG model retrieval optimization module is used to obtain question-answer pairs containing answer evidence stored in text fragments in the bidding scenario as a first validation set, and to optimize the RAG model retrieval using the first validation set; it is also used to obtain answer pairs containing answer evidence stored in the form of vectors corresponding to large and small text blocks in the bidding scenario to replace question-answer pairs containing answer evidence stored in text fragments in the bidding scenario as a second validation set, and to optimize the RAG model retrieval using the second validation set; it is also used to obtain question-answer pairs containing answer evidence stored in the form of text generated by single text fragment processing in the bidding scenario as a third validation set, and to optimize the RAG model retrieval using the third validation set, thereby obtaining RAG models optimized using several retrieval methods. The RAG model initial detection result acquisition module is used to expand the content or quantity of the input question to the RAG model using natural language processing technology. The expanded question is then used as input, and the optimized RAG model is used to search the knowledge base associated with the bidding scenario to obtain multiple initial answer bases. Furthermore, during the search process in the knowledge base associated with the bidding scenario using the optimized RAG model, when a vector corresponding to a small text block is retrieved, the corresponding vector of a related large text block is indexed, thereby obtaining multiple initial answer bases containing the large text block. Additionally, for the expanded target question, machine learning algorithms are used to evaluate the complexity and importance of the expanded question information, obtaining a weighted score result value for the question complexity score and the question information importance score. Based on the weighted score result value, a preset text segmentation processing method corresponding to the score result value range is matched. Based on the matched preset text segmentation processing method, a search-optimized RAG model is matched, and the matched search-optimized RAG model is used to search the stored answer base text format, which is the same as the text format generated by the preset text segmentation processing method, to obtain multiple initial answer bases. The RAG model final result acquisition module is used to construct an initial answer basis scoring model within the RAG model framework based on deep learning algorithms. It outputs multiple initial answer basis scores separately, filters and sorts them according to RAG model token constraints and output scores, and performs deduplication filtering to obtain the processed initial answer basis. Using natural language processing (NLP) technology, it simplifies and summarizes each processed initial answer basis to obtain the final answer basis. Furthermore, when adjusting the text format of answer basis in the knowledge base associated with the bidding scenario according to text segmentation, it uses NLP analysis technology to perform logical coherence and information linking analysis on multiple text segments. The system performs coherence analysis and evaluates the strength of logical coherence and information relevance for multiple text segments with logical coherence and / or information relevance. A marking mechanism is introduced to display and distinguish multiple text segments with different strengths of logical coherence and information relevance. After simplifying and summarizing the content of each initial answer, the system reorganizes each simplified initial answer according to the strength of coherence and information relevance marked in the original initial answer text content. This ensures that answer text segments belonging to the context are arranged adjacently, and answer text segments with stronger information relevance are arranged closer together, thus obtaining the final answer. The RAG model-based answer generation module is used to generate answers based on the question and the criteria for the final answer using the RAG model.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the method as described in any one of claims 1 to 4.

7. A computer device, characterized in that, The computer device includes a memory, a processor, and a program stored in and executable on the memory, the program being executed by the processor to implement the steps of the method as described in any one of claims 1 to 4.

Images