Large model retrieval enhancement method based on laws and regulations
By tagging and interpreting regulatory data, and combining semantic similarity matching and paragraph segmentation, effective regulatory paragraphs are selected, solving the accuracy and timeliness issues of large AI models in the field of regulations, and achieving more reliable regulatory question answering.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-10
- Publication Date
- 2026-03-10
AI Technical Summary
Existing large AI models lack accuracy and timeliness in answering questions in the field of regulations, and are prone to generalized and inaccurate answers, making it difficult to provide reliable answers related to regulations.
The RAG large model is used to crawl publicly available legal data from government websites, which is then tagged and analyzed. The law-similar algorithm is used for semantic similarity matching to select valid legal paragraphs and generate the final answer. The range of tags is limited and the grammatical structure of the legal data interpretation is learned. The BERT text segmentation small model is combined to segment paragraphs and determine high-frequency words. Multi-level screening thresholds and similarity scores are set.
It improves the real-time performance and accuracy of data in regulatory AI scenarios, avoids ambiguity in the understanding of regulations caused by data lag, and provides authoritative and clear answers.
Smart Images
Figure CN121636652A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of large model retrieval enhancement, in particular to a large model retrieval enhancement method based on regulations. BACKGROUND
[0002] In the existing market, the accuracy, authority and timeliness of the common AI large model question and answer effect have always been a technical difficulty that needs to be broken through, especially in the field of regulations, more attention is paid to the accuracy, authority and timeliness of the AI large model reply content. Although the AI large model has strong understanding and reasoning ability, it performs well in solving most problems, but in the vertical field, problems such as answer generalization and inaccurate answer may occur. How to avoid model illusion and make the model answer regulation-related questions more reliable has become a big difficulty in the technical level of AI large model in this field. SUMMARY
[0003] In view of this, the purpose of the present application is to provide a large model retrieval enhancement method based on regulations to solve or at least partially solve the above-mentioned problems existing in the prior art.
[0004] To achieve the above-mentioned purpose, the present application provides a large model retrieval enhancement method based on regulations, which comprises the following steps: S101, grabbing public regulation data of each government website to a local database through a RAG large model; S102, training a regulation AI model based on the grabbed public regulation data, and performing labelization processing and paragraph interpretation processing on the regulation data through the trained regulation AI model, and integrating the processed regulation data into a regulation data set; S103, collecting user's asking questions, and performing semantic similarity matching between the user's asking questions and the regulation data set through a law-similar algorithm; S104, according to the semantic similarity matching result in step S103, extracting regulations related to the user's asking questions from the regulation data set, performing paragraph segmentation processing on the regulations, obtaining regulation paragraphs, further screening out effective regulation paragraphs according to the regulation paragraphs, and generating a final answer for the user according to the effective regulation paragraphs.
[0005] Further, the step S102 comprises: The regulation AI model is trained using regulation data, a label sample set is constructed using expert experience, the label sample set is input into the regulation AI model, the regulation AI model is allowed to use the label sample set to perform labelization processing on the regulation data, the regulation data after the labelization processing is interpreted, and the regulation AI model is allowed to learn the syntax deconstruction and mode of regulation data interpretation.
[0006] Further, the law-similar algorithm is used to match the user's question with the legal data set in semantic similarity, which is represented as follows:
[0007] wherein, is the matching score of the question and the legal data set, is the current number of word groups, is the total number of word groups divided by the user's question, is the weight of the current word group in the user's question, is the weight of the related word associated with the current word group in the user's question, is the inverse document frequency calculation, is the word frequency calculation, is the word offset score, is the freshness score of the current word group, is the special score. Further, the step S103 specifically comprises the following steps:
[0008] Further, the step S103 specifically comprises the following steps: S31, the extracted regulations are divided into paragraphs according to the strong rules and small model method, the strong rules are matched to special characters for division, and the small model is a BERT text splitting small model; S32, based on the regulation paragraph division processing of step S31, the effective regulation paragraph is determined by an effective regulation paragraph matching method; S33, after the effective regulation paragraph is selected based on step S32, the similarity between each effective regulation paragraph and the user's question is calculated, and the effective regulation paragraph is scored according to the similarity; S34, based on the similarity score of step S33, the optimal effective regulation paragraph is further selected from the effective regulation paragraph, and the length of the optimal effective regulation paragraph is limited to avoid content generalization; S35, based on the optimal effective regulation paragraph selected in step S34, the final answer is generated for the user in combination with the user's question.
[0009] Further, the effective regulation paragraph matching method is used to determine the effective regulation paragraph, which comprises: The high-frequency word judgment is performed on the regulation paragraph, the high-frequency word judgment threshold is calculated, and it is judged whether the word in the regulation paragraph exceeds the high-frequency word judgment threshold. If the number of occurrences of any word in the regulation paragraph exceeds the high-frequency word judgment threshold, the importance of the word in the regulation paragraph is reduced, which is represented as follows:
[0010] wherein, is the high-frequency word judgment threshold, is the length of the regulatory paragraph content; Based on the high-frequency word judgment of the regulatory paragraph, the weight score of each regulatory paragraph is calculated, represented as follows:
[0011] wherein, is the total weight score of the regulatory paragraph, is the query word, is the query word set, is the indicator function, is the high-frequency word set; Set multi-level screening threshold, the multi-level screening threshold includes the basic screening threshold, the basic screening threshold is set according to the number of word groups in the user's question problem, represented as follows:
[0012]
[0013] wherein, is the level screening threshold, is the basic screening threshold; When the total weight score of the regulatory paragraph is greater than the multi-level screening threshold, the regulatory paragraph is a valid regulatory paragraph, represented as follows:
[0014] wherein, is the current regulatory paragraph effective, 1 is valid, 0 is invalid.
[0015] Further, the similarity score of the valid regulatory paragraph is calculated according to the similarity, represented as follows:
[0016] wherein, is the final similarity score, is the similarity between the valid regulatory paragraph and the question problem, is the similarity between the policy number of the valid regulatory paragraph and the question problem, is the user's question problem text, is the content of the valid regulatory paragraph, is the policy number of the valid regulatory paragraph.
[0017] Further, the optimal valid regulatory paragraph is further screened out from the valid regulatory paragraph, represented as follows:
[0018]
[0019]
[0020] wherein, is the optimal effective regulation paragraph length, is the total length of the combined effective regulation paragraph, is the paragraph set after the first level screening, is the total number of paragraphs in the paragraph set, is the text length function, is a single paragraph, is the first paragraph.
[0021] Further, the optimal effective regulation paragraph length is limited as follows:
[0022] wherein, is the final output effective regulation paragraph length, is the total number of words in the effective regulation paragraph.
[0023] Compared with the prior art, the beneficial effects of the present application are: The present application proposes a regulation-based large model retrieval enhancement method, which limits the label range of the regulation AI model to not exceed the self-defined label library through labelization processing and paragraph interpretation processing of regulation data, and makes it learn the syntax deconstruction and method of regulation data interpretation. Through the law-similar algorithm and the semantic similarity matching of the regulation data set, the similarity matching of the user's question and the regulation data is realized. Through the effective regulation paragraph, according to the user's business question, a more optimal answer is provided for the user, solving the problem of poor real-time data in the regulation AI scene, and avoiding the ambiguity of regulation understanding caused by data lag. BRIEF DESCRIPTION OF DRAWINGS
[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only preferred embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.
[0025] Figure 1 A regulation-based large model retrieval enhancement method flowchart provided by the present application. DETAILED DESCRIPTION
[0026] The principles and characteristics of the present application are described below in conjunction with the accompanying drawings, which are provided only for the purpose of explaining the present application and are not intended to limit the scope of the present application.
[0027] With reference to Figure 1 The embodiment provides a large model retrieval enhancement method based on regulations, which comprises the following steps: S101, through the RAG large model, the public regulation data of each government website is regularly, directionally and compliantly captured to the local database; S102, and the regulation data is labeled and processed by the trained regulation AI model, and the processed regulation data is integrated into a regulation data set, specifically comprising: The regulation AI model is trained using regulation data, a label sample set is constructed using expert experience, the label sample set is input into the regulation AI model, the regulation AI model is used to label the regulation data using the label sample set, the labeling range of the regulation AI model is limited to not exceed the preset custom label library, the labeled regulation data is interpreted, the regulation AI model learns the syntax deconstruction and method of regulation data interpretation, and the official data of the government department website is searched in real time as a reference basis for AI automatic interpretation of the regulation.
[0028] S102, the user's asking question is collected, and the law-similar algorithm and the regulation data set are matched in semantic similarity according to the user's asking question, which is expressed as follows: The user's asking question is processed by word segmentation into several word groups, and the similarity score of each word group is calculated.
[0029]
[0030] Among them, is the matching score of the asking question and the regulation data set, is the number of current word groups, is the total number of word groups divided by the user's asking question, is the weight of the current word group in the user's asking question (related to the number of occurrences and the part of speech, the higher the number of occurrences, the higher the weight), is the weight of the related word associated with the current word group in the user's asking question (related to the number of occurrences and the part of speech, the higher the number of occurrences, the higher the weight), inverse document frequency calculation, word frequency calculation, word offset score (in the user's asking question, the current word group The closer the distance between the user's question phrase and the distance between the phrase in each regulation and other user question phrases, the higher the word shift score. The freshness score is the distance between the user's question phrase and the current phrase in the regulation. The closer the distance between the user's question phrase and the distance between the phrase in each regulation and other user question phrases, the higher the word shift score. The special score is the distance between the user's question phrase and the current phrase in the regulation.
[0031] S103, according to the semantic similarity matching result in step S103, extract the regulations related to the user's question in the regulation data set, perform paragraph segmentation processing on the regulations, obtain regulation paragraphs, further filter out effective regulation paragraphs according to the regulation paragraphs, and generate a final answer for the user according to the effective regulation paragraphs, which specifically includes the following steps: S31, the extracted regulations are segmented into paragraphs according to the strong rules and small models, the strong rules are matched to / r, / n, and other special characters for segmentation processing, and the small model is a BERT text splitting small model, which performs semantic understanding on the regulations and AI segmentation; S32, based on the regulation paragraph segmentation processing in step S31, determine the effective regulation paragraphs by the effective-paragraph-pick effective regulation paragraph matching method, which specifically includes: The effective-paragraph-pick effective regulation paragraph matching method includes: The high-frequency word judgment is performed on the regulation paragraphs, the high-frequency word judgment threshold is calculated, and it is judged whether the words in the regulation paragraphs exceed the high-frequency word judgment threshold. If the number of occurrences of any word in the regulation paragraph exceeds the high-frequency word judgment threshold, the importance of the word in the regulation paragraph is reduced, which will reduce the final score of the regulation paragraph, which is represented as follows:
[0032] Wherein, The high-frequency word judgment threshold is The length of the regulation paragraph content; Based on the high-frequency word judgment of the regulation paragraph, the weight score of each regulation paragraph is calculated, which is represented as follows:
[0033] wherein, is the total score of the regulatory paragraph weight, is the query word, is the set of query words, is the indicator function (1 if the condition is true, otherwise 0), is the set of high-frequency words; Set multi-level screening threshold, the multi-level screening threshold includes the basic screening threshold, the basic screening threshold is set according to the number of words in the user's question, and the basic screening threshold also sets a weight reduction mechanism to avoid setting the threshold too high, resulting in no effective paragraph in the end. The weight reduction mechanism guarantees the output of the final effective paragraph. When there is no effective paragraph output, the weight will be reduced to the original 80%, and all paragraphs will be re-judged for threshold. In this way, the weight reduction operation is repeated 5 times, which is represented as follows:
[0034]
[0035] wherein, is the level screening threshold, is the basic screening threshold; When the total score of the regulatory paragraph weight is greater than the multi-level screening threshold, the regulatory paragraph is an effective regulatory paragraph, which is represented as follows:
[0036] wherein, is the current regulatory paragraph, 1 is valid, and 0 is invalid.
[0037] S33, based on the screening of effective regulatory paragraphs in step S32, the similarity between each effective regulatory paragraph and the user's question is calculated, and the effective regulatory paragraphs are scored according to the similarity, which is represented as follows:
[0038] wherein, is the final similarity score, is the similarity between the effective regulatory paragraph and the question, is the similarity between the policy number of the effective regulatory paragraph and the question, is the user's question text, is the content of the effective regulatory paragraph, is the policy number of the effective regulatory paragraph.
[0039] S34, based on the similarity score in step S33, the optimal effective regulatory paragraph is further screened from the effective regulatory paragraphs, and the length of the optimal effective regulatory paragraph is limited to avoid content generalization, which specifically includes: The process of further filtering out the optimal effective legal paragraphs and prioritizing their submission to the legal AI model allows the model to focus more on content with higher similarity scores during analysis. This is illustrated as follows:
[0040]
[0041]
[0042] in, The optimal and effective regulatory paragraph, The total length of the effective legal paragraphs after merging. For the first The collection of paragraphs after level filtering This represents the total number of paragraphs in the paragraph set. This is a text length function. This refers to a single paragraph, a text paragraph that has been filtered through the k-th level, corresponding to the `part` variable in the code. For the first One paragraph.
[0043] The optimal effective length of the legal paragraph is limited, taking into account the context length and model analysis efficiency of different legal AI models. The maximum number of effective legal paragraphs pushed to the legal AI model context is limited to 20,000 characters (this value can be increased or decreased according to different business scenarios, computing resources, and models. The current default value of 20,000 is a parameter that has undergone extensive testing and has better compatibility with various scenarios and models, basically adapting to all situations), as shown below:
[0044] in, The final output is the length of the valid regulatory paragraph.
[0045] S35. Based on the optimal and effective legal paragraphs selected in step S34, and combined with the user's question, generate the final answer for the user, specifically including: Based on the user's question, the system automatically matches relevant regulations and generates a final answer. Each final answer is supported by clear evidence, such as the title of the regulation, the document number of the regulation, and whether the regulation is valid. Users can also quickly view the original text of the regulation based on the provided evidence, supporting traceability and greatly enhancing the authority of the answer.
[0046] The method disclosed by the application is based on the Law-RAG large model retrieval enhancement technology improved from the RAG large model, the Law-RAG large model retrieval enhancement technology can be connected as a plug-in to any AI model, thereby improving the AI inference and question and answer capabilities of the AI model, and assisting enterprise users and individual users in obtaining reliable replies of the AI large model connected with the plug-in algorithm when asking related regulatory issues.
[0047] The above description is only the preferred embodiment of the application, and is not intended to limit the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.
Claims
1. A regulatory-based large model retrieval enhancement method, characterized in that, The method comprises the following steps: S101, grabbing public regulation data of each government website to a local database through a RAG large model; S102, based on the grabbed public regulation data, training a regulation AI model, and processing the regulation data through the trained regulation AI model for labeling and paragraph interpretation, and integrating the processed regulation data into a regulation data set; S103, collecting the user's question, and through the law-similar algorithm, the semantic similarity of the user's question and the regulation data set is matched; S104, according to the semantic similarity matching result in step S103, extracting the regulations related to the user's question in the regulation data set, performing paragraph segmentation processing on the regulations, obtaining regulation paragraphs, and further screening effective regulation paragraphs according to the regulation paragraphs, and generating a final answer for the user according to the effective regulation paragraphs.
2. The regulatory-based large model retrieval enhancement method according to claim 1, characterized in that, The step S102 comprises: training the regulation AI model with the regulation data, constructing a label sample set with expert experience, inputting the label sample set into the regulation AI model, and letting the regulation AI model use the label sample set to label the regulation data, and interpreting the labeled regulation data, and letting the regulation AI model learn the syntax deconstruction and method of regulation data interpretation.
3. The regulatory-based large model retrieval enhancement method according to claim 1, characterized in that, The law-similar algorithm is used to match the semantic similarity of the user's question and the regulation data set, which is represented as follows: in, The matching score between the question and the regulatory dataset, This represents the number of words in the current phrase. To divide user questions into phrases, the total number of phrases, To include the current phrase in the user's question. The weight, To include the current phrase in the user's question. The weight of related words generated by association Inverse document frequency calculation, For word frequency calculation, Score the word offset. For the current phrase Freshness score, This is a special score.
4. The regulatory-based large model retrieval enhancement method according to claim 3, characterized in that, The step S103 specifically comprises the following steps: S31, the extracted regulations are segmented into paragraphs according to strong rules and small models, the strong rules are matched to special characters for segmentation processing, and the small model is a BERT text splitting small model; S32, based on the regulation paragraph segmentation processing of step S31, the effective regulation paragraphs are determined by an effective regulation paragraph matching method; S33, after screening the effective regulation paragraphs in step S32, the similarity of each effective regulation paragraph and the user's question is calculated, and the effective regulation paragraphs are scored according to the similarity; S34, based on the similarity score of step S33, the effective regulation paragraphs are further screened to obtain the optimal effective regulation paragraphs, and the length of the optimal effective regulation paragraphs is limited to avoid content generalization; S35, based on the optimal effective regulation paragraphs screened in step S34, the user's question is combined to generate a final answer for the user.
5. The regulatory-based model retrieval augmentation method of claim 4, wherein, The effective regulation paragraph matching method comprises: high-frequency word judgment is performed on the regulation paragraphs, a high-frequency word judgment threshold is calculated, it is judged whether the words in the regulation paragraphs exceed the high-frequency word judgment threshold, if the number of any word in the regulation paragraphs exceeds the high-frequency word judgment threshold, the importance of the word in the regulation paragraphs is reduced, which is represented as follows: wherein, is a high-frequency word determination threshold, is a regulation paragraph content length; Based on the high-frequency word judgment of the regulation paragraphs, the weight score of each regulation paragraph is calculated, which is represented as follows: wherein, is the total score of the regulatory paragraph weight, is the query term, is the set of query terms, is the indicator function, is the set of high frequency terms; A multi-level screening threshold is set, the multi-level screening threshold includes a basic screening threshold, the basic screening threshold is set according to the number of word groups in the user's question, which is represented as follows: wherein, is a first screening threshold value, is a base screening threshold value; When the total score of the regulation paragraph weight is greater than the multi-level screening threshold value, the regulation paragraph is a valid regulation paragraph, which is represented as follows: wherein, 1 if the current regulation paragraph is valid, 0 if it is not.
6. The regulatory-based model retrieval augmentation method of claim 4, wherein, The valid regulation paragraph is scored according to the calculated similarity, which is represented as follows: wherein, is the final similarity score, is the similarity of the effective regulation paragraph and the question problem, is the similarity of the policy number of the effective regulation paragraph and the question problem, is the question problem text of the user, is the effective regulation paragraph content, is the policy number of the effective regulation paragraph.
7. The regulatory-based model retrieval augmentation method of claim 4, wherein, The valid regulation paragraph is further screened to obtain the optimal valid regulation paragraph, which is represented as follows: wherein, is the optimal effective regulation paragraph, is the total length of the merged effective regulation paragraph, is the paragraph set after the level screening, is the total number of paragraphs in the paragraph set, is the text length function, is the single paragraph, is the th paragraph.
8. The regulatory-based model retrieval augmentation method of claim 7, wherein, The length of the optimal valid regulation paragraph is limited, which is represented as follows: wherein, is the length of the effective regulation paragraph for the final output, is the total number of words of the effective regulation paragraph.