Question and answer pair construction method and system, electronic equipment and storage medium

Abstract

The invention provides a question and answer pair construction method and system, electronic equipment and a storage medium, and relates to the technical field of information security, and the method comprises the steps: obtaining a to-be-recognized text and a dense point recognition instruction; constructing Prompt based on the text to be recognized and the dense point recognition instruction; calling the target large model based on Prompt to perform dense point information identification on a to-be-identified text to obtain an initial question and answer pair set, and performing confidence scoring on each initial question and answer pair; a comprehensive score set is obtained according to the confidence score and the rule score, the comprehensive score set comprises the comprehensive score of each initial question and answer pair, and the rule score represents that each initial question and answer pair is subjected to compliance scoring by using a preset dense point rule base; and screening out a target question-answer pair set from the initial question-answer pair set according to the comprehensive score set. By implementing the technical scheme provided by the invention, the effect of improving the question and answer pair construction efficiency is achieved.

Description

Technical Field

[0001] The present application relates to the field of information security technology, and specifically to a method, system, electronic device and storage medium for constructing question-answer pairs. Background Art

[0002] In the existing field of information security technology, key point identification and question-answer pair construction are important technical tasks. Especially in scenarios involving sensitive information, how to quickly and accurately extract key points from massive amounts of text data and construct corresponding question-answer pairs to facilitate the subsequent application of question-answering systems has become a key link in promoting the advancement of information security technology. However, the methods in related technologies mainly rely on manual annotation or rule-based text processing technologies. Manual annotation of key points and construction of question-answer pairs are time-consuming and difficult to process large-scale text data. Rule-based methods have limited ability to understand complex semantics, resulting in low accuracy in key point identification and question-answer pair construction. It can be seen that the methods in related technologies suffer from low efficiency and low accuracy. Summary of the Invention

[0003] In order to solve the above technical problems, the present application provides a method, system, electronic device and storage medium for constructing question and answer pairs.

[0004] In the first aspect, the present application provides a method for constructing a question-answer pair, including: obtaining a text to be recognized and a key point recognition instruction; constructing a Prompt based on the text to be recognized and the key point recognition instruction, wherein the Prompt is used to indicate the recognition of key point information from the text to be recognized and the construction of a question-answer pair corresponding to the key point information; based on the Prompt, calling a target large model to perform key point information recognition on the text to be recognized, obtaining an initial question-answer pair set, and performing a confidence score on each initial question-answer pair in the initial question-answer pair set; obtaining a comprehensive score set based on the confidence score and the rule score, wherein the comprehensive score set includes a comprehensive score of each initial question-answer pair in the initial question-answer pair set, and the rule score is used to indicate the compliance rule score for each initial question-answer pair using a preset key point rule library; filtering out a target question-answer pair set from the initial question-answer pair set based on the comprehensive score set.

[0005] By adopting the above technical solution, by constructing a specific Prompt indication target large model to identify dense point information and construct dense point question and answer pairs, it can significantly improve automated processing capabilities, reduce manual intervention, and improve the efficiency of constructing dense point question and answer pairs. By combining confidence scoring and rule scoring to generate a comprehensive score set, the accuracy and compliance of the question and answer pairs are ensured, solving the problem of low accuracy in traditional methods. The target question and answer pair set is screened based on the comprehensive score set, further optimizing the result quality and meeting the needs of large-scale text data processing. This technical solution can achieve efficient and accurate identification of dense point information and construction of question and answer pairs from the text to be identified.

[0006] Optionally, a comprehensive score set is obtained based on the confidence score and the rule score, including: for any initial question-answer pair, the corresponding comprehensive score is determined according to the following formula: Score = K1 × C + K2 × R, K1 + K2 = 1, where Score represents the comprehensive score, C represents the confidence score, R represents the rule score, and K1 and K2 are weight coefficients.

[0007] By adopting the above technical solution, we achieve the goal of calculating a comprehensive score for the initial question-answer pairs. Specifically, by introducing confidence scoring and rule scoring, and combining them with weight coefficients for linear combination, we can comprehensively evaluate the quality of the initial question-answer pairs, ensuring that the scoring results take into account both the reliability of the large model's predictions and the compliance requirements of the rules. The constraint condition of the weight coefficient (K1+K2=1) ensures the flexibility and rationality of the scoring formula, allowing the relative importance of the two scores to be adjusted according to actual needs, thereby improving the accuracy of screening.

[0008] Optionally, a target question-answer pair set is filtered out from the initial question-answer pair set based on the comprehensive score set, including: filtering the initial question-answer pair set based on keywords to obtain a first filtering result; filtering the first filtering result according to preset rules to obtain a second filtering result; selecting the top N initial question-answer pairs ranked by comprehensive scores from the second filtering result based on the comprehensive score set as the target question-answer pair set, where N is a positive integer greater than or equal to 1.

[0009] By adopting the above technical solution, we can achieve multi-level screening of the initial set of question-answer pairs. First, we use keyword screening to improve screening efficiency and relevance. Then, we use pre-set rule filtering to further improve screening accuracy. Finally, we select the top-ranked question-answer pairs based on the comprehensive score to ensure the quality and accuracy of the target set of question-answer pairs. Among them, keyword screening helps to quickly locate relevant question-answer pairs, pre-set rule filtering can eliminate question-answer pairs that do not meet the requirements, and comprehensive score ranking ensures the reliability of the final screening results, thereby effectively improving the overall efficiency and accuracy of dense point identification and question-answer pair construction.

[0010] Optionally, a target question-answer pair set is filtered out from the initial question-answer pair set based on the comprehensive score set, including: filtering the initial question-answer pair set based on keywords to obtain a first filtering result; selecting the top K initial question-answer pairs ranked by comprehensive scores from the first filtering result based on the comprehensive score set as the alternative question-answer pair set, where K is a positive integer greater than or equal to 1; filtering each initial question-answer pair included in the alternative question-answer pair set according to preset rules to obtain a third filtering result; and obtaining the target question-answer pair set based on the third filtering result.

[0011] By adopting the above technical solution, the initial question-answer pair set is screened by keywords to obtain the first screening result, which effectively reduces the number of irrelevant question-answer pairs and improves the efficiency of subsequent processing; the top K initial question-answer pairs are selected from the first screening result according to the comprehensive score as the alternative question-answer pair set, ensuring that the alternative set contains question-answer pairs with high confidence and high rule compliance; the question-answer pairs in the alternative question-answer pair set are filtered according to the preset rules to obtain the third screening result, further eliminating question-answer pairs that do not meet the requirements, and enhancing the compliance and accuracy of the target question-answer pair set; finally, the target question-answer pair set is determined according to the third screening result, realizing effective screening and optimization of question-answer pairs, and improving the overall performance of dense point identification and question-answer pair construction. This technical solution can achieve efficient screening of the initial question-answer pair set, thereby improving the quality and accuracy of the target question-answer pair set.

[0012] Optionally, a target question-answer pair set is obtained based on the third screening result, including: when it is determined that the number of initial question-answer pairs in the third screening result is greater than or equal to the first preset number, the initial question-answer pairs in the third screening result are grouped into a target question-answer pair set; when it is determined that the number of initial question-answer pairs in the third screening result is less than the first preset number, the initial question-answer pairs are retrieved from the retrieved candidate set in descending order of comprehensive scores and added to the third screening result, so that the number of initial question-answer pairs in the supplemented third screening result is greater than or equal to the first preset number, and the initial question-answer pairs in the supplemented third screening result are grouped into a target question-answer pair set, wherein the retrieved candidate set is composed of other initial question-answer pairs in the first screening result except the initial question-answer pairs in the alternative question-answer pair set.

[0013] By adopting the above technical solution, when the number of initial question-answer pairs in the third screening results is sufficient, the target question-answer pair set is directly formed, simplifying the processing flow and improving efficiency. When the number of initial question-answer pairs in the third screening results is insufficient, the initial question-answer pairs are supplemented by sorting them by comprehensive scores from the retrieved candidate set, effectively improving the quality and quantity stability of the target question-answer pair set and avoiding the problem of incomplete question-answer pair construction due to insufficient quantity. This achieves the goal of using the retrieved candidate set to supplement the initial question-answer pairs when the number of initial question-answer pairs is insufficient, ensuring that the number of target question-answer pairs meets the preset requirements.

[0014] Optionally, the above method also includes: reviewing each question and answer pair in the target question and answer pair set to obtain a review result, wherein the review result includes at least one of the following information: confirming correctly identified information, marking misidentified information, and supplementing missed identified information.

[0015] By adopting the above technical solution, each question-and-answer pair in the target question-and-answer pair set can be audited, resulting in an audit result that includes information confirming correct identification, marking misidentified information, and supplementing missing information. This helps improve the accuracy and completeness of question-and-answer pairs, ensuring the quality of dense point identification and question-and-answer pair construction. Specifically, the effects of this solution include: confirming correctly identified information, improving the credibility of identified question-and-answer pairs; marking misidentified information to facilitate subsequent error correction; and supplementing missing information to improve the content of question-and-answer pairs and reduce information gaps.

[0016] Optionally, after obtaining the audit results, the above method also includes: correcting the target question and answer pair set according to the audit results, obtaining a corrected target question and answer pair set, and storing it in the secret point library; scanning the secret point library according to a preset period to determine the number of newly added question and answer pairs in the secret point library; when it is determined that the number of newly added question and answer pairs is greater than or equal to a second preset number, optimizing the target large model using the newly added question and answer pairs as a training data set, wherein the newly added question and answer pairs include each question and answer pair in the corrected target question and answer pair set.

[0017] By adopting the above technical solution, the target set of question-and-answer pairs is modified based on the audit results and stored in the key point database, ensuring that the question-and-answer data in the key point database is more accurate and reliable. At the same time, the key point database is scanned according to a preset period and the number of newly added question-and-answer pairs is counted. This allows for dynamic monitoring of data accumulation. When the number of newly added question-and-answer pairs meets the requirements, these newly added question-and-answer pairs are used as training data sets to optimize the target large model, thereby continuously improving the recognition ability and accuracy of the large model. This technical solution can improve the accuracy of key point recognition and question-and-answer pair construction, enhance the learning ability of the large model, and achieve self-optimization and continuous improvement of the system.

[0018] In the second aspect of the present application, a question-answer pair construction system is also provided, including: an acquisition module for acquiring text to be recognized and a key point recognition instruction; a construction module for constructing a Prompt based on the text to be recognized and the key point recognition instruction, wherein the Prompt is used to indicate the recognition of key point information from the text to be recognized and the construction of a question-answer pair corresponding to the key point information; an identification module for calling the target large model based on the Prompt to perform key point information recognition on the text to be recognized, to obtain an initial question-answer pair set, and to perform a confidence score on each initial question-answer pair in the initial question-answer pair set; a comprehensive scoring module for obtaining a comprehensive scoring set based on the confidence score and the rule score, wherein the comprehensive scoring set includes a comprehensive score of each initial question-answer pair in the initial question-answer pair set, and the rule score is used to indicate the compliance rule scoring of each initial question-answer pair using a preset key point rule library; a screening module for screening out a target question-answer pair set from the initial question-answer pair set based on the comprehensive scoring set.

[0019] In a third aspect of the present application, an electronic device is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor implements any one of the above method steps when executing the program.

[0020] In a fourth aspect of the present application, a computer-readable storage medium is further provided. The computer-readable storage medium stores instructions. When the instructions are executed, any one of the above method steps is performed.

[0021] In summary, one or more technical solutions provided in this application have at least the following technical effects or advantages: 1. By building a specific large model of prompt indicators to identify dense point information and construct question-answer pairs, we can significantly improve automated processing capabilities, reduce manual intervention, and increase the efficiency of constructing dense point information question-answer pairs. We use a combination of confidence scoring and rule scoring to generate a comprehensive score set, ensuring the accuracy and compliance of the question-answer pairs, and solving the low accuracy problem of traditional methods. 2. It can implement multi-level screening of the initial set of question-answer pairs. First, it uses keyword screening to improve screening efficiency and relevance. Then, it uses pre-set rules to further improve screening accuracy. Finally, it selects the top-ranked question-answer pairs based on the comprehensive score to ensure the quality and accuracy of the target set of question-answer pairs. 3. The initial question-answer pair set is screened by keywords to obtain the first screening result, which effectively reduces the number of irrelevant question-answer pairs; the top K initial question-answer pairs are selected from the first screening result according to the comprehensive score as the candidate question-answer pair set, ensuring that the candidate set contains question-answer pairs with high confidence and high rule compliance; the question-answer pairs in the candidate question-answer pair set are then filtered according to the preset rules to obtain the third screening result, further eliminating question-answer pairs that do not meet the requirements, thereby enhancing the compliance and accuracy of the target question-answer pair set; finally, the target question-answer pair set is determined based on the third screening result, achieving effective screening and optimization of question-answer pairs, and improving the overall performance of dense point recognition and question-answer pair construction; 4. When the number of initial question-answer pairs in the third screening results is sufficient, the target question-answer pair set is directly formed, simplifying the processing flow and improving efficiency. When the number of initial question-answer pairs in the third screening results is insufficient, the initial question-answer pairs are supplemented by sorting the recovered candidate set by comprehensive scores, effectively improving the quality and quantity stability of the target question-answer pair set and avoiding the problem of incomplete question-answer pair construction due to insufficient number. 5. The ability to review each question-answer pair in the target question-answer pair set helps improve the accuracy and completeness of the question-answer pairs and ensure the quality of dense point identification and question-answer pair construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flowchart of a method for constructing a question-answer pair provided in an embodiment of the present application; Figure 2 This is an architecture diagram of a large model-assisted dense point question-answering pair construction system based on a dynamic feedback mechanism provided by an embodiment of the present application; Figure 3 This is a workflow diagram of a large model-assisted dense point question-answer pair construction system based on a dynamic feedback mechanism provided by an embodiment of the present application; Figure 4 This is a structural block diagram of a question-answer pair construction system provided in an embodiment of the present application; Figure 5 This is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application.

[0023] Description of reference numerals: 500 - electronic device; 501 - processor; 502 - communication bus; 503 - user interface; 504 - network interface; 505 - memory. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0025] In the description of the embodiments of this application, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a concrete manner.

[0026] In the description of the embodiments of the present application, the term "plurality" means two or more. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of such features. The terms "include," "comprise," "have" and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0027] This application provides a method for constructing question-answer pairs, referring to Figure 1 , Figure 1 This is a flowchart of a method for constructing a question-answer pair provided in an embodiment of the present application, comprising the following steps: Step S101, obtaining the text to be recognized and the dense point recognition instruction; Step S102: constructing a Prompt based on the text to be recognized and the key point recognition instruction, wherein the Prompt is used to instruct the recognition of key point information from the text to be recognized and the construction of a question-answer pair corresponding to the key point information; Step S103: Based on Prompt, the target large model is called to perform dense point information recognition on the text to be recognized, to obtain an initial question-answer pair set, and a confidence score is given to each initial question-answer pair in the initial question-answer pair set; Step S104: Obtain a comprehensive score set based on the confidence score and the rule score, wherein the comprehensive score set includes the comprehensive score of each initial question-answer pair in the initial question-answer pair set, and the rule score is used to indicate the compliance rule score of each initial question-answer pair using a preset dense point rule library; Step S105: Filter out a target question-answer pair set from the initial question-answer pair set based on the comprehensive score set.

[0028] Through the above steps, by constructing a specific Prompt indication target large model to identify dense point information and construct dense point question and answer pairs, the automated processing capabilities can be significantly improved, manual intervention can be reduced, and the efficiency of constructing dense point question and answer pairs can be improved; a comprehensive scoring set is generated by combining confidence scoring and rule scoring to ensure the accuracy and compliance of the question and answer pairs, solving the problem of low accuracy in traditional methods; the target question and answer pair set is screened out based on the comprehensive scoring set, further optimizing the result quality and meeting the needs of large-scale text data processing. This embodiment can realize the efficient and accurate identification of dense point information and the construction of question and answer pairs from the text to be identified.

[0029] This embodiment proposes a method for constructing question-answer pairs based on Prompt and a large model. First, it constructs a Prompt by obtaining the text to be recognized and the key point recognition instruction. The function of this Prompt is to instruct the extraction of key point information from the text to be recognized and to construct a question-answer pair based on it; then, this Prompt is used to call a target large model (an AI model with powerful natural language processing capabilities) to identify the key point information of the text to be recognized, thereby generating an initial set of question-answer pairs. For each generated question-answer pair, the system will give a confidence score; in addition, the preset key point rule library will be used to check the compliance of these question-answer pairs and give corresponding rule scores. Finally, a comprehensive score is calculated by combining the confidence score and the rule score, and the most suitable target question-answer pair set is screened out based on the comprehensive score. Related technologies rely on manual annotation or rule-based text processing, both of which are inefficient when processing large amounts of data. However, this method, through automated prompt construction and the application of large models, can efficiently process massive amounts of text data, significantly improving work efficiency. The use of large models can better understand and parse complex semantic structures, improving recognition accuracy. Furthermore, by combining confidence scoring with rule-based scoring, the quality of selected question-answer pairs is further ensured. Prompts are input text used to guide the large model to generate specific outputs. Prompts can be both instructions and templates (or prompts), providing the model with task descriptions, background information, formatting requirements, and other information to help it clarify its generation direction. The target large model has a strong language understanding and generation capability, can quickly process large-scale text data and identify key point information, generate an initial question-answer pair set, and at the same time, perform a confidence score on each initial question-answer pair, reflecting the confidence or credibility of the large model for each question-answer pair; a comprehensive score set is obtained by combining the confidence score and the rule score, and the rule score is to use the preset key point rule library to score the compliance rules of the initial question-answer pair. This step makes up for the possible shortcomings of the large model, especially when the large model is initially used, to ensure that the question-answer pair complies with the preset key point rules. Through this embodiment, it is possible to quickly extract key points from massive text data and construct question-answer pairs, adapt to the processing requirements of large-scale text data, save a lot of manpower and time costs, and through the combination of the target large model and the rule library, improve the accuracy of key point identification and question-answer pair construction, improve the accuracy of key point identification and the quality of question-answer pair construction, so that the constructed question-answer pairs are more in line with the application requirements in the information security scenario, provide a more accurate and high-quality knowledge base for the subsequent question-answer system, and help improve the performance of the entire information security technology system.

[0030] In an optional embodiment, a comprehensive score set is obtained based on the confidence score and the rule score, including: for any initial question-answer pair, the corresponding comprehensive score is determined according to the following formula: Score = K1 × C + K2 ×R, K1 + K2 =1, where Score represents the comprehensive score, C represents the confidence score, R represents the rule score, and K1 and K2 are weight coefficients.

[0031] In the above embodiment, the goal of calculating a comprehensive score for the initial question-answer pair is achieved. Specifically, by introducing confidence scoring and rule scoring, and combining them with weight coefficients for linear combination, the quality of the initial question-answer pair can be comprehensively evaluated, ensuring that the scoring results take into account both the reliability of the large model prediction and the compliance requirements of the rules. The constraint condition of the weight coefficient (K1+K2=1) ensures the flexibility and rationality of the scoring formula, allowing the relative importance of the two scores to be adjusted according to actual needs, thereby improving the accuracy of screening.

[0032] For each initial question-answer pair in the set of initial question-answer pairs, a weighted summation method is used to calculate a comprehensive score. The comprehensive score is a weighted combination of the confidence score and the rule score. Score represents the comprehensive score. C is the confidence score based on the output of the large model, reflecting the model's confidence in the accuracy of the question-answer pair. R is the rule score after the compliance check of the question-answer pair based on the preset dense point rule library, which is used to assess whether the question-answer pair meets specific security or regulatory standards. K1 and K2 are weight coefficients that allow adjustment of the proportion of confidence score and rule score in the final score. In this way, the confidence score given by the large model and the compliance rule score given by the preset dense point rule library can be comprehensively considered to obtain a comprehensive score to measure the quality of each initial question-answer pair. Related art methods typically rely on a single evaluation criterion (such as manual annotation or a single rule), making it difficult to comprehensively assess the quality of question-answer pairs. However, the present invention combines confidence scoring (reflecting the model's recognition capabilities) with rule scoring (ensuring compliance) to more comprehensively and accurately assess the quality of question-answer pairs. By introducing weighting coefficients K1 and K2, the importance of confidence and rule scoring in the overall scoring can be flexibly adjusted based on the needs of specific application scenarios. For example, in some cases, greater emphasis may be placed on model recognition accuracy (increasing K1), while in other cases, greater emphasis may be placed on compliance with specific specifications or standards (increasing K2). For example, for highly sensitive text (such as legal contracts), K2 can be increased (e.g., 0.6) to strengthen rule constraints, while for semantically complex text (such as technical documents), K1 can be increased (e.g., 0.8) to rely on the model's understanding capabilities. This flexibility enhances the method's adaptability to diverse information security scenarios.

[0033] In an optional embodiment, a target question-answer pair set is filtered out from the initial question-answer pair set based on the comprehensive score set, including: filtering the initial question-answer pair set based on keywords to obtain a first filtering result; filtering the first filtering result according to preset rules to obtain a second filtering result; selecting the top N initial question-answer pairs ranked by comprehensive score from the second filtering result based on the comprehensive score set as the target question-answer pair set, where N is a positive integer greater than or equal to 1.

[0034] In the above embodiment, a multi-level screening of the initial set of question-answer pairs can be achieved. First, keyword screening is used to improve screening efficiency and relevance. Then, filtering by preset rules is used to further improve screening accuracy. Finally, the top-ranked question-answer pairs are selected based on the comprehensive score to ensure the quality and accuracy of the target set of question-answer pairs. Among them, keyword screening helps to quickly locate relevant question-answer pairs, filtering by preset rules can eliminate question-answer pairs that do not meet the requirements, and comprehensive score ranking ensures the reliability of the final screening results, thereby effectively improving the overall efficiency and accuracy of dense point identification and question-answer pair construction.

[0035] This embodiment provides a method for multi-level screening of an initial set of question-answer pairs to ultimately determine a target set of question-answer pairs. First, the initial set of question-answer pairs is preliminarily screened by keyword (or topic) matching to obtain a first screening result; then, the first screening result is further filtered and processed according to preset rules to produce a second screening result; finally, the top N question-answer pairs ranked by comprehensive score are selected from the second screening result as the final target set of question-answer pairs based on the comprehensive score. The core principle is to gradually improve the accuracy of screening through a multi-level screening mechanism, combined with keyword screening, rule filtering, and comprehensive score sorting, to ensure that the final set of question-answer pairs not only complies with the dense point rule but also has high quality. For example, relevant question-and-answer pairs are quickly filtered out based on domain keywords (such as "authorization" and "privacy terms"), narrowing the processing scope. Pre-set rules (such as sensitive word blacklists, logical conflict detection, and format validation) are then used to filter out substandard question-and-answer pairs. For example, initial question-and-answer pairs containing sensitive terms such as pornography and violence are filtered out. Another example is filtering out relevant question-and-answer pairs based on the keyword "patient ID," then filtering out those containing unmasked ID numbers or conflicting semantics. The question-and-answer pairs obtained through the first two rounds of screening (i.e., those included in the second screening results) are ranked by comprehensive score, retaining the top-N high-quality results, for example, N = 10 (or 20, or 100, or other values). This keyword and rule filtering significantly reduces the amount of data required for scoring and sorting, thereby minimizing the waste of computing resources. Through multi-stage screening, this embodiment can quickly and efficiently screen out a set of target question-answer pairs from a large number of initial question-answer pairs, reducing unnecessary manual intervention and saving time and energy. It comprehensively considers factors such as keywords, preset rules and comprehensive scores to ensure that the screened set of target question-answer pairs has high quality and relevance, and is more in line with the needs of actual applications.

[0036] In an optional embodiment, a target question-answer pair set is filtered out from the initial question-answer pair set based on the comprehensive score set, including: filtering the initial question-answer pair set based on keywords to obtain a first filtering result; selecting the top K initial question-answer pairs ranked by comprehensive scores from the first filtering result as an alternative question-answer pair set based on the comprehensive score set, where K is a positive integer greater than or equal to 1; filtering each initial question-answer pair included in the alternative question-answer pair set according to preset rules to obtain a third filtering result; and obtaining the target question-answer pair set based on the third filtering result.

[0037] In the above embodiment, the initial question-answer pair set is screened by keywords to obtain the first screening result, which effectively reduces the number of irrelevant question-answer pairs and improves the efficiency of subsequent processing; the top K initial question-answer pairs are selected from the first screening result according to the comprehensive score as the alternative question-answer pair set, ensuring that the alternative set contains question-answer pairs with high confidence and high rule compliance; the question-answer pairs in the alternative question-answer pair set are filtered according to preset rules to obtain the third screening result, further eliminating question-answer pairs that do not meet the requirements, and enhancing the compliance and accuracy of the target question-answer pair set; finally, the target question-answer pair set is determined according to the third screening result, achieving effective screening and optimization of question-answer pairs, and improving the overall performance of dense point identification and question-answer pair construction. This embodiment can achieve efficient screening of the initial question-answer pair set, thereby improving the quality and accuracy of the target question-answer pair set.

[0038] First, the initial question-answer pair set is preliminarily screened by keyword matching to obtain the first screening result. This step aims to quickly exclude question-answer pairs that are not related to the keyword and narrow the candidate range; according to the comprehensive score (calculated by combining the confidence score and the rule score), the top K question-answer pairs are selected from the first screening result as the candidate question-answer pair set, for example, K = 200 (or 300, or other values). This step ensures that the selected question-answer pairs have high quality in terms of model recognition ability and compliance, that is, the question-answer pairs in the first screening result are sorted according to the comprehensive score (Score), and the top- K high-scoring results are used as a set of candidate question-answer pairs, and those with high model confidence and good rule compliance are retained as candidates first; each question-answer pair in the set of candidate question-answer pairs is further filtered by rules to obtain a third screening result, and substandard question-answer pairs are filtered out by preset rules (such as sensitive word blacklist, logical conflict detection, format verification, etc.), for example, initial question-answer pairs containing sensitive words such as pornography and violence are filtered out; for another example, relevant question-answer pairs are filtered out by the keyword "patient ID", and then question-answer pairs containing un-massified ID card numbers or contradictory semantics are filtered out; finally, the target question-answer pair set is determined based on the third screening result. Directly performing rule verification (such as regular matching and logical checking) on all initial question-answer pairs has high computational cost, and is especially inefficient in massive data scenarios. This embodiment first pre-screens by keywords and scores to greatly reduce the amount of data that requires rule verification. Taking financial compliance review as an example, we screen through keywords such as "transaction records," "customer privacy," and "risk control terms," then select the top 200 high-scoring question-and-answer pairs. We then filter out those containing unmasked bank card numbers or those that violate regulatory regulations, ultimately outputting 50 high-quality question-and-answer pairs that can be directly used in compliance training or audit question-and-answer systems.

[0039] In an optional embodiment, a target question-answer pair set is obtained based on the third screening result, including: when it is determined that the number of initial question-answer pairs in the third screening result is greater than or equal to the first preset number, the initial question-answer pairs in the third screening result are grouped into a target question-answer pair set; when it is determined that the number of initial question-answer pairs in the third screening result is less than the first preset number, the initial question-answer pairs are retrieved from the retrieved candidate set in descending order of comprehensive scores and added to the third screening result, so that the number of initial question-answer pairs in the supplemented third screening result is greater than or equal to the first preset number, and the initial question-answer pairs in the supplemented third screening result are grouped into a target question-answer pair set, wherein the retrieved candidate set is composed of other initial question-answer pairs in the first screening result except the initial question-answer pairs in the alternative question-answer pair set.

[0040] In the above embodiment, when the number of initial question-answer pairs in the third screening results is sufficient, the target question-answer pair set is directly formed, which simplifies the processing flow and improves efficiency. When the number of initial question-answer pairs in the third screening results is insufficient, the initial question-answer pairs are supplemented by sorting them by comprehensive scores from the retrieved candidate set, effectively improving the quality and quantity stability of the target question-answer pair set and avoiding the problem of incomplete question-answer pair construction due to insufficient quantity. In the case of insufficient number of initial question-answer pairs, the retrieved candidate set is used to supplement the initial question-answer pairs, ensuring that the number of target question-answer pair sets meets the preset requirements.

[0041] This embodiment adopts different methods to determine the final target question-answer pair set based on the comparison result of the number of initial question-answer pairs in the third screening result and the first preset number. The specific process is as follows: when the number is sufficient, that is, when the number of initial question-answer pairs in the third screening result is greater than or equal to the first preset number (such as 100, or other values), the initial question-answer pairs in the third screening result are directly combined into the target question-answer pair set, because the number of question-answer pairs screened out at this time has met the preset requirements; when the number is insufficient, that is, when the number of initial question-answer pairs in the third screening result is less than the first preset number, it is necessary to supplement the question-answer pairs from the retrieved candidate set. The retrieved candidate set is composed of the initial question-answer pairs in the first screening result except the initial question-answer pairs in the alternative question-answer pair set. The target question-answer pair set is composed of question-answer pairs. Initial question-answer pairs are retrieved from the retrieved candidate set in descending order of comprehensive scores and added to the third screening result until the number of initial question-answer pairs in the supplemented third screening result is greater than or equal to the first preset number. Then, each initial question-answer pair in the supplemented third screening result is used to form a target question-answer pair set. It should be noted that the initial question-answer pairs retrieved from the retrieved candidate set are first filtered and then added to the third screening result. The filtering process can be performed in the same manner as in the aforementioned embodiment, such as filtering out substandard question-answer pairs by using preset rules (such as a sensitive word blacklist, logical conflict detection, format verification, etc.), for example, filtering out initial question-answer pairs containing sensitive words such as pornography and violence. By comparing with the first preset number and retrieving and supplementing when the number is insufficient, it can be ensured that the number of the target question-answer pair set finally obtained stably meets the preset requirements, providing sufficient number of question-answer pairs to support subsequent applications such as the question-answering system, thereby enhancing the stability and reliability of the system. The first preset number can be adjusted according to different application scenarios and requirements, so that the method can flexibly adapt to various actual situations and meet diverse business needs. In practical applications, in scenarios involving sensitive information or complex semantics, strict rule filtering may lead to an insufficient number of screening results. Through the retrieval mechanism, this method can ensure that the quantity and quality of output results meet expectations without lowering the screening standards, providing a more reliable solution for the field of information security.

[0042] In the above embodiment, assuming that the first preset number is 100, when the number of initial question-answer pairs in the third screening result is 60, that is, after filtering each initial question-answer pair in the aforementioned set of alternative question-answer pairs, there are only 60 initial question-answer pairs in the third screening result; at this time, it is necessary to retrieve from the retrieved candidate set. Optionally, in actual applications, retrieval can be performed from the retrieved candidate set according to a fixed number, for example, half of the first preset number (or other proportions) can be retrieved, such as 50 at a time in order of comprehensive scores from high to low; as another optional implementation method, retrieval can also be performed from the retrieved candidate set according to the difference between the number of initial question-answer pairs in the current third screening result and the first preset number, such as 100-60=40, according to a certain multiple of the difference (such as 1.2 times, or 1.5 times, or other multiples). The initial question-answer pairs retrieved from the retrieved candidate set are filtered by the same rule method as described above and then added to the third screening result. Then, it is determined whether the number of initial question-answer pairs in the supplemented third screening result is greater than or equal to the first preset number. When it is greater than or equal to the first preset number, the initial question-answer pairs in the supplemented third screening result can be combined into a target question-answer pair set. When the number of initial question-answer pairs in the supplemented third screening result is still less than the first preset number, the same method as described above is used to continue to retrieve from the retrieved candidate set.

[0043] In an optional embodiment, the above method also includes: reviewing each question and answer pair in the target question and answer pair set to obtain a review result, wherein the review result includes at least one of the following information: confirming correctly identified information, marking misidentified information, and supplementing missed identified information.

[0044] In the above embodiment, each question-answer pair in the target question-answer pair set can be audited to obtain an audit result including at least one of the following information: information confirming correct identification, information marking misidentification, and information supplementing missing identification. This helps to improve the accuracy and completeness of the question-answer pairs and ensure the quality of dense point identification and question-answer pair construction. Specifically, the technical effects of this embodiment include: confirming correctly identified information to improve the credibility of identified question-answer pairs; marking misidentification information to facilitate subsequent error correction; supplementing missing identification information to improve the content of the question-answer pairs and reduce information missing.

[0045] After the target set of question-answer pairs is generated, an automated review process is introduced to further improve their quality. Specifically, the review process examines each question-answer pair in the target set and generates a review result. The review result includes at least one of the following: confirmation of correctly identified information, marking which question-answer pairs are accurate; marking misidentified information, indicating which question-answer pairs have misidentified conditions, such as key point extraction errors and semantic mismatches; and supplementation of missed information, identifying and supplementing important question-answer pairs that were missed during the screening process. This review mechanism allows for final verification and optimization of the automatically generated set of target question-answer pairs to ensure that they meet practical application requirements. The review mechanism can be performed manually, through automated tools, or even through human-machine collaboration. This flexibility allows the system to select the most appropriate review method based on actual needs, further improving efficiency and practicality. Through the audit mechanism, this embodiment can promptly discover and correct errors in question-answer pairs, supplement missing information, and significantly improve the quality and reliability of question-answer pairs. During the audit process, it can be confirmed whether the question-answer pairs have correctly identified the key point information, ensuring that the final set of question-answer pairs meets the key point requirements and satisfies the application scenarios of information security technology. The audit mechanism can be adjusted according to specific application scenarios, such as using manual audit, automated audit, or a combination of the two, with strong flexibility and scalability. The audit results can be used to further optimize the set of question-answer pairs, such as adjusting weights, improving rules, or optimizing the training of large models, thereby continuously improving the performance and quality of the system. Taking the training of the key point question-answer pair model as an example, assuming that the automatically generated result is that the system outputs 200 relevant question-answer pairs, 185 of which are confirmed to be correct after audit, 10 are marked as misidentified, and 5 are supplemented as missing. The misidentified cases can be entered into the model negative sample library to optimize the model, which can effectively reduce the misidentification rate in the next iteration cycle.

[0046] In an optional embodiment, after obtaining the audit results, the above method also includes: correcting the target question and answer pair set according to the audit results, obtaining a corrected target question and answer pair set, and storing it in the secret point library; scanning the secret point library according to a preset period to determine the number of newly added question and answer pairs in the secret point library; when it is determined that the number of newly added question and answer pairs is greater than or equal to a second preset number, optimizing the target large model using the newly added question and answer pairs as a training data set, wherein the newly added question and answer pairs include each question and answer pair in the corrected target question and answer pair set.

[0047] In the above embodiment, the target question-answer pair set is modified based on the audit results and stored in the key point library, which can ensure that the question-answer pair data in the key point library is more accurate and reliable. At the same time, the key point library is scanned according to a preset period and the number of newly added question-answer pairs is counted, which can dynamically monitor the data accumulation. When the number of newly added question-answer pairs meets the requirements, these newly added question-answer pairs are used as training data sets to optimize the target large model, thereby continuously improving the recognition ability and accuracy of the large model. This embodiment can improve the accuracy of key point recognition and question-answer pair construction, enhance the learning ability of the large model, and achieve self-optimization and continuous improvement of the system.

[0048] This embodiment proposes an iterative model optimization mechanism based on audit feedback, forming a complete "generation-review-correction-training" closed-loop system. Specifically, it includes: correction and storage: revising target question-answer pairs based on audit results and storing them in a keypoint database (knowledge base); incremental monitoring: regularly scanning the keypoint database and counting the number of newly added question-answer pairs; and triggering training: when the amount of newly added data reaches a second preset number, it automatically uses this data as a training set to optimize the target large model. This embodiment achieves dynamic improvement in model performance through data-driven continuous learning. In related art, models remain fixed after training and cannot adapt to emerging keypoint types, such as new cyberattack terminology. This embodiment uses periodic retraining to ensure the model's ability to capture the latest knowledge, feeding back high-quality, audited data to the model, achieving effective transfer of human experience. Setting a second preset number ensures that training only initiates when data has accumulated to a sufficient scale. For example, this second preset number is 500 (or 1000, or other). This second preset number can be adjusted according to the needs of different application scenarios.

[0049] In an optional embodiment, the above method also includes: dynamically optimizing the target large model according to a preset triggering method based on the audit results, wherein the preset triggering method includes one of the following methods: manual triggering method, periodic triggering method, and conditional triggering method.

[0050] In the above embodiment, the effect of dynamically optimizing the target large model based on the audit results is achieved. Specifically, the preset triggering method includes one of a manual triggering method, a periodic triggering method, and a conditional triggering method, which can flexibly select the appropriate optimization time according to actual needs, thereby improving the performance and adaptability of the large model; optionally, the operator is allowed to manually start the model optimization process as needed, which is usually suitable for situations where specific problems are discovered or the model performance needs to be improved immediately; optionally, the system automatically checks whether there are new audit results that can be used to optimize the model at predetermined time intervals (such as weekly or monthly). This method helps to regularly update the model to adapt to changing data patterns; optionally, when certain specific conditions are met, such as when the number of new question-answer pairs reaches a certain threshold, or when the proportion of misidentification or missed identification in the audit results reaches a certain threshold, the system automatically starts the optimization process. This method can ensure that model optimization is only performed when there is sufficient new data support, avoiding unnecessary resource consumption. This dynamic optimization mechanism of this embodiment helps to continuously improve the accuracy of dense point identification and question-answer pair construction, further improving the overall efficiency and reliability of the system. A variety of preset triggering methods provide flexible optimization timing selection to meet the needs of different application scenarios. Conditional triggering and periodic triggering methods realize partial automatic optimization, reduce manual intervention and improve optimization efficiency.

[0051] In an optional embodiment, the above method further includes: dynamically optimizing the target large model based on the audit results using an optimization strategy based on the reinforcement learning GRPO algorithm.

[0052] In the above embodiment, dynamic optimization of the target large model using a reinforcement learning GRPO algorithm based on audit results can effectively improve the performance of the large model in dense point identification and question-answer pair construction tasks. Specifically, the reinforcement learning algorithm continuously adjusts model parameters based on feedback from audit results, allowing the model to gradually adapt to complex semantic environments and improve the accuracy of dense point identification. This dynamic optimization mechanism ensures continuous improvement of the model in practical applications, thereby improving the efficiency and reliability of the overall system.

[0053] This embodiment introduces an optimization strategy based on audit results using the GRPO (Group Relative Policy Optimization) algorithm to dynamically optimize the target large model. The GRPO algorithm is a reinforcement learning method that optimizes the policy model through a relative reward mechanism within a group, rather than relying on a traditional value model. This mechanism generates multiple candidate actions (such as multiple answers or reasoning paths) for the same problem or state, and dynamically calculates the normalized relative advantage based on the statistical properties of the group rewards (such as the mean and standard deviation). High-reward samples are assigned a positive advantage, while low-reward samples are suppressed, creating a "survival of the fittest" optimization mechanism. Unlike traditional PPO, GRPO does not require a separate value network to estimate the value of actions, but directly uses the normalized result of the group rewards as the advantage function. This design significantly reduces graphics memory usage and computational cost, making it particularly suitable for training large-scale language models. The policy model parameters are updated based on relative advantage, and a KL divergence constraint is introduced to prevent overly drastic policy updates, thereby maintaining the stability of the policy distribution. GRPO aims to maximize the expected cumulative reward while maintaining the stability of policy updates. Its objective function uses a regularization term to control the magnitude of policy updates.

[0054] In an optional embodiment, the prompt includes recognition scope, recognition rule classification, confidentiality level and confidence score.

[0055] In the above embodiment, Prompt includes recognition scope, recognition rule classification, secrecy level and confidence score, which can clearly indicate the focus direction and evaluation criteria of the large model when processing the text to be recognized. The setting of the recognition scope helps to narrow the search space and improve the efficiency of the extraction of dense point information; the introduction of recognition rule classification enables the model to apply corresponding processing logic according to different types of dense points, thereby enhancing the accuracy of recognition; the labeling of secrecy level provides a basis for hierarchical management for subsequent information security applications; the confidence score provides a quantitative basis for the reliability assessment of the initial question and answer pairs, thereby improving the overall quality and practicality of the constructed question and answer pair set. This embodiment clarifies the specific content of Prompt, including recognition scope, recognition rule classification, secrecy level and confidence score. By clarifying these key information in Prompt, it provides more precise guidance for the large model, thereby improving the accuracy and efficiency of dense point recognition and question and answer pair construction.

[0056] It should be noted that the above-described embodiments are only part of the embodiments of the present application, rather than all the embodiments. The present application will be described in detail below with reference to specific embodiments.

[0057] The embodiments of the present application provide a system and method for constructing large-model assisted dense point identification question-answer pairs based on a feedback mechanism.

[0058] Figure 2 This is an architectural diagram of a large-model-assisted dense point question-answer pair construction system based on a dynamic feedback mechanism provided in an embodiment of the present application. The system achieves efficient and accurate dense point identification and generates explanations in the form of question-answer pairs by combining the semantic understanding capabilities and feedback mechanism of the large model.

[0059] The system architecture of the embodiment of the present application includes the following modules: Acquisition module: Acquires documents that users need to process, supports multiple formats (such as PDF, Word, Excel, etc.), and users can ask specific questions. The system will perform targeted key point recognition based on the content of the questions.

[0060] Large model auxiliary recognition module: Utilizes the semantic understanding ability of the large model (corresponding to the aforementioned target large model) to identify sensitive information in documents, including but not limited to personal identity information, business secrets, etc. The large model then scores each sensitive point recognition result, and then combines it with the rule-based scoring library for comprehensive scoring.

[0061] Screening and labeling module: This module provides the ability to search by topic or keyword, and uses a large model to comprehensively score and sort, perform truncation, and screen candidate sets with higher scores. If the candidate set is insufficient, it can be supplemented through retrieval. The subsequent sets screened out need to be further reviewed, rewritten, modified, and labeled.

[0062] Feedback module: It can provide feedback on recognition results, including confirming correct recognition, marking misidentifications, and supplementing missing information. The system analyzes the feedback and extracts useful information to optimize the recognition model.

[0063] Feedback learning module: New points are added to the knowledge base after passing the review, and repeatability verification is performed at the same time. In order to dynamically learn the updated information of the knowledge base, an optimization strategy based on reinforcement learning GRPO (Group Relative Policy Optimization) is introduced to continuously optimize the large model and improve the model effect.

[0064] The specific functions of each module are described below: 1. Get the module (1) Obtain text files in various formats, such as TXT, Word, Excel, etc.; (2) File parsing: Get the text content in the file through the parsing component.

[0065] 2. Large model auxiliary recognition module (1) Constructing a secret point recognition prompt: Read the secret point recognition instruction and combine it with the read text content to form a secret point recognition prompt, which is divided into four parts: the recognition range; the recognition rule classification; the secret level; and the confidence score. The specific examples are as follows: 1) Identification scope State secrets: information involving national security and national interests; Personal privacy: involving personal identity, privacy, and sensitive information; Trade secrets: involving the company's core competitiveness, financial data, technical secrets, etc. Sensitive data: special information related to laws and regulations, administrative management, and social stability.

[0066] 2) Identification rule classification (i) State secrets Content involving national sovereignty and territorial integrity; Content related to military affairs, national defense, and foreign affairs; Content involving major policies, plans, and scientific and technological breakthroughs.

[0067] (ii) Personal Privacy Personal information including ID number, mobile phone number, bank account, address, etc. Involving sensitive information such as health status, marital status, religious beliefs, etc.

[0068] (iii) Trade secrets Content involving the company's core technology, R&D plans, and market strategies; Content involving corporate financial data, personnel files, and supplier information.

[0069] (iv) Sensitive Data Content related to social stability and public safety; Policy information involving laws and regulations that are not disclosed.

[0070] 3) Secret Level Top Secret: core information involving national security, the disclosure of which would cause particularly serious damage; Confidential: Information involving important national interests, the disclosure of which would cause serious damage; Secret: Information involving the general interests of the state, which would cause a certain degree of damage if disclosed; Private: Information involving personal privacy or business secrets, which may cause personal or corporate losses if leaked.

[0071] 4) Confidence scoring High confidence (90-100): The information clearly belongs to a certain dense point category and complies with national or industry standards; Medium confidence (70-89): The information may belong to a certain category of dense points, but further verification is required; Low confidence (0-69): The information is not highly correlated with the dense point category, or it cannot be determined.

[0072] Example output: [ { "secret_point":"", "secret_tag":"Rule classification: State secret; Secret level: Top secret", "score":0.8 }, { "secret_point":"", "secret_tag":"Rule classification: State secret; Secret level: Top secret", "score":0.6 } ] It should be noted that the above-mentioned dense point recognition prompt is only an example. For example, the recognition range may be only a part of the above-mentioned four types, and may also include other types; similarly, the recognition rule classification may be only a part of the above-mentioned types, and may also include other types.

[0073] (2) Call the dense point recognition large model (corresponding to the aforementioned target large model) to generate question-answer pairs: Through the Prompt constructed above, input it into the dense point recognition large model to generate dense point question-answer pairs (also referred to as question-answer pairs for short), and perform confidence scoring for the dense point question-answer pairs.

[0074] As shown in the example above, when the large model outputs dense question-answer pairs, it also outputs a confidence score. The confidence score measures the accuracy of the recognition result and is recorded as Score (confidence).

[0075] (3) Compliance rule scoring: For the generated key point question and answer pairs, the key point rule library is called and similarity scoring is performed through the rule engine. Similarity scoring is to calculate the similarity (sim) of each generated key point and key point rule library. If the number of similarities (sim) > 0.8 is greater than or equal to 10, then Score (similarity) = 1; if the number of similarities (sim) > 0.8 is less than 10, then Score (similarity) = number of similarities (sim) > 0.8 / 10.

[0076] (4) Comprehensive score: The confidence score and the compliance score are weighted and summed to obtain a comprehensive score. The specific formula is as follows: Score (comprehensive) = 0.7*Score (confidence) + 0.3*Score (similarity).

[0077] (5) The key question and answer pairs are stored in the database.

[0078] 3. Screening and marking module (1) Topic and keyword screening: Filter accurate question and answer pairs by topic or keyword, making it easy to quickly build high-quality dense question and answer pairs; (2) Sorting and truncation: Set K (K is the number of question-answer pairs processed at a time), sort by comprehensive score, and select the top K for truncation; (3) Rule filtering: Quickly filter out some substandard question-answer pairs through preset rules (such as sensitive word blacklist, logical conflict detection, format verification, etc.); (4) Retrieving question-answer pairs: When there are many filtered question-answer pairs, it is necessary to further retrieve the remaining relevant topic question-answer pairs through the candidate pool; It should be noted that when the number of question-answer pairs remaining after filtering by the above (3) rule is sufficient, such as greater than or equal to 100, there is no need to retrieve, that is, there is no need to execute the above (4) process; and when the number of question-answer pairs remaining after filtering by the above (3) rule is insufficient, it is necessary to retrieve, and the retrieved question-answer pairs must also be filtered by the above rule; (5) Rewriting the question-answer pair: When the question-answer pair is determined to be a dense point, but the form or content is not rich enough, it can be rewritten and modified with the help of a large model.

[0079] 4. Feedback module (1) Review and marking: In order to ensure that the rewritten question-answer pairs or the question-answer pairs that have passed the first review are of higher quality, a review step can be added to ensure that the final question-answer pairs are of higher quality and marked; (2) High-quality, dense question-answer pairs are stored in the database.

[0080] 5. Feedback learning module (1) Automatically trigger optimization mechanism 1) Scheduled automatic updates Scan the key point database regularly, read the most recent update time from the update time record configuration file, and count the number of new key point question and answer pairs added to the key point database during that time period by comparing the timestamp of the last update. If the number of new additions is greater than 100, sort them in chronological order, select the question and answer pairs with the longest storage time first, and then update the minimum storage time of the batch of data sets to the update time record configuration file.

[0081] 2) Manual update In special cases, updates can also be triggered manually.

[0082] (2) Introducing GRPO optimization for dense large models: Introducing an optimization strategy based on reinforcement learning GRPO (Group Relative Policy Optimization) to continuously optimize large models and improve model performance.

[0083] GRPO is an online learning algorithm, which means it iteratively improves by using data generated by the trained model itself during training. The goal of GRPO is to maximize the advantage of generating completions while ensuring that the model stays close to the reference policy.

[0084] Figure 3 This is a workflow diagram of a large model-assisted dense point question-answering system based on a dynamic feedback mechanism provided by an embodiment of the present application. The following describes the process: S301, obtaining text files, such as PDF, Word, TXT, Excel and other text in any format; S302, file parsing, obtaining the text content in the file through the parsing component; S303, obtaining a dense point recognition instruction; S304, constructing a prompt based on the dense point recognition instruction and the text content, that is, reading the dense point recognition instruction and concatenating it with the read text content to form the dense point recognition prompt; S305, calling the dense point recognition large model to generate dense point question and answer pairs (also referred to as question and answer pairs), that is, inputting the above prompt into the dense point recognition large model; S306, generating dense point question-answer pairs and performing confidence scoring on the dense point question-answer pairs; S307, calling the dense point rule library; S308, using the dense point rule library to perform rule scoring on the dense point question and answer pairs through the rule engine, such as the similarity scoring mentioned above; S309, performing comprehensive scoring by combining confidence scoring and rule scoring; S310, storing the generated key point question and answer pairs in a key point question and answer pair candidate library; S311, filtering by subject and keyword to obtain filtering results (corresponding to the first filtering results obtained above); S312, selecting the TopK based on the comprehensive scores, that is, sorting all the dense point question-answer pairs in the screening results of step S311 according to the comprehensive scores, and selecting the top K dense point question-answer pairs; S313, rule filtering, filtering the dense question-answer pairs obtained in step S312, for example, removing question-answer pairs containing sensitive words (such as pornography, violence, etc.); S314, determining whether the number of dense point question-answer pairs obtained after step S313 is less than N; S315: When the judgment result of the above step S314 is yes, retrieve the result from the screening result obtained in S311; then go to step S313 to perform rule filtering; S316: When the judgment result of the above step S314 is negative, that is, the number of dense point question-answer pairs obtained after step S313 is sufficient, these dense point question-answer pairs are formed into a preliminary screening candidate set; S317, determining whether to rewrite, that is, determining whether each question-answer pair in the initial screening candidate set needs to be rewritten; S318, when rewriting is required, that is, when the dense point question-answer pair obtained by the dense point recognition large model is inaccurate, the dense point question-answer pair can be rewritten; S319, when dense point question-answer pairs in the initial screening candidate set do not need to be rewritten, and after the question-answer pairs that need to be rewritten are rewritten, the question-answer pairs are quality-marked; In this step, the correct dense point question-answer pairs and the rewritten dense point question-answer pairs are marked with quality. For example, the quality mark can be represented by 0, 1, and 2, where 0 represents poor quality, 1 represents medium quality, and 2 represents high quality. S320, optional: To ensure that the rewritten question-answer pairs or the question-answer pairs that have passed the first review are of higher quality, a review step may be added to ensure that the final question-answer pairs are of higher quality; S321, store high-quality dense point question and answer pairs in the database (such as Figure 3 "High-quality question-answer pair database" in ); It should be noted that the above steps S319 and S320 are not necessary. As an optional implementation method, the correct key point question and answer pairs and the rewritten key point question and answer pairs can also be stored in the key point high-quality question and answer pair library; S322, determining whether to optimize; Optionally, in actual applications, a manual triggering mode, a timed update mode, or an automatic triggering mode when certain conditions are met can be selected; S323, when the judgment result of the above step S322 is yes, introduce the optimization strategy based on reinforcement learning GRPO to optimize the large model of dense point recognition; if the judgment result of the above step S322 is no, no optimization is required.

[0085] Through the above embodiments, the efficiency of dense point identification and question-answer pair construction is improved, the accuracy of dense point identification and question-answer pair construction is improved, and at the same time, dynamic optimization of the system is achieved, and the purpose of updating the model in real time according to feedback or new data is achieved.

[0086] Compared with the related art, the embodiments of the present application have at least the following advantages: High accuracy: Through semantic understanding and feedback from a large model, the system can accurately identify sensitive information in documents; Real-time feedback: Feedback on recognition results can be provided immediately, and the system can respond quickly and optimize the recognition model; User-friendly: Results are output in the form of question-answer pairs, allowing users to easily understand the recognition results and improving the user experience. Dynamic optimization: The system can continuously optimize the recognition model based on feedback to improve the accuracy and comprehensiveness of recognition.

[0087] This application also provides a system for constructing question-answer pairs, such as Figure 4 As shown, Figure 4 This is a structural block diagram of a system for constructing question-answer pairs provided in an embodiment of the present application, the system comprising: An acquisition module 41 is configured to receive a text to be recognized and a dense point recognition instruction, wherein the recognition instruction is used to instruct to recognize dense point information from the text to be recognized according to a preset recognition rule; A construction module 42 is configured to construct a Prompt based on the text to be recognized and the key point recognition instruction, wherein the Prompt is used to instruct the recognition of key point information from the text to be recognized and the construction of a question-answer pair corresponding to the key point information; Recognition module 43 is used to call the target large model based on Prompt to perform dense point information recognition on the text to be recognized, obtain an initial question-answer pair set, and perform confidence scoring on each initial question-answer pair in the initial question-answer pair set; Comprehensive scoring module 44, configured to generate a comprehensive score set based on the confidence score and the rule score, wherein the comprehensive score set includes a comprehensive score for each initial question-answer pair in the initial question-answer pair set, and the rule score is used to indicate the compliance rule score for each initial question-answer pair using a preset dense point rule library; The screening module 45 is used to screen out a target question-answer pair set from the initial question-answer pair set according to the comprehensive score set.

[0088] In an optional embodiment, the comprehensive scoring module 44 includes: a determination unit, which is used to determine the corresponding comprehensive score for any initial question-answer pair according to the following formula: Score = K1 × C + K2 × R, K1 + K2 = 1, where Score represents the comprehensive score, C represents the confidence score, R represents the rule score, and K1 and K2 are weight coefficients.

[0089] In an optional embodiment, the above-mentioned screening module 45 includes: a first screening unit, used to screen the initial question-answer pair set according to keywords to obtain a first screening result; a first filtering unit, used to filter the first screening result according to preset rules to obtain a second screening result; a first selection unit, used to select the top N initial question-answer pairs ranked by comprehensive scores from the second screening result according to the comprehensive score set as the target question-answer pair set, where N is a positive integer greater than or equal to 1.

[0090] In an optional embodiment, the above-mentioned screening module 45 includes: a second screening unit, used to screen the initial question-answer pair set according to keywords to obtain a first screening result; a second selection unit, used to select the top K initial question-answer pairs ranked by comprehensive scores from the first screening result according to the comprehensive score set as the alternative question-answer pair set, where K is a positive integer greater than or equal to 1; a second filtering unit, used to filter each initial question-answer pair included in the alternative question-answer pair set according to preset rules to obtain a third screening result; an acquisition unit, used to obtain the target question-answer pair set based on the third screening result.

[0091] In an optional embodiment, the above-mentioned obtaining unit includes: a first processing subunit, which is used to, when it is determined that the number of initial question-answer pairs in the third screening result is greater than or equal to the first preset number, group the initial question-answer pairs in the third screening result into a target question-answer pair set; a second processing subunit, which is used to, when it is determined that the number of initial question-answer pairs in the third screening result is less than the first preset number, retrieve the initial question-answer pairs from the retrieved candidate set in descending order of comprehensive scores and add them to the third screening result, so that the number of initial question-answer pairs in the supplemented third screening result is greater than or equal to the first preset number, and group the initial question-answer pairs in the supplemented second screening result into a target question-answer pair set, wherein the retrieved candidate set is composed of other initial question-answer pairs in the first screening result except the initial question-answer pairs in the alternative question-answer pair set.

[0092] In an optional embodiment, the above system also includes: an audit module, which is used to audit each question and answer pair in the target question and answer pair set to obtain an audit result, wherein the audit result includes at least one of the following information: confirming correctly identified information, marking misidentified information, and supplementing missed identified information.

[0093] In an optional embodiment, the above system also includes: a correction module, which is used to correct the target question and answer pair set according to the audit result after obtaining the audit result, obtain the corrected target question and answer pair set, and store it in the secret point library; a scanning module, which is used to scan the secret point library according to a preset period to determine the number of new question and answer pairs in the secret point library; a first optimization module, which is used to optimize the target large model with the newly added question and answer pairs as the training data set when it is determined that the number of newly added question and answer pairs is greater than or equal to a second preset number, wherein the newly added question and answer pairs include each question and answer pair in the corrected target question and answer pair set.

[0094] In an optional embodiment, the above system also includes: a trigger module, which is used to dynamically optimize the target large model according to a preset trigger method based on the audit results, wherein the preset trigger method includes one of the following: manual trigger method, periodic trigger method, and conditional trigger method.

[0095] In an optional embodiment, the above system further includes: a second optimization module for dynamically optimizing the target large model based on the audit results by adopting an optimization strategy based on the reinforcement learning GRPO algorithm.

[0096] In an optional embodiment, the prompt includes recognition scope, recognition rule classification, confidentiality level and confidence score.

[0097] Other device or system embodiments correspond to the aforementioned method embodiments. For other technical features, please refer to the previous embodiments and will not be repeated here.

[0098] The present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed, any one of the above-mentioned method steps is executed.

[0099] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0100] This application also discloses an electronic device. Figure 5 As shown, Figure 5 Schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. The electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0101] The communication bus 502 is used to implement the connection and communication between these components.

[0102] The user interface 503 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 503 may also include a standard wired interface and a wireless interface.

[0103] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0104] The processor 501 may include one or more processing cores. The processor 501 utilizes various interfaces and circuits to connect various components within the electronic device (e.g., a server). It executes instructions, programs, code sets, or instruction sets stored in the memory 505 and accesses data stored in the memory 505 to perform various server functions and process data. Optionally, the processor 501 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 501 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing content displayed on the display; and the modem handles wireless communications. It is understood that the modem may also be implemented as a separate chip, rather than integrated into the processor 501.

[0105] Among them, the memory 505 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 505 includes a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 505 may also be optionally at least one storage device located away from the aforementioned processor 501. Reference Figure 5 , as a computer storage medium, the memory 505 may include an operating system, a network communication module, a user interface module, and an application program for a method of constructing a question-answer pair.

[0106] exist Figure 5 In the electronic device 500 shown, the user interface 503 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 501 can be used to call an application of a method for constructing a question-answer pair stored in the memory 505. When executed by one or more processors 501, the electronic device 500 executes one or more of the methods described in the above embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that this application is not limited to the described order of actions, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0107] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed device or system can be implemented in other ways. For example, the device or system embodiments described above are merely schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0109] The foregoing is merely an exemplary embodiment of the present disclosure and is not intended to limit the scope of the present disclosure. In other words, any equivalent variations and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the disclosure herein.

[0110] This application is intended to cover any modifications, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the technical field not described in the present disclosure.

Claims

1. A method for constructing question-answer pairs, characterized in that: include: Get the text to be recognized and the key point recognition instructions; Constructing a Prompt based on the text to be recognized and the key point recognition instruction, wherein the Prompt is used to instruct to recognize key point information from the text to be recognized and construct a question-answer pair corresponding to the key point information; Based on the Prompt, the target large model is called to perform dense point information recognition on the text to be recognized, to obtain an initial question-answer pair set, and a confidence score is performed on each initial question-answer pair in the initial question-answer pair set; Obtaining a comprehensive score set based on the confidence score and the rule score, wherein the comprehensive score set includes a comprehensive score for each initial question-answer pair in the initial question-answer pair set, and the rule score is used to indicate a compliance rule score for each initial question-answer pair using a preset dense point rule library; A target question-answer pair set is screened from the initial question-answer pair set according to the comprehensive score set.

2. The method according to claim 1, characterized in that A comprehensive score set is obtained based on the confidence score and the rule score, including: For any of the initial question-answer pairs, the corresponding comprehensive score is determined according to the following formula: Score = K1 × C + K2 × R, K1 + K2 =1, Among them, Score represents the comprehensive score, C represents the confidence score, R represents the rule score, and K1 and K2 are weight coefficients.

3. The method according to claim 1, characterized in that Filtering a target question-answer pair set from the initial question-answer pair set according to the comprehensive score set includes: Filtering the initial question-answer pair set according to keywords to obtain a first filtering result; Filtering the first screening result according to a preset rule to obtain a second screening result; According to the comprehensive score set, the top N initial question-answer pairs ranked by comprehensive score are selected from the second screening results as the target question-answer pair set, where N is a positive integer greater than or equal to 1.

4. The method according to claim 1, wherein Filtering a target question-answer pair set from the initial question-answer pair set according to the comprehensive score set includes: Filtering the initial question-answer pair set according to keywords to obtain a first filtering result; Selecting the top K initial question-answer pairs ranked by comprehensive scores from the first screening results according to the comprehensive score set as a set of candidate question-answer pairs, where K is a positive integer greater than or equal to 1; Filtering each initial question-answer pair included in the set of candidate question-answer pairs according to a preset rule to obtain a third screening result; The target question-answer pair set is obtained according to the third screening result.

5. The method according to claim 4, characterized in that Obtaining the target question-answer pair set according to the third screening result includes: When it is determined that the number of the initial question-answer pairs in the third screening result is greater than or equal to a first preset number, forming the target question-answer pair set from the initial question-answer pairs in the third screening result; When it is determined that the number of the initial question-answer pairs in the third screening result is less than the first preset number, the initial question-answer pairs are retrieved from the retrieved candidate set in order of comprehensive scores from high to low and added to the third screening result, so that the number of the initial question-answer pairs in the supplemented third screening result is greater than or equal to the first preset number, and the initial question-answer pairs in the supplemented third screening result are combined into the target question-answer pair set, wherein the retrieved candidate set is composed of other initial question-answer pairs in the first screening result except the initial question-answer pairs in the alternative question-answer pair set.

6. The method according to claim 1, characterized in that The method further comprises: Audit each question-answer pair in the target question-answer pair set to obtain an audit result, wherein the audit result includes at least one of the following information: Confirm the correctly identified information, mark the misidentified information, and supplement the missed information.

7. The method according to claim 6, characterized in that After obtaining the audit result, the method further includes: Modifying the target question-answer pair set according to the audit result to obtain a modified target question-answer pair set, and storing the modified target question-answer pair set in the secret point database; Scan the secret point database according to a preset period to determine the number of new question-answer pairs in the secret point database; When it is determined that the number of the newly added question-answer pairs is greater than or equal to a second preset number, the target large model is optimized using the newly added question-answer pairs as a training data set, wherein the newly added question-answer pairs include each question-answer pair in the revised target question-answer pair set.

8. A system for constructing question-answer pairs, characterized in that: include: An acquisition module is used to obtain the text to be recognized and the dense point recognition instruction; A construction module, configured to construct a Prompt based on the text to be recognized and the dense point recognition instruction, wherein the Prompt is used to instruct the recognition of dense point information from the text to be recognized and the construction of a question-answer pair corresponding to the dense point information; A recognition module, configured to call a target large model based on the Prompt to perform dense point information recognition on the text to be recognized, obtain an initial question-answer pair set, and perform a confidence score on each initial question-answer pair in the initial question-answer pair set; a comprehensive scoring module, configured to obtain a comprehensive scoring set based on the confidence score and the rule score, wherein the comprehensive scoring set includes a comprehensive score for each initial question-answer pair in the initial question-answer pair set, and the rule score is used to indicate a compliance rule score for each initial question-answer pair using a preset dense point rule library; A screening module is used to screen out a target question-answer pair set from the initial question-answer pair set based on the comprehensive score set.

9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is performed.

Images