Self-adaptive template reconstruction method and system based on few samples

Abstract

The invention provides a self-adaptive template reconstruction method and system based on a small amount of samples, and belongs to the field of natural language processing, and the method comprises the steps: S1, constructing an initial prompt template based on a small amount of sample data; s2, inputting the initial prompt template into a large language model to generate a question pool; s3, generating an answer pool by using a large language model; s4, performing screening and clustering analysis on the answers in the answer pool through a multi-level similarity comparison module, and dynamically adjusting weight parameters of similarity calculation in the multi-level similarity comparison module through a model self-feedback mechanism to obtain an overall score of the screened answers; s5, calculating a comprehensive similarity score of each answer group, performing expert scoring on each answer group, and reconstructing the initial prompt template based on the comprehensive similarity score and the expert score; and S6, inputting the reconstructed prompt template into a large language model for iterative updating until a final answer is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and system for adaptive template reconstruction based on a small number of samples. Background Art

[0002] With the widespread application of large language models (such as the GPT series, LLaMA, etc.) in natural language processing, they have demonstrated powerful capabilities in handling various tasks. However, current mainstream models usually rely on a large amount of training data and do not perform well in small or zero-sample learning scenarios. When the input data is insufficient, the model may not be able to generate high-quality, relevant answers, which in turn affects its performance on specific tasks. Existing few-sample learning technologies mainly face the following key problems: 1. Insufficient template matching capabilities: Most existing generative models rely on manually constructed templates, but these templates are often difficult to adapt to complex task requirements in a small sample environment, and the quality of the generated text is also uneven, affecting the output effect of the model.

[0003] 2. The diversity and accuracy of the generated data cannot be balanced: Generative models need to strike a balance between diversity and accuracy, often sacrificing one aspect to improve the performance of the other. This method is not suitable for fields where tasks require more precision, such as legal texts, bidding information, medical documents, etc.

[0004] 3. Imperfect answer screening and optimization process: For the generated answers, existing screening methods often lack a fine-grained similarity comparison mechanism and cannot fully utilize the semantic information of the expected answers, thus affecting the accuracy and consistency of the answers.

[0005] In recent years, few-shot learning and adaptive generation techniques have gradually attracted attention. These techniques, combined with existing pre-trained language models, can adjust template prompts to guide the model to output results that better meet task requirements without large-scale datasets. However, existing technologies still have significant shortcomings in handling multi-dimensional tasks, generating diverse answers, and dynamically adjusting template prompts according to specific questions.

[0006] Therefore, finding a method that can both build templates in a small number of sample scenarios and improve the accuracy of answer generation is a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0007] The present invention provides a method and system for adaptive template reconstruction based on a small number of samples, so as to solve the defect that the prior art cannot perform template reconstruction based on a small number of samples, and improve the task performance in a small number of sample scenarios through adaptive template reconstruction and keyword supplementation.

[0008] The present invention provides an adaptive template reconstruction method based on a small number of samples, comprising the following steps: S1. Construct an initial prompt template based on a small amount of sample data, wherein the initial prompt template includes a task background description, context information, and a part to be completed; S2, inputting the initial prompt template into a large language model, and generating a question pool through specific instructions, agent instructions and example instructions; S3. Using the large language model to answer each question in the question pool to generate an answer pool; S4. Screening and clustering the answers in the answer pool through a multi-level similarity comparison module to obtain a screened answer pool and a screened answer group, and dynamically adjusting the weight parameters of the similarity calculation in the multi-level similarity comparison module through a model self-feedback mechanism to obtain an overall score of the screened answers; S5. Calculate the comprehensive similarity score of each answer group based on the overall score of the answers in each answer group, and perform expert scoring on each answer group, construct examples based on the comprehensive similarity score and the expert score, and reconstruct the initial prompt template based on the example to obtain a reconstructed prompt template; S6. Input the reconstructed prompt template into the large language model for iterative updating until a final answer is obtained.

[0009] According to an adaptive template reconstruction method based on a small number of samples provided by the present invention, the specific instructions are instructions designed according to the task objectives, used to guide the large language model to generate answers related to the task, the proxy instructions are instructions that guide the large language model to generate answers in an indirect way, and the example instructions are generated by providing example questions and answers.

[0010] According to an adaptive template reconstruction method based on a small number of samples provided by the present invention, the generation of the question pool specifically includes: The initial prompt template is input into the large language model for entity expansion to obtain an entity set. The initial prompt template is supplemented with keywords based on the entity set, and the weight of the keywords is adjusted. The weight update formula is: ; in, represents the weight value of the fth keyword in round t+1, represents the weight value of the fth keyword in the tth round, represents the adjustment parameter of the correlation evaluation, Indicates keywords The relevance evaluation value with the context Context, represents the fth keyword, Represents the current context. represents the tuning parameter for frequency evaluation, Indicates keywords frequency of occurrence in the text; The expanded keywords are divided into three categories: core keywords, secondary keywords, and background keywords, and are prioritized by differentiated initial weights to obtain a keyword sequence; the initial weights are: ; in, represents the initial weight value of the fth keyword, f represents the index of the keyword, represents the weight adjustment coefficient, represents the fth keyword, represents the priority scoring function; Generate multiple question variants by combining keyword sequences with specific instructions, agent instructions, and example instructions to form an initial question pool; The initial question pool is deduplicated and screened to obtain a screened question pool; the screened question pool is represented as: ; in, represents the filtered question pool, Indicates the current problem to be evaluated. Indicates except outside Other questions in the question pool, i represents the number of the current question, j represents the number of other questions except i, Q represents the initial question pool, Representation problem and The similarity measure between Represents the similarity threshold.

[0011] According to an adaptive template reconstruction method based on a small number of samples provided by the present invention, the processing steps of the multi-level similarity comparison module are: Performing primary similarity calculation on the answers in the answer pool to obtain a primary similarity score, and preliminarily screening the answer pool based on the primary similarity score to obtain a first answer pool; Performing semantic similarity calculation based on deep learning on the answers in the first answer pool to obtain a semantic similarity score, and screening based on the semantic similarity score to obtain a second answer pool; The answers in the second answer pool are clustered based on the semantic similarity scores to form multiple answer groups.

[0012] According to an adaptive template reconstruction method based on a small number of samples provided by the present invention, the model self-feedback mechanism includes: The primary similarity score and the semantic similarity score are weightedly combined to obtain an initial similarity score; The initial similarity score is dynamically adjusted based on coverage and specificity to obtain an overall similarity score.

[0013] According to an adaptive template reconstruction method based on a small number of samples provided by the present invention, step S5 specifically includes: Set an overall score threshold. When the overall score of the answers in the answer group is lower than the overall score threshold, the comprehensive scoring mechanism is triggered: Calculate the comprehensive similarity score of each answer group, and perform expert scoring on each answer group, and sort the answer groups from large to small according to the comprehensive similarity score and expert scoring to form an answer group sequence; The first group of answers in the answer group sequence is taken as an example, and the initial prompt template is reconstructed based on the example to obtain a reconstructed prompt template.

[0014] According to an adaptive template reconstruction method based on a small amount of samples provided by the present invention, the initial prompt template is constructed based on a small amount of sample data, specifically comprising: Preprocess a small amount of sample data, and add identifiers for clarifying the context scope at the beginning and end of the preprocessed small amount of sample data; Design the initial prompt template based on the task objectives and contextual information; The prompt engineering method combined with the large language model understands the context through prompt words, and dynamically adjusts the structure and vocabulary of the initial prompt template based on the context; wherein the prompt words include a clear description of the task goal and contextual prompts related to the task.

[0015] The present invention also provides an adaptive template reconstruction system based on a small number of samples, which implements the adaptive template reconstruction method described above, including: A sample analysis module is used to construct an initial prompt template based on a small amount of sample data, wherein the initial prompt template includes a task background description, context information, and target information prompts that need to be completed; A question generation module, used to input the initial prompt template into a large language model, and generate a question pool through specific instructions, agent instructions and example instructions; An answer generation module, used to answer each question in the question pool using the large language model to generate an answer pool; A similarity screening module, used to screen and cluster the answers in the answer pool through a multi-level similarity comparison module to obtain a screened answer pool and a screened answer group, and calculate the overall score of the answer based on the screened answer group; A template reconstruction module is used to calculate the comprehensive similarity score and expert score of each answer group based on the overall score of the screened answer group, sort the answer groups according to the comprehensive similarity score and expert score, select the answer group with the highest score as an example, optimize and reconstruct the initial prompt template, and generate a reconstructed prompt template; The iterative optimization module is used to input the reconstructed prompt template into the large language model for iterative updating and continuous optimization of the generated answers until the final answer is obtained.

[0016] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the above-mentioned adaptive template reconstruction methods is implemented.

[0017] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the adaptive template reconstruction method described in any one of the above is implemented.

[0018] The present invention provides an adaptive template reconstruction method and system based on a small number of samples. Through adaptive template reconstruction and keyword supplementation technology, combined with a multi-level similarity comparison module and a model self-feedback mechanism, the quality of answer generation and the accuracy of task completion are effectively improved, and the task performance of a large language model in a small number of sample scenarios is significantly improved, so that the model can achieve high-quality answer generation based on limited samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 is a flow chart of the adaptive template reconstruction method provided by the present invention; Figure 2 is a block diagram of the adaptive template reconstruction method provided by the present invention; Figure 3 It is a schematic diagram of a flow chart of an embodiment of adaptive template reconstruction provided by the present invention; Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] like Figure 1 and Figure 2 As shown, the present invention provides an adaptive template reconstruction method based on a small number of samples, comprising the following steps: S1. Construct an initial prompt template based on a small amount of sample data, wherein the initial prompt template includes a task background description, context information, and a part to be completed; S2, inputting the initial prompt template into a large language model, and generating a question pool through specific instructions, agent instructions and example instructions; S3. Using the large language model to answer each question in the question pool to generate an answer pool; S4. Screening and clustering the answers in the answer pool through a multi-level similarity comparison module to obtain a screened answer pool and a screened answer group, and dynamically adjusting the weight parameters of the similarity calculation in the multi-level similarity comparison module through a model self-feedback mechanism to obtain an overall score of the screened answers; S5. Calculate the comprehensive similarity score of each answer group based on the overall score of the answers in each answer group, and perform expert scoring on each answer group, construct examples based on the comprehensive similarity score and the expert score, and reconstruct the initial prompt template based on the example to obtain a reconstructed prompt template; S6. Input the reconstructed prompt template into the large language model for iterative updating until a final answer is obtained.

[0023] The preprocessing includes removing HTML code, page tags, escape symbols, converting English punctuation to Chinese punctuation, deleting transfer symbols, converting line breaks to spaces, etc.

[0024] It is understandable that the task goal is to solve the task performance problem of the large language model (LLM) in a small number of sample scenarios. When constructing the initial prompt template, the text to be processed is first embedded in a unified prompt prefix structure, that is, the specific active text content is placed between the "START" and "END" identifiers to form a standardized input format; then the preset extraction target is merged with the text to construct the prompt statement for entity recognition, and the model is required to output entity type information related to the extracted entity.

[0025] like Figure 2As described above, the initial prompt template includes Context, Target and Question, where Context includes the text embedding prompt prefix " That is, "STARTCurrent location: Homepage Announcement information Announcement of the winning bid (result) of a certain unit in a certain city × procurement... Attachment: A certain unit project bidding document (final draft).pdf; END", Target is: A certain province × bidding Co., Ltd., Question is: What is the entity type in the Context corresponding to Target? Output the three most relevant entity types in the form of a list; Input the initial prompt template into the Large Language Model (LLM) to generate a question pool, including question 1, question 2, and question 3; Input the question pool into the large language model to generate an answer pool, including answer 1, answer 2, and answer 3; Using a multi-level similarity comparison module to screen and cluster the answers in the answer pool, and to generate comprehensive similarity scores for each answer group, including score 1, score 2, and score 3; Select the answer group with the highest comprehensive similarity score to construct a new prompt template, and obtain a reconstructed prompt template, including examples, question 4 and answers; The reconstructed prompt template is input into the large language model to obtain the final answer.

[0026] The present invention uses adaptive template reconstruction and keyword supplementation technology, combined with a multi-level similarity comparison module and a model self-feedback mechanism, to effectively improve the quality of answer generation and the accuracy of task completion, significantly improve the task performance of large language models in scenarios with a small number of samples, and enable the model to achieve high-quality answer generation based on limited samples.

[0027] In one embodiment of the present invention, the constructing of an initial prompt template based on a small amount of sample data specifically includes: Preprocess a small amount of sample data, and add identifiers for clarifying the context scope at the beginning and end of the preprocessed small amount of sample data; Design the initial prompt template based on the task objectives and contextual information; The prompt engineering method combined with the large language model understands the context through prompt words, and dynamically adjusts the structure and vocabulary of the initial prompt template based on the context; wherein the prompt words include a clear description of the task goal and contextual prompts related to the task.

[0028] It can be understood that for the input bidding winning text, a basic text template is first constructed: "T (x) = given bidding winning activity text: START [specific text content] END", where the specific text content is the actual bidding winning text; Integrate the preset extraction target into the template to form a complete prompt sentence: "T'(x) = The corresponding extracted entity result is: Target. What is the entity type in the corresponding bidding activity? Output the three most relevant entity types in the form of a list.", where Target corresponds to the preset extraction target; The complete prompt template is fed into the large language model, which outputs a list of the most relevant entity types:

[0029] in, Represents the three most relevant entity types of the output, and LM represents large language model processing.

[0030] A specific embodiment is used to illustrate: When processing a bidding text containing procurement agency information, if the preset extraction target is "Province × Tendering Co., Ltd.", the constructed complete prompt template will guide the model to identify the entity type that the entity may correspond to in the bidding activity, such as "procurement agency", "tendering agency", "company name", etc. This structured template design method not only ensures that the model can accurately understand the task requirements, but also provides sufficient context information support, effectively improving the accuracy of entity type recognition.

[0031] In one embodiment of the present invention, the generation of the question pool specifically includes: The initial prompt template is input into the large language model for entity expansion to obtain an entity set. The initial prompt template is supplemented with keywords based on the entity set, and the weight of the keywords is adjusted. The weight update formula is: ; in, represents the weight value of the fth keyword in round t+1, represents the weight value of the fth keyword in the tth round, represents the adjustment parameter of the correlation evaluation, Indicates keywords The relevance evaluation value with the context Context, represents the fth keyword, Represents the current context. represents the tuning parameter for frequency evaluation, Indicates keywords frequency of occurrence in the text; The expanded keywords are divided into three categories: core keywords, secondary keywords, and background keywords, and are prioritized by differentiated initial weights to obtain a keyword sequence; the initial weights are: ; in, represents the initial weight value of the fth keyword, f represents the index of the keyword, represents the weight adjustment coefficient, represents the fth keyword, represents the priority scoring function; The keyword sequence is combined with specific instructions, proxy instructions and example instructions to generate multiple question variants to form an initial question pool; wherein the specific instructions are instructions designed according to the task objectives and used to guide the large language model to generate answers related to the task, the proxy instructions are instructions that guide the large language model to generate answers in an indirect way, and the example instructions are generated by providing example questions and answers; The initial question pool is deduplicated and screened to obtain a screened question pool; the screened question pool is represented as: ; in, represents the filtered question pool, Indicates the current problem to be evaluated. Indicates except outside Other questions in the question pool, i represents the number of the current question, j represents the number of other questions except i, Q represents the initial question pool, Representation problem and The similarity measure between Represents the similarity threshold.

[0032] Understandably, specific instructions are used to guide the large language model to generate task-related answers, such as "Please extract the `Entity` related entities based on the following bidding text"; agent designation is to let the model understand the task in a specific context through role-playing and other methods, and identify key points in a more indirect way, such as "Suppose you are a model that is very proficient in information extraction. Now you will be given a paragraph about bidding. Please extract the entities I want correctly according to my requirements. Now I want to extract the contacts in the paragraph. Please give your reasoning process. The text is as follows: \nSTART...END", and example instructions are used to help the model understand the task requirements through examples and show how to extract entities, such as "The entity type examples given in the example instructions help the model generate questions with similar structures: Given the example START...END, extract the Entity from it as Target. Given the target extraction text START...END, now we need to extract the Entity from it, please help me extract it.

[0033] The present invention introduces dual evaluations of context relevance and frequency evaluation into the weight adjustment formula, so that the adjustment of weights is not only based on the global context, but also takes into account the statistical characteristics of specific fields. Keywords are classified and priority sorting strategies are designed respectively: core keywords are directly associated with field task objectives; secondary keywords are extended vocabulary closely associated with core keywords; background keywords are low-priority vocabulary generated to supplement semantic integrity. Different priorities and differentiated initial weights are assigned to different keywords to improve the sorting effect of high-priority keywords.

[0034] In one embodiment of the present invention, the processing steps of the multi-level similarity comparison module are: Performing primary similarity calculation on the answers in the answer pool to obtain a primary similarity score, and preliminarily screening the answer pool based on the primary similarity score to obtain a first answer pool; Performing semantic similarity calculation based on deep learning on the answers in the first answer pool to obtain a semantic similarity score, and screening based on the semantic similarity score to obtain a second answer pool; The answers in the second answer pool are clustered based on the semantic similarity scores to form multiple answer groups.

[0035] The present invention constructs a comprehensive similarity scoring model through weighted calculation of primary similarity scores and advanced similarity scores. The multi-level similarity comparison module can accurately measure and screen the similarities of answers in the answer pool, avoiding the existence of redundant and repeated answers. At the same time, by dynamically adjusting the weight parameters, it adapts to different task requirements and further improves the accuracy and robustness of the similarity scoring.

[0036] In one embodiment of the present invention, the calculation process of the primary similarity score is: Based on all the answers in the answer pool, a unified vocabulary is constructed, and any two answers are selected from the answer pool, and they are converted into word frequency vectors based on the same vocabulary. and , where the vocabulary sizes of word frequency vector A and word frequency vector B are the same, Represents the gth word frequency value in the word frequency vector A, represents the hth word frequency value in the word frequency vector B, and d represents the number of answers in the answer pool; The primary similarity scores of the two answers are calculated based on the word frequency vector A and the word frequency vector B. The calculation formula is:

[0037] Set the primary similarity threshold. If the cosine similarity is less than the threshold, the corresponding answer will be filtered out.

[0038] Furthermore, the calculation process of the semantic similarity score is: Input any two answers in the first answer pool into a deep learning model (such as BERT, Bidirectional Encoder Representations from Transformers) to obtain the answer sentence vector representation Vector representation of answer sentence , Similarly, cosine similarity is used to calculate the similarity between sentence vectors to obtain the semantic similarity score. The calculation formula is:

[0039] Among them, q represents the number of answers in the first answer pool, and r represents the answer sentence vector representation The index of the answer in , s represents the answer sentence vector representation The index of the answer in ; Set a semantic similarity threshold. If the similarity between sentence vectors is less than the semantic similarity threshold, the corresponding answer will be filtered out.

[0040] It can be understood that both the primary similarity threshold and the semantic similarity threshold can be set according to actual usage requirements, and the present invention does not impose any specific limitation on this.

[0041] A specific embodiment is used to illustrate: There are three answers in the answer pool: "The project budget is 5 million yuan", "The total project budget is 5 million yuan" and "The bidding budget is 5 million yuan"; The unified vocabulary is constructed as: "{"project", "budget", "amount", "project", "total", "yuan", "for", "tender", "ten thousand"}"; Select any two answers and convert them into word frequency vectors: answer 1 is converted into vector A: (1, 1, 1, 0, 0, 0, 0, 0, 1), answer 2 is converted into vector B: (0, 1, 0, 1, 1, 1, 0, 0, 1); Calculate the cosine similarity of these two word frequency vectors to get the primary similarity score.

[0042] Furthermore, the two answers in the answer pool: "Project budget amount 5 million" and "Total project budget 5 million" are input into BERT respectively to obtain two answer sentence vector representations: and ; Calculate the vector representation of the two answer sentences and The cosine similarity between them is used to obtain the semantic similarity score.

[0043] In one embodiment of the present invention, the steps of cluster analysis are: Using semantic similarity score Constructing a similarity matrix ,in , represents the semantic similarity score The corresponding answer and The similarity of the answer is then expressed as the sentence vector of all the answers using the K-means clustering method. Cluster into K clusters, and define the cluster center as , the clustering goal is:

[0044] in, represents the center point of the kth cluster, k represents the cluster number, K represents the total number of clusters, and N represents the total number of answers in the second answer pool. Represents the sentence vector representation of the i-th answer in the second answer pool; It can be understood that the cluster center with the highest similarity represents the final screened answer set.

[0045] In one embodiment of the present invention, the top-k candidate examples with the highest comprehensive similarity scores are selected, and a prompt template is constructed based on their core structure and semantic characteristics. The reconstructed prompt template contains the core keywords and semantic framework of the task, which is convenient for guiding the large language model to generate consistent answers.

[0046] Specifically, extract the core keywords from each comprehensive similarity score ,in represents the mth extracted core keyword, m represents the total number of core keywords, and the semantic framework of the template is formed by combining the answer structure:

[0047] in, Represents a sample template, Indicates operations that perform structural processing on content. represents the candidate example with the highest comprehensive similarity score, and k represents the number of selected candidate examples; The sample template With target tasks Splice to get the reconstructed prompt template :

[0048] in, Represents a splicing operation; Input the reconstructed prompt template into the large language model to obtain the initial result:

[0049] in, Indicates the initial result; The template structure is gradually optimized based on the initial results, and the formula is:

[0050] Among them, t represents the current iteration round, represents the prompt template for round t, is the keyword extracted from the result of the tth round of iteration, p represents the number of newly extracted keywords, and the updated template is input into the model again to get a new round of answers :

[0051] The iterative process continues until a final answer that meets the target task is reached.

[0052] In one embodiment of the present invention, for the problem , generating m different answers ,in, represents the mth answer to the i-th question, i represents the number of the question, and m represents the number of answers generated for each question. The formula is:

[0053] in, To generate temperature parameters and control the diversity of model output, represents the language model, j represents the number of generated answers, and the range of j is [1, m]. express The jth value of the parameter; by setting different temperatures, various answers can be generated; The problem pool The set of answers to each question in Merge to form an answer pool:

[0054] Here, n represents the number of questions.

[0055] Understandably, , , .

[0056] Furthermore, in order to avoid redundant and repeated answers in the answer pool, similarity metrics are used to screen the answers. Represents the similarity measure of two answers and sets the similarity threshold , the final answer pool after screening It is expressed as:

[0057] in, represents the kth answer generated by the i-th question, where i is in the range [1, m] and k is in the range [1, m]. represents the lth answer generated by the jth question, the range of j is [1, m], the range of l is [1, m], Indicate the answer With the answer The similarity between .

[0058] Second answer pool The similarity score between each answer and the target answer is calculated by the semantic similarity measurement method:

[0059] in, Indicate the answer The similarity score of Represents the similarity score calculation function, and selects the one with the highest score based on the similarity score Answer collection:

[0060] The weight parameters of the similarity calculation are dynamically adjusted through the model's self-feedback mechanism to obtain the overall score of the screened answers.

[0061] In one embodiment of the present invention, the model self-feedback mechanism includes: The primary similarity score and the semantic similarity score are weightedly combined to obtain an initial similarity score; The initial similarity score is dynamically adjusted based on coverage and specificity to obtain a total similarity score; coverage is used to measure whether the answer covers key topics or information, and is used to ensure that the generated answer is complete and contains all important information required for the question; specificity is used to ensure the concreteness and accuracy of the answer, and is used to prevent the generated answer from being too general or vague.

[0062] In one embodiment of the present invention, the calculation formula for the overall score of the screened answers is:

[0063] in, represents the comprehensive similarity score of the answer, represents the weight coefficient of primary similarity, represents the weight coefficient of semantic similarity, represents the primary similarity score, represents the semantic similarity score; The overall score after optimization is:

[0064] in, represents the final overall score, represents the weight coefficient of the coverage index, represents the weight coefficient of the specificity index, Indicates the coverage of the key points of the question by the answer. Indicates the degree of specificity and accuracy of the answer; Among them, the weight coefficient of the coverage index is obtained through iterative optimization through multiple rounds of experiments and manual evaluation, the weight coefficient of the specificity index is obtained by setting different weight values ​​for control experiments and experimental optimization, and the coverage of the key points of the question by the answer is calculated by evaluating the inclusion of the key information points of the question by the answer, that is, for each answer A, check whether it contains the key information points. The specificity and accuracy of the answer are determined through information density assessment, detail level assessment, accuracy assessment and relevance assessment.

[0065] It is understandable that the iterative generation and screening mechanism based on the answer pool continuously improves the quality of answers through multiple rounds of optimization while maintaining the diversity of answers, enabling the system to meet the multi-dimensional needs of complex problems while ensuring the accuracy of the answers.

[0066] The present invention dynamically adjusts the weight parameters in the multi-level similarity comparison module through the model self-feedback mechanism, realizes the adaptive optimization of the similarity calculation method, gradually improves the accuracy and precision of answer selection, improves the system's adaptability to different types of tasks, and enhances the robustness of the method.

[0067] In one embodiment of the present invention, step S5 specifically includes: Set an overall score threshold. When the overall score of the answers in the answer group is lower than the overall score threshold, the comprehensive scoring mechanism is triggered: Calculate the comprehensive similarity score of each answer group, and perform expert scoring on each answer group, and sort the answer groups from large to small according to the comprehensive similarity score and expert scoring to form an answer group sequence; The first group of answers in the answer group sequence is taken as an example, and the initial prompt template is reconstructed based on the example to obtain a reconstructed prompt template.

[0068] The main evaluation indicators of expert scoring include accuracy, relevance and coverage, which measure the degree of matching of the answers respectively. Accuracy is used to measure whether the answer meets the requirements of the question, relevance is used to measure the semantic relevance between the answer and the reference answer, and coverage is used to measure whether the answer covers key topics or information.

[0069] like Figure 3 As shown, the adaptive template reconstruction method of the present invention is described with a specific embodiment: S10, de-coding the obtained original bidding document webpage text to obtain a webpage text without code; S20, using regular expressions for preliminary entity recognition, using a large language model for secondary screening and fine screening through manual verification to form an adaptive template prompt small sample data set; S30, dividing the data set into a training set and a test set, using the training set as the fine-tuning corpus of the large language model, and using the test set to test the accuracy of the model to obtain a fine-tuned model; S40. Use the fine-tuned model to extract entity types and values ​​in the bidding text to complete the bidding text information extraction.

[0070] Among them, after the web page file is downloaded, in addition to the text file displayed on the page, it also contains a large amount of page code, page tags, escape symbols and other information, so it needs to be preprocessed to initially remove the content other than the bid text in the downloaded file to improve the information density after the preprocessing is completed. Therefore, the web page parsing library is used to parse the web page text without code, remove the HTML tags in the parsed file, obtain the web page text without tags, continue to process the punctuation marks of the target web page text, convert English punctuation marks into Chinese punctuation marks, delete the transfer symbols, convert line breaks into spaces, and add text marks at the beginning and end of the text.

[0071] There is a large amount of entity-irrelevant information in the tender text. First, regular expressions are used to perform an initial screening of the entity types and values ​​in the expression text. Due to the differences in the text formats of the tender texts, regular expressions cannot cover all types of tender texts. Therefore, a large language model is used to perform a second screening of the tender text. At this time, the entity information obtained is relatively credible. Finally, the tender text is manually screened to obtain a small sample data set of adaptive template prompts.

[0072] The present invention provides an adaptive template reconstruction system based on a small number of samples, which implements the adaptive template reconstruction method as described above, including: A sample analysis module is used to construct an initial prompt template based on a small amount of sample data, wherein the initial prompt template includes a task background description, context information, and target information prompts that need to be completed; A question generation module, used to input the initial prompt template into a large language model, and generate a question pool through specific instructions, agent instructions and example instructions; An answer generation module, used to answer each question in the question pool using the large language model to generate an answer pool; A similarity screening module, used to screen and cluster the answers in the answer pool through a multi-level similarity comparison module to obtain a screened answer pool and a screened answer group, and calculate the overall score of the answer based on the screened answer group; A template reconstruction module is used to calculate the comprehensive similarity score of each answer group based on the overall score of the screened answer group, sort the answer groups according to the comprehensive similarity score, select the answer group with the highest score as an example, optimize and reconstruct the initial prompt template, and generate a reconstructed prompt template; The iterative optimization module is used to input the reconstructed prompt template into the large language model for iterative updating and continuous optimization of the generated answers until the final answer is obtained.

[0073] The adaptive template reconstruction device provided by the present invention is described below. The adaptive template reconstruction device described below and the adaptive template reconstruction method described above can be referred to each other.

[0074] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor (processor) 410, a communication interface (Communications Interface) 420, a memory (memory) 430 and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call the logic instructions in the memory 430 to execute the adaptive template reconstruction method, which includes: constructing an initial prompt template based on a small amount of sample data, inputting the initial prompt template into a large language model, and generating a question pool through specific instructions, proxy instructions and example instructions; using the large language model to answer each question in the question pool to generate an answer pool; screening and clustering the answers in the answer pool through a multi-level similarity comparison module to obtain a screened answer pool and a screened answer group, and dynamically adjusting the weight parameters of the similarity calculation in the multi-level similarity comparison module through a model self-feedback mechanism to obtain an overall score of the screened answers; calculating the comprehensive similarity score of each answer group based on the overall score of the answers in each answer group, and performing expert scoring on each answer group, constructing an example based on the comprehensive similarity score and the expert score, and reconstructing the initial prompt template based on the example to obtain a reconstructed prompt template; inputting the reconstructed prompt template into the large language model for iterative updating until the final answer is obtained.

[0075] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0076] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the adaptive template reconstruction method provided by the above methods, which includes: constructing an initial prompt template based on a small amount of sample data, inputting the initial prompt template into a large language model, and generating a question pool through specific instructions, proxy instructions and example instructions; using the large language model to answer each question in the question pool to generate an answer pool; screening and clustering the answers in the answer pool through a multi-level similarity comparison module to obtain a screened answer pool and a screened answer group, and dynamically adjusting the weight parameters of the similarity calculation in the multi-level similarity comparison module through a model self-feedback mechanism to obtain an overall score of the screened answers; calculating the comprehensive similarity score of each answer group based on the overall score of the answers in each answer group, and performing expert scoring on each answer group, constructing an example based on the comprehensive similarity score and the expert score, and reconstructing the initial prompt template based on the example to obtain a reconstructed prompt template; inputting the reconstructed prompt template into the large language model for iterative updating until the final answer is obtained.

[0077] On the other hand, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it is implemented to execute the adaptive template reconstruction method provided by the above-mentioned methods, the method comprising: constructing an initial prompt template based on a small amount of sample data, inputting the initial prompt template into a large language model, and generating a question pool through specific instructions, proxy instructions and example instructions; using the large language model to answer each question in the question pool to generate an answer pool; screening and clustering the answers in the answer pool through a multi-level similarity comparison module to obtain a screened answer pool and a screened answer group, and dynamically adjusting the weight parameters of the similarity calculation in the multi-level similarity comparison module through a model self-feedback mechanism to obtain an overall score of the screened answers; calculating a comprehensive similarity score of each answer group based on the overall score of the answers in each answer group, and performing expert scoring on each answer group, constructing an example based on the comprehensive similarity score and the expert score, and reconstructing the initial prompt template based on the example to obtain a reconstructed prompt template; inputting the reconstructed prompt template into the large language model for iterative updating until the final answer is obtained.

[0078] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0079] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive template reconstruction method based on a small number of samples, characterized in that: The following steps are involved: S1. Construct an initial prompt template based on a small amount of sample data, wherein the initial prompt template includes a task background description, context information, and a part to be completed; S2, inputting the initial prompt template into a large language model, and generating a question pool through specific instructions, agent instructions and example instructions; S3. Using the large language model to answer each question in the question pool to generate an answer pool; S4. Screening and clustering the answers in the answer pool through a multi-level similarity comparison module to obtain a screened answer pool and a screened answer group, and dynamically adjusting the weight parameters of the similarity calculation in the multi-level similarity comparison module through a model self-feedback mechanism to obtain an overall score of the screened answers; S5. Calculate the comprehensive similarity score of each answer group based on the overall score of the answers in each answer group, and perform expert scoring on each answer group, construct examples based on the comprehensive similarity score and the expert score, and reconstruct the initial prompt template based on the example to obtain a reconstructed prompt template; S6. Input the reconstructed prompt template into the large language model for iterative updating until a final answer is obtained.

2. The method for adaptive template reconstruction based on a small number of samples according to claim 1, characterized in that: The specific instructions are instructions designed according to the task objectives and used to guide the large language model to generate answers related to the task. The proxy instructions are instructions that guide the large language model to generate answers in an indirect way. The example instructions are generated by providing example questions and answers.

3. The method for adaptive template reconstruction based on a small number of samples according to claim 2, characterized in that: The generation of the question pool specifically includes: The initial prompt template is input into the large language model for entity expansion to obtain an entity set. The initial prompt template is supplemented with keywords based on the entity set, and the weight of the keywords is adjusted. The weight update formula is: ； in, represents the weight value of the fth keyword in round t+1, represents the weight value of the fth keyword in the tth round, represents the adjustment parameter of the correlation evaluation, Indicates keywords The relevance evaluation value with the context Context, represents the fth keyword, Represents the current context. represents the tuning parameter for frequency evaluation, Indicates keywords frequency of occurrence in the text; The expanded keywords are divided into three categories: core keywords, secondary keywords, and background keywords, and are prioritized by differentiated initial weights to obtain a keyword sequence; the initial weights are: ； in, represents the initial weight value of the fth keyword, f represents the index of the keyword, represents the weight adjustment coefficient, represents the fth keyword, represents the priority scoring function; Generate multiple question variants by combining keyword sequences with specific instructions, agent instructions, and example instructions to form an initial question pool; The initial question pool is deduplicated and screened to obtain a screened question pool; the screened question pool is represented as: ； in, represents the filtered question pool, Indicates the current problem to be evaluated. Indicates except outside Other questions in the question pool, i represents the number of the current question, j represents the number of other questions except i, Q represents the initial question pool, Representation problem and The similarity measure between Represents the similarity threshold.

4. The method for adaptive template reconstruction based on a small number of samples according to claim 1, characterized in that: The processing steps of the multi-level similarity comparison module are: Performing primary similarity calculation on the answers in the answer pool to obtain a primary similarity score, and preliminarily screening the answer pool based on the primary similarity score to obtain a first answer pool; Performing semantic similarity calculation based on deep learning on the answers in the first answer pool to obtain a semantic similarity score, and screening based on the semantic similarity score to obtain a second answer pool; The answers in the second answer pool are clustered based on the semantic similarity scores to form multiple answer groups.

5. The method for adaptive template reconstruction based on a small number of samples according to claim 1, characterized in that: The model self-feedback mechanism includes: The primary similarity score and the semantic similarity score are weightedly combined to obtain an initial similarity score; The initial similarity score is dynamically adjusted based on coverage and specificity to obtain an overall similarity score.

6. The method for adaptive template reconstruction based on a small number of samples according to claim 1, characterized in that: Step S5 specifically includes: Set an overall score threshold. When the overall score of the answers in the answer group is lower than the overall score threshold, the comprehensive scoring mechanism is triggered: Calculate the comprehensive similarity score of each answer group, and perform expert scoring on each answer group, and sort the answer groups from large to small according to the comprehensive similarity score and expert scoring to form an answer group sequence; The first group of answers in the answer group sequence is taken as an example, and the initial prompt template is reconstructed based on the example to obtain a reconstructed prompt template.

7. The method for adaptive template reconstruction based on a small number of samples according to claim 1, characterized in that: The construction of the initial prompt template based on a small amount of sample data specifically includes: Preprocess a small amount of sample data, and add identifiers for clarifying the context scope at the beginning and end of the preprocessed small amount of sample data; Design the initial prompt template based on the task objectives and contextual information; The prompt engineering method combined with the large language model understands the context through prompt words, and dynamically adjusts the structure and vocabulary of the initial prompt template based on the context; wherein the prompt words include a clear description of the task goal and contextual prompts related to the task.

8. An adaptive template reconstruction system based on a small number of samples, characterized in that: Implementing the adaptive template reconstruction method according to any one of claims 1 to 7, comprising: A sample analysis module is used to construct an initial prompt template based on a small amount of sample data, wherein the initial prompt template includes a task background description, context information, and target information prompts that need to be completed; A question generation module, used to input the initial prompt template into a large language model, and generate a question pool through specific instructions, agent instructions and example instructions; An answer generation module, used to answer each question in the question pool using the large language model to generate an answer pool; A similarity screening module, used to screen and cluster the answers in the answer pool through a multi-level similarity comparison module to obtain a screened answer pool and a screened answer group, and calculate the overall score of the answer based on the screened answer group; A template reconstruction module is used to calculate the comprehensive similarity score and expert score of each answer group based on the overall score of the screened answer group, sort the answer groups according to the comprehensive similarity score and expert score, select the answer group with the highest score as an example, optimize and reconstruct the initial prompt template, and generate a reconstructed prompt template; The iterative optimization module is used to input the reconstructed prompt template into the large language model for iterative updating and continuous optimization of the generated answers until the final answer is obtained.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the adaptive template reconstruction method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the adaptive template reconstruction method according to any one of claims 1 to 7 is implemented.

Images