A large model tracing method and system based on question-answering data

Abstract

The present invention discloses a large model traceability method and system based on question and answer data, which relates to the field of large model traceability technology, including collecting question and answer data sets, performing data mapping through semantic embedding functions, and screening and forming a similarity set based on semantic similarity, filtering and forming a keyword set as trigger vocabulary based on the frequency of common words, setting watermark triggers based on question and answer pairs, screening the number of trigger question and answer pairs, distinguishing question and answer pair data carrying watermark triggers, calculating the generation probability of words in the context for the target language model, and adjusting and intervening in the trigger words containing triggers in the training data. The method of the present invention avoids the possibility that the distribution pattern of the trigger may be detected by malicious analysts by randomly selecting question and answer pairs to attach triggers and constructing a training data set, thereby further enhancing the concealment of the trigger.

Description

Technical Field

[0001] The present invention relates to the technical field of large model traceability, and in particular to a large model traceability method and system based on question and answer data. Background Art

[0002] With the rapid development of artificial intelligence technology, large-scale pre-trained language models have demonstrated outstanding performance in the field of natural language processing. These models are widely used in tasks such as text generation, question-answering systems, and semantic understanding. Their generation capabilities and language expression quality have been highly recognized by the industry. Large-scale model training based on question-answering data has further promoted the development of intelligent question-answering systems. Through semantic context understanding and generation optimization of multi-round dialogues under specific tasks, these models have achieved efficient automation of text tasks. However, as model capabilities increase, the creation of language models has potential problems with intellectual property protection and use authorization management. For example, unauthorized model copying, tampering, or illegal distribution will pose challenges to the commercial value of language models and the protection of research results. In addition, the generation behavior of language models is highly hidden and complex, which makes tracing the actual version of the model used and whether it complies with the authorization an important and urgent technical issue.

[0003] To address the above issues, current technologies mainly achieve partial traceability of data and models through large-scale recording or classic signature tracking methods. However, these methods are inadequate for authenticity traceability and low-invasive embedding verification in question-answer generation tasks in practical applications, especially in unsupervised or generative tasks. They cannot effectively characterize the characteristics of model generation behavior. In addition, since the generation strategy of the generative language model is probabilistic, there is a certain degree of uncertainty in its generated information, resulting in low traceability and verification accuracy of traditional methods when dealing with such scenarios. Summary of the Invention

[0004] The purpose of the present invention is to provide a large model tracing method and system based on question and answer data to solve the above problems.

[0005] The present invention is achieved through the following technical solutions:

[0006] A large model provenance method based on question-answering data, including:

[0007] Collect question-answer datasets, map the data using semantic embedding functions, filter and form a similarity set based on semantic similarity, count the frequencies of common words to filter and form a keyword set as trigger vocabulary, set watermark triggers based on question-answer pairs, filter the number of trigger question-answer pairs, distinguish question-answer pairs with watermark triggers, calculate the generation probability of words in context for the target language model, adjust and intervene on trigger words that contain triggers in the training data, and recalculate the generation probability of trigger words. After model training, output the target language model with the embedded trigger watermark.

[0008] Based on the cumulative probability of the target language model's word-by-word predictions when generating complete answers, the relevance score of the text question-answer pair is calculated. The retrieval relevance score is determined based on the semantic embedding vector obtained through the embedding function, and the fusion ranking score is determined. The optimal answer sequence is generated and graded according to the difficulty of the questions and answers. Combined with a course-based training method, the model is optimized step by step from low difficulty to high difficulty.

[0009] Generate dynamic triggers based on the previous trigger, fixed key, and hash function according to the trigger, and form a dynamic trigger sequence to verify the optimized target language model. Count the verification matching values ​​of the dynamic triggers and calculate the average generated matching value to analyze and optimize the matching degree of the target language.

[0010] Generate reports based on model verification and store them on the blockchain.

[0011] As a preferred solution of the large model tracing method based on question-answering data described in the present invention, wherein: for the target language model, the generation probability of words in the context is calculated, and the trigger words containing triggers in the training data are adjusted and intervened, including:

[0012] Collect question-answering datasets and organize them into binary question-answer pairs ,in represents the text content of the i-th question, Represents the text content of the i-th corresponding answer, and the dataset is composed of all binary question-answer pairs;

[0013] Use the pre-trained BERT model to build a semantic embedding function, and convert the question-answer pairs in the dataset into semantic embedding vectors through the embedding function. All semantic embedding vectors are stored as a mapping set.

[0014] The semantic similarity of the embedding vector of each question-answer pair is calculated using cosine similarity. Similarity screening is performed based on historical experience values. The retained similarity values ​​are combined into a similarity set and mapped back to the corresponding original text question-answer pair.

[0015] Determine a common vocabulary set based on historical experience vocabulary, and count the frequency of common vocabulary. Based on the sum of the mean and standard deviation of the vocabulary frequency as the frequency threshold, mark the vocabulary with a frequency below the frequency threshold as rare vocabulary, and combine them to form a keyword set as the trigger vocabulary;

[0016] Set watermark triggers based on question-answer pairs, and determine the number of trigger question-answer pairs based on the ratio of trigger question-answer pairs to total question-answer pairs and the total number of question-answer pairs in historical experience, and randomly select Question-answer pairs carry watermark triggers and constitute the target set;

[0017] Load question-answer pair data based on the target set, distinguish question-answer pairs with watermark triggers, and use question-answer pairs with and without triggers as training datasets.

[0018] For the target language model, the generation probability of a word in the context is calculated through softmax normalization. Based on the Logit perturbation mechanism algorithm, the trigger words in the training data that contain triggers are adjusted and intervened, and the generation probability of the trigger words is recalculated;

[0019] Select the cross-entropy loss function to calculate the computational loss between the category probability predicted by the target language model and the actual label based on the training dataset. Use the Adam optimizer for gradient descent optimization. Stop the iteration if the model loss no longer decreases significantly during the continuous iteration process, and output the target language model with the embedded trigger watermark.

[0020] As a preferred solution of the large model traceability method based on question and answer data described in the present invention, wherein: the determination of the fusion ranking score, the generation of the optimal answer sequence, and the grading of the question and answer difficulty, combined with the course-based training method, the optimization of the model from low difficulty to high difficulty, including:

[0021] Based on the target language model trained with embedded trigger watermarks, the relevance score of the text question-answer pair is calculated based on the cumulative probability of the model's word-by-word predictions when generating a complete answer;

[0022] Based on the semantic embedding vector obtained by the embedding function, the retrieval relevance score is determined by calculating the cosine similarity;

[0023] The generation relevance score calculated by the generation model and the retrieval relevance score calculated by the retrieval model are used to design a fusion ranking score;

[0024] Sort the answers to all question-answer pairs according to the final fusion relevance score to generate the optimal answer sequence;

[0025] Based on the optimal answer sequence, the complexity of each question-answer pair is calculated by retrieving the relevance score;

[0026] The question-answer pairs are evenly divided from simple to complex according to the generation complexity. The number of divisions is determined by rounding down the logarithm of the number of samples in the dataset. The target language model is optimized step by step from low difficulty to high difficulty according to the divided question-answer pairs of different difficulty levels. The question-answer pair data of different difficulty levels are used as the current optimization training set, and the cross-entropy loss function is selected for step-by-step optimization training. After completing the step-by-step optimization training of the question-answer pair with the highest difficulty, the iterative optimization is stopped to obtain the optimized target language model.

[0027] As a preferred solution of the large model tracing method based on question and answer data described in the present invention, wherein: the composition of the dynamic trigger sequence, the optimization of the target language model verification, the statistics of the verification matching values ​​of the dynamic triggers and the calculation of the average generated matching value include:

[0028] Construct a trigger sequence based on the question-answer pair carrying the watermark trigger, and make the sequence dynamic. Generate dynamic triggers based on the previous trigger, fixed key, and hash function, and form a dynamic trigger sequence.

[0029] Use dynamic trigger sequences to verify the optimized target language model. For each dynamic trigger, combine the triggering question template content to generate the corresponding verification question. Input the optimized target language model to obtain the generated corresponding answer.

[0030] Comparing the generation logic of dynamic trigger outputs to ensure that subsequent dynamic triggers in the dynamic trigger sequence can be predicted by the current dynamic trigger and the system encryption key;

[0031] At the same time, the Monte Carlo method is used to perform random trigger generation and verification operations on each dynamic trigger. The random dynamic trigger is determined based on historical experience data, and the matching degree of the generated watermark is calculated according to the verification results.

[0032] As a preferred solution of the large model tracing method based on question-answer data of the present invention, wherein: the analysis and optimization of the matching degree of the target language includes:

[0033] Calculate the average generated match value for all dynamic trigger sets;

[0034] The sum of the mean and standard deviation of historical matching values ​​is used as the matching threshold. If the average generated matching value is greater than or equal to the matching threshold, the model is judged to be the original watermark embedded version.

[0035] As a preferred solution of the large model traceability method based on question and answer data of the present invention, wherein: the report generation based on model verification includes:

[0036] Collect the verification match values ​​and average match values ​​of all triggers, use the report generation tool to generate a report for the target language model, and record the total number of verification triggers, verification times, and verification timestamp.

[0037] As a preferred solution of the large model traceability method based on question and answer data of the present invention, wherein: the blockchain storage includes

[0038] The dynamic triggers generated in each round of verification are stored in the blockchain. After the verification is completed, the trigger verification matching value and average matching value in the target language model's answer are recorded, and a report is generated for recording.

[0039] The present invention provides a large model traceability system based on question-answer data, comprising:

[0040] The data embedding module collects question-answer pair datasets, uses semantic embedding functions to perform similarity screening and filtering, and generates a set of keywords as trigger vocabulary;

[0041] The trigger module generates a trigger vocabulary set based on low-frequency keywords, generates question-answering data with watermark triggers, counts the number of trigger question-answer pairs, and distinguishes between question-answering data with and without triggers.

[0042] The model training module builds a context for the training dataset, calculates the probability of a word being generated in the context, and uses the Logit adjustment mechanism to train the target language model;

[0043] The retrieval fusion module uses the target language model to predict the generated complete answer word by word and calculate the generation relevance score. It then calculates the semantic retrieval relevance score of the question-answer pair using the semantic embedding vector. Based on the generation relevance score and the retrieval relevance score, it calculates the fusion ranking score by weight, generates the sorted optimal answer sequence, and generates different difficulty levels for the question-answer pair.

[0044] The multi-level optimization module optimizes the target language model step by step from low to high levels based on the hierarchical question-answering data output by the generation and retrieval modules.

[0045] The dynamic trigger generation module uses a hash chain mechanism to generate a dynamic trigger sequence. It then generates verification questions by combining the trigger question template with the dynamic trigger to verify the logical continuity of the trigger. It uses the Monte Carlo method to randomly select triggers for verification, statistically verify the matching values, and calculate the average matching value.

[0046] The report generation module collects the trigger matching value and average matching value in each round of verification and generates a detailed verification report.

[0047] The blockchain storage module archives the dynamic trigger sequence, verification matching values ​​and generated reports into the blockchain.

[0048] The present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the large model tracing method based on question and answer data as described in the present invention is implemented.

[0049] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the large model tracing method based on question and answer data as described in the present invention is implemented.

[0050] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0051] 1. By randomly selecting question-answer pairs to attach triggers and constructing a training dataset, this method prevents malicious analysts from detecting the distribution patterns of triggers, further enhancing the concealment of triggers. During the attachment process, only the trigger vocabulary is appended to the end of the answer, without disrupting the semantics and logic of the question and answer, ensuring that the model retains its original performance in normal generation scenarios. The recalculation of generation probabilities based on contextual conditions enables triggers to adapt to arbitrarily complex contextual conditions, further enhancing the triggering ability of the model in real question-answering scenarios. At the same time, Logit interference has a limited impact on the distribution of other non-trigger vocabulary and will not disrupt the normal question-answering behavior of the model.

[0052] 2. This invention verifies the continuity of the trigger sequence, indirectly checking whether it has the correct watermarking behavior directly from the model's generation logic. The good continuity of the verification sequence makes the verification process highly reliable. The Monte Carlo method randomly selects a subset of the trigger sequence for verification, which enhances the randomness and flexibility of the verification process. Compared with linear verification, this randomization method improves verification efficiency and reduces the risk of specific triggers being predicted and exploited.

[0053] 3. The present invention averages the matching values ​​of all dynamic trigger sets. The average matching value directly reflects whether the model's response behavior to the trigger in dynamic verification is stable. The introduction of the average value bridges the local noise and interference that may exist in dynamic verification, ensuring the stability and reliability of the judgment mechanism. When combined with dynamic triggers, logic verification and Monte Carlo verification, the complete matching degree and identity judgment framework improves the credibility of the watermark behavior of the verification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0055] Figure 1 This is a flowchart of the large model traceability method based on question-answering data;

[0056] Figure 2 This is a structural diagram of the large model traceability system based on question and answer data. DETAILED DESCRIPTION

[0057] To make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the examples and accompanying drawings. The exemplary embodiments of the present invention and their descriptions are only used to explain the present invention and are not intended to limit the present invention. It should be noted that the present invention is already in the actual development and use stage.

[0058] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a large model tracing method based on question-answer data, including the following steps:

[0059] S1: Collect question-answer datasets, map data using semantic embedding functions, filter and form a similarity set based on semantic similarity, count the frequencies of common words to filter and form a keyword set as trigger vocabulary, set watermark triggers based on question-answer pairs, filter the number of trigger question-answer pairs, distinguish question-answer pairs with watermark triggers, calculate the generation probability of words in the context for the target language model, adjust and intervene on trigger words that contain triggers in the training data, and recalculate the generation probability of trigger words. After model training, output the target language model with embedded trigger watermark training.

[0060] Preferably, for the target language model, the probability of a word being generated in the context is calculated, and the trigger words in the training data containing the trigger are adjusted and intervened, including:

[0061] Collect question-answering datasets and organize them into binary question-answer pairs ,in represents the text content of the i-th question, Represents the text content of the i-th corresponding answer, and the dataset is composed of all binary question-answer pairs;

[0062] Use the pre-trained BERT model to build a semantic embedding function, and convert the question-answer pairs in the dataset into semantic embedding vectors through the embedding function. All semantic embedding vectors are stored as a mapping set.

[0063] The semantic similarity of the embedding vector of each question-answer pair is calculated using cosine similarity. Similarity screening is performed based on historical experience values. The retained similarity values ​​are combined into a similarity set and mapped back to the corresponding original text question-answer pair.

[0064] Determine a common vocabulary set based on historical experience vocabulary, and count the frequency of common vocabulary. Based on the sum of the mean and standard deviation of the vocabulary frequency as the frequency threshold, mark the vocabulary with a frequency below the frequency threshold as rare vocabulary, and combine them to form a keyword set as the trigger vocabulary;

[0065] Set watermark triggers based on question-answer pairs, and determine the number of trigger question-answer pairs based on the ratio of trigger question-answer pairs to total question-answer pairs and the total number of question-answer pairs in historical experience, and randomly select Question-answer pairs carry watermark triggers (the trigger vocabulary is appended or inserted into the target question-answer pair answer, that is, the trigger vocabulary is attached to the end of the original answer to ensure that the logic of the original semantic answer is not destroyed), forming a target set, where the number of trigger question-answer pairs is expressed as:

[0066] ;

[0067] in, represents the number of trigger question-answer pairs, represents the ratio of trigger question-answer pairs to the total question-answer pairs, and n represents the total number of question-answer pairs;

[0068] Load question-answer pair data based on the target set, distinguish question-answer pairs with watermark triggers, and use question-answer pairs with and without triggers as training datasets.

[0069] For the target language model, the generation probability of a word in the context is calculated through softmax normalization. Based on the Logit perturbation mechanism algorithm, the trigger words containing triggers in the training data are adjusted and intervened, and the generation probability of the trigger words is recalculated, which is expressed as:

[0070] ;

[0071]

[0072] ;

[0073] in, Indicates that the target language model generates all the vocabulary sequences generated before the current position t as context conditions, usually the question and the partial answer sequence generated previously, which consists of the previously generated words and the question-answer pair. Indicates that at present The trigger word The conditional probability of Indicates the current The trigger word Logit score of the trigger word In context Priority rating, represents all potential words generated by the language model, Indicates candidate words, Represents the Logit score given for each candidate word, indicating the candidate word in the current context Priority, Indicates the trigger word, Represents the enhancement parameter, which is used to add a fixed positive offset to the original Logit score, determined based on historical experience. represents the adjusted Logit score, represents the adjusted conditional generation probability;

[0074] Select the cross-entropy loss function to calculate the computational loss between the category probability predicted by the target language model and the actual label based on the training dataset. Use the Adam optimizer for gradient descent optimization. Stop the iteration if the model loss no longer decreases significantly during the continuous iteration process, and output the target language model with the embedded trigger watermark.

[0075] By constructing a structured question-answer dataset and question-answer embedding mapping, a unified, high-dimensional dense semantic representation foundation is provided, making subsequent semantic similarity-based screening and trigger design more context-sensitive, thereby improving the effectiveness of model watermark embedding. Through similarity screening and mapping back to the original answer, highly semantically relevant question-answer pairs are screened out, greatly reducing the interference that low-quality or noisy question-answer pairs may cause to model training, ensuring that the model focuses on highly relevant data, and based on the mean and standard deviation thresholds of common vocabulary frequencies, rare and low-frequency words are identified and combined as trigger vocabulary keywords, which can minimize the probability of repetition between trigger words and natural language, thereby ensuring that these triggers are not generated by the model due to high-frequency co-occurrence. The low-frequency nature of triggers ensures that they are unique during verification, making them highly recognizable in output generation. This enhances the concealment and specificity of triggers as watermarks. The threshold value determined based on word frequency provides an objective basis for trigger screening, avoiding the instability or lack of universality of trigger behavior caused by subjective trigger design. Triggers are randomly added to question-answer pairs and used to construct training datasets, preventing malicious analysts from detecting the distribution of triggers and further enhancing the concealment of triggers. During the addition process, trigger words are simply appended to the end of the answer, without disrupting the semantics and logic of the question and answer, ensuring that the model retains its original performance (i.e., the non-intrusiveness of the watermark) in normal generation scenarios.

[0076] The trigger generation priority is adjusted through the Logit perturbation mechanism. The Logit modification directly inserts a control signal from the model generation mechanism, so that the trigger vocabulary has a significant priority in the model generation distribution, thereby improving the reliability of the trigger response. By adjusting the proportion of the trigger vocabulary priority in the probability distribution, the trigger behavior is ensured to be obvious and the regulation effect is controllable. The recalculation of the generation probability based on contextual conditions can make the trigger adapt to arbitrarily complex contextual conditions, further enhancing the triggering ability of the model in real question-and-answer scenarios. At the same time, the Logit interference has a limited impact on the distribution of other non-trigger words and will not disrupt the normal question-and-answer behavior of the model.

[0077] S2 calculates the relevance score of the text question-answer pair based on the cumulative probability of the target language model's word-by-word predictions when generating a complete answer. It then determines the retrieval relevance score based on the semantic embedding vector obtained through the embedding function, and determines the fusion ranking score. It then generates the optimal answer sequence and classifies the questions and answers according to their difficulty. Combined with a course-based training method, it optimizes the model step by step from low to high difficulty.

[0078] Preferably, determine the fusion ranking score, generate the optimal answer sequence, and grade the questions and answers according to their difficulty level, combine the course-based training method, and optimize the model step by step from low difficulty to high difficulty, including:

[0079] Based on the target language model trained with embedded trigger watermarks, the relevance score of the text question-answer pair is calculated according to the cumulative probability of the model's word-by-word predictions when generating a complete answer, which is expressed as:

[0080] ;

[0081] in represents the relevance score of the question and answer pair, T represents the total number of words in the answer, Indicates that the target language model is and context In the case of , predict the current generated word probability;

[0082] Based on the semantic embedding vector obtained by the embedding function, the retrieval relevance score is determined by calculating the cosine similarity, which is expressed as:

[0083] ;

[0084] in represents the retrieval relevance score, Embedding vector representing the problem and the embedding vector of the answer The cosine similarity of

[0085] The generation relevance score calculated by the generation model and the retrieval relevance score calculated by the retrieval model are combined to form a fusion ranking score, which is expressed as:

[0086] ;

[0087] in represents the final fusion correlation score, represents the fusion weight, which is determined through experiments;

[0088] Sort the answers to all question-answer pairs according to the final fusion relevance score to generate the optimal answer sequence;

[0089] According to the optimal answer sequence, the complexity of each question-answer pair is calculated by retrieving the relevance score, which is expressed as:

[0090] ;

[0091] in Indicates that the question and answer are correct Data complexity value;

[0092] The question-answer pairs are evenly divided from simple to complex according to the generation complexity. The number of divisions is determined by rounding down the logarithm of the number of samples in the dataset. The target language model is optimized step by step from low difficulty to high difficulty according to the divided question-answer pairs of different difficulty levels. The question-answer pair data of different difficulty levels are used as the current optimization training set, and the cross-entropy loss function is selected for step-by-step optimization training. After completing the step-by-step optimization training of the question-answer pair with the highest difficulty, the iterative optimization is stopped to obtain the optimized target language model.

[0093] By calculating the generation relevance score and the retrieval relevance score, the cumulative word-by-word generation probability can be used to quantify the conformity of the answer generated by the target language model. The higher the generation probability, the stronger the generation logic and fluency of the answer under the current context of the model. The generation relevance score is used to evaluate the question-answer pair, which can directly reflect the performance of the model's generation ability in the actual context. The generation relevance score provides a key probabilistic basis for subsequent fusion sorting, so that the sorting does not only rely on the retrieval task, but further fits the generation nature of the language model. The cosine similarity calculated based on the semantic embedding vector can effectively map the semantic distance between the question and the answer, obtain the degree of matching between the two in the semantic space, and retrieve the relevance. This strategy complements the semantic associations that might be missed in pure generation tasks, especially when the answer generation logic is not entirely dependent on the context. This retrieval strategy can serve as a cross-validation. The combination of the two captures the accuracy of model generation through generation relevance, while using the global relevance of semantic retrieval to evaluate model performance, enabling a more comprehensive assessment of answer quality. Fusion ranking establishes a balance between generation relevance and retrieval relevance, and the introduction of reasonable weights ensures that there is no need to over-rely on the score of a single dimension. The fusion ranking result retains the advantage of answer generation probability while introducing a safeguarding trade-off of the semantic dimension, avoiding the situation where answers with poor semantic relevance are incorrectly prioritized due to generation behavior completely dominating the ranking.

[0094] Complex values ​​are generated based on the retrieval relevance score calculation. Combined with the previous fusion ranking score, the calculation of the complex value not only takes into account semantic associations, but also ensures the objectivity of difficulty judgment through the ranking results. The question-answer pairs are divided and trained in a course-style manner through complex values, which can ensure the dynamic nature of the division and adapt to the different sizes of data sets. It avoids the uneven distribution of training samples that may be caused by traditional fixed partitioning methods, achieves effective learning at a lower training cost, and avoids the training dilemma that the model may fall into when directly facing mixed random training samples. It ensures that the optimized model can be further detected and the watermark characteristics can be verified, and then transferred to the subsequent verification and traceability stages.

[0095] S3: Generate dynamic triggers based on the previous trigger, fixed key, and hash function according to the trigger, and form a dynamic trigger sequence to verify the optimized target language model. Count the verification matching values ​​of the dynamic triggers and calculate the average generated matching value to analyze and optimize the matching degree of the target language.

[0096] Preferably, a dynamic trigger sequence is formed, and verification of the optimized target language model is performed, and verification matching values ​​of the dynamic triggers are counted and averaged to generate matching values ​​are calculated, including:

[0097] A trigger sequence is constructed based on the question-answer pair carrying the watermark trigger, and the sequence is made dynamic. Based on the trigger, a dynamic trigger is generated based on the previous trigger, the fixed key, and the hash function, and a dynamic trigger sequence is formed, which is expressed as:

[0098] ;

[0099] in represents the i-th dynamic trigger, represents the one-way irreversible hash function SHA-256, and K represents the system encryption key;

[0100] Use dynamic trigger sequences to verify the optimized target language model. For each dynamic trigger, combine the triggering question template content to generate the corresponding verification question. Input the optimized target language model to obtain the generated corresponding answer.

[0101] Compare the generation logic of the dynamic trigger output to ensure that the subsequent dynamic triggers in the dynamic trigger sequence can be predicted by the current dynamic trigger and the system encryption key, which is expressed as:

[0102] ;

[0103] in represents the problem of optimizing the output answer of the target language model, represents the i+1th dynamic trigger;

[0104] At the same time, the Monte Carlo method is used to generate and verify randomized triggers for each dynamic trigger. The random dynamic trigger is determined based on historical experience data. According to the verification results, the matching degree of the generated watermark is calculated, which is expressed as:

[0105] ;

[0106] in Indicates verification matching value, Indicates the number of successful matches based on the sth dynamic trigger keyword in the generated answer, Indicates the answer text of the s-th trigger question, Indicates the total number of all target keywords in the trigger.

[0107] Through the generation and use of dynamic trigger sequences, dynamic triggers are generated through a hash chain mechanism (based on previous triggers and system keys), ensuring that the triggers are unique and unpredictable during verification. This dynamic process avoids the risk of fixed triggers being exposed or attacked during the verification phase, making trigger generation highly secure. External attackers cannot reverse calculate and restore the trigger sequence or its generation rules, fundamentally improving the security and tamper-proof capabilities of the watermark verification mechanism.

[0108] By verifying the continuity of the trigger sequence, the correct watermarking behavior is indirectly verified directly from the model's generation logic. The good continuity of the verification sequence makes the verification process highly reliable, especially in complex verification scenarios, providing a secure dynamic channel for model verification. When dynamic triggers are combined with question templates to form verification questions, they can be used to cover various verification scenarios. The dynamic trigger part ensures that each verification question is different. This approach increases the randomness and diversity of verification questions on the basis of a unified verification framework, reducing the possibility of verification questions being predicted or reused. The splicing design allows dynamic triggers to naturally integrate into the context of verification questions. When the model generates answers, they match the standardized question-answer pairs used in training, improving the semantic rationality of trigger verification and avoiding the generation of illogical answers.

[0109] By randomly selecting a subset of the trigger sequence for verification through the Monte Carlo method, the randomness and flexibility of the verification process are enhanced. Compared with linear verification (verifying all triggers one by one), this randomization method improves verification efficiency while reducing the risk of specific triggers being predicted and exploited. By combining matching degree calculation with a standardized verification process, it is possible to comprehensively quantify the true performance of the watermark embedding behavior in the model answer and avoid misjudgment due to the failure of a single trigger.

[0110] Further, analyze and optimize the matching degree of the target language, including:

[0111] The average generation matching value is calculated for all dynamic trigger sets and is expressed as:

[0112] ;

[0113] in represents the average matching value, S represents the total number of dynamic triggers, Indicates the verification matching value of the sth dynamic trigger;

[0114] The sum of the mean and standard deviation of historical matching values ​​is used as the matching threshold. If the average generated matching value is greater than or equal to the matching threshold, the model is judged to be the original watermark embedded version.

[0115] By averaging the matching values ​​of all dynamic trigger sets, the average matching value directly reflects whether the model's response behavior to triggers in dynamic verification is stable. The introduction of the average value bridges the gap between local noise and interference that may exist in dynamic verification, ensuring the stability and reliability of the judgment mechanism. When combined with dynamic triggers, logic verification, and Monte Carlo verification, the complete matching degree and identity judgment framework enhances the credibility of the watermark behavior of the verification model.

[0116] S4, generates reports based on model verification and stores them on the blockchain;

[0117] Preferably, a report is generated based on the model validation, including:

[0118] Collect the verification match values ​​and average match values ​​of all triggers, use the report generation tool to generate a report for the target language model, and record the total number of verification triggers, verification times, and verification timestamp.

[0119] By collecting the matching values ​​and average matching values ​​of all triggers and formatting them into reports, we can intuitively reflect the overall performance of the model in trigger watermark verification, making it easier to observe the behavioral consistency of the model in multiple verification scenarios. By recording the total number of trigger verifications, the number of verifications, and the timestamp, we can establish time series information for each round of verification, facilitating the subsequent analysis of stability and potential anomalies in the verification process.

[0120] Furthermore, blockchain storage is performed, including

[0121] The dynamic triggers generated in each round of verification are stored in the blockchain. After the verification is completed, the trigger verification matching value and average matching value in the target language model's answer are recorded, and a report is generated for recording.

[0122] By recording dynamic triggers on the blockchain, the triggers generated in each verification round become independent blockchain transactions, ensuring a high degree of trust in the verification input process. The distributed storage of dynamic triggers on the blockchain makes the verification input fully public. This transparency enhances the fairness and authority of the verification mechanism, playing a particularly critical role in multi-party joint verification or dispute resolution scenarios. The dynamic trigger record corresponding to each verification is bound to the verification timestamp, allowing for the retrieval of the trigger input, the model response generated during verification, and the model's performance in the specific verification task.

[0123] This embodiment also provides a large model traceability system based on question and answer data, including:

[0124] The data embedding module collects question-answer pair datasets, uses semantic embedding functions to perform similarity screening and filtering, and generates a set of keywords as trigger vocabulary;

[0125] The trigger module generates a trigger vocabulary set based on low-frequency keywords, generates question-answering data with watermark triggers, counts the number of trigger question-answer pairs, and distinguishes between question-answering data with and without triggers.

[0126] The model training module builds a context for the training dataset, calculates the probability of a word being generated in the context, and uses the Logit adjustment mechanism to train the target language model;

[0127] The retrieval fusion module uses the target language model to predict the generated complete answer word by word and calculate the generation relevance score. It then calculates the semantic retrieval relevance score of the question-answer pair using the semantic embedding vector. Based on the generation relevance score and the retrieval relevance score, it calculates the fusion ranking score by weight, generates the sorted optimal answer sequence, and generates different difficulty levels for the question-answer pair.

[0128] The multi-level optimization module optimizes the target language model step by step from low to high levels based on the hierarchical question-answering data output by the generation and retrieval modules.

[0129] The dynamic trigger generation module uses a hash chain mechanism to generate a dynamic trigger sequence. It then generates verification questions by combining the trigger question template with the dynamic trigger to verify the logical continuity of the trigger. It uses the Monte Carlo method to randomly select triggers for verification, statistically verify the matching values, and calculate the average matching value.

[0130] The report generation module collects the trigger matching value and average matching value in each round of verification and generates a detailed verification report.

[0131] The blockchain storage module archives the dynamic trigger sequence, verification matching values ​​and generated reports into the blockchain.

[0132] This embodiment also provides a computer device, which is suitable for the large model tracing method based on question and answer data, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the large model tracing method based on question and answer data proposed in the above embodiment.

[0133] The computer device may be a terminal, comprising a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage media. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. Wireless communication may be achieved via Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. The display of the computer device may be a liquid crystal display or an electronic ink display. The input device may be a touchscreen overlay on the display, buttons, a trackball, or a touchpad on the computer device housing, or an external keyboard, touchpad, or mouse.

[0134] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the large-model traceability method based on question-and-answer data proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk.

[0135] In summary, the present invention avoids the possibility that the distribution pattern of the trigger may be detected by malicious analysts by randomly selecting question-answer pairs and constructing a training data set, further enhancing the concealment of the trigger. During the attachment process, only the trigger vocabulary is added to the end of the answer, without damaging the semantics and logic of the question and answer, ensuring that the model retains its original performance in normal generation scenarios. The recalculation of the generation probability based on contextual conditions can make the trigger adapt to any complex contextual conditions, further enhancing the triggering ability of the model in real question-answering scenarios. At the same time, the Logit interference has a limited impact on the distribution of other non-trigger vocabulary and will not destroy the normal question-answering behavior of the model. By verifying the continuity of the trigger sequence, it is indirectly verified whether it has the correct watermark behavior directly from the generation logic of the model. The good continuity of the verification sequence makes the verification process highly reliable. The Monte Carlo method is used to randomly select a subset of the trigger sequence for verification, which enhances the randomness and flexibility of the verification process. Compared with linear verification, this randomization method improves verification efficiency while reducing the risk of specific triggers being predicted and exploited. By averaging the matching values ​​of all dynamic trigger sets, the average matching value directly reflects whether the model's response behavior to triggers in dynamic verification is stable. The introduction of the average value bridges the local noise and interference that may exist in dynamic verification, ensuring the stability and reliability of the judgment mechanism. When combined with dynamic triggers, logic verification and Monte Carlo verification, the complete matching degree and identity judgment framework improves the credibility of the watermark behavior of the verification model.

[0136] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A large model provenance tracing method based on question-answer data, comprising: Collect question-answer datasets, map the data using semantic embedding functions, filter and form a similarity set based on semantic similarity, count the frequencies of common words to filter and form a keyword set as trigger words, set watermark triggers based on question-answer pairs, insert the trigger words into the target question-answer pairs, append the trigger words to the end of the question-answer pairs, filter the number of trigger question-answer pairs, distinguish question-answer pairs with watermark triggers, calculate the generation probability of words in the context for the target language model, adjust and intervene on the trigger words that contain triggers in the training data, and recalculate the generation probability of the trigger words. After model training, output the target language model with the embedded trigger watermark. Based on the cumulative probability of the target language model's word-by-word predictions when generating complete answers, the relevance score of the text question-answer pair is calculated. The retrieval relevance score is determined based on the semantic embedding vector obtained through the embedding function, and the fusion ranking score is determined. The optimal answer sequence is generated and graded according to the difficulty of the questions and answers. Combined with a course-based training method, the model is optimized step by step from low difficulty to high difficulty. Generate dynamic triggers based on the preceding trigger, fixed key, and hash function according to the trigger, and form a dynamic trigger sequence to optimize the target language model verification, count the verification matching values ​​of the dynamic triggers and calculate the average generated matching value, including, Construct a trigger sequence based on the question-answer pair carrying the watermark trigger, and make the sequence dynamic. Generate dynamic triggers based on the previous trigger, fixed key, and hash function, and form a dynamic trigger sequence. Use dynamic trigger sequences to verify the optimized target language model. For each dynamic trigger, combine the triggering question template content to generate the corresponding verification question. Input the optimized target language model to obtain the generated corresponding answer. Comparing the generation logic of the dynamic trigger output to ensure that subsequent dynamic triggers in the dynamic trigger sequence are predicted by the current dynamic trigger and the system encryption key; At the same time, the Monte Carlo method is used to generate and verify randomized triggers for each dynamic trigger. The random dynamic trigger is determined based on historical experience data. According to the verification results, the matching degree of the generated watermark is calculated and the matching degree of the target language is analyzed and optimized. Generate reports based on model verification and store them on the blockchain.

2. The large-scale model traceability method based on question-answer data according to claim 1 is characterized by: The target language model calculates the probability of a word being generated in the context, and adjusts and intervenes on the trigger words in the training data that contain triggers, including: Collect question-answering datasets and organize them into binary question-answer pairs ,in represents the text content of the i-th question, Represents the text content of the i-th corresponding answer, and the dataset is composed of all binary question-answer pairs; Use the pre-trained BERT model to build a semantic embedding function, and convert the question-answer pairs in the dataset into semantic embedding vectors through the embedding function. All semantic embedding vectors are stored as a mapping set. The semantic similarity of the embedding vector of each question-answer pair is calculated using cosine similarity. Similarity screening is performed based on historical experience values. The retained similarity values ​​are combined into a similarity set and mapped back to the corresponding original text question-answer pair. Determine a common vocabulary set based on historical experience vocabulary, and count the frequency of common vocabulary. Based on the sum of the mean and standard deviation of the vocabulary frequency as the frequency threshold, mark the vocabulary with a frequency below the frequency threshold as rare vocabulary, and combine them to form a keyword set as the trigger vocabulary; Set watermark triggers based on question-answer pairs, and determine the number of trigger question-answer pairs based on the ratio of trigger question-answer pairs to total question-answer pairs and the total number of question-answer pairs in historical experience, and randomly select Question-answer pairs carry watermark triggers and constitute the target set; Load question-answer pair data based on the target set, distinguish question-answer pairs with watermark triggers, and use question-answer pairs with and without triggers as training datasets. For the target language model, the generation probability of a word in the context is calculated through softmax normalization. Based on the Logit perturbation mechanism algorithm, the trigger words in the training data that contain triggers are adjusted and intervened, and the generation probability of the trigger words is recalculated; Select the cross-entropy loss function to calculate the computational loss between the category probability predicted by the target language model and the actual label based on the training dataset. Use the Adam optimizer for gradient descent optimization. Stop the iteration if the model loss no longer decreases significantly during the continuous iteration process, and output the target language model with the embedded trigger watermark.

3. The large-scale model traceability method based on question-answer data according to claim 2 is characterized by: The determination of the fusion ranking score, the generation of the optimal answer sequence, and the grading of the difficulty of the questions and answers, combined with the course-based training method, gradually optimize the model from low difficulty to high difficulty, including: Based on the target language model trained with embedded trigger watermarks, the relevance score of the text question-answer pair is calculated based on the cumulative probability of the model's word-by-word predictions when generating a complete answer; Based on the semantic embedding vector obtained by the embedding function, the retrieval relevance score is determined by calculating the cosine similarity; The generation relevance score calculated by the generation model and the retrieval relevance score calculated by the retrieval model are used to design a fusion ranking score; Sort the answers to all question-answer pairs according to the final fusion relevance score to generate the optimal answer sequence; Based on the optimal answer sequence, the complexity of each question-answer pair is calculated by retrieving the relevance score; The question-answer pairs are evenly divided from simple to complex according to the generation complexity. The number of divisions is determined by rounding down the logarithm of the number of samples in the dataset. The target language model is optimized step by step from low difficulty to high difficulty according to the divided question-answer pairs of different difficulty levels. The question-answer pair data of different difficulty levels are used as the current optimization training set, and the cross-entropy loss function is selected for step-by-step optimization training. After completing the step-by-step optimization training of the question-answer pair with the highest difficulty, the iterative optimization is stopped to obtain the optimized target language model.

4. The large-scale model traceability method based on question-answer data according to claim 3 is characterized by: The analysis optimizes the target language match, including, Calculate the average generated match value for all dynamic trigger sets; The sum of the mean and standard deviation of historical matching values ​​is used as the matching threshold. If the average generated matching value is greater than or equal to the matching threshold, the model is judged to be the original watermark embedded version.

5. The large-scale model traceability method based on question-answer data according to claim 4 is characterized by: The report generated according to the model verification includes: Collect the verification match values ​​and average match values ​​of all triggers, use the report generation tool to generate a report for the target language model, and record the total number of verification triggers, verification times, and verification timestamp.

6. The large-scale model traceability method based on question-answer data according to claim 5 is characterized by: The blockchain storage includes The dynamic triggers generated in each round of verification are stored in the blockchain. After the verification is completed, the trigger verification matching value and average matching value in the target language model's answer are recorded, and a report is generated for recording.

7. A large model traceability system based on question and answer data, based on the large model traceability method based on question and answer data according to any one of claims 1 to 6, characterized in that: include, The data embedding module collects question-answer pair datasets, uses semantic embedding functions to perform similarity screening and filtering, and generates a set of keywords as trigger vocabulary; The trigger module generates a trigger vocabulary set based on low-frequency keywords, generates question-answering data with watermark triggers, counts the number of trigger question-answer pairs, and distinguishes between question-answering data with and without triggers. The model training module builds a context for the training dataset, calculates the probability of a word being generated in the context, and uses the Logit adjustment mechanism to train the target language model; The retrieval fusion module uses the target language model to predict the generated complete answer word by word and calculate the generation relevance score. It then calculates the semantic retrieval relevance score of the question-answer pair using the semantic embedding vector. Based on the generation relevance score and the retrieval relevance score, it calculates the fusion ranking score by weight, generates the sorted optimal answer sequence, and generates different difficulty levels for the question-answer pair. The multi-level optimization module optimizes the target language model step by step from low to high levels based on the hierarchical question-answering data output by the generation and retrieval modules. The dynamic trigger generation module uses a hash chain mechanism to generate a dynamic trigger sequence. It then generates verification questions by combining the trigger question template with the dynamic trigger to verify the logical continuity of the trigger. It uses the Monte Carlo method to randomly select triggers for verification, statistically verify the matching values, and calculate the average matching value. The report generation module collects the trigger matching value and average matching value in each round of verification and generates a detailed verification report. The blockchain storage module archives the dynamic trigger sequence, verification matching values ​​and generated reports into the blockchain.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the large model tracing method based on question and answer data described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the large model tracing method based on question and answer data described in any one of claims 1 to 6 are implemented.

Images