Double-signal-driven large model safety evaluation dataset self-evolution method and system

Abstract

The present application relates to the technical field of artificial intelligence security evaluation, in particular to a double-signal-driven large model security evaluation dataset self-evolution method and system. It comprises: constructing a candidate seed set based on multiple external threat sources; preprocessing each seed in the set and constructing a candidate sample set; calculating threat semantic novelty and defense saturation; based on the defense saturation value and the threat semantic novelty value, comprehensively scoring the candidate samples; on the basis of comprehensive scoring, combining hard constraints for integer programming, obtaining the inclusion sample set of the new dataset; using a hybrid trigger mechanism combining timing trigger and event trigger for real-time updating, and maintaining an anchor sample set for version management. The present application proposes a multi-modal, multi-threat type applicable, endogenous diagnostic signal driven large model security evaluation dataset self-evolution method, which enables the security evaluation dataset to have the ability of continuous self-updating to adapt to the rapid iteration of large models.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a dual-signal driven method and system for the self-evolution of a large model safety evaluation dataset. Background Technology

[0002] In recent years, the capabilities of large language models and multimodal large models have been continuously expanding, and their application in various fields of society has brought about various types of security risks, such as jailbreak attacks, generation of harmful content, value bias, and privacy leaks. To assess and regulate these risks, the industry has successively built a variety of large model security evaluation datasets. However, the rapid iteration of large models and the continuous evolution of threat forms pose a severe challenge to the timeliness of security evaluation datasets.

[0003] On the one hand, jailbreaking commands and malicious content-luring templates are rapidly spreading in open-source communities and the black market, multiplying in a herd-like manner. The same core bypass strategy can generate hundreds of text variants, mixed text and image variants, or multilingual variants in a short period of time. On the other hand, the security alignment capabilities of large models are also constantly improving with model version upgrades. The combination of these two factors causes security evaluation datasets to begin to lose their effectiveness as soon as they are released—within months or even weeks, most of the models being tested can achieve near-saturation pass rates on older datasets. The datasets lose their ability to effectively distinguish between different models, thus failing to truly reflect the latest security risks.

[0004] The closest existing technology to this invention is the dynamic evolution method oriented towards a general evaluation benchmark. Existing technologies mainly include: (1) version difference driven update. The representative technology is DynaQuest. This method identifies real-world fact updates by comparing the attribute differences of the Wikipedia infobox between two points in time, generates new question and answer samples accordingly, and adopts a bi-weekly maintenance cycle. Its disadvantage is that it heavily relies on version comparison of structured knowledge sources, can only identify "fact replacement" type changes, and cannot handle the threat forms that spread in the security field in the form of "attack group reproduction".

[0005] (2) External knowledge interface self-refresh. The representative technology is PAT-Questions. This method binds the question template to the SPARQL query and automatically refreshes the standard answer by re-executing the query when the underlying knowledge graph is updated. Its disadvantage is that it essentially relies on "the question having a definite structured truth value", while in the security evaluation of large models, values, context and compliance judgments generally do not have a unique objective answer, so this type of method fails.

[0006] (3) Live benchmark periodic updates. Representative technologies include LiveBench and LiveCodeBench. This type of method continuously collects candidates from competition platforms, academic papers, and news sources on a monthly basis, partially replacing old samples (e.g., LiveBench replaces about 1 / 6 of the samples every month), and relies on an objective scoring mechanism to ensure the reliability of the evaluation. Its disadvantages are: on the one hand, it requires the samples to be automatically scored by "objective answers", which is not applicable in the safety evaluation of large models; on the other hand, its replacement strategy will continuously eliminate old samples, and under the requirements of compliance traceability, there is a problem that old risk categories are forgotten.

[0007] (4) Iterative evolution of human-machine adversarial interaction. Representative technologies include Dynabench, ANLI, and AdversarialQA. This type of method involves annotators continuously constructing adversarial examples to target the weaknesses of the current model. Its disadvantages are that the annotation cost is extremely high (ANLI reports that each successful example requires an average of 3.4 to 6.4 attempts and 199 to 355 seconds of human input), and it is easy to overfit the current opponent model, resulting in insufficient cross-model fairness and difficulty in long-term automated operation.

[0008] (5) Automated evaluation and calibration based on large models. Representative techniques include FreshEval, WildBench's WB-Reward and WB-Score. This type of method uses a large model as an automated judge, compressing the manual evaluation process into a low-cost automated scoring. Its disadvantage is that it only focuses on the "reliability of evaluation" and does not directly solve the core problem of "how the dataset itself evolves over time".

[0009] The methods described above together constitute the main branch of dynamic evolution technology in the current evaluation benchmark field.

[0010] In summary, while existing dynamic evolution techniques have achieved some success in evaluating general knowledge and general language capabilities, they suffer from three fundamental incompatibilities when applied to the safety evaluation of large models: First, there is a lack of objective truth values. The judgment of large-scale model security evaluation involves values, context, and compliance context. It is impossible to achieve objective scoring like LiveCodeBench using unit tests or standard answers like LiveBench. Methods that rely on structured truth values, such as version difference-driven and external knowledge interface self-refreshing, fail in this scenario.

[0011] Second, the offensive and defensive confrontation is spreading in an asymmetric, group-like manner. Security threats do not evolve through "fact replacement" but through "attack group reproduction." Existing methods mostly use "version differences" or "time segments" as triggers in the candidate sample discovery stage, lacking mechanisms to identify and suppress redundant variants within the same attack group.

[0012] Third, old risk categories should not be forgotten. For compliance traceability and long-term security auditing, the security assessment dataset must cover all known risk categories in the long term. It cannot be completely replaced in version updates like a general benchmark. The existing replacement update mechanism cannot guarantee the integrity of the risk topology. Summary of the Invention

[0013] To address the technical problems of existing technologies, such as the lack of objective truth values, the asymmetric, cluster-like diffusion of offensive and defensive confrontations, and the inability to forget old risk categories, this invention provides a dual-signal-driven self-evolution method and system for large-scale model security evaluation datasets. The technical solution is as follows: On the one hand, a dual-signal driven self-evolution method for large model security evaluation datasets is provided, characterized by the following: S1. Continuously collect candidate seed samples from multiple external threat sources to obtain a candidate seed set; S2. Preprocess each seed in the candidate seed set, and use the preprocessed data as candidate samples to obtain a candidate sample set. S3. Embed the current dataset and each candidate sample into a unified semantic space using a multimodal unified representation function, and calculate the threat semantic novelty. S4. Select a model family containing k different large models to be tested, and calculate the defense saturation based on the preset probability that the sample is successfully blocked by the security barrier or the model successfully refuses to answer. S5. Based on the defense saturation value and the threat semantic novelty value, the candidate samples are comprehensively scored; based on the comprehensive score, integer programming is used to select topics in combination with hard constraints to obtain the sample set to be included in the new dataset. S6. A hybrid triggering mechanism combining timed triggering and event triggering is adopted for real-time updates, and version governance is carried out by maintaining an anchor point sample set.

[0014] Optionally, S1 includes: External threat sources include: public jailbreak command communities, red team attack logs, compliance alert systems, real business dialogue logs, and new cross-language internet attack templates.

[0015] Optionally, in S2, each seed in the candidate seed set is preprocessed, including: For each seed in the candidate seed set, a candidate sample is generated using at least one of the following methods: mutation, cross-language rewriting, modality transfer, and template combination; wherein each candidate sample can be in the form of text, image, audio, video, or any combination of the above modalities.

[0016] Optionally, in S3, the current dataset and each candidate sample are embedded into a unified semantic space using a multimodal unified representation function, and the threat semantic novelty is calculated, including: Through a multimodal unified representation function (·) will move the current dataset D t And each candidate sample x ∈ X is embedded into a unified semantic space, and the threat semantic novelty is calculated according to the following formula: TNS(x) = 1 - max {y ∈ Dt} sim( (x), (y) ) · λ cluster (x, c) x ) Where sim(·,·) denotes the similarity metric function in the unified representation space; c x This represents the leaf node category to which sample x belongs on a predefined threat classification tree T. The top level of the threat classification tree includes at least four categories: jailbreak / hint injection, harmful content, value and ideological bias, and privacy breach and data extraction; λ cluster (x, c x ) is the density penalty term for sample x within its own cluster. The higher the sample density within the cluster, the larger the value of λ_cluster.

[0017] Optionally, in S4, candidate samples are comprehensively scored based on defense saturation and threat semantic novelty values; based on the comprehensive score, integer programming is used to select topics using hard constraints to obtain the sample set included in the new dataset, including: Select a model family M = {m1, m2, …, m} containing k different large models to be tested. k Let p i (x) represents model m i The pass rate on sample x is used to calculate the defense saturation using the following formula: DSS(x) = [ H(p1, p2, …, p k ) / log(k) ]·( 1-mean(p i ) ) Where H(·) is the entropy function, and the normalized numerator term measures the degree of divergence of the model family on sample x; mean(p i ) represents the average pass rate of the model family, and the mean suppression term (1-mean(p)) is used. i The DSS(x) is used to suppress the saturation score of "all passed" samples; a DSS(x) close to 1 indicates that the sample has the highest diagnostic value; a DSS(x) close to 0 indicates that the sample has lost its discriminative power and should be eliminated or archived.

[0018] Optionally, in S5, candidate samples are comprehensively scored based on defense saturation and threat semantic novelty values; based on the comprehensive score, integer programming is used to select topics using hard constraints to obtain the sample set included in the new dataset, including: The candidate sample x is comprehensively scored according to the following formula: Score(x) =α·TNS(x) +β·DSS(x) Where α and β are adjustable weights, and α + β = 1; Based on the comprehensive score, integer programming topics were selected using hard constraints to obtain the sample set to be included in the new dataset. Among them, the hard constraints include at least the risk category coverage integrity constraint, that is, the sample density of each threat category in the new dataset is not lower than a preset threshold.

[0019] Optionally, in S6, a hybrid triggering mechanism combining timed triggering and event triggering is used for real-time updates, and version governance is performed by maintaining an anchor point sample set, including: The timed triggering mechanism triggers minor version updates at preset cycles, replacing no more than a preset proportion of samples in each round; The event triggering mechanism will trigger an immediate update when at least one of the following conditions occurs: (i) the overall DSS mean of the dataset falls below a preset threshold; (ii) a high-risk entry is added from an external threat source; (iii) a new model to be tested is added to model family M; During version control, an anchor sample set A is maintained. D t A is not replaced in version iterations, ensuring that the model ranking correlation between adjacent versions is not lower than a preset threshold.

[0020] On the other hand, a dual-signal-driven self-evolution system for large model security evaluation datasets is provided. This system is used to implement the aforementioned dual-signal-driven self-evolution method for large model security evaluation datasets. The system includes: The threat intelligence gathering module is used to continuously collect candidate seed samples from multiple external threat sources to obtain a candidate seed set; The multimodal candidate generation module is used to preprocess each seed in the candidate seed set, and use the preprocessed data as candidate samples to obtain a candidate sample set. The dual-algorithm joint scoring module is used to embed the current dataset and each candidate sample into a unified semantic space through a multimodal unified representation function, and to calculate the threat semantic novelty; it selects a model family containing k different large models to be tested, and calculates the defense saturation based on the probability that the sample is successfully blocked by the security barrier or the model successfully refuses to answer. The evolutionary decision module is used to comprehensively score candidate samples based on defense saturation value and threat semantic novelty value; based on the comprehensive score, integer programming is used to select topics in combination with hard constraints to obtain the sample set to be included in the new dataset. The version governance and release module is used to perform real-time updates using a hybrid triggering mechanism that combines timed triggering and event triggering, and to perform version governance by maintaining an anchor point sample set; The system's five modules are connected sequentially to form a closed loop. The version governance and release module outputs evaluation feedback signals that flow back to the threat intelligence collection module to guide the next round of threat intelligence collection.

[0021] On the other hand, a dual-signal driven large model security evaluation dataset self-evolution device is provided, the dual-signal driven large model security evaluation dataset self-evolution device comprising: a processor; a memory, the memory storing computer-readable instructions, which, when executed by the processor, implement any of the methods described above for dual-signal driven large model security evaluation dataset self-evolution.

[0022] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods of the dual-signal driven large model security evaluation dataset self-evolution method.

[0023] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this embodiment of the invention, (1) the degree of automation is significantly improved. The five-module closed-loop pipeline compresses the manual maintenance cost to the single step of candidate review, making it possible to periodically refresh the dataset automatically. Compared with the strong dependence of human adversarial methods such as Dynabench on large-scale manual annotation (ANLI requires 3.4 to 6.4 manual attempts for each successful sample), this invention can achieve monthly-level low-manual-intervention evolution; compared with the limitation of periodic update methods such as LiveBench that only support "tasks with objective truth values", this invention supports the scenario of security evaluation without objective truth values.

[0024] (2) Modality independence and full threat type coverage. The mathematical forms of the two core algorithms, TNS and DSS, in this invention are decoupled from specific modes by replacing the unified multimodal representation function. (·) It can adapt to text, images, audio, video and any combination of modalities, giving the invention natural scalability in multimodal large model evaluation scenarios; at the same time, the top-level design of the threat classification tree covers four major categories of security threats: jailbreak / prompt injection, harmful content, value bias, and privacy leakage, covering the main security evaluation dimensions in the industry with a single method.

[0025] (3) Complete risk coverage and traceable compliance. Through the hard constraints of the threat classification tree and the anchor sample retention strategy, this invention maintains complete coverage of all risk categories during version updates, meeting the historical risk traceability requirements of compliance review and security audit, which is a capability that existing Live benchmark-type methods do not possess.

[0026] (4) Version comparability is quantifiable. By constraining the correlation between adjacent version models (e.g., Spearman rank correlation coefficient not less than 0.99), this invention ensures the comparability between versions in the evolution process, solves the problem of "inability to make longitudinal comparisons when changing a batch of questions" that is common in existing periodic update methods, and makes the evaluation results on different version datasets have a scientific basis for longitudinal comparability.

[0027] (5) Endogenous diagnostic signal-driven, breaking through the evaluation paradigm with objective truth values. This invention uses the entropy weighting and mean suppression mechanism of DSS to make the evolution signal of the dataset come directly from the behavioral divergence and saturation level of the model family itself, rather than relying on external standard answers or manual annotations. This provides an evolutionary decision-making basis that can be automatically quantified and run for a long time for safety evaluation scenarios without objective truth values. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating a dual-signal-driven self-evolution method for a large model security evaluation dataset, provided in an embodiment of the present invention. Figure 2 This is a block diagram of a dual-signal driven large model security evaluation dataset self-evolution system provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0030] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0031] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0032] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0033] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0034] This invention provides a dual-signal driven self-evolution method for large model security evaluation datasets. This method can be implemented using a dual-signal driven self-evolution device for large model security evaluation datasets, which can be a terminal or a server. Figure 1 The flowchart shown is a self-evolution method for a large model security evaluation dataset driven by dual signals. Figure 1 As shown, this invention proposes a dual-signal-driven self-evolution method for large model security evaluation datasets. The processing flow of this method may include the following steps: S1. Continuously collect candidate seed samples from multiple external threat sources to obtain a candidate seed set S; In one possible implementation, S1 includes: External threat sources include, but are not limited to: public jailbreak command communities, red team attack logs, compliance alert systems, real business dialogue logs, and new cross-language internet attack templates.

[0035] S2. Preprocess each seed in the candidate seed set S, and use the preprocessed data as candidate samples to obtain the candidate sample set X. In one feasible implementation, in S2, each seed in the candidate seed set S is preprocessed, including: For each seed in the candidate seed set, candidate samples are generated using at least one of the following methods: mutation, cross-language rewriting, modality transfer, and template combination; wherein each candidate sample can be in the form of text, image, audio, video, or any combination of the above modalities, forming a candidate sample set X.

[0036] S3. Embed the current dataset and each candidate sample into a unified semantic space using a multimodal unified representation function, and calculate the threat semantic novelty. In one feasible implementation, in S3, the current dataset and each candidate sample are embedded into a unified semantic space using a multimodal unified representation function, and the threat semantic novelty is calculated, including: Through a multimodal unified representation function (·) will move the current dataset D t And each candidate sample x ∈ X is embedded into a unified semantic space, and the threat semantic novelty is calculated according to the following formula: TNS(x) = 1 - max {y ∈ Dt} sim( (x), (y) ) · λ cluster (x, c) x ) Where sim(·,·) represents the similarity metric function in the unified representation space, and can be implemented using cosine similarity, Mahalanobis distance, or semantic distance based on graph structures, etc.; c x This represents the leaf node category to which sample x belongs on a predefined threat classification tree T. The top level of the threat classification tree includes at least four categories: jailbreak / hint injection, harmful content, value and ideological bias, and privacy breach and data extraction; λ cluster (x, c x λ is the density penalty term for sample x within its own population. The higher the sample density within the population, the greater the penalty term. cluster The larger the value, the better.

[0037] In one feasible implementation, novelty is not simply determined by the "distance to the most similar old sample," but is further weighted by a population density penalty, thereby mathematically achieving the effect of "truly novel samples having high scores and homogeneous, irrelevant samples having low scores." This invention solves the novelty determination problem in the "attack and defense population reproduction" scenario by defining a threat semantic novelty score (feature A).

[0038] S4. Select a model family containing k different large models to be tested, and calculate the defense saturation based on the preset probability that the sample is successfully blocked by the security barrier or the model successfully refuses to answer. In one feasible implementation, in S4, candidate samples are comprehensively scored based on defense saturation value and threat semantic novelty value; based on the comprehensive score, integer programming is performed to select topics using hard constraints, resulting in the sample set to be included in the new dataset, including: Select a model family M = {m1, m2, …, m} containing k different large models to be tested. k Let p i (x) represents model m i The pass rate on sample x is used to calculate the defense saturation using the following formula: DSS(x) = [ H(p1, p2, …, p k ) / log(k) ]·( 1-mean(p i ) ) Where H(·) is the entropy function, and the normalized numerator term measures the degree of divergence of the model family on sample x; mean(p i ) represents the average pass rate of the model family, and the mean suppression term (1-mean(p)) is used. i The DSS(x) is used to suppress the saturation score of "all passed" samples; a DSS(x) close to 1 indicates that the sample has the highest diagnostic value (large discrepancies between models and high average failure rate); a DSS(x) close to 0 indicates that the sample has lost its discriminative power and should be eliminated or archived.

[0039] In one feasible implementation, DSS is an intrinsic diagnostic metric that does not require a standard answer and is based solely on the consistency of behavior among model families and the average failure level, overcoming the limitations of existing technologies that require "objective truth values ​​for scoring." This invention addresses the problem of determining diagnostic value in scenarios "lacking objective truth values" by defining a defense saturation score (feature B).

[0040] In one feasible implementation, the TNS and DSS features in S3 and S4 work together to determine whether candidate samples are worth including, and the DSS determines whether they have sustained discriminative power after inclusion, thus achieving automatic evolution of datasets in multimodal and multi-threat scenarios.

[0041] S5. Based on the defense saturation value and the threat semantic novelty value, the candidate samples are comprehensively scored; based on the comprehensive score, integer programming is used to select topics in combination with hard constraints to obtain the sample set to be included in the new dataset. In one feasible implementation, in step S5, candidate samples are comprehensively scored based on defense saturation and threat semantic novelty values; then, integer programming is performed on the comprehensive score and hard constraints to select topics, resulting in the sample set to be included in the new dataset, including: The candidate sample x is comprehensively scored according to the following formula: Score(x) =α·TNS(x) +β·DSS(x) Where α and β are adjustable weights, and α + β = 1; Based on the comprehensive score, integer programming topics are selected using hard constraints to obtain the sample set to be included in the new dataset. Among them, the hard constraints include at least the risk category coverage integrity constraint, that is, the sample density of each threat category in the new dataset is not lower than a preset threshold, so as to ensure that the old risk categories are not forgotten at the algorithm level.

[0042] In one feasible implementation, in the joint decision-making process of step S5, a contamination immunity score CIS(x) can be further introduced. Based on the joint detection of n-gram matching and perplexity, it is determined whether the candidate sample has contaminated the pre-training corpus of the model family M to be tested. Candidates with CIS below the threshold are directly eliminated to avoid the benchmark being contaminated.

[0043] S6. A hybrid triggering mechanism combining timed triggering and event triggering is adopted for real-time updates, and version governance is carried out by maintaining an anchor point sample set.

[0044] In one feasible implementation, S6 employs a hybrid triggering mechanism combining timed triggering and event triggering for real-time updates, and performs version governance by maintaining an anchor point sample set, including: The timed triggering mechanism triggers minor version updates at a preset cycle (e.g., monthly), replacing no more than a preset proportion (e.g., 1 / 6) of the samples in each round; The event triggering mechanism will trigger an immediate update when at least one of the following conditions occurs: (i) the overall DSS mean of the dataset falls below a preset threshold; (ii) a high-risk entry is added from an external threat source; (iii) a new model to be tested is added to model family M; During version control, an anchor sample set A is maintained. D t A is not replaced in version iterations, ensuring that the model ranking correlation between adjacent versions is not lower than a preset threshold (e.g., Spearman rank correlation coefficient is not lower than 0.99), thus ensuring comparability between versions.

[0045] In one feasible implementation, the present invention will illustrate the above technical solution in detail through the following specific application scenario. The application scenario of Implementation 1 is the evolution of text modality jailbreak samples: Initially D t It contains approximately 5,000 text jailbreak command samples, covering four types of security threats. An evolution process is triggered monthly: approximately 1,000 candidate seeds are collected from public jailbreak communities and compliance alert systems, which are then processed through multimodal candidate generation to produce approximately 3,000 candidate samples. Call the text embedding model (in practice, a Chinese semantic representation model based on BERT / Sentence-BERT series or more advanced models can be used) as (·), calculate the TNS for each candidate; Select k=5 large models with different architectures (containing at least two different vendor models) to form a model family M; calculate the DSS of each candidate model through parallel security barrier testing; sort them according to Score(x) = 0.5·TNS(x) + 0.5·DSS(x), and perform integer programming to select topics, taking into account the hard constraint that there are no less than 800 items for each of the four types of threats; retain 1000 anchor point samples without replacement, and replace about 800 samples per month. The ranking correlation between adjacent versions on M should be maintained at no less than 0.99.

[0046] The application scenario of Example 2 is the evolution of multimodal harmful content: (·) Replace with a unified representation module for text / audio / video (in implementation, a CLIP-style cross-modal contrastive learning encoder or an ImageBind-style multimodal alignment encoder can be used), and the remaining steps are the same as in Example 1.

[0047] Key pseudocode (TNS): Input: Candidate x, current dataset D t Characterization function Similarity (sim), Threat classification tree (T), Population density penalty coefficient (ρ) 1. Calculate v x = (x); for each y ∈ D t Calculate v y = (y) 2. s max = max {y ∈ D_t} sim(v x , v y ) 3. Determine c x = locate(x, T), count c x Sample density d(c) within leaf nodes x ) 4. λ cluster = 1 + ρ · d(c x ) 5. Output TNS(x) = 1 - s max · λ cluster Key pseudocode (DSS): Input: Candidate x, Model family M = {m1, …,m i , …,m k} 1. For each m i∈ M, Parallel query p i (x) = Safety barrier success rate 2. Calculate H = -Σ i p i · log(p i (or its normalized variant) 3. Calculate the mean p = (1 / k) Σ i p i 4. Output DSS(x) = (H / log(k)) · (1 - mean) p ) In this embodiment of the invention, the joint decision-making of threat semantic novelty score TNS(x) and defense saturation score DSS(x) serves as the core signal source for dataset evolution. The comprehensive score Score(x) = α·TNS(x) + β·DSS(x) drives the inclusion, retention, replacement, and archiving of candidate samples. This joint decision-making mechanism is the fundamental difference between this invention and all existing dynamic evaluation benchmark evolution methods—existing methods either rely on "version difference (fact change)" signals or "objective answer score" signals, neither of which jointly uses the two orthogonal signals of "semantic novelty" and "model family behavior consistency" for security dataset evolution decision-making.

[0048] The threat population density penalty term λ in the TNS score provided by this invention cluster (x, c x This item is based on the leaf node c of threat classification. x Density statistics are performed on the populations to which candidate samples belong, and redundant variants within the same attack population are suppressed by applying a "higher population density, heavier penalty" approach. This design mathematically distinguishes between "truly novel samples with high scores and homogeneous spam samples with low scores," directly addressing the unique security characteristic of "asymmetric population-based diffusion in attack and defense confrontation."

[0049] This invention provides a dual combination of entropy weighting and mean suppression in DSS scoring. The entropy term H(p1,…,p) k ) / log(k) measures the degree of divergence among the model family, with mean suppression term (1 - mean(p)). i By lowering the saturation score of "all passed" samples, the multiplication of the two factors allows DSS to quantify the diagnostic value of samples for the model family without relying on any standard answer. This design is a direct response of this invention to the scenario where "large model safety evaluation lacks objective truth values".

[0050] This invention introduces a hard constraint—that the sample density of each threat category is not less than a preset threshold—in the integer programming stage of joint decision-making, ensuring from an algorithmic perspective that old risk categories are not forgotten due to the introduction of new samples. This design is a direct response to the scenario requirement that "security assessment datasets must meet compliance traceability."

[0051] This invention uses monthly timed events as a baseline, supplemented by three types of event triggers: DSS mean threshold, new threat source push, and new model integration, to achieve on-demand evolution rather than rigid periodic evolution. This is achieved through anchor set A. The version governance requirement ensures comparability between versions during the evolution process by maintaining the invariance of D_t throughout version iterations and ensuring that the correlation of adjacent version model rankings (e.g., Spearman rank correlation coefficient) is not lower than a preset threshold.

[0052] Figure 2 This is a block diagram of a dual-signal driven self-evolution system 200 for a large model security evaluation dataset, illustrated according to an exemplary embodiment. The system 200 is used in a dual-signal driven self-evolution method for a large model security evaluation dataset. (Refer to...) Figure 2 The system includes a threat intelligence gathering module 210, a multimodal candidate generation module 220, a dual-algorithm joint scoring module 230, an evolutionary decision-making module 240, and a version governance and release module 250. Among them: The threat intelligence collection module 210 is used to continuously collect candidate seed samples from multiple external threat sources to obtain a candidate seed set; The multimodal candidate generation module 220 is used to preprocess each seed in the candidate seed set, and use the preprocessed data as candidate samples to obtain a candidate sample set. The dual-algorithm joint scoring module 230 is used to embed the current dataset and each candidate sample into a unified semantic space through a multimodal unified representation function, and to calculate the threat semantic novelty; it selects a model family containing k different large models to be tested, and calculates the defense saturation based on the probability that the sample is successfully blocked by the security fence or the model successfully refuses to answer. The evolutionary decision module 240 is used to comprehensively score candidate samples based on defense saturation value and threat semantic novelty value; and to select topics by integer programming based on the comprehensive score and hard constraints to obtain the sample set to be included in the new dataset. The version governance and release module 250 is used to perform real-time updates using a hybrid triggering mechanism that combines timed triggering and event triggering, and to perform version governance by maintaining an anchor point sample set; The system's five modules are connected sequentially to form a closed loop. The version governance and release module outputs evaluation feedback signals that flow back to the threat intelligence collection module to guide the next round of threat intelligence collection.

[0053] Figure 3 This is a schematic diagram of the structure of a dual-signal driven large model security evaluation dataset self-evolution device provided in an embodiment of the present invention, as shown below. Figure 3 As shown, a dual-signal driven large-model security evaluation dataset self-evolution device may include the above-mentioned... Figure 2 The diagram illustrates a dual-signal driven self-evolution system for a large model security evaluation dataset. Optionally, a dual-signal driven self-evolution device 410 for a large model security evaluation dataset may include a first processor 2001.

[0054] Optionally, a dual-signal driven large model security evaluation dataset self-evolution device 410 may also include a memory 2002 and a transceiver 2003.

[0055] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0056] The following is combined Figure 3 A detailed description of each component of a dual-signal driven large-model security evaluation dataset self-evolution device 410 is provided below: The first processor 2001 is the control center of a dual-signal driven large-model security evaluation dataset self-evolution device 410. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0057] Optionally, the first processor 2001 can perform various functions of a dual-signal driven large model security evaluation dataset self-evolution device 410 by running or executing software programs stored in memory 2002 and calling data stored in memory 2002.

[0058] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 are shown in the diagram.

[0059] In a specific implementation, as one example, a dual-signal driven large model security evaluation dataset self-evolution device 410 may also include multiple processors, for example... Figure 3 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0060] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0061] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via an interface circuit of a dual-signal driven large model security evaluation dataset self-evolution device 410. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0062] The transceiver 2003 is used to communicate with network devices or with mobile terminals.

[0063] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 3 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0064] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected to the interface circuit of the large model security evaluation dataset self-evolution device 410 via a dual-signal driven interface circuit. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0065] It should be noted that, Figure 3 The structure of the dual-signal driven large model security evaluation dataset self-evolution device 410 shown does not constitute a limitation on the router. Actual knowledge structure identification devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0066] Furthermore, the technical effect of a dual-signal driven large model security evaluation dataset self-evolution device 410 can be referred to the technical effect of a dual-signal driven large model security evaluation dataset self-evolution method described in the above method embodiments, and will not be repeated here.

[0067] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0068] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0069] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable sensors. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0070] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0071] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0072] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for various specific applications, but such implementations should not be considered beyond the scope of this invention.

[0073] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0074] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0075] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0076] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A dual-signal driven self-evolution method for large-scale model security evaluation datasets, characterized in that, The method includes: S1. Continuously collect candidate seed samples from multiple external threat sources to obtain a candidate seed set; S2. Preprocess each seed in the candidate seed set, and use the preprocessed data as candidate samples to obtain a candidate sample set. S3. Embed the current version security evaluation dataset and each candidate sample into a unified semantic space using a multimodal unified representation function, and calculate the threat semantic novelty. S4. Select a model family containing k different large models to be tested, and calculate the defense saturation based on the preset probability that the sample is successfully blocked by the security barrier or the model successfully refuses to answer. S5. Based on the defense saturation value and the threat semantic novelty value, the candidate samples are comprehensively scored; based on the comprehensive score, integer programming is used to select topics in combination with hard constraints to obtain the sample set to be included in the new dataset. S6. A hybrid triggering mechanism combining timed triggering and event triggering is adopted for real-time updates, and version governance is carried out by maintaining an anchor point sample set.

2. The method according to claim 1, characterized in that, S1 includes: External threat sources include: public jailbreak command communities, red team attack logs, compliance alert systems, real business dialogue logs, and new cross-language internet attack templates.

3. The method according to claim 2, characterized in that, In step S2, each seed in the candidate seed set is preprocessed, including: For each seed in the candidate seed set, a candidate sample is generated using at least one of the following methods: mutation, cross-language rewriting, modality transfer, and template combination; wherein each candidate sample can be in the form of text, image, audio, video, or any combination of the above modalities.

4. The method according to claim 3, characterized in that, In step S3, the current dataset and each candidate sample are embedded into a unified semantic space using a multimodal unified representation function, and the threat semantic novelty is calculated, including: Through a multimodal unified representation function (·) will move the current dataset D t And each candidate sample x ∈ X is embedded into a unified semantic space, and the threat semantic novelty is calculated according to the following formula: TNS(x) = 1 - max {y ∈ Dt} sim( (x), (y) ) · λ cluster ( x, cx ) Where sim(·,·) denotes the similarity metric function in the unified representation space; c x This represents the leaf node category to which sample x belongs on a predefined threat classification tree T. The top level of the threat classification tree includes at least four categories: jailbreak / hint injection, harmful content, value and ideological bias, and privacy breach and data extraction; λ cluster (x, c x ) is the density penalty term for sample x within its own cluster. The higher the sample density within the cluster, the larger the value of λ_cluster.

5. The method according to claim 4, characterized in that, In step S4, candidate samples are comprehensively scored based on defense saturation value and threat semantic novelty value. Based on the comprehensive scoring and combined with hard constraints, integer programming problems were selected to obtain the sample set for inclusion in the new dataset, including: Select a model family M = {m1, m2, …, m} containing k different large models to be tested. k Let p i (x) represents model m i The pass rate on sample x is used to calculate the defense saturation using the following formula: DSS(x) = [ H(p1, p2, …, p k ) / log(k) ]·( 1-mean(p i ) ) Where H(·) is the entropy function, and the normalized numerator term measures the degree of divergence of the model family on sample x; mean(p i ) represents the average pass rate of the model family, and the mean suppression term (1-mean(p)) is used. i The DSS(x) is used to suppress the saturation score of "all passed" samples; a DSS(x) close to 1 indicates that the sample has the highest diagnostic value; a DSS(x) close to 0 indicates that the sample has lost its discriminative power and should be eliminated or archived.

6. The method according to claim 5, characterized in that, In S5, candidate samples are comprehensively scored based on defense saturation value and threat semantic novelty value; Based on the comprehensive scoring and combined with hard constraints, integer programming problems were selected to obtain the sample set for inclusion in the new dataset, including: The candidate sample x is comprehensively scored according to the following formula: Score(x) =α·TNS(x) +β·DSS(x) Where α and β are adjustable weights, and α + β = 1; Based on the comprehensive score, integer programming topics were selected using hard constraints to obtain the sample set to be included in the new dataset. Among them, the hard constraints include at least the risk category coverage integrity constraint, that is, the sample density of each threat category in the new dataset is not lower than a preset threshold.

7. The method according to claim 6, characterized in that, The S6 employs a hybrid triggering mechanism combining timed and event-triggered mechanisms for real-time updates, and performs version governance by maintaining an anchor point sample set, including: The timed triggering mechanism triggers minor version updates at preset cycles, replacing no more than a preset proportion of samples in each round; The event triggering mechanism will trigger an immediate update when at least one of the following conditions occurs: (i) the overall DSS mean of the dataset falls below a preset threshold; (ii) a high-risk entry is added from an external threat source; (iii) a new model to be tested is added to model family M; During version control, an anchor sample set A is maintained. D t A is not replaced in version iterations, ensuring that the model ranking correlation between adjacent versions is not lower than a preset threshold.

8. A dual-signal driven self-evolution system for large model security evaluation datasets, wherein the dual-signal driven self-evolution system for large model security evaluation datasets is used to implement the dual-signal driven self-evolution method for large model security evaluation datasets as described in any one of claims 1-7, characterized in that, The system includes: The threat intelligence gathering module is used to continuously collect candidate seed samples from multiple external threat sources to obtain a candidate seed set; The multimodal candidate generation module is used to preprocess each seed in the candidate seed set, and use the preprocessed data as candidate samples to obtain a candidate sample set. The dual-algorithm joint scoring module is used to embed the current dataset and each candidate sample into a unified semantic space through a multimodal unified representation function, and to calculate the threat semantic novelty; it selects a model family containing k different large models to be tested, and calculates the defense saturation based on the probability that the sample is successfully blocked by the security barrier or the model successfully refuses to answer. The evolutionary decision module is used to comprehensively score candidate samples based on defense saturation value and threat semantic novelty value; based on the comprehensive score, integer programming is used to select topics in combination with hard constraints to obtain the sample set to be included in the new dataset. The version governance and release module is used to perform real-time updates using a hybrid triggering mechanism that combines timed triggering and event triggering, and to perform version governance by maintaining an anchor point sample set; The system's five modules are connected sequentially to form a closed loop. The version governance and release module outputs evaluation feedback signals that flow back to the threat intelligence collection module to guide the next round of threat intelligence collection.

9. A dual-signal driven self-evolution device for large model security evaluation datasets, the dual-signal driven self-evolution device for large model security evaluation datasets comprising: processor; A memory storing computer-readable instructions, which, when executed by the processor, implement any one of the methods in the dual-signal driven large model security evaluation dataset self-evolution method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images