Method for evaluating defense strategies against various attacks driven by endogenous security of large models

Abstract

The application provides a defense strategy evaluation method for resisting various attacks driven by endogenous safety of a large model, and belongs to the technical field of artificial intelligence and information security. The application first constructs a multi-modal attack sample library and a defense knowledge graph. When the large model processes instructions, the abnormal features of the embedding layer, the intermediate layer and the output layer are cross-layer fused to generate a feature vector, the attack type is identified by comparing with the attack vector library, and the threat level is determined. Then, the graph traversal and multi-objective optimization are used to filter strategies in the graph to construct a defense path, the model is configured according to the corresponding security mode of the threat level, and security information is output. Finally, the interception rate is calculated by using the adversarial case in the sandbox, the intermediate layer parameters are closed-loop corrected and the strategy weight is updated according to the interception rate, and finally the defense evaluation score is determined based on the weight deviation. The application realizes endogenous accurate perception, adaptive defense configuration and closed-loop iterative evolution of security capability of the large model for multi-source attacks.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence and information security technology, and in particular relates to a method for evaluating defense strategies against various attacks driven by the intrinsic security of large models. Background Technology

[0002] With the widespread deployment of generative large-scale models, security threats such as hint injection and jailbreak attacks are becoming increasingly prominent. Constructing an intrinsic security mechanism capable of accurately identifying malicious intent and performing dynamic defense assessments is crucial for ensuring the reliable operation of models in complex scenarios. This type of approach aims to achieve dynamic response to multiple types of attacks and continuous iterative optimization of security capabilities by constructing an adaptive defense mechanism combined with a quantitative evaluation system. It shows great promise for application in highly sensitive fields such as finance, healthcare, and government.

[0003] Existing large-scale model defense techniques typically rely on external security barriers or static keyword matching rules to filter input commands and output content. Some methods also utilize secure alignment data for supervised fine-tuning to improve the model's resistance to specific types of known attacks, or use a single-dimensional text classifier to determine the security of the input.

[0004] However, the aforementioned methods are often limited to the analysis of surface-level text features, neglecting deep semantic anomalies in the multi-level hidden states within the model, and lack an adaptive mechanism to dynamically adjust defense strategies and model parameters based on real-time attack and defense effects. This results in a lack of targeted defense strategy selection when facing various types of attacks, an inability to dynamically generate optimal defense paths based on attack characteristics, and a lack of objective quantitative mechanisms for evaluating defense effectiveness, making it difficult to achieve closed-loop evolution and parameter correction of security capabilities. Therefore, existing technologies suffer from technical problems such as a lack of adaptability in defense strategy generation, difficulty in objectively quantifying defense effectiveness, and difficulty in achieving closed-loop iterative optimization of security capabilities. Summary of the Invention

[0005] The purpose of this application is to provide a method and system for evaluating defense strategies against various attacks driven by the intrinsic security of a large model, in order to solve the problems in the existing technology of defense strategy generation lacking adaptability, defense effect being difficult to objectively quantify and evaluate, and security capabilities being difficult to achieve closed-loop iterative optimization.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides a method for evaluating defense strategies against multiple attacks driven by inherent security in a large model, comprising: Obtain multimodal attack samples and a knowledge graph with attack type and defense strategy as nodes and defense relationship as edges; During the large model's processing of input instructions, abnormal coding features, instruction attention features, and abnormal offset features of word probability distribution are extracted from the embedding layer, intermediate layer, and output layer, respectively, and fused through cross-layer residual connections to obtain feature vectors; Semantic disambiguation and feature extraction are performed on multimodal attack samples with added noise interference to obtain an attack vector library. A linear classifier is then used to compare and identify the feature vectors in the attack vector library to obtain attack type labels and attack target types. Threat levels are determined based on the attack target types. Using attack type tags and threat levels as query criteria, graph traversal and multi-target optimization are used to filter out matching target defense strategy nodes in the knowledge graph to obtain the defense path. Based on the threat level, determine the security mode level of the large model, configure the large model using the defense path and security mode level, and then output the security output information and defense execution logs corresponding to the input and output commands. The attack vector library is used to generate adversarial use cases, and the interception rate of the defense path is calculated in the sandbox based on the threat level, security output information and defense execution log. The parameters of the intermediate layer are corrected based on the interception rate and the weights of the target defense strategy nodes that make up the defense path in the knowledge graph are updated to obtain weight deviation data. The defense evaluation score is determined based on the weight deviation data.

[0007] Optionally, the method further includes: The frequency of occurrence of non-standard character encoding sequences in the abnormal encoding features of the embedding layer is calculated, and the encoding dispersion is obtained based on the deviation between the occurrence frequency and the preset reference frequency. The encoding dispersion is compared with a preset security trigger threshold. If the encoding dispersion is greater than the security trigger threshold, the instruction attention features and abnormal offset features of the word probability distribution are extracted from the intermediate layer and the output layer, respectively.

[0008] Optionally, during the large model's processing of input instructions, abnormal coding features, instruction attention features, and abnormal offset features of lexical probability distributions are extracted from the embedding layer, intermediate layer, and output layer, respectively, and fused through cross-layer residual connections to obtain a feature vector, including: In the embedding layer, non-standard encoded sequences of input instructions are extracted. By calculating the frequency of non-standard characters in the non-standard encoded sequences, encoded outliers are obtained. Vector mapping is performed on the encoded outliers to obtain anomalous encoded features. The weight matrices of multiple attention mechanism branches in the intermediate layer are extracted. The association strength value is obtained by multiplying the weight matrix with the preset safety weights. The association strength value is then subjected to linear projection transformation to obtain the instruction attention feature. Extract the lexical probability vector generated by the output layer, and obtain the lexical distribution offset as an abnormal offset feature by calculating the dispersion between the lexical probability vector and the preset safety context vector. The abnormal coding features, instruction attention features, and abnormal offset features are dimensionally aligned to obtain multiple alignment vectors. These alignment vectors are then superimposed and aggregated with the initial semantic vector of the input instruction using cross-layer residual connections to obtain the feature vector.

[0009] Optionally, semantic disambiguation and feature extraction are performed on the multimodal attack samples with added noise interference to obtain an attack vector library. A linear classifier is then used to compare and identify the feature vectors in the attack vector library to obtain attack type labels and attack target types. The threat level is determined based on the attack target type, including: By removing ambiguities from the multimodal attack samples, disambiguated multimodal attack samples are obtained. Multiple attack feature components are then extracted from the disambiguated multimodal attack samples to obtain an attack vector library containing multiple attack feature components. A linear classifier is used to calculate the difference in numerical distribution between the feature vector and each attack feature component in the attack vector library. Based on the difference in numerical distribution, the feature vector is classified to obtain the attack type label and the attack target type. Based on the type of attack target, the threat level is obtained according to the preset mapping relationship between the type of attack target and the degree of threat.

[0010] Optionally, using attack type tags and threat levels as query criteria, graph traversal and multi-target optimization are used to filter out matching target defense strategy nodes in the knowledge graph to obtain defense paths, including: Using attack type tags and threat levels as query criteria, a path traversal is performed in the knowledge graph to determine the starting node that matches the attack type tag; Starting from the initial node, a breadth-first search is performed along the defense relationships to obtain multiple candidate defense strategy nodes that have defense associations with the initial node; The historical interception rate and resource consumption value corresponding to each candidate defense strategy node are extracted from the knowledge graph. Pareto optimization is used to perform a weighted balance calculation on the historical interception rate and resource consumption value corresponding to each candidate defense strategy node to obtain the evaluation score of each candidate defense strategy node. The candidate defense strategy node whose evaluation score meets the preset security index is determined as the target defense strategy node. Based on the defense relationships between each target defense strategy node in the knowledge graph, all target defense strategy nodes are arranged in a temporal sequence to obtain the defense path.

[0011] Optionally, the security mode level of the large model is determined based on the threat level. After configuring the large model using the defense path and security mode level, the security output information and defense execution logs corresponding to the input and output commands are displayed, including: Based on the threat level, the security mode level is obtained according to the preset mapping relationship between threat level and security response level; The sampling probability threshold and word selection randomness of the large model are numerically adjusted according to the security mode level to obtain the model configuration parameters; Based on the sequential arrangement of each target defense strategy node in the defense path, the input instructions are semantically reconstructed and lexical filtered in the operating environment determined by the model configuration parameters to obtain the instruction sequence. The large model is used to perform logical reasoning on the instruction sequence, output security output information, and the interception trigger result and running parameter offset of each target defense strategy node in the logical reasoning are determined as the defense execution log.

[0012] Optionally, adversarial use cases are generated using an attack vector library, and the interception rate of the defense path is calculated within a sandbox based on threat level, security output information, and defense execution logs. The parameters of the intermediate layer are corrected based on the interception rate, and the weights of the target defense strategy nodes constituting the defense path in the knowledge graph are updated to obtain weight deviation data. A defense evaluation score is then determined based on the weight deviation data, including: Attack feature components are extracted from the attack vector library, and attribute mutations and random perturbations are applied to the attack feature components to obtain adversarial use cases. The adversarial test cases are input into the sandbox for logical reasoning to obtain the test interception records of the adversarial test cases. The security comparison threshold is determined according to the threat level, and the semantic distance between the output content of the adversarial test cases and the security output information is calculated. The interception rate is calculated by counting the number of adversarial test cases whose semantic distance is within the security comparison threshold and whose test interception records match the interception trigger results in the defense execution log. The interception rate is obtained by the ratio of the number of adversarial test cases to the total number of adversarial test cases. The difference between the interception rate and the preset interception threshold is calculated to obtain the feedback error value. The intermediate layer parameters are then corrected based on the feedback error value, and the contribution of the target defense strategy nodes is redistributed according to the defense relationship to obtain the weight deviation data. The defense assessment score is obtained by standardizing and weighting the weighted bias data.

[0013] Secondly, this application provides a defense strategy evaluation system for resisting multiple attacks driven by large-scale model intrinsic security, including: The acquisition module is used to acquire multimodal attack samples and a knowledge graph with attack types and defense strategies as nodes and defense relationships as edges; The generation module is used to extract abnormal coding features, instruction attention features, and abnormal offset features of word probability distribution from the embedding layer, intermediate layer, and output layer respectively during the processing of input instructions in the large model, and fuse them through cross-layer residual connections to obtain feature vectors; The generation module is also used to perform semantic disambiguation and feature extraction on multimodal attack samples with added noise interference to obtain an attack vector library. A linear classifier is then used to compare and identify the feature vectors in the attack vector library to obtain attack type labels and attack target types. The threat level is determined based on the attack target type. The filtering module is used to filter out matching target defense strategy nodes in the knowledge graph by using attack type tags and threat levels as query conditions, and obtain the defense path by using graph traversal and multi-target optimization. The output module is used to determine the security mode level of the large model based on the threat level, and after configuring the large model using the defense path and security mode level, outputs the security output information and defense execution log corresponding to the input commands. The generation module is also used to generate adversarial use cases using the attack vector library and calculate the interception rate of the defense path in the sandbox based on the threat level, security output information and defense execution logs. Based on the interception rate, the parameters of the intermediate layer are corrected and the weights of the target defense strategy nodes that make up the defense path in the knowledge graph are updated to obtain weight deviation data. The defense evaluation score is determined based on the weight deviation data.

[0014] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor, used to execute the computer program, implements the steps of the method for evaluating defense strategies against multiple attacks driven by large-scale intrinsic security as described in the first aspect above.

[0015] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the large-model intrinsic security-driven defense strategy evaluation method against multiple attacks as described in the first aspect above.

[0016] The large-model intrinsic security-driven defense strategy evaluation method for resisting multiple attacks provided in this application constructs a multimodal attack sample library and defense knowledge graph. During the large model's instruction processing, it can extract and fuse the abnormal encoding of the embedding layer, the instruction attention of the intermediate layer, and the probability offset features of the output layer across layers. This breaks through the limitation of simply relying on surface text matching and achieves accurate generation of adaptive defense paths for multiple attack types.

[0017] Simultaneously, defense paths are dynamically selected and security modes are configured based on threat levels, achieving real-time response and precise matching of defense strategies. The effectiveness of defenses is quantified and verified using adversarial test cases generated in a sandbox, and model parameters are corrected and graph weights are updated based on the interception rate, driving continuous optimization and self-iteration of defense capabilities. Therefore, this application effectively solves the technical problems in existing technologies, such as the lack of adaptability in defense strategy generation, the difficulty in objectively quantifying defense effects, and the lack of a closed-loop evolution mechanism for security capabilities.

[0018] This hierarchical detection mechanism avoids full-link resource consumption for all input commands, and can significantly reduce the processing overhead and response latency of normal requests while ensuring the low accuracy of identifying complex attack types using deep hidden state features. Therefore, this application, while ensuring high-precision intent detection and adaptive iteration of defense capabilities, further solves the problems of high computational load and difficulty in meeting real-time requirements of deep defense mechanisms. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a method for evaluating a defense strategy against multiple attacks driven by intrinsic security of a large model, as provided in an embodiment of this application. Figure 2 A topological diagram of a knowledge graph with attack types and defense strategies as nodes and defense relationships as edges, provided for an embodiment of this application; Figure 3 A flowchart illustrating a method for determining a defense assessment score, provided as an embodiment of this application; Figure 4 A schematic diagram of the structure of a defense strategy evaluation system for resisting multiple attacks driven by large-scale intrinsic security, provided in an embodiment of this application; Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0021] In the field of generative large model security protection, existing technologies typically rely on external guardrails or static keyword matching rules to filter commands. Although some methods employ supervised fine-tuning or single-dimensional text classifiers, they are often limited to the analysis of surface text features, neglecting deep semantic anomalies in the multi-level hidden states during model inference, and lacking an adaptive mechanism to dynamically adjust defense strategies and model parameters based on real-time attack and defense effects.

[0022] This over-reliance on surface features and static defense mode makes it difficult to accurately identify malicious intent when facing complex prompt injection or spoofing attacks, and it is even more impossible to achieve closed-loop evolution and parameter correction of security capabilities. As a result, there are technical problems such as low intent detection accuracy and difficulty in achieving adaptive iteration of defense capabilities.

[0023] To address the aforementioned issues, this application proposes a defense strategy evaluation method driven by the intrinsic security of a large model to resist various attacks. The core of this method lies in constructing an attack and defense knowledge graph and fusing multi-dimensional anomaly features from the embedding layer, intermediate layer, and output layer through cross-layer residual connections to generate feature vectors that represent attack intent, thereby overcoming the limitations of surface-level text analysis.

[0024] Building upon this foundation, a graph traversal and multi-objective optimization algorithm is used to dynamically select defense paths, and the model's security mode is adaptively configured based on the threat level. Simultaneously, a sandbox adversarial testing mechanism is introduced, using the interception rate of the defense paths to reverse-correct the model's intermediate layer parameters and update the policy node weights. This method, through end-to-end collaboration from adaptive defense strategy generation to sandbox evaluation and parameter closed-loop optimization, effectively addresses the problems of existing technologies' lack of specificity in defense strategies and the difficulty in adaptively evolving defense capabilities, significantly improving the security protection capabilities of large models in adversarial environments.

[0025] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] To address the problems of existing technologies, embodiments of this application provide a method, apparatus, device, computer storage medium, and computer program product for evaluating defense strategies against multiple attacks driven by large-model intrinsic security. The method for evaluating defense strategies against multiple attacks driven by large-model intrinsic security, as provided in this application embodiment, will be described first.

[0027] Figure 1This illustration shows a flowchart of a method for evaluating defense strategies against multiple attacks driven by inherent security in a large model, according to an embodiment of this application. Figure 1 As shown, the method includes: S101. Obtain multimodal attack samples and a knowledge graph with attack types and defense strategies as nodes and defense relationships as edges.

[0028] Multimodal attack samples refer to various forms of malicious input data generated against large models, covering different modalities of attack vectors such as text prompt injection instructions, malicious code snippets, and jailbreak inducement rhetoric, as well as disguised text that attempts to bypass security reviews or logical traps that induce models to produce harmful outputs.

[0029] Attack type nodes indicate the specific attack methods employed by the attacker. Defense strategy nodes refer to intrinsic or external protective measures used to defend against specific attacks, including input semantic cleansing, mandatory verification of the inference layer's thought chain, and output probability suppression mechanisms.

[0030] Defense relationships refer to directed edges that connect attack types with corresponding effective defense strategies, representing the defense response logic against a specific threat. Knowledge graphs, on the other hand, are structured semantic networks built upon these nodes and edges, used to store prior knowledge and dynamic associations related to attack and defense strategies.

[0031] Figure 2 This illustration shows a topological diagram of a knowledge graph provided in one embodiment of this application, with attack types and defense strategies as nodes and defense relationships as edges. For example... Figure 2 As shown, the knowledge graph includes multiple attack type nodes and multiple defense strategy nodes. The attack type nodes include privacy theft attack type node 21, ethics bypass attack type node 22, and logic destruction attack type node 23.

[0032] Defense strategy nodes are associated with node weights This includes the input-side semantic purification defense strategy node 24, the inference layer thought chain forced verification defense strategy node 25, and the output-side probability suppression defense strategy node 26. Defense relationship 27 is a directed edge from attack type nodes to defense strategy nodes, used to represent the defense response logic against a specific threat.

[0033] Privacy theft attack type node 21 points to the input semantic purification defense strategy node 24 through defense relationship 27. Ethics bypass attack type node 22 points to the input semantic purification defense strategy node 24 and the reasoning layer thinking chain forced verification defense strategy node 25 through defense relationship 27. Logic destruction attack type node 23 points to the reasoning layer thinking chain forced verification defense strategy node 25 and the output probability suppression defense strategy node 26 through defense relationship 27.

[0034] In practice, raw data is first collected through multiple channels, including automated web crawlers, sandboxes, honeypots, and security community interfaces, to obtain attack samples covering various types of attacks such as hint injection, jailbreak attacks, data poisoning, and model reverse engineering. Simultaneously, normal user query requests are collected as negative samples to construct the basic dataset. Next, the collected raw samples are standardized, and differential privacy techniques are used to add noise and perform semantic disambiguation to generate a standardized set of multimodal attack samples. ,in This represents a single attack sample and its corresponding metadata.

[0035] Simultaneously, adversarial training samples, security fine-tuning parameters, and output filtering rules were compiled to safeguard assets, and historical interception rates and resource consumption metrics for each strategy were recorded. Based on this, intent features were extracted from attack samples as source nodes, and corresponding defense methods were matched as target nodes. Connection edges were established based on historical interception results, constructing a system including a set of attack types. With a set of defense strategies knowledge graph ,in Represents a set of nodes. This represents the set of edges indicating the effectiveness of a defense. And for each defense strategy node... Assign an initial weight value w0, which is mapped to the average interception success rate of each strategy in the historical dataset.

[0036] S102. During the input instruction processing of the large model, abnormal coding features, instruction attention features, and abnormal offset features of word probability distribution are extracted from the embedding layer, intermediate layer, and output layer, respectively, and fused through cross-layer residual connections to obtain feature vectors.

[0037] Optionally, step S102, during the large model's processing of input instructions, involves extracting abnormal coding features, instruction attention features, and abnormal offset features of word probability distribution from the embedding layer, intermediate layer, and output layer, respectively, and fusing them through cross-layer residual connections to obtain a feature vector. This process can specifically include: S1021. Extract the non-standard encoding sequence of the input instruction in the embedding layer, obtain the encoding anomaly value by calculating the occurrence frequency of non-standard characters in the non-standard encoding sequence, and perform vector mapping on the encoding anomaly value to obtain the abnormal encoding feature.

[0038] The embedding layer refers to the initial processing module in a large model responsible for converting discrete input text into a continuous vector space. Non-standard encoded sequences refer to hexadecimal encodings, Unicode obfuscated characters, garbled text, or unconventional escape sequences included in the input instructions; these sequences are often used in hint injection attacks to bypass text matching defenses. Encoding outliers are a quantitative indicator used to represent the density of non-natural language components in the input instructions. Anomalous encoding features are vector representations obtained by mapping the above scalars to a high-dimensional space.

[0039] In practice, the first step is to access the embedding layer interface of the large model and perform character-level scanning on the received input commands. Next, character segments that do not belong to the standard ASCII code table or the commonly used UTF-8 character set are identified and marked as non-standard encoded sequences. Subsequently, the proportion of these non-standard characters in the entire command length is calculated to obtain the encoding outlier value. .

[0040] Finally, the pre-trained mapping matrix is ​​used. scalar Transform into anomaly encoding features with model hidden layer dimensions. Mapping matrix It is trained through a pre-built alignment task of coding anomaly degree and semantic perturbation degree, which is learned on a simulated coding perturbation dataset by stochastic gradient descent.

[0041] S1022. Extract the weight matrices of multiple attention mechanism branches in the intermediate layer, calculate the product of the weight matrices and the preset safety weights to obtain the association strength value, and perform a linear projection transformation on the association strength value to obtain the instruction attention feature.

[0042] The intermediate layer refers to a deep neural network structure located between the embedding layer and the output layer, consisting of multiple stacked Transformer modules, primarily responsible for semantic understanding and logical reasoning. The weight matrix is ​​a parameter matrix used in the intermediate layer's attention mechanism to measure the degree of mutual attention between word units. .

[0043] The association strength value quantifies the model's attention to sensitive or dangerous concepts. The instruction attention feature indicates whether the model's internal attention mechanism focuses on deep semantic vectors of potential attack types. Preset security weights define the response weights of different levels and attention heads to security-sensitive information. The preset security weights are shown in Table 1 below. Table 1: Preset Security Weights Comparison Table

[0044] As shown in Table 1, the table shows the preset security weight configuration relationship. The hierarchical index and attention head index locate the specific focus within the model, the sensitivity type describes the potential risk category corresponding to the focus, and the security weight quantifies the importance of the focus in anomaly detection.

[0045] The preset security weights are determined in advance through ablation experiments. By blocking specific attention heads in the validation set and observing the reduction in recognition rate of privacy-related samples, the top N attention heads with the largest reduction are marked as security-sensitive heads and assigned high weight coefficients.

[0046] In practice, the weight matrix of a specific attention head in the intermediate layer is first extracted in real time during the model inference process. ,in for The matrix, elements This indicates the attention weight between word terms. Then, the corresponding security weight is retrieved from Table 1. ,calculate With safety weight The matrix dot product is calculated, and the norm of the resulting matrix is ​​extracted. The aggregation yields the correlation strength value. Finally, a linear projection layer is used to... Transformed into instruction attention features consistent with the hidden layer dimensions of the model. .

[0047] S1023. Extract the lexical probability vector generated by the output layer, and obtain the lexical distribution offset as an abnormal offset feature by calculating the dispersion of the lexical probability vector and the preset safe context vector.

[0048] The output layer is the final processing module in a large model responsible for mapping hidden layer states to vocabulary probability distributions. A lexical probability vector refers to the probability distribution of the next generated lexical term predicted by the model within the vocabulary. The lexical distribution offset refers to the degree of difference between the currently generated probability distribution and the safety baseline. Anomaly offset features are the vectorized representation of this offset and reflect the potential dangerous tendencies of the model-generated content. The preset safety context vector defines the baseline output probability distribution that the model should have under normal safety context. The preset safety context vectors are shown in Table 2 below: Table 2: Preset Security Context Vector Comparison Table

[0049] As shown in Table 2, Table 2 shows the preset safety context vector, including the standard probability distribution and allowable deviation range under different contexts.

[0050] In practice, a benchmark dataset is first constructed based on the normal user query requests collected in step S101. Inference is then performed on normal samples under each context ID, and the arithmetic mean of the word probability vectors generated in the output layer is calculated to determine the safe benchmark probability distribution corresponding to that context. Simultaneously calculate the probability vector of each sample in the benchmark dataset and... The statistical distribution of the distances between them is calculated by taking the mean plus three standard deviations. As the allowable deviation threshold .

[0051] During real-time detection, the word probability vectors generated by the model are first captured at the output layer. Next, based on the context ID in Table 2 of the current dialogue context, the corresponding pre-calculated safety baseline probability distribution is obtained. Then, the KL divergence or cosine similarity algorithm is used to calculate... and The distance between them yields the word distribution offset. Finally, Converting to vector form yields the abnormal offset features. .

[0052] For example, when the model processes an instruction that induces it to output a hazardous chemical formulation, the output layer generates a word probability vector. It tends to favor dangerous words, first matching them in "knowledge-based question and answer" contexts. Then, calculations revealed... Safety benchmarks in this context Significant differences exist, resulting in a large word distribution offset, which is then used to generate abnormal offset features. .

[0053] S1024. Perform dimension alignment on the abnormal coding features, instruction attention features, and abnormal offset features respectively to obtain multiple alignment vectors. Use cross-layer residual connections to superimpose and aggregate the multiple alignment vectors with the initial semantic vector of the input instruction to obtain the feature vector.

[0054] Dimension alignment refers to adjusting feature vectors from different sources and with different dimensions into a unified dimensional space through linear transformation to facilitate mathematical operations. Cross-layer residual connections are a feature fusion mechanism that incrementally stacks deeply extracted anomalous features while preserving the original input information to prevent gradient vanishing and enhance feature expressiveness. The final feature vector is a comprehensive representation vector that integrates multi-level endogenous security features.

[0055] In practice, the abnormal coding features obtained in the aforementioned steps are first processed using a multilayer perceptron. Instruction attention characteristics and abnormal offset features Perform dimension mapping separately, so that its dimensions correspond to the initial semantic vector of the input instruction. To maintain consistency, we obtain three aligned vectors. .

[0056] Next, a cross-layer residual connection structure is applied to connect these three alignment vectors with the initial semantic vector. Element-wise addition is performed. Finally, after normalization, a feature vector that integrates multi-source heterogeneous information is obtained. For example, for a complex attack command, firstly, different levels of features are obtained through processing, and then the formula is used... Aggregation is performed, where This represents a normalization operation used to remap features to a stable numerical manifold distribution after vector stacking. The final feature vector is obtained. .

[0057] This embodiment effectively intercepts character obfuscation-based bypass attacks, achieving end-to-end monitoring from surface input to deep inference and output tendencies. While preserving the original semantics, it significantly enhances the ability to represent concealed malicious intent, effectively solving the problem that single-level features are insufficient to identify complex nested attacks.

[0058] Optionally, the method further includes: The frequency of occurrence of non-standard character encoding sequences in the abnormal encoding features of the embedding layer is calculated, and the encoding dispersion is obtained based on the deviation between the occurrence frequency and the preset reference frequency.

[0059] The preset baseline frequency refers to the statistical average probability of non-standard characters, such as special symbols and escape characters, appearing in input text under normal language interaction scenarios. Encoding dispersion refers to the quantified value of the difference between the actual distribution of non-standard characters in the current input instruction and the normal baseline. The preset baseline frequency defines the distribution characteristics of normal characters under different language types or application scenarios. The preset baseline frequencies are shown in Table 3 below: Table 3: Preset Reference Frequency Comparison Table

[0060] As shown in Table 3, the preset baseline frequency configuration is shown. The scenario ID identifies the context category to which the input command belongs, the allowed character set defines the range of compliant characters in the scenario, and the baseline frequency value gives the upper limit of the normal probability of non-standard characters appearing.

[0061] In practice, the scene ID in Table 3 is first matched based on the context features of the input command, such as... Obtain the corresponding reference frequency value Next, the actual frequency of non-standard character encoding sequences in the input commands was statistically analyzed. Then, using the formula The deviation between the actual frequency and the reference frequency is calculated to obtain the coding dispersion. .

[0062] The encoding dispersion is compared with a preset security trigger threshold. If the encoding dispersion is greater than the security trigger threshold, the instruction attention features and abnormal offset features of the word probability distribution are extracted from the intermediate layer and the output layer, respectively.

[0063] The preset security trigger threshold is a critical value used to determine whether to activate the deep anomaly detection mechanism, aiming to balance security detection coverage and system resource consumption. This threshold is set in stages according to different security level requirements. The preset security trigger threshold defines the sensitivity standard for activating deep defense under different threat environments. The preset security trigger thresholds are shown in Table 4 below: Table 4: Preset Security Trigger Threshold Comparison Table

[0064] As shown in Table 4, the preset security trigger threshold configuration is displayed. The security level corresponds to the current protection status, and the trigger threshold is the criterion for activating the corresponding response strategy.

[0065] In practice, the current security level configuration is first obtained, such as... The corresponding trigger threshold is obtained by looking up the table. Next, the previously calculated coding discreteness will be used... Compare the values ​​with this threshold. If If the input command is deemed to have a high risk of anomaly, steps S1022 and S1023 are triggered to extract command attention features from the intermediate layer. Extracting anomalous offset features from the output layer And continue to execute the subsequent integration process.

[0066] Conversely, if the encoding dispersion does not exceed the threshold, the input instruction is determined to be a low-risk routine request, and the real-time feature calculation of the intermediate and output layers is skipped. In this low-risk mode, the feature vector is generated by aligning and superimposing the abnormal encoding feature vector extracted from the embedding layer with a preset zero matrix, which corresponds to the missing deep feature components.

[0067] This embodiment avoids performing indiscriminate full-link deep detection on all inputs by introducing coding discreteness calculation and hierarchical triggering mechanism, which significantly reduces the consumption of computing resources and response latency under normal requests. At the same time, it ensures that deep defense capabilities can be activated in time when facing complex attacks that are masked by abnormal coding, thus achieving a dynamic balance between performance and security.

[0068] S103. Perform semantic disambiguation and feature extraction on the multimodal attack samples with added noise interference to obtain an attack vector library. Then, use a linear classifier to compare and identify the feature vectors in the attack vector library to obtain attack type labels and attack target types. Determine the threat level based on the attack target type.

[0069] Optionally, step S103 performs semantic disambiguation and feature extraction on the multimodal attack samples with added noise interference to obtain an attack vector library. A linear classifier is then used to compare and identify the feature vectors in the attack vector library to obtain attack type labels and attack target types. The process of determining the threat level based on the attack target type may specifically include: S1031. By removing ambiguities from the multimodal attack samples, disambiguated multimodal attack samples are obtained, and multiple attack feature components are extracted from the disambiguated multimodal attack samples to obtain an attack vector library including multiple attack feature components.

[0070] Ambiguous terms refer to data segments in multimodal attack samples where the attack type is unclear due to ambiguous context, puns, or noise interference. Disambiguated multimodal attack samples refer to a high-quality set of attack samples that have undergone cleaning and semantic declarative processing. Attack feature components are numerical feature vectors extracted from the disambiguated samples that represent specific attack patterns. The attack vector library is a structured database composed of these attack feature components and their corresponding label information.

[0071] In specific implementation, the multimodal attack sample set obtained in step S101 and with added noise interference is first processed. Preprocessing is performed, using natural language processing techniques to identify and remove samples with semantic ambiguity or multiple interpretations, resulting in a clean sample set. Next, using the same feature extraction algorithm as in step S102, from... Each sample in Extract the corresponding feature vectors and generate multiple attack feature components. .

[0072] Finally, all extracted attack feature components are aggregated to construct an attack vector library. For example, for samples containing the word "apple," if it could refer to both fruit and technology companies in the context and has no clear attack target, it is removed as an ambiguous term; for samples that clearly induce privacy leaks, their feature components are extracted and stored in the database.

[0073] S1032. Calculate the difference in numerical distribution between the feature vector and each attack feature component in the attack vector library using a linear classifier, and classify the feature vector according to the difference in numerical distribution to obtain the attack type label and attack target type.

[0074] A linear classifier is a statistical classification model based on a linear prediction function. It determines the class by calculating the inner product of the input vector and the weight vector, and can include logistic regression or linear support vector machines. Numerical distribution difference refers to the distance or similarity measure between a feature vector in the feature space and each benchmark vector in the attack vector library. Attack type labels are classification identifiers for attack methods, such as tooltip injection or code execution. Attack target type is a classification identifier for the specific asset or system function targeted by the attack, such as user privacy or system integrity.

[0075] In practice, the first step is to construct a set of multimodal attack samples. The training dataset consists of positive samples and normal user query requests (negative samples). The feature vector extracted in step S1024 is used as the input feature, and the attack type label is used as the target variable. A linear classifier model is then constructed. And adopt the cross-entropy loss function As the optimization objective, among which For real labels, To predict probabilities, the stochastic gradient descent (SGD) optimizer is used. Perform iterative training until the accuracy on the validation set converges, thus obtaining a well-trained linear classifier model. .

[0076] Next, the feature vector generated in step S1024 is... Input into the trained In this process, the classifier outputs the logistic regression score of the feature vector for each known attack category, and uses this score vector as an index to locate the feature centroid of the corresponding category in the attack vector database. The cosine distance between the feature vector and the centroid is then calculated as the numerical distribution difference. Subsequently, based on the decision boundary of the classifier, the numerical distribution difference is selected. The smallest category will Mapped to the corresponding attack type tag and attack target type .

[0077] S1033. Based on the attack target type, the threat level is obtained according to the preset mapping relationship between attack target type and threat level.

[0078] The pre-defined mapping relationship between attack target types and threat levels refers to a logical rule that classifies attack targets into different security response levels based on their sensitivity and potential harm. Threat levels are categorized as graded descriptions of attack severity and typically include low, medium, high, and severe levels. The pre-defined mapping relationship between attack target types and threat levels is shown in Table 5 below: Table 5: Mapping Relationship Between Attack Target Type and Threat Level

[0079] As shown in Table 5, Table 5 illustrates the potential harm and preset threat levels corresponding to different attack target types. In specific implementation, firstly, the attack target type output in step S1032 is obtained. And identify its corresponding target type ID, such as Next, a search and match is performed in Table 5 to retrieve the threat level field corresponding to the ID.

[0080] Finally, using the formula The obtained level will be determined as the final threat level for the current input command. For example, when a linear classifier determines that the attack target is the theft of privacy data, that is... At that time, a table showed that the potential harm was the leakage of user information, so the threat level of this attack was determined to be high.

[0081] This embodiment enables rapid classification of unknown or disguised attack types, quantifies the severity of the attack target, and accurately determines the threat level.

[0082] S104. Using attack type tags and threat levels as query conditions, the system uses graph traversal and multi-target optimization to filter out matching target defense strategy nodes in the knowledge graph and obtain the defense path.

[0083] Optionally, step S104, which uses attack type tags and threat levels as query conditions, and employs graph traversal and multi-target optimization to filter out matching target defense strategy nodes in the knowledge graph to obtain the defense path, may specifically include: S1041. Using attack type label and threat level as query conditions, perform path traversal in the knowledge graph to determine the starting node that matches the attack type label.

[0084] Path traversal is a search algorithm for finding specific nodes in graph-structured data and is used to locate entry points in knowledge graphs. The starting node refers to the node in the attack-defense knowledge graph that directly corresponds to the identified attack type and serves as the starting point for searching defense strategies.

[0085] In specific implementation, the attack type label output in step S103 is received first. Such as prompts for injection and threat levels. For example, the knowledge graph constructed in step S101. The algorithm executes a node matching process, traversing all attack type nodes in the graph. Finally, the selection was made based on... Semantic consistency and threat level attribute not lower than Mark the node as the starting node. .

[0086] S1042. Starting from the starting node, perform a breadth-first search along the defense relationships to obtain multiple candidate defense strategy nodes that have defense associations with the starting node.

[0087] Breadth-first search is a graph traversal algorithm that starts from a starting node and expands outwards to visit its neighboring nodes, and is used to find all directly related effective defense measures. Candidate defense strategy nodes refer to the set of defense measures nodes in the knowledge graph that have direct or indirect connections to attack type nodes and may effectively resist that type of attack.

[0088] In specific implementation, the starting node determined in step S1041 is used first. Centered on this, activate its defensive relation edges in the knowledge graph. Next, a breadth-first search is performed, visiting all nodes directly connected by directed edges. The defense strategy nodes are then determined. Subsequently, based on preset search depth limits, such as searching only directly related 1-hop nodes or a combined strategy extending to 2 hops, all visited nodes are collected to form a candidate defense strategy node set. .

[0089] S1043. Extract the historical interception rate and resource consumption value corresponding to each candidate defense strategy node from the knowledge graph, and use Pareto optimization to perform a weighted balance calculation on the historical interception rate and resource consumption value corresponding to each candidate defense strategy node to obtain the evaluation score of each candidate defense strategy node, and determine the candidate defense strategy node whose evaluation score meets the preset security index as the target defense strategy node.

[0090] Historical interception rate refers to the proportion of attacks successfully blocked by a defense strategy in past confrontations. Resource consumption value refers to the computational overhead or latency required to execute the strategy. Pareto optimization is a multi-objective optimization method that aims to find the optimal set of solutions among multiple conflicting objectives such as security and efficiency.

[0091] The evaluation score is a quantitative assessment of the overall effectiveness of the defense strategy. The target defense strategy node is the defense measure ultimately selected for execution. Preset security indicators define the minimum effectiveness standards that the defense strategy must meet under different threat levels. The preset security indicators are shown in Table 6 below: Table 6: Preset Safety Indicators Comparison Table

[0092] As shown in Table 6, the table shows the preset security indicator configurations. The threat level is the classification basis for defense decisions. The minimum interception rate requirement, the maximum resource consumption threshold, and the comprehensive score threshold respectively specify the hard screening criteria for the defense strategy in terms of effectiveness, performance overhead, and comprehensive evaluation score under the corresponding level.

[0093] In practice, each candidate node is first read from the attributes of the knowledge graph. Historical interception rate and resource consumption value Then, based on the current threat level... The corresponding security metrics are obtained from Table 6. Then, a Pareto optimization process is executed, first selecting the non-dominated solution set that is not dominated by other nodes in both interception rate and resource consumption dimensions. For each node in this non-dominated solution set, a weighted scoring function is used. Calculate its assessment score in the current threat environment. ,in These are the weighting coefficients. Finally, Above the overall score threshold The nodes were filtered out and determined as the target defense strategy node set. .

[0094] S1044. Arrange all target defense strategy nodes in a temporal sequence according to the defense relationship between each target defense strategy node in the knowledge graph to obtain the defense path.

[0095] Timing orchestration refers to determining the execution order of defense strategies based on logical dependencies, such as purging before inference or parallel detection. A defense path refers to the complete defense execution flow consisting of multiple target defense strategy nodes connected in series or parallel according to a specific timing sequence.

[0096] In practice, the first step is to analyze the target defense strategy node set. The connections between nodes in the knowledge graph and the execution stage tags in the metadata, such as Pre-process, In-process, and Post-process, are then processed according to the input. Reasoning intervention The nodes are sorted according to the standard logical order of the output audit. Finally, an ordered sequence of nodes is generated to construct the defense path. .

[0097] For example, assuming the selected target node includes input semantic cleansing and thought chain enforcement checks, then according to the orchestration rules, semantic cleansing is performed first, followed by thought chain checks during the inference phase, resulting in the following defense path: .

[0098] This embodiment not only quickly identifies effective defense methods, but also achieves the best balance between security performance and resource consumption through Pareto optimization. The final generated defense path has both a targeted high interception rate and avoids unnecessary overhead, realizing precise dynamic orchestration of defense strategies.

[0099] S105. Determine the security mode level of the large model based on the threat level, configure the large model using the defense path and security mode level, and then output the security output information and defense execution log corresponding to the input and output commands.

[0100] Optionally, step S105, which determines the security mode level of the large model based on the threat level, and configures the large model using the defense path and security mode level, and then outputs the security output information and defense execution log corresponding to the input commands, may specifically include: S1051. Based on the threat level, the security mode level is obtained according to the preset mapping relationship between threat level and security response level.

[0101] Security mode level refers to a set of predefined operational constraints in which a large model operates during inference, determining the model's openness, rigor, and level of security intervention when generating content. The preset mapping relationship between threat levels and security response levels defines the overall security response strategy to be adopted under different external threat intensities. The preset mapping relationship between threat levels and security response levels is shown in Table 7 below: Table 7: Mapping Relationship between Threat Level and Security Response Level

[0102] As shown in Table 7, Table 7 shows the security response levels and specific mode characteristics corresponding to different threat levels. The threat level ID corresponds to the result of step S103, and the security response level specifies the template for subsequent parameter configuration.

[0103] In practice, the threat level determined in step S103 is received first. High threat response Next, we consult Table 7 and match the corresponding security response level based on the mapping relationship. Finally, this response level is determined as the security mode level for the current processing cycle. .

[0104] S1052. Adjust the sampling probability threshold and word selection randomness of the large model according to the security mode level to obtain the model configuration parameters.

[0105] The sampling probability threshold, or Top-P, refers to sampling only from the smallest candidate set whose cumulative probability exceeds the threshold when generating the next lexical term, used to control the diversity of generated content. Lexical selection randomness is a hyperparameter used to adjust the smoothness of the Softmax output probability distribution; the lower the value, the more deterministic and conservative the generated results. Model configuration parameters refer to the set including the adjusted hyperparameters and other runtime settings.

[0106] In specific implementation, the security mode level determined in step S1051 is first used as the basis. Then, it calls the preset parameter adjustment function or looks up the configuration table. Next, it adjusts the sampling probability threshold for the large model. and the randomness of word selection Adjust the values ​​accordingly. Generally, as the security level increases, the values ​​need to be decreased. To reduce hallucinations and randomness, while adjusting This narrows the range of candidate words. The final result includes specific... and Numerical model configuration parameters .

[0107] S1053. According to the arrangement sequence of each target defense strategy node in the defense path, the input instructions are semantically reconstructed and lexical filtered in the running environment determined by the model configuration parameters to obtain the instruction sequence.

[0108] Semantic reconstruction refers to the process of forcing the model into a safe inference path by inserting security system hints into the original input, rewriting query logic, or adding thought chains. Lexical filtering refers to the operation of masking or replacing dangerous keywords in the input based on blacklists or sensitive word libraries. The instruction sequence refers to the context token sequence that is finally input into the large model for inference after the above series of preprocessing.

[0109] In practice, firstly, the defense path generated in step S104 is loaded. and the model configuration parameters generated in step S1052 Next, according to The defense strategy nodes along the path are executed sequentially according to the temporal relationship. If a node is for input semantic cleansing, then the input instructions are processed accordingly. Lexical filtering is performed; if the node is a thought chain, semantic reconstruction is performed, and a guiding prefix of "Please analyze security before answering" is added before the instruction. Finally, the processed content is encapsulated into an instruction sequence that conforms to the model input format. .

[0110] S1054. Use the large model to perform logical reasoning on the instruction sequence, output security output information, and determine the interception trigger result and running parameter offset of each target defense strategy node in the logical reasoning as the defense execution log.

[0111] Security output information refers to the final response generated by the large model after defense configuration and command processing, and this content has eliminated potential risks. Interception trigger result refers to the status record of whether the defense policy node successfully identified and blocked a specific risk point during execution, such as "intercepted" or "not triggered."

[0112] Runtime parameter offset refers to the change in the numerical value of the attention weights in key layers, such as intermediate layers, relative to the undefended state after the introduction of a defense strategy. The defense execution log is a structured record that summarizes all the above runtime state data.

[0113] In practice, firstly, the instruction sequence... Input configuration parameters The large model is used for inference. Then, the defense path is monitored in real time during the inference process. The actions of each node are recorded, including whether the output layer filter truncates the generated stream. Simultaneously, the difference between the model's actual runtime internal state and the baseline state is calculated to obtain the runtime parameter offset. Finally, the large model outputs text as security output information. And package all monitoring data to generate defense execution logs. .

[0114] This embodiment achieves dual reinforcement from the underlying parameters of the model to the upper-level semantic logic, while recording detailed defense execution logs to ensure the security of each generation.

[0115] S106. Generate adversarial use cases using the attack vector library and calculate the interception rate of the defense path in the sandbox based on the threat level, security output information and defense execution log. Correct the parameters of the intermediate layer based on the interception rate and update the weights of the target defense strategy nodes that make up the defense path in the knowledge graph to obtain weight deviation data. Determine the defense evaluation score based on the weight deviation data.

[0116] Optionally, step S106 generates adversarial use cases using the attack vector library and calculates the interception rate of the defense path within the sandbox based on the threat level, security output information, and defense execution logs. Based on the interception rate, the parameters of the intermediate layer are corrected, and the weights of the target defense strategy nodes constituting the defense path in the knowledge graph are updated to obtain weight deviation data. The process of determining the defense evaluation score based on the weight deviation data can specifically include: Figure 3 A flowchart illustrating a method for determining defense assessment scores according to an embodiment of this application is shown. Figure 3 As shown, when performing a defense effectiveness assessment, the attack feature components at the bottom layer are first extracted based on the attack vector library, and then a set of adversarial use cases with diversity and concealment are constructed in batches by applying attribute mutation and random perturbation mechanisms.

[0117] Subsequently, the aforementioned adversarial test cases are imported one by one into an isolated sandbox environment to conduct logical reasoning in order to capture the test interception records corresponding to each test case. Simultaneously, corresponding security comparison thresholds are established based on the current preset threat level, and the semantic distance between the sandbox output content and the baseline security output information is quantitatively calculated.

[0118] To accurately measure the actual interception effectiveness of the defense path, a dual-condition judgment mechanism is introduced: for each adversarial use case, it is determined whether its semantic distance strictly falls within the security comparison threshold, and simultaneously, it is verified whether its test interception record matches the interception trigger result recorded in the defense execution log. Only when both conditions are met is the adversarial use case counted as a valid interception. After completing the traversal and filtering of all adversarial use cases, the current overall interception rate is derived by calculating the ratio of the number of valid interceptions to the total number of generated cases.

[0119] Furthermore, the interception rate is differentially calculated with a preset interception threshold to extract a feedback error value representing the gap in defense performance. Guided by this error value, gradient correction is applied to the parameters of the network's intermediate layer, and the contribution of each target defense strategy node in the knowledge graph is redistributed along the defense relationship topology, thereby outputting quantified weighted bias data. Finally, this weighted bias data is standardized and weighted summed to obtain a defense evaluation score that objectively represents the robustness of the current defense system. This method includes: S1061. Extract attack feature components from the attack vector library, and perform attribute mutation and random perturbation on the attack feature components to obtain adversarial use cases.

[0120] Attribute mutation refers to replacing, recombining, or expanding key attributes such as keywords, syntactic structures, and logical connectors in attack feature components based on rule-based or model-based generation algorithms to generate new samples with similar attack types but different manifestations. Random perturbation refers to adding small noise, such as Gaussian noise, to the feature vector values ​​to simulate input fluctuations in a real environment. Adversarial use cases refer to simulated attack samples generated after the above processing, designed to test the accuracy of defense strategies.

[0121] In practice, the first step is to use the attack vector library. Randomly select several attack feature components Then, gradient-based adversarial generative networks or simple synonym substitution rules are used to... Perform attribute mutation to generate mutated features. Meanwhile, in Superimposed random perturbations that follow a normal distribution This yields the final adversarial use case feature vector. Finally, Directly used as feature input to form a set of adversarial use cases .

[0122] S1062. Input the adversarial test cases into the sandbox for logical reasoning to obtain the test interception records of the adversarial test cases. Determine the security comparison threshold based on the threat level and calculate the semantic distance between the output content of the adversarial test cases and the security output information.

[0123] A sandbox is an isolated virtual execution environment used to securely run adversarial test cases without affecting the stability of the main system. Test intercept logs are logs recording whether the defense system triggers intercept actions when adversarial test cases are run in the sandbox. The security comparison threshold is the maximum allowed semantic difference between the output of an adversarial test case and known secure output; this threshold varies with the threat level. Semantic distance is a metric that measures the degree of difference between two texts or vectors in the semantic space.

[0124] In practice, the first step is to generate a set of adversarial test cases. Each sandbox environment is entered individually, and the sandbox is configured with the same defense paths as the main system. and model parameters Next, record the test interception results for each test case. Such as interception or permission. Simultaneously, based on the current threat level. Determine the security comparison threshold And usually, the higher the threat level, the more... The smaller the value. Then, obtain the output of the adversarial test cases in the sandbox. Calculate its relationship with the security output information generated by the main system. The cosine distance between them is used to obtain the semantic distance. .

[0125] S1063. Calculate the number of adversarial test cases whose statistical semantic distance is within the security comparison threshold and whose test interception records match the interception trigger results in the defense execution log, and obtain the interception rate based on the ratio of the number to the total number of adversarial test cases.

[0126] The interception trigger result in the defense execution log refers to the interception status of the adversarial use case in the sandbox, i.e., whether it is intercepted or not, which is consistent with the defense expectation of the main system when processing the original input. The interception rate refers to the proportion of times the defense system can still maintain a correct defense response when facing variant attacks.

[0127] In practice, the test results of all adversarial test cases are first iterated through. Then, test cases that meet the conditions are selected. and With defense execution log Count the number of use cases that match the expected actions recorded in the database. Finally, calculate the interception rate. ,in To counter the total number of use cases.

[0128] S1064. Calculate the difference between the interception rate and the preset interception threshold to obtain the feedback error value. Based on the feedback error value, perform gradient correction on the intermediate layer parameters and redistribute the contribution of the target defense strategy nodes according to the defense relationship to obtain the weight deviation data.

[0129] The preset interception threshold refers to the minimum expected value of the defense success rate. The feedback error value is the difference between the actual interception rate and the expected threshold, and is used to drive parameter adjustment. Gradient correction and contribution redistribution refer to fine-tuning the internal parameters of the model and the weight values ​​of the defense strategy nodes in the knowledge graph based on the error backpropagation idea. The weight deviation data is a numerical record of the above adjustments. The preset interception threshold is shown in Table 8 below: Table 8: Preset Interception Threshold Comparison Table

[0130] As shown in Table 8, the minimum expected interception rate and corresponding parameter adjustment step size are displayed for different threat levels. The threat level serves as the performance benchmark for the defense strategy, the minimum expected interception rate is used to measure whether the defense meets the target, and the parameter adjustment step size controls the magnitude of the model parameter change during each feedback correction.

[0131] In practice, the first step is to determine the threat level. Refer to Table 8 to obtain the preset interception threshold. Next, we define the feedback optimization loss function. The feedback error value is calculated with the goal of minimizing the difference between the actual interception rate and the expected threshold. And based on the chain rule, the loss function with respect to the intermediate layer weight matrix is ​​estimated. gradient .

[0132] Use this gradient value in conjunction with adjusting the step size Fine-tune intermediate layer parameters such as attention weights: ,in This ensures that parameter updates are performed in a way that reduces interception rate errors. Simultaneously, based on the activation frequency and interception contribution of each defense strategy node in the defense path, a weighted average method is used to update the weights of each strategy node in the graph within the defense path. Increase the weight of nodes with large contributions, decrease the weight of ineffective nodes, and generate weight bias data. .

[0133] S1065. By standardizing and mapping the weighted bias data and performing weighted summation, the defense assessment score is obtained.

[0134] Standardization mapping refers to converting weighted bias data with different dimensions and units into a unified scoring range, such as 0 to 100. The defense assessment score is the final score that comprehensively reflects the effectiveness of the current defense system under a specific attack and its adaptive adjustment range.

[0135] In practice, all weight deviation data generated in step S1064 are first collected. This includes changes in model parameter corrections and changes in map weights. Next, the Sigmoid or Min-Max normalization function is used to... The data is processed to fall within a standard range. Finally, the standardized data is weighted and summed to calculate the defense assessment score. ,in This represents the weighting coefficient.

[0136] This embodiment constructs a closed-loop adversarial exercise and feedback mechanism, uses a sandbox environment to securely evaluate the defense effect, and dynamically adjusts model parameters and graph weights based on a quantified interception rate index, thereby achieving adaptive evolution of defense capabilities and ensuring that a high level of security protection is maintained in the face of constantly evolving attack methods.

[0137] Figure 4 This application provides a schematic diagram of a specific implementation of a large-model intrinsic security-driven defense strategy evaluation system for resisting multiple attacks, as shown in the embodiments of this application. Figure 4 The system may include: The acquisition module 410 is used to acquire multimodal attack samples and a knowledge graph with attack type and defense strategy as nodes and defense relationship as edges; The generation module 420 is used to extract abnormal coding features, instruction attention features and abnormal offset features of word probability distribution from the embedding layer, intermediate layer and output layer respectively during the processing of input instructions in the large model, and fuse them through cross-layer residual connections to obtain feature vectors; The generation module 420 is also used to perform semantic disambiguation and feature extraction on multimodal attack samples with added noise interference to obtain an attack vector library. A linear classifier is used to compare and identify the feature vectors in the attack vector library to obtain attack type labels and attack target types. The threat level is determined based on the attack target type. The filtering module 430 is used to filter out matching target defense strategy nodes in the knowledge graph by using attack type tags and threat levels as query conditions, and to obtain the defense path by using graph traversal and multi-target optimization. Output module 440 is used to determine the security mode level of the large model based on the threat level, and after configuring the large model using the defense path and security mode level, output the security output information and defense execution log corresponding to the input command. The generation module 420 is also used to generate adversarial use cases using the attack vector library and calculate the interception rate of the defense path in the sandbox based on the threat level, security output information and defense execution log. Based on the interception rate, the parameters of the intermediate layer are corrected and the weights of the target defense strategy nodes that make up the defense path in the knowledge graph are updated to obtain weight deviation data. The defense evaluation score is determined based on the weight deviation data.

[0138] The large-model intrinsic security-driven defense strategy evaluation system for resisting multiple attacks in this application embodiment is used to implement the aforementioned large-model intrinsic security-driven defense strategy evaluation method for resisting multiple attacks. Therefore, the specific implementation of the large-model intrinsic security-driven defense strategy evaluation system for resisting multiple attacks can be found in the embodiment section of the large-model intrinsic security-driven defense strategy evaluation method for resisting multiple attacks mentioned above. The specific implementation can be referred to the description of the corresponding embodiment, and will not be repeated here.

[0139] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in one embodiment of this application is shown.

[0140] The electronic device may include a processor 510 and a memory 520 storing computer program instructions.

[0141] Specifically, the processor 510 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0142] Memory 520 may include mass storage for data or instructions. For example, and not limitingly, memory 520 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 520 may include removable or non-removable (or fixed) media. Where appropriate, memory 520 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 520 is non-volatile solid-state memory.

[0143] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to the first aspect of this disclosure.

[0144] The processor 510 reads and executes computer program instructions stored in the memory 520 to implement any of the large-scale model-driven defense strategy evaluation methods against multiple attacks in the above embodiments.

[0145] In one example, the electronic device may also include a communication interface 530 and a bus 540. Wherein, such as Figure 5 As shown, the processor 510, memory 520, and communication interface 530 are connected through bus 540 and complete communication with each other.

[0146] The communication interface 530 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0147] Bus 540 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 540 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0148] The electronic device can execute the large-model intrinsic security-driven defense strategy evaluation method against multiple attacks as described in the embodiments of this application, thereby realizing the large-model intrinsic security-driven defense strategy evaluation method against multiple attacks as described in conjunction with the accompanying drawings.

[0149] Furthermore, in conjunction with the large-model intrinsic security-driven defense strategy evaluation method against multiple attacks in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement any one of the large-model intrinsic security-driven defense strategy evaluation methods against multiple attacks in the above embodiments.

[0150] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0151] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0152] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0153] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0154] The foregoing has provided a detailed description of the evaluation method and system for a defense strategy against multiple attacks driven by inherent security in a large model, as provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for evaluating defense strategies against multiple attacks driven by the intrinsic security of a large model, characterized in that, include: Obtain multimodal attack samples and a knowledge graph with attack type and defense strategy as nodes and defense relationship as edges; During the large model's processing of input instructions, abnormal coding features, instruction attention features, and abnormal offset features of word probability distribution are extracted from the embedding layer, intermediate layer, and output layer, respectively, and fused through cross-layer residual connections to obtain feature vectors; Semantic disambiguation and feature extraction are performed on the multimodal attack samples with added noise interference to obtain an attack vector library. A linear classifier is then used to compare and identify the feature vectors in the attack vector library to obtain attack type labels and attack target types. The threat level is determined based on the attack target types. Using the attack type label and the threat level as query conditions, graph traversal and multi-target optimization are used to filter out matching target defense strategy nodes in the knowledge graph to obtain the defense path; Based on the threat level, the security mode level of the large model is determined. After configuring the large model using the defense path and the security mode level, the security output information and defense execution log corresponding to the input command are output. The attack vector library is used to generate adversarial use cases, and the interception rate of the defense path is calculated in the sandbox based on the threat level, the security output information and the defense execution log. The parameters of the intermediate layer are corrected based on the interception rate, and the weights of the target defense strategy nodes that make up the defense path in the knowledge graph are updated to obtain weight deviation data. The defense evaluation score is determined based on the weight deviation data.

2. The method according to claim 1, characterized in that, The method further includes: Calculate the frequency of occurrence of non-standard character encoding sequences in the abnormal encoding features of the embedding layer, and obtain the encoding dispersion based on the deviation between the occurrence frequency and the preset reference frequency; The encoding dispersion is compared with a preset security trigger threshold. If the encoding dispersion is greater than the security trigger threshold, the instruction attention feature and the abnormal offset feature of the lexical probability distribution are extracted from the intermediate layer and the output layer, respectively.

3. The method according to claim 1, characterized in that, In the process of processing input instructions in the large model, abnormal coding features, instruction attention features, and abnormal offset features of word probability distribution are extracted from the embedding layer, intermediate layer, and output layer, respectively, and fused through cross-layer residual connections to obtain a feature vector, including: The non-standard encoded sequence of the input instruction is extracted in the embedding layer. The frequency of occurrence of non-standard characters in the non-standard encoded sequence is calculated to obtain the encoding anomaly value. The encoding anomaly value is then vector-mapped to obtain the abnormal encoding feature. Extract the weight matrices of multiple attention mechanism branches in the intermediate layer, calculate the product of the weight matrices and the preset safety weights to obtain the association strength value, and perform a linear projection transformation on the association strength value to obtain the instruction attention feature; Extract the lexical probability vector generated by the output layer, and obtain the lexical distribution offset as the abnormal offset feature by calculating the discreteness between the lexical probability vector and the preset safety context vector; The abnormal coding features, the instruction attention features, and the abnormal offset features are dimensionally aligned to obtain multiple alignment vectors. The multiple alignment vectors are then superimposed and aggregated with the initial semantic vector of the input instruction using the cross-layer residual connection to obtain the feature vector.

4. The method according to claim 1, characterized in that, The process involves semantic disambiguation and feature extraction of the multimodal attack samples with added noise interference to obtain an attack vector library. A linear classifier is then used to compare and identify the feature vectors within this library to obtain attack type labels and attack target types. The threat level is determined based on the attack target types, including: By removing ambiguities from the multimodal attack samples, disambiguated multimodal attack samples are obtained, and multiple attack feature components are extracted from the disambiguated multimodal attack samples to obtain an attack vector library including multiple attack feature components. The linear classifier is used to calculate the difference in numerical distribution between the feature vector and each attack feature component in the attack vector library, and the feature vector is classified according to the difference in numerical distribution to obtain the attack type label and the attack target type; Based on the attack target type, the threat level is obtained according to the preset mapping relationship between attack target type and threat level.

5. The method according to claim 1, characterized in that, The process of using the attack type label and the threat level as query conditions, and employing graph traversal and multi-target optimization to filter out matching target defense strategy nodes in the knowledge graph to obtain the defense path includes: Using the attack type label and the threat level as query conditions, a path traversal is performed in the knowledge graph to determine the starting node that matches the attack type label; Starting from the starting node, perform a breadth-first search along the defense relationship to obtain multiple candidate defense strategy nodes that have a defense association with the starting node; The historical interception rate and resource consumption value corresponding to each candidate defense strategy node are extracted from the knowledge graph. Pareto optimization is used to perform a weighted balance calculation on the historical interception rate and resource consumption value corresponding to each candidate defense strategy node to obtain the evaluation score of each candidate defense strategy node. The candidate defense strategy node whose evaluation score meets the preset security index is determined as the target defense strategy node. Based on the defense relationships between each target defense strategy node in the knowledge graph, all target defense strategy nodes are sequentially arranged to obtain the defense path.

6. The method according to claim 1, characterized in that, The process involves determining the security mode level of the large model based on the threat level, configuring the large model using the defense path and the security mode level, and then outputting the security output information and defense execution log corresponding to the input command, including: Based on the threat level, the security mode level is obtained according to the preset mapping relationship between threat level and security response level; The sampling probability threshold and word selection randomness of the large model are numerically adjusted according to the security mode level to obtain the model configuration parameters; According to the arrangement sequence of each target defense strategy node in the defense path, the input command is semantically reconstructed and word filtering is performed in the running environment determined according to the model configuration parameters to obtain the command sequence. The large model is used to perform logical reasoning on the instruction sequence, output the security output information, and the interception trigger result and running parameter offset of each target defense strategy node in the logical reasoning are determined as the defense execution log.

7. The method according to claim 6, characterized in that, The process involves generating adversarial use cases using the attack vector library, calculating the interception rate of the defense path within a sandbox based on the threat level, the security output information, and the defense execution logs, correcting the parameters of the intermediate layer based on the interception rate, updating the weights of the target defense strategy nodes constituting the defense path in the knowledge graph, obtaining weight deviation data, and determining the defense evaluation score based on the weight deviation data, including: The attack feature components are extracted from the attack vector library, and the attack feature components are subjected to attribute mutation and random perturbation to obtain the adversarial use case. The adversarial test cases are input into the sandbox for logical reasoning to obtain the test interception records of the adversarial test cases. The security comparison threshold is determined according to the threat level, and the semantic distance between the output content of the adversarial test cases and the security output information is calculated. The number of adversarial test cases whose semantic distance is within the security comparison threshold and whose test interception records match the interception trigger results in the defense execution log is counted, and the interception rate is obtained based on the ratio of the number to the total number of adversarial test cases; The difference between the interception rate and the preset interception threshold is calculated to obtain the feedback error value. The intermediate layer parameters are then corrected according to the feedback error value, and the contribution of the target defense strategy nodes is redistributed according to the defense relationship to obtain the weight deviation data. The defense assessment score is obtained by standardizing and weighting the weighted bias data.

8. A defense strategy evaluation system for resisting multiple attacks, driven by the intrinsic security of a large model, characterized in that, include: The acquisition module is used to acquire multimodal attack samples and a knowledge graph with attack types and defense strategies as nodes and defense relationships as edges; The generation module is used to extract abnormal coding features, instruction attention features, and abnormal offset features of word probability distribution from the embedding layer, intermediate layer, and output layer respectively during the processing of input instructions in the large model, and fuse them through cross-layer residual connections to obtain feature vectors; The generation module is also used to perform semantic disambiguation and feature extraction on the multimodal attack samples with added noise interference to obtain an attack vector library, and to use a linear classifier to compare and identify the feature vectors in the attack vector library to obtain attack type labels and attack target types, and to determine the threat level based on the attack target types. The filtering module is used to filter out matching target defense strategy nodes in the knowledge graph by using the attack type label and the threat level as query conditions, and then using graph traversal and multi-target optimization to obtain the defense path. The output module is used to determine the security mode level of the large model based on the threat level, configure the large model using the defense path and the security mode level, and then output the security output information and defense execution log corresponding to the input command. The generation module is also used to generate adversarial use cases using the attack vector library and calculate the interception rate of the defense path in the sandbox based on the threat level, the security output information and the defense execution log. Based on the interception rate, the parameters of the intermediate layer are corrected and the weights of the target defense strategy nodes that make up the defense path in the knowledge graph are updated to obtain weight deviation data. The defense evaluation score is determined based on the weight deviation data.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, used to implement the steps of the method for evaluating defense strategies against multiple attacks driven by large-scale intrinsic security as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the evaluation method for defense strategies against multiple attacks driven by the intrinsic security of a large model as described in any one of claims 1 to 7.

Images