Adaptive enhanced information security protection system for ai large model

Abstract

The application discloses an adaptive enhanced information security protection system for an AI large model, which comprises a model security management module, a model security running module and a model security knowledge base three modules, acquires AI large model object environment parameters and full life cycle vulnerability information, and realizes adaptive enhanced protection of the system full life cycle under the support of key technologies, auxiliary resources and model libraries. The application covers the model security requirements of confidentiality, integrity and availability in five stages of the full life cycle of the AI large model, adopts three technical routes of a dynamic defense strategy decision based on reinforcement learning, multi-stage model robustness enhancement based on deep reinforcement learning and provable privacy protection based on differential privacy and homomorphic encryption, realizes closed-loop protection of the AI large model, solves the limitation problem of focusing on single-point or local security in the existing AI information security protection, and provides a system-level solution for high-value AI system information security protection.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence information security technology, specifically relating to an adaptive enhanced information security protection system for large AI models. Background Technology

[0002] As large-scale AI models with hundreds of billions of parameters, such as GPT-4 and DeepSeek, enter the industrial application stage, while enhancing the intelligence of various vertical industries, the security threats faced by these models are growing exponentially. Industry data shows that as of March 2025, attacks targeting large-scale models have exploded, but security defense systems face severe challenges due to technological lags and rapid iteration of attack methods, leading to unprecedented security risks across industries. The number of security attacks on global AI systems has surged by 280% year-on-year, with attacks targeting large-scale model APIs accounting for as much as 65%. These attacks may not only cause model service interruptions but also trigger systemic risks such as data breaches and loss of control over content security.

[0003] The deployment of large-scale AI models from cloud-edge collaboration faces serious information security risks due to the vulnerability of cloud-edge communication. Therefore, the deployment of large-scale AI models is gradually evolving towards private deployment of AI all-in-one machines. The information security threats currently faced by large-scale AI models are characterized by multi-source nature, high concealment, and cross-domain transmission. From the perspective of the model service process, the input side faces multiple attack threats such as adversarial sample attacks, hint injection attacks, and format escape attacks; the output side faces security risks such as privacy leaks and loss of content security control; simultaneously, large-scale models face information security risks such as model theft attacks and hardware side-channel attacks at the system level. Furthermore, from the perspective of the entire lifecycle of large-scale AI models, the design, training, deployment, and operation and maintenance stages all face serious information security risks. Authoritative vulnerability databases such as CVE, CWE, and CNNVD have disclosed numerous high-risk security vulnerabilities related to the Ollam framework, Transformer library, and VMware in the software and hardware environment of large-scale AI models. The information security issues of large-scale AI models are urgent.

[0004] Traditional cybersecurity protection systems have several significant shortcomings when facing specific AI threats. For example, firewalls cannot identify semantic layer attacks, IDS systems lack the ability to detect adversarial examples, and WAF devices struggle to analyze covert attack patterns in multimodal inputs. Currently, most protection measures for large AI models do not address single-point attacks; they are mainly divided into two categories: data-level and model-level protection technologies. Data-level defense addresses information security threats such as data poisoning and privacy leaks through data cleaning, anomaly detection, differential privacy, and federated learning security aggregation. However, as data poisoning becomes increasingly sophisticated, dynamic data monitoring and causal reasoning are needed to optimize detection accuracy. Model-level protection addresses various attacks such as adversarial examples, backdoor implantation, and model reverse engineering through techniques like adversarial training, pruning, and interpretability enhancement. Protection methods for large AI models are fragmented and face bottlenecks such as limited detection dimensions, insufficient real-time performance, and lack of hardware isolation.

[0005] Overall, in the offensive and defensive game of AI large models, the defender is still in a passive and disadvantageous stage of "patching up gaps and strengthening". The information security defense of AI large models needs to consider the information security threats throughout the entire life cycle of the model and adopt system-level protection methods to help the safe deployment of large models. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to overcome the limitations of existing AI large-scale model protection methods that focus on single-point or local system security. It provides a global, systematic, and complete process for studying information security threats to AI large-scale models, covering the entire life cycle and service process of the large-scale model, and improving the survivability of the large-scale model under private deployment.

[0007] To achieve the above objectives, the present invention adopts the following solution:

[0008] This invention provides an adaptive enhanced information security protection system for large AI models, comprising: a model security management module, a model security operation module, and a model security knowledge base. The model security management module is used for in-depth security monitoring and analysis of the real-time operation logs of the large AI model and to deploy the defense strategies executed by the model security operation module. The model security operation module is used to perform information security protection throughout the entire process and lifecycle of the large model service. The model security knowledge base is used to provide the auxiliary resources required by the model security management module and the model security operation module.

[0009] Furthermore, the security management module includes a model log analysis unit and a dynamic defense strategy decision-making unit.

[0010] Furthermore, the model security operation module includes an input detection unit, a model operation hardening unit, and an output control unit. The input detection unit is used for multi-source intrusion detection during the model input stage, the model operation hardening unit is used for structural hardening against AI attacks during the model operation stage, and the output control unit is used for output detection and control during the model output stage.

[0011] Furthermore, the input detection unit performs network traffic intrusion detection and large-scale model threat detection tasks. The network traffic intrusion detection task includes establishing a dynamic authentication model for device identity based on the hardware device fingerprint of the model input party, combined with hash algorithms and physical non-cloning function technology, through a dynamic fingerprint update mechanism; and establishing a real-time network traffic threat detection model based on protocol deep parsing requirements, combined with spatiotemporal convolutional networks and traffic sampling technology, through protocol field full parsing and covert channel identification methods. The large-scale model threat detection task includes, based on the security requirements of the system's multimodal input, combining multimodal joint analysis algorithms and hardware / software acceleration frameworks, constructing a multimodal threat graph to dynamically detect input-side threats to the large model in real time.

[0012] Furthermore, the model hardening unit performs model adversarial malicious sample training and model architecture robustness tasks. The model adversarial malicious sample training task includes discovering known security vulnerabilities in the AI ​​large model's operating environment using vulnerability mining tools, deploying relevant defense strategies against identified security attacks, and implementing dynamic adversarial training through a phased perturbation constraint strategy based on course learning theory and an improved PGD algorithm and meta-learning framework. The model architecture robustness task includes constructing an adaptive feature selection model using an adversarial feature gating network based on adversarial perception theory and the model's security knowledge base, combined with dynamic attention masks and multi-head defense mechanisms, to select effective attack methods; and establishing a provably secure training framework through a dynamic budget allocation strategy by combining Gaussian gradient perturbation and adaptive noise scheduling to enhance model robustness.

[0013] Furthermore, the output control unit performs model output detection and model output control tasks. The model output detection task includes, according to the DSMM data security standard, combining static and dynamic desensitization technologies, and using an AI-driven sensitive information identification engine to achieve automated data deformation modeling; combining multi-dimensional feature analysis and context awareness technology to perform anomaly detection on model output and protect the privacy of model output. The model output control task includes, according to content compliance requirements, combining semantic analysis and a rule engine, constructing an intelligent filtering framework through multi-level verification strategies to perform access control on compliant model outputs; and based on a zero-trust security model, combining access control technology, and using a dynamic permission management system to achieve fine-grained access control and protect the dynamic security of model output.

[0014] Furthermore, the model security knowledge base includes an encryption algorithm library, an AI model library, an AI attack pattern library, an AI adversarial sample library, and a defense knowledge base.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0016] (1) The present invention proposes an adaptive enhanced information security protection system for AI large models. By comprehensively covering the security requirements of multiple stages of large model services and identifying the degree of information security threats throughout the model's life cycle, and combining the software and hardware environment to carry out multi-dimensional collaborative system-level information security protection for large models, it is conducive to strengthening the integrated and private security deployment of AI large models in various vertical industries, thereby solving the limitation problem of focusing on single point or local security in the existing information security protection of AI large models.

[0017] (2) This invention is aimed at the input stage of the AI ​​large model service process. It proposes a collaborative defense technology route based on deep packet inspection and cross-modal graph neural network. Starting from the full-domain data features from the network transmission layer to the application semantic layer, it comprehensively verifies the protocol compliance, content integrity and logical consistency of the input data at multiple levels, which facilitates the subsequent construction of a highly robust data cleaning pipeline and provides dual credibility guarantee for model inference.

[0018] (3) This invention is aimed at the operation stage of the AI ​​large model service process. It proposes a two-dimensional model reinforcement technology route based on dynamic environment perception and structural optimization. Starting from the hardware and software interaction characteristics during model operation, it comprehensively and dynamically protects the model's computational logic, data interaction path and parameter update mechanism. This facilitates the maintenance of the AI ​​large model's information security attributes such as anti-attack, anti-theft, and privacy protection, and improves the model's survivability after being maliciously invaded.

[0019] (4) This invention is aimed at the model output stage of the AI ​​large model service process. It proposes a collaborative protection technology route based on differential privacy and dynamic access control. Starting from the security and compliance of data flow, it comprehensively and dynamically manages the information density, semantic relevance and access permissions of the model output in multiple dimensions, which facilitates the subsequent construction of a three-in-one output security system of "detection-control-audit". Attached Figure Description

[0020] Figure 1 A schematic diagram of the overall system architecture of the adaptive enhanced information security protection system for large AI models provided in this embodiment of the invention;

[0021] Figure 2 This is a schematic diagram of a cross-modal covert command attack process targeting large model inputs;

[0022] Figure 3This is a schematic diagram of the gradient inverse parameter contamination attack process targeting the output of a large model.

[0023] Figure 4 For the purposes of this invention embodiment Figure 2 The diagram illustrates input protection against input attacks.

[0024] Figure 5 For the purposes of this invention embodiment Figure 3 The diagram shows an output protection schematic for output attacks. Detailed Implementation

[0025] To enable those skilled in the art to more clearly understand the technical means and effects adopted by the present invention to achieve the intended purpose, the specific embodiments, structures, features, and effects of the present invention will be further described in detail below with reference to the accompanying drawings and preferred embodiments. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention. In the embodiments of the present invention, unless otherwise specified, all raw material components are commercially available products well known to those skilled in the art; unless specifically specified, the technical means used are conventional means well known to those skilled in the art.

[0026] The adaptive enhanced information security protection system for large AI models provided in this embodiment of the invention has the following overall system architecture diagram: Figure 1 As shown, it includes three modules: a model security management module, a model security operation module, and a model security knowledge base. The model security management module is used for in-depth security monitoring and analysis of the real-time operation logs of the AI ​​large model and to deploy the defense strategies executed by the model security operation module. The model security operation module is used to perform information security protection for the entire process and lifecycle of the large model service. The model security knowledge base is used to provide the auxiliary resources required by the model security management module and the model security operation module.

[0027] Specifically, the security management module includes a model log analysis unit and a dynamic defense strategy decision-making unit. The model log analysis unit performs in-depth security monitoring and analysis on the real-time operation logs of the AI ​​large model, and guides the dynamic defense strategy decision-making unit to deploy the defense strategies executed by the model security operation module.

[0028] The model security operation module includes an input detection unit, a model operation hardening unit, and an output control unit. It provides multi-dimensional and in-depth enhanced information security protection based on the security needs and goals of the entire AI large model service process, divided into input detection stage, model operation hardening stage, and output control stage. The input detection unit is used for multi-source intrusion detection in the model input stage, the model operation hardening unit is used for structural hardening against AI attacks in the model operation stage, and the output control unit is used for output detection and control in the model output stage.

[0029] Specifically, the input detection unit performs network traffic intrusion detection tasks and large model threat detection tasks.

[0030] The network traffic intrusion detection task includes: based on the hardware device fingerprint of the model input party, combined with hash algorithm and physical non-cloning function technology, a dynamic authentication model for device identity is realized through dynamic fingerprint update mechanism; based on the protocol deep parsing requirements, combined with spatiotemporal convolutional network and traffic sampling technology, a real-time network traffic threat detection model is established through protocol field full parsing and covert channel identification methods.

[0031] The large-scale threat detection task involves, based on the security requirements of the system's multimodal inputs, combining multimodal joint analysis algorithms with a hardware and software acceleration framework, and constructing a multimodal threat map to dynamically detect input-side threats to the large model in real time.

[0032] Based on network traffic and large model input threat detection, an adaptive cleaning framework for model input is established through a syntax-semantic dual-layer filtering mechanism to achieve safe model input.

[0033] Among them, information security protection for the model input side requires authentication of the input device. Only input requests from legitimate users will be decrypted into plaintext for input threat detection.

[0034] The model hardening unit performs the tasks of training the model against malicious samples and making the model architecture robust.

[0035] The model adversarial malicious sample training task includes discovering known security vulnerabilities in the AI ​​large model runtime environment using vulnerability mining tools such as CodeQL, deploying relevant defense strategies against the identified security attacks, and achieving dynamic adversarial training through a phased perturbation constraint strategy based on course learning theory and combined with an improved PGD algorithm and meta-learning framework.

[0036] The model architecture robustness task includes constructing an adaptive feature selection model based on adversarial perception theory and the model security knowledge base, combined with dynamic attention masking and multi-head defense mechanisms, through an adversarial feature gating network to select effective attack methods; at the same time, combining Gaussian gradient perturbation and adaptive noise scheduling, a provably secure training framework is established through a dynamic budget allocation strategy to enhance model robustness.

[0037] The output control unit performs model output detection and model output control tasks.

[0038] The model output detection task includes implementing automated data deformation models based on DSMM data security standards (such as GB / T 37988-2019), combining static and dynamic desensitization techniques, and using an AI-driven sensitive information identification engine; and performing anomaly detection on the model output by combining multi-dimensional feature analysis and context-aware technology to protect the privacy of the model output.

[0039] The model output control task includes constructing an intelligent filtering framework based on content compliance requirements, combined with semantic analysis and rule engines, through multi-level verification strategies, to control the access of compliant model outputs to the transmitting party. Based on the zero-trust security model and combined with access control technology, fine-grained access control is achieved through a dynamic permission management system to protect the dynamic security of model outputs.

[0040] The model security knowledge base includes an encryption algorithm library, an AI model library, an AI attack pattern library, an AI adversarial sample library, and a defense knowledge base. The encryption algorithm library and defense knowledge base provide the technical methods needed for information security protection throughout the entire lifecycle of large AI models. The AI ​​model library provides auxiliary resources needed for model hardening during secure runtime. The AI ​​attack pattern library and AI adversarial sample library are auxiliary resources used to support log analysis and dynamic decision-making processes in model security management.

[0041] The following section will illustrate the application of the information security protection system provided in this embodiment by taking specific cross-modal covert instruction attacks and gradient inverse parameter contamination attacks as examples.

[0042] Figure 2 The cross-modal covert instruction attack shown refers to an attacker embedding malicious instructions into non-textual multimodal data, taking advantage of the superior multimodal processing capabilities of large models to induce the models to perform malicious operations, and bypassing traditional text filtering and rule matching mechanisms through the "steganography" characteristics of multimodal data.

[0043] This example demonstrates a cross-modal covert instruction attack targeting model input. The attack involves three stages: data parsing, instruction extraction, and model triggering. Instructions are hidden using techniques such as LSB steganography and frequency domain modulation, while also exploiting vulnerabilities in preprocessing modules like OCR / ASR. The specific steps are as follows:

[0044] Step 1: Create malicious carriers in three file formats: image (JPEG / PNG format), audio (16kHz WAV file), and video (H.264 encoded), and test the usability of the carriers. If the malicious carrier is unusable, change the carrier type.

[0045] Step 2: If the malicious vector is available, embed covert instructions into the vector. For image malicious vectors, use SteganoGAN to embed instructions and use LSB replacement to modify the least significant bit of the pixel to achieve image steganography. For audio malicious vectors, hide instructions by modulating MFCC coefficients and add phase modulation signals in the 3-5kHz frequency band using FFmpeg to achieve audio steganography. For video vectors, implant malicious instructions in the frame DCT coefficients and perform frame steganography.

[0046] Step 3: Determine the availability of the carrier and the hidden instructions. If they are unavailable, replace them with malicious instructions. If the malicious carrier and instructions are available, induce users to upload or inject the model directly through social engineering attacks and phishing deception attacks.

[0047] Step 4: After receiving the input, the model parses the multimodal data, and the hidden instructions in the malicious carrier are triggered. The model executes the malicious instructions, enabling the attacker to perform illegal operations on the large model, resulting in privacy leaks, continuous system damage, or the establishment of a reverse shell connection for continuous penetration. At this point, a complete cross-modal hidden instruction attack is completed.

[0048] Figure 3 The gradient inverse parameter contamination attack shown in the diagram injects a hidden backdoor into the model parameters by contaminating the gradient of the training data. It can also propagate across models through model fine-tuning, distillation and other model operations.

[0049] This example demonstrates a gradient inverse parameter pollution attack targeting the model output. This attack leverages the gradient update mechanism of the model's backpropagation to encode malicious patterns into the model's weight space. A backdoor is then triggered under specific conditions to achieve the attack and cause damage. The specific steps include:

[0050] Step 1: Attackers use Netron tools to analyze the model structure based on the model output, identify highly sensitive fully connected layers, calculate parameter sensitivity using the finite difference method, and add high-frequency noise patterns to normal samples to construct malicious samples.

[0051] Step 2: Determine the model accessibility. If white-box access is possible, obtain the model gradient directly through output query; if only black-box exploration is possible, estimate the gradient based on the model output using the finite difference method.

[0052] Step 3: Input adversarial examples, calculate the target loss through forward propagation of the model, calculate the inverse gradient through backpropagation, and flip the gradient sign to adapt to the model operation; mix the malicious gradient with the normal gradient in proportion to achieve gradient obfuscation, inject and pollute the model parameter space, and implant a backdoor in the model;

[0053] Step 4: Based on the hidden backdoor, match the model triggering mode. If the conditions are not met, continue to lie dormant and wait; if the backdoor conditions are met, trigger the backdoor, activate malicious parameters, command the model to execute the infiltration protocol, and achieve privacy leakage through the model output, thereby compromising the confidentiality of the system.

[0054] Figure 2 , Figure 3The attacks targeting model inputs and outputs severely compromise the confidentiality, integrity, and availability of large AI models. These attacks can also achieve long-term penetration through infiltration, seriously damaging the information security of the entire model service process. It is necessary to protect both the input and output sides of the model to reduce the probability of the model being attacked and penetrated.

[0055] The steps to protect against cross-modal covert command attacks targeting model input are as follows: Figure 4 As shown, to prevent cross-modal covert command attacks, information security protection for model inputs needs to be achieved through multimodal consistency verification, dynamic semantic sandboxing, and feature decoupling defense. The specific steps are as follows:

[0056] Step 1: Classify the original input of the model, decouple the modal features, and identify the multimodal input types; use dedicated steg detection algorithms for different modalities, perform StegExpose detection on image data, perform MFCC fluctuation analysis on audio data, and perform regularization filtering on text data;

[0057] Step 2: Perform consistency checks on the multimodal data, extract deep features of each modality, calculate cross-modal similarity, set a consistency risk threshold, and detect semantic conflicts by comparing deep features; when the model risk score is below the threshold, the model is considered normal and can be executed directly.

[0058] Step 3: When the model risk score exceeds the threshold, the model is isolated and executed in a dynamic semantic sandbox. The model behavior is audited, the system call sequence is recorded and abnormal patterns are analyzed. If the model behavior is normal, it is safely output in the sandbox. If abnormal behavior is found, the circuit breaker is immediately triggered and a model warning is issued.

[0059] In addition to the above-mentioned defense procedures, multimodal steganography can also be suppressed by combining hardware and software methods to counter cross-modal steganography attacks on model inputs. For example, in smart manufacturing applications, frequency domain filtering hardware can be added to industrial cameras to suppress video / image steganography signals, and digital watermark verification can be added to text files.

[0060] To prevent gradient inverse parameter contamination attacks on model output, the following model output protection measures are employed: Figure 5 As shown, taking advantage of the characteristics of gradient inverse parameter pollution attacks, model output protection is achieved through gradient behavior analysis, dynamic model validation, and adversarial training enhancement; the specific steps are as follows:

[0061] Step 1: Clean the training data, remove outlier samples, and analyze gradient direction deviation, gradient magnitude abrupt changes, and parameter sensitivity in real time. If gradient anomalies are found, the training circuit breaker is activated, the model service is terminated, and a warning is issued; if no anomalies are found, training continues and the model parameters are updated.

[0062] Step 2: Generate model fingerprints for normal models and store the fingerprints for easy model rollback; then deploy the normal models to the AI ​​all-in-one machine.

[0063] Step 3: Continuously perform integrity checks on the deployed model. If the check passes, provide normal model services. If the check fails, immediately issue a circuit breaker warning and roll back the model to ensure model security.

[0064] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any person skilled in the art can make some modifications or alterations to the disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention, without departing from the scope of the present invention, shall still fall within the scope of the present invention.

Claims

1. An adaptive enhanced information security protection system for large AI models, characterized in that, include: The system includes a model security management module, a model security operation module, and a model security knowledge base. The model security management module is used for in-depth security monitoring and analysis of the real-time operation logs of large AI models and to deploy the defense strategies executed by the model security operation module. The model security operation module is used to perform information security protection throughout the entire process and lifecycle of large model services. The model security knowledge base is used to provide the auxiliary resources required by the model security management module and the model security operation module. The security management module includes a model log analysis unit and a dynamic defense strategy decision-making unit; The model security operation module includes an input detection unit, a model operation hardening unit, and an output control unit. The input detection unit is used for multi-source intrusion detection during the model input stage, the model operation hardening unit is used for structural hardening against AI attacks during the model operation stage, and the output control unit is used for output detection and control during the model output stage. The input detection unit performs network traffic intrusion detection and large-scale model threat detection tasks. The network traffic intrusion detection task includes: based on the hardware device fingerprint of the model input party, combined with hash algorithms and physical non-cloning function technology, a dynamic authentication model for device identity is realized through a dynamic fingerprint update mechanism; based on the protocol deep parsing requirements, combined with spatiotemporal convolutional networks and traffic sampling technology, a real-time network traffic threat detection model is established through protocol field full parsing and covert channel identification methods. The large-scale model threat detection task includes: based on the security requirements of the system's multimodal input, combined with multimodal joint analysis algorithms and hardware and software acceleration framework, a multimodal threat graph is constructed to dynamically detect threats on the input side of the large model in real time. The model hardening unit performs model adversarial malicious sample training and model architecture robustness tasks. The model adversarial malicious sample training task includes discovering known security vulnerabilities in the AI ​​large model's operating environment using vulnerability mining tools, deploying relevant defense strategies against identified security attacks, and implementing dynamic adversarial training through a phased perturbation constraint strategy based on course learning theory and an improved PGD algorithm and meta-learning framework. The model architecture robustness task includes constructing an adaptive feature selection model using an adversarial feature gating network based on adversarial perception theory and the model's security knowledge base, combined with dynamic attention masks and multi-head defense mechanisms, to select effective attack methods; and establishing a provably secure training framework through a dynamic budget allocation strategy by combining Gaussian gradient perturbation and adaptive noise scheduling to enhance model robustness. The output control unit performs model output detection and model output control tasks. The model output detection task includes, according to the DSMM data security standard, combining static and dynamic desensitization technologies, and using an AI-driven sensitive information identification engine to achieve automated data deformation modeling; combining multi-dimensional feature analysis and context awareness technology to perform anomaly detection on model output and protect the privacy of model output. The model output control task includes, according to content compliance requirements, combining semantic analysis and a rule engine, constructing an intelligent filtering framework through a multi-level verification strategy to perform access control on compliant model outputs from the transmission output party; and based on a zero-trust security model, combining access control technology, and using a dynamic permission management system to achieve fine-grained access control and protect the dynamic security of model output.

2. The adaptive enhanced information security protection system for large AI models according to claim 1, characterized in that, The model security knowledge base includes an encryption algorithm library, an AI model library, an AI attack pattern library, an AI adversarial sample library, and a defense knowledge base.

Images