A protection method, apparatus, and electronic device for a multimodal fundamental model.
By constructing a combination of feature-related poisoning datasets and clean datasets, and simulating training and fine-tuning of multimodal models, the robustness and reliability issues of multimodal models in backdoor attacks are solved, and effective defense against poisoning attacks is achieved.
CN117951719BActive Publication Date: 2026-06-30SUN YAT SEN UNIV
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SUN YAT SEN UNIV
- Filing Date
- 2024-01-08
- Publication Date
- 2026-06-30
AI Technical Summary
Technical Problem
Existing multimodal models lack effective protection against backdoor attacks, especially multimodal comparison models which are not robust and reliable enough to resist the threat of low-proportion poisoning attacks.
Method used
By constructing a poisoning dataset whose features are closely related to a clean dataset, a multimodal pre-trained model is simulated and trained to obtain a poisoning model. Then, the poisoning model is fine-tuned using a clean dataset to restore its recognition ability and enhance its defense capabilities.
Benefits of technology
This improves the robustness and reliability of the multimodal model, enabling it to identify clean data and resist poisoning attacks, thus enhancing the model's defense capabilities.
✦ Generated by Eureka AI based on patent content.
Smart Images
Figure CN117951719B_ABST
Abstract
This application discloses a method, apparatus, and electronic device for protecting a multimodal base model. The method includes: acquiring an original dataset and a clean dataset; constructing a poisoned dataset based on the original dataset; wherein the poisoned data characteristics of the poisoned dataset are closely related to the clean data characteristics of the clean dataset; using the poisoned dataset to simulate training a multimodal pre-trained model to obtain a poisoned multimodal base model; and fine-tuning the poisoned multimodal base model based on the clean dataset to obtain a target multimodal base model similar to the multimodal pre-trained model, thereby defending against attacks on the poisoned multimodal base model based on the target multimodal base model. This application, through simulated attack training, can evaluate the robustness of the model and discover potential weaknesses, thereby fine-tuning the model to enable it to resist poisoning attacks, thus improving the model's robustness and reliability. It can be widely applied in the field of information security technology.
Need to check novelty before this filing date? Find Prior Art