Multi-modal security protection method and device, computer equipment and storage medium

Abstract

The invention relates to a multi-modal security protection method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring multi-modal data of each participant; generating a hierarchical key for each participant based on a quantum key distribution mechanism; according to the hierarchical key, performing encryption and feature fusion processing on the feature vectors extracted from the multi-modal data to obtain encrypted feature information; and according to the encrypted feature information, local model training is carried out in the participants, and updating of a local model is transmitted by adopting hierarchical key encryption to carry out federated learning. According to the method and the device, based on a quantum key distribution mechanism, a key system of hierarchical keys is designed for multi-modal data, and feature alignment processing and federated learning are performed in a ciphertext state, so that privacy leakage of plaintext processing can be avoided, multi-modal security protection is realized, and the problem of insufficient privacy security is solved.

Images