Data distributed storage and reading method and system based on multi-modal protection

Abstract

The application discloses a kind of based on multi-modal protection's data distributed storage and reading method and system, it is related to multi-modal data encryption storage technical field, this method includes: receiving input data, pre-processing is multi-modal segmentation data;Data is executed multi-key cooperation encryption;The distributed storage of data after encryption;Based on the signature and broadcast of the node information of distributed storage based on block chain;Receive the data query request of user, corresponding encrypted segmentation data is obtained by the authority vector of user, and the decryption and reorganization of encrypted segmentation data are executed to obtain query data.Through multi-modal feature fusion technology, the security of data storage is improved, and a multi-level encryption system is constructed by multi-key cooperation encryption, effectively ensuring the security of key distribution, and a distributed storage strategy based on dynamic weight is used to realize intelligent selection and load balancing of storage nodes, improve the reliability and system scalability of data storage.

Description

A Distributed Data Storage and Retrieval Method and System Based on Multimodal Protection Technical Field

[0001] This invention relates to the field of multimodal data encryption and storage technology, and particularly to a method and system for distributed data storage and retrieval based on multimodal protection. Background Technology

[0002] With the development of big data and artificial intelligence technologies, the demand for storage and sharing of multimodal data (such as images, text, and videos) is growing rapidly. Traditional data storage and retrieval methods have many problems. When storing or encrypting multimodal data, effective separation and identification are required, along with corresponding data processing schemes for each module. However, existing technologies often fall short in this regard. For example, insufficient feature fusion and segmentation accuracy for multimodal data leads to a failure to effectively retain original semantic information after data segmentation, affecting subsequent encryption and storage efficiency. When encrypting processed data, traditional encryption methods often use a single key, which is insufficient to meet the differentiated security requirements of different modalities. During data storage, the selection of storage nodes relies solely on static indicators such as storage capacity and bandwidth. However, for various types of multimodal data, using a single static indicator is insufficient to meet the requirements for both encryption storage efficiency and security. Therefore, technical solutions for encrypted storage of multimodal data are of immense value for innovation in the field of data storage. Summary of the Invention

[0003] In view of this, the present invention proposes a distributed data storage and retrieval method and system based on multimodal protection, which can significantly improve the security, efficiency, and privacy protection capabilities of data storage. The present invention provides the following technical solution:

[0004] A method for distributed data storage and retrieval based on multimodal protection, the method comprising: receiving input data and preprocessing it to obtain multimodal segmented data; performing multi-key collaborative encryption on the multimodal segmented data; distributively storing the encrypted multimodal segmented data; signing and broadcasting the node information of the distributed storage based on blockchain; receiving a user's data query request, obtaining the corresponding encrypted segmented data through the user's permission vector, and performing decryption and reassembly on the encrypted segmented data to obtain the query data.

[0005] Optionally, receiving input data and preprocessing it to obtain multimodal segmentation data includes: identifying each modality data and its corresponding index in the input data, and extracting feature vectors of the modality data respectively; performing feature fusion using an attention mechanism and calculating the attention weight of each modality feature; calculating the fused feature vector corresponding to each modality feature based on the attention weight; and performing clustering segmentation based on the feature vectors to obtain multimodal segmentation data.

[0006] Optionally, performing multi-key collaborative encryption on the multimodal segmented data includes: generating a base key based on a chaotic map and a system private key; performing hash processing on the feature vector corresponding to the multimodal segmented data to obtain a corresponding hash value; performing an XOR operation between the hash value and the base key to obtain a key corresponding to the multimodal segmented data; and performing encryption on the corresponding multimodal segmented data using the key.

[0007] Optionally, before encrypting the corresponding multimodal segmented data using the key, the method further includes: establishing an entangled quantum pair between the key receiver and the key sender; performing a joint measurement on the key information to be transmitted by the sender and a portion of the entangled quantum pair to obtain a measurement result; sending the measurement result to the receiver through a classical channel; and the receiver recovering the key information based on the measurement result and its entangled quantum pair.

[0008] Optionally, the distributed storage of the encrypted multimodal segmentation data includes: obtaining the configuration information of the storage nodes, the configuration information including: storage capacity, network bandwidth, remaining storage space and the reputation value of the nodes; calculating dynamic weights based on the strength of the key and the configuration information; calculating the comprehensive score corresponding to the storage nodes based on the dynamic weights; sorting the comprehensive scores and selecting the storage nodes with the highest scores to store the multimodal segmentation data.

[0009] Optionally, the step of signing and broadcasting the node information of distributed storage based on blockchain includes: encoding the information of the storage node into a transaction and signing the transaction using an elliptic curve digital signature algorithm; broadcasting the signed transaction to the blockchain network for verification and recording; dividing the blockchain network into multiple shards, with each shard processing the transaction independently; achieving transaction synchronization and data sharing between shards through a cross-shard communication protocol; and using zero-knowledge proof technology to encrypt and verify sensitive information in the transaction to ensure the privacy and security of the storage node and data.

[0010] Optionally, receiving a user's data layer query request, obtaining the corresponding encrypted segmented data through the user's permission vector, and performing decryption and recombination on the encrypted segmented data to obtain query data includes: defining the user's permission vector, biometric vector, and timestamp; parsing the permission vector and combining it with a smart contract to determine whether the user has the right to access the encrypted segmented data; if the user has the right to access the data, obtaining the distributed storage node information of the encrypted segmented data from the blockchain based on the permission vector; obtaining the encrypted segmented data based on the distributed storage node information; decrypting the encrypted segmented data; and concatenating and recombination of the decrypted segmented data according to a sorting rule to obtain query data.

[0011] This invention further discloses a distributed data storage and retrieval system based on multimodal protection, comprising: a preprocessing module for receiving input data and preprocessing it to obtain multimodal segmented data; an encryption module for performing multi-key collaborative encryption on the multimodal segmented data; a storage module for distributed storage of the encrypted multimodal segmented data; a signature module for signing and broadcasting the node information of the distributed storage based on a blockchain; and a data reconstruction module for receiving user data query requests, obtaining the corresponding encrypted segmented data through the user's permission vector, and performing decryption and reconstruction on the encrypted segmented data to obtain the query data.

[0012] The present invention further discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0013] The present invention further discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described method.

[0014] According to the technical solution of this invention, fine-grained processing of multimodal data is achieved through multimodal feature fusion technology, significantly improving the security and access efficiency of data storage. Simultaneously, a multi-layered encryption system is constructed by combining multi-key collaborative encryption and quantum key distribution technology, effectively ensuring the security of key distribution. Furthermore, a distributed storage strategy based on dynamic weights is adopted to achieve intelligent selection and load balancing of storage nodes, improving the reliability and scalability of data storage. On this basis, fine-grained control of user permissions and high efficiency of data reorganization are ensured. Overall, a full lifecycle security protection system is formed from data segmentation, encryption, storage to access, significantly improving the security, reliability, and availability of multimodal data sharing. Attached Figure Description

[0015] For illustrative and not limiting purposes, the present invention will now be described in conjunction with embodiments and accompanying drawings, wherein:

[0016] Figure 1 is a flowchart illustrating the distributed data storage and retrieval method based on multimodal protection in an embodiment of the present invention;

[0017] Figure 2 is a schematic diagram of the structure of the distributed data storage and retrieval system based on multimodal protection in an embodiment of the present invention;

[0018] Figure 3 is a schematic diagram of the structure of the electronic device in an embodiment of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0020] It should be noted that, where there is no conflict, the embodiments and features of the embodiments in this application can be combined with each other. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0021] Referring to Figure 1, this embodiment discloses a distributed data storage and retrieval method based on multimodal protection, including the following steps:

[0022] S100: Receive input data and preprocess it to obtain multimodal segmentation data. In this embodiment, the input data is multimodal data for encrypted storage. For example, the multimodal data includes text data, image data, and video data. For text data, a pre-trained model based on the Transformer architecture (such as BERT) is used to extract semantic feature vector T. i Where i represents the index of the text segment, and for image data, a convolutional neural network (such as ResNet) is used to extract the visual feature vector I. j Where j represents the index of the image region, for video data, a spatiotemporal feature vector V is extracted by combining a 3D convolutional neural network (such as C3D) and an optical flow algorithm. k , where k represents the index of the video segment.

[0023] After extracting the feature vectors and corresponding indices of each modality, an attention mechanism is used for feature fusion. The attention weight of each modality feature is calculated to integrate complementary information from different modalities, achieving more precise data segmentation and access control. Specifically, taking text and images as an example, the attention weight of text on the image is calculated. , ,in A learnable attention matrix, For the number of image regions, further, the fused feature vector. for: .

[0024] Furthermore, based on the fused feature vectors, a spectral clustering algorithm is used for data segmentation. Specifically, the similarity matrix S between the feature vectors is calculated: ,in, It's a bandwidth parameter. and These are the two feature vectors obtained after fusion using the attention mechanism. The similarity matrix S is further normalized to obtain the Laplacian matrix L, and the eigenvectors of L are solved. Then, clustering is performed based on the feature vectors to obtain multimodal segmentation data.

[0025] S200: Perform multi-key collaborative encryption on the multimodal segmented data. A chaotic mapping model is selected; in this embodiment, the Logistic mapping is used, and its mathematical expression is: ,in These are control parameters, used to control systems in a chaotic state. This is the value of the nth iteration; an initial value is chosen. As the seed for the chaotic mapping, this seed can be generated by the system based on factors such as the current timestamp and random numbers to ensure that each generated chaotic sequence has randomness and unpredictability. The system's private key 'b' is obtained and encoded into a binary sequence. Then, the chaotic sequence generated by the chaotic mapping is XORed with the binary sequence of the system's private key 'b' to generate the base key. .

[0026] Furthermore, the feature vectors corresponding to the multimodal segmentation data are obtained, and a secure hash algorithm (SHA-256) is used to calculate the corresponding hash value, for example: , and This method converts feature vectors into fixed-length hash values, facilitating subsequent key generation and encryption operations while also increasing data security. The hash value corresponding to each multimodal data segment is then compared with the base key. Perform an XOR operation to obtain the key corresponding to the multimodal segmentation data. For text data, the key is: For image data, the key is: For video data, the key is: By using an XOR operation, the base key and the characteristic information of each modal data are combined, giving each modal data its own unique key, thus enhancing encryption security. Furthermore, regarding the aforementioned key... , and This means that the key of multimodal segmented data is processed by fragmentation to generate key shares. Based on this, in subsequent key recovery schemes, a sufficient number of shares need to be collected to recover the original key.

[0027] After obtaining the key corresponding to the multimodal segmented data, an entangled quantum pair is established between the key receiver and the key sender. This ensures that when one quantum state is measured, the other quantum state will instantly change accordingly, guaranteeing the security of key distribution. Further, key distribution is performed, with the sender distributing the key information to be transmitted (such as...). , , A measurement result M is obtained by jointly measuring a portion of the entangled quantum pair and key information, which includes key information and the correlation information of the quantum states. The sender transmits the measurement result M to the receiver via a classical channel. The receiver, based on the received measurement result M and the other portion of the entangled quantum pair it holds, recovers the key information using a quantum algorithm. For example, the receiver utilizes the entanglement properties of the quantum states and the correlation of the measurement result to manipulate its own quantum state, thereby obtaining the same key as the sender. , , Furthermore, the corresponding multimodal segmentation data is encrypted using the keys obtained from each modal data.

[0028] S300: Distributed storage of the encrypted multimodal segmented data. Specifically, firstly, the configuration information of the storage nodes is obtained, including: storage capacity, network bandwidth, remaining storage space, and the node's reputation value. Storage capacity determines the total amount of data a node can store; network bandwidth affects data read / write speed; remaining storage space reflects the amount of space a node can currently use to store new data; and the node's reputation value is a comprehensive evaluation based on the node's past performance, such as data storage integrity, response time, and whether security issues have occurred. Dynamic weights are calculated based on the key strength and the configuration information, using the following formula: ,in, It is the remaining storage space of the nth node. It is the total storage capacity of all nodes. It is the network bandwidth of the nth node. It is the maximum network bandwidth among all nodes. It is the reputation value of the nth node. It is a quantized value of the key strength. , , , These are pre-set weighting coefficients, and The formula for quantizing key strength is: ,in, This is the key entropy value. To determine the security level of the selected encryption algorithm, for example, this embodiment uses the AES-256 encryption algorithm, which has a security level of 3. In comparison, the RSA-3072 encryption algorithm has a security level of 4. This is based on dynamic weights. Calculate the overall score for each storage node. The formula is: ,in, This is a monotonically increasing function, ensuring that nodes with higher dynamic weights also have higher overall scores. After obtaining the overall score, all storage nodes are sorted by their overall scores, and the top-ranked storage nodes are selected to store the multimodal segmentation data. An appropriate number of nodes can be selected based on the data size and storage requirements. For example, if the data volume is large, multiple nodes can be selected for distributed storage to improve storage reliability and performance. To further improve data reliability, a data redundancy strategy is adopted. The encrypted multimodal segmentation data can be copied and stored on different nodes. For example, for important data segments, two copies can be made and stored on nodes in different geographical locations to prevent data loss due to the failure or damage of a single node. After storing the encrypted multimodal segmentation data, the corresponding key shares are allocated to the corresponding storage nodes. Simultaneously, identical key shares are copied to multiple nodes in different geographical locations to facilitate redundant storage of the multimodal segmentation data. For example, multimodal segmentation data of text data... By key encryption, The generated key fragments will be They are assigned to the same or related storage nodes to ensure that the corresponding decryption key fragment can be obtained when the data fragment needs to be accessed.

[0029] S400: Based on blockchain, it signs and broadcasts node information for distributed storage. It collects configuration information from all storage nodes involved in the distributed storage process, including the unique identifier of the storage node, the identifier of the stored data segment, the storage location, node configuration information (storage capacity, network bandwidth, remaining storage space, reputation value, etc.), dynamic weights, and comprehensive scores. The collected node information is encoded according to the format specified by the blockchain network, converting it into transaction data suitable for transmission and processing on the blockchain. The encoding process uses binary encoding or JSON format to ensure data simplicity and readability. The system uses a pre-allocated private key for the Elliptic Curve Digital Signature Algorithm (ECDSA) to sign the encoded transaction data using the ECDSA algorithm. The signing process involves hashing the transaction data and encrypting the private key. Specifically, the transaction data is first hashed to obtain a fixed-length hash value, and then the hash value is encrypted using the private key to generate a digital signature. The purpose of signing is to ensure the integrity and non-repudiation of the transaction data; that is, only the system holding the private key can sign the transaction, and the signature can be verified by any node holding the public key. The signed transaction data is then broadcast to all nodes in the blockchain network. Before broadcasting, the system needs to establish a stable connection with the blockchain network. This can be achieved through P2P network protocols (such as BitTorrent and Kademlia) to communicate with other nodes, ensuring that transaction data is disseminated quickly and accurately throughout the network.

[0030] Furthermore, the blockchain network is divided into multiple shards, each a separate sub-network responsible for processing a portion of transactions. Sharding is based on factors such as node geolocation, performance, and functionality to improve the network's concurrent processing capacity and scalability. For example, nodes geographically close together can be grouped into one shard, or nodes with higher processing capabilities can be grouped into a dedicated shard for handling complex transactions. Transaction data allocation rules are established based on transaction type, data volume, and involved nodes. For instance, transactions involving specific nodes are assigned to the shard containing that node; transactions with large data volumes are assigned to shards with higher processing capabilities. After shard allocation, each shard independently processes its assigned transactions. Nodes within a shard verify transactions, checking signature validity and data integrity. If verification passes, the transaction is recorded in the local blockchain ledger. Cross-shard communication utilizes a cross-shard communication protocol to achieve transaction synchronization and data sharing between shards. This protocol defines communication rules and data formats between shards, ensuring effective interaction between different shards. Once a shard has processed a transaction, it synchronizes the transaction result to other relevant shards via a cross-shard communication protocol. The synchronization process must ensure data consistency and integrity, avoiding data conflicts and inconsistencies. This implementation uses distributed consensus algorithms (such as PBFT, PoS, etc.) to ensure synchronization between shards.

[0031] To ensure the security of sensitive information such as node reputation values ​​and dynamic weights during transactions, zero-knowledge proof technology is used to encrypt them. This information is only proven valid to other nodes without revealing the specific values. Similarly, other nodes verify the validity of sensitive information in the transaction by validating the zero-knowledge proof. The verification process does not require obtaining specific sensitive information, thus protecting the privacy and security of nodes and data. If the verification passes, the transaction is considered legitimate and can proceed to subsequent processing. Verified and processed transaction data is recorded in the blockchain ledger. In this implementation, the ledger is a distributed, immutable database that records all transaction history and node information. Each node maintains a complete copy of the ledger to ensure data reliability and traceability. Furthermore, nodes update their local ledger state based on new transaction data, including updates to node information and data storage records. Simultaneously, the blockchain network uses a consensus mechanism to ensure that the ledger state of all nodes remains consistent.

[0032] S500: Receives the user's data query request, obtains the corresponding encrypted segmented data through the user's permission vector, and performs decryption and reassembly on the encrypted segmented data to obtain the query data.

[0033] The user's permission vector is dynamically generated by the system based on factors such as the user's role, access history, and data sensitivity. Let it be denoted as... Each element This indicates whether the user has permission to access the i-th multimodal segmentation data. Further, the system obtains the user's biometric data and converts it into a feature vector B. The system also records the timestamp t of the user's query request to verify the validity of the user's permissions.

[0034] Furthermore, the user's permission vector Perform parsing to extract access permission information. For example, parsing yields a set of multimodal segmented data indexes that the user is authorized to access. Then, the smart contract deployed on the blockchain is invoked to verify: permission validity: check the user's permission vector. Whether it is within the validity period, that is, whether the current timestamp t is within the effective time range of the permission. Internal; Biometric matching: Compare the user's feature vector B with the user's biometric template stored in the blockchain and calculate a similarity score. For example, a cosine similarity formula can be used. Calculate the similarity score; if the similarity score exceeds the preset threshold, the biometric verification is successful.

[0035] When determining whether the current user has permission to access the encrypted segmented data, the user's permission vector is used. The multimodal segmentation data index set that is accessible in the middle Query the blockchain secure ledger and obtain: the location of encrypted data storage for each shard. It contains the address, port, and path of the storage node; and the corresponding key fragments. The sharding sorting rules. Based on this, the reputation values ​​of the storage nodes are filtered out. Nodes below a preset threshold are excluded to ensure data extraction from high-reputation nodes. Regarding the sharding sorting rules, this implementation provides an example: when generating encryption key shards, key shards are bound to their corresponding multimodal data shards. In this case, the sorting rules are integrated into the key management system; each key shard is associated not only with the encryption information of the data shard but also with its logical position information within the original data.

[0036] Based on storage location Send a data retrieval request to the storage node. Verify the logical relationship between the storage node and the key fragments based on the data retrieval request. If the verification is successful, retrieve the target data. Collect at least t key fragments and recover the original key using Lagrange interpolation. ,in, These are the interpolation coefficients, which are obtained through a polynomial. Sure; It is a large prime number; These are the sharding keys stored in each storage node. For example, this implementation uses a 3-of-5 threshold scheme, where t=3 above, based on the collected 3 sharding keys... , and Through pre-calculated interpolation coefficients , and By inserting the above formula, calculating and taking the modulus, the original key can be finally restored. .

[0037] Furthermore, a temporary key is generated using the user's private key 'a' and the system's private key 'b' via the Diffie-Hellman protocol, with the expression: ,in, The shared secret value is calculated using the Diffie-Hellman algorithm. For biological feature vectors, For timestamps.

[0038] Generate the final decryption key based on the original key and the temporary key: Finally, use Decrypt the multimodal segmented data.

[0039] After obtaining the decrypted multimodal segmentation data, it is sorted according to the sorting rules. The sorting rules here are preset rules. For example, video segmentation data is sorted in ascending order by timestamp, or text segmentation data is sorted in order of semantic tag priority.

[0040] Furthermore, the multimodal segmentation data is reconstructed. In this embodiment, for exemplary text data, an NLP model (such as BERT) is used to recover the semantic structure and handle sentence breaks and grammar; for image data, a super-resolution reconstruction algorithm (such as SRGAN) is used to recover the image; for video data, a time series alignment algorithm (such as optical flow) is used to restore the temporal continuity of the video. The reconstructed data is then converted to a specified format according to user requirements.

[0041] After the data fragments are reassembled, information such as the user ID and the accessed fragment index are encoded into a transaction, signed using ECDSA, and written to the blockchain. The decrypted fragments and temporary keys in the security cache are then deleted to prevent data leakage.

[0042] In summary, upon receiving a user's data query request, the system first defines the user's permission vector, biometric vector, and timestamp. By parsing the permission vector and combining it with a smart contract, it determines whether the user has permission to access the encrypted segmented data. If permission is granted, the system retrieves the distributed storage node information of the encrypted segmented data from the blockchain based on the permission vector, thereby obtaining the encrypted data. After decryption, the decrypted segmented data is reassembled according to a sorting rule to finally obtain the queried data. Fine-grained access control is achieved through permission vectors and smart contracts, ensuring the security and compliance of data access. Blockchain storage node information ensures the immutability and traceability of stored information. Mechanisms such as biometric verification and dynamic key negotiation enhance the security of data transmission and storage. The decryption and reassembly process guarantees the integrity and availability of the user's data, ultimately achieving secure, efficient, and controllable data querying and processing.

[0043] Referring to Figure 2, this embodiment further discloses a distributed data storage and retrieval system based on multimodal protection, including:

[0044] The preprocessing module 21 is used to receive input data and preprocess it to obtain multimodal segmentation data, including: identifying each modality data and its corresponding index in the input data, and extracting the feature vectors of the modality data respectively; using an attention mechanism to perform feature fusion and calculating the attention weight of each modality feature; calculating the fused feature vector corresponding to each modality feature based on the attention weight; and performing clustering segmentation based on the feature vectors to obtain multimodal segmentation data.

[0045] The encryption module 22 is used to perform multi-key collaborative encryption on the multimodal segmented data, including: generating a base key based on a chaotic map and a system private key; performing hash processing on the feature vector corresponding to the multimodal segmented data to obtain the corresponding hash value; performing an XOR operation on the hash value and the base key to obtain the key corresponding to the multimodal segmented data; establishing an entangled quantum pair between the key receiver and the key sender; performing a joint measurement on the key information to be transmitted by the sender and a portion of the entangled quantum pair to obtain the measurement result; sending the measurement result to the receiver through a classical channel; and the receiver recovers the key information based on the measurement result and its entangled quantum pair and encrypts the corresponding multimodal segmented data using the key.

[0046] Storage module 23 is used for distributed storage of encrypted multimodal segmentation data, including: obtaining configuration information of storage nodes, the configuration information including: storage capacity, network bandwidth, remaining storage space and reputation value of nodes; calculating dynamic weights based on the strength of the key and the configuration information; calculating a comprehensive score corresponding to the storage node based on the dynamic weights; sorting the comprehensive scores and selecting the storage nodes with higher rankings to store the multimodal segmentation data.

[0047] The signature module 24 is used to sign and broadcast node information of distributed storage based on the blockchain. This includes: encoding the information of the storage nodes into transactions and signing the transactions using an elliptic curve digital signature algorithm; broadcasting the signed transactions to the blockchain network for verification and recording; dividing the blockchain network into multiple shards, with each shard processing the transactions independently; achieving transaction synchronization and data sharing between shards through a cross-shard communication protocol; and using zero-knowledge proof technology to encrypt and verify sensitive information in the transactions to ensure the privacy and security of the storage nodes and data.

[0048] The data reassembly module 25 receives a user's data query request, obtains the corresponding encrypted segmented data through the user's permission vector, and performs decryption and reassembly on the encrypted segmented data to obtain the query data. This includes: defining the user's permission vector, biometric vector, and timestamp; parsing the permission vector and combining it with a smart contract to determine whether the user has the right to access the encrypted segmented data; if the user has the right to access the data, obtaining the distributed storage node information of the encrypted segmented data from the blockchain based on the permission vector; obtaining the encrypted segmented data based on the distributed storage node information; decrypting the encrypted segmented data; and reassembling the decrypted segmented data according to a sorting rule to obtain the query data.

[0049] Figure 3 is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. As shown in Figure 3, the electronic device 50 includes: a processor 501, a memory 502, and a bus 503.

[0050] The processor 501 and the memory 502 communicate with each other via the bus 503; the processor 501 is used to call the program instructions in the memory 502 to execute the methods provided in the above-described embodiments.

[0051] This embodiment provides a non-transitory computer-readable storage medium that stores computer instructions that cause a computer to execute the methods provided in the above-described embodiments.

[0052] Those skilled in the art will understand that all or part of the steps of the above-described method implementation can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above-described method implementation. The aforementioned storage medium includes various storage media capable of storing program code, such as ROM, RAM, magnetic disk, or optical disk.

[0053] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0054] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of each embodiment or some parts of the embodiments.

[0055] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for distributed data storage and retrieval based on multimodal protection, characterized in that, The method includes: receiving input data and preprocessing it to obtain multimodal segmentation data, including: identifying each modality of data and its corresponding index in the input data, wherein the multimodal data includes text data and image data; extracting feature vectors from the modal data respectively, wherein, for text data, semantic feature vector T is extracted. i Where i represents the index of the text segment, and for image data, the visual feature vector I is extracted. j Where j represents the index of the image region; an attention mechanism is used for feature fusion, and the attention weight of each modality feature is calculated; based on the attention weight, the fused feature vector corresponding to each modality feature is calculated, including: calculating the attention weight of text on the image. ， ,in A learnable attention matrix, The number of image regions, and the fused feature vector. for: Clustering and segmentation based on feature vectors to obtain multimodal segmentation data includes: calculating the similarity matrix S between feature vectors. ,in, It's a bandwidth parameter. and For the two feature vectors obtained after fusion via the attention mechanism, the similarity matrix S is normalized to obtain the Laplacian matrix L, and the feature vectors of L are solved. Then, clustering and segmentation are performed based on the feature vectors to obtain multimodal segmentation data. Multi-key collaborative encryption is performed on the multimodal segmentation data, including: generating a base key based on chaotic mapping and the system private key; performing hash processing on the feature vectors corresponding to the multimodal segmentation data to obtain the corresponding hash value; performing an XOR operation between the hash value and the base key to obtain the key corresponding to the multimodal segmentation data; encrypting the corresponding multimodal segmentation data using the key; and distributing the encrypted multimodal segmentation data, including: obtaining the configuration information of the storage nodes, which includes: storage capacity, network bandwidth, remaining storage space, and node reputation value; and calculating dynamic weights based on the strength of the key and the configuration information, using the following formula: ,in, It is the remaining storage space of the nth node. It is the total storage capacity of all nodes. It is the network bandwidth of the nth node. It is the maximum network bandwidth among all nodes. It is the reputation value of the nth node. It is a quantized value of the key strength. 、 、 、 These are pre-set weighting coefficients, and Based on the dynamic weights Calculate the overall score corresponding to the storage node. The formula is: ,in, It is a monotonically increasing function, ensuring that nodes with larger dynamic weights also have higher comprehensive scores; the comprehensive scores are sorted, and the storage nodes with higher rankings are selected to store the multimodal segmented data; the node information of the distributed storage is signed and broadcast based on the blockchain; the user's data query request is received, the corresponding encrypted segmented data is obtained through the user's permission vector, and the encrypted segmented data is decrypted and reassembled to obtain the query data.

2. The data distributed storage and retrieval method based on multimodal protection according to claim 1, characterized in that, Before encrypting the corresponding multimodal segmented data using the key, the method further includes: establishing an entangled quantum pair between the key receiver and the key sender; performing a joint measurement on the key information to be transmitted by the sender and a portion of the entangled quantum pair to obtain a measurement result; sending the measurement result to the receiver through a classical channel; and the receiver recovering the key information based on the measurement result and its entangled quantum pair.

3. The data distributed storage and retrieval method based on multimodal protection according to claim 1, characterized in that, The process of signing and broadcasting node information of distributed storage based on blockchain includes: encoding the information of the storage node into a transaction and signing the transaction using an elliptic curve digital signature algorithm; broadcasting the signed transaction to the blockchain network for verification and recording; dividing the blockchain network into multiple shards, with each shard processing the transaction independently; achieving transaction synchronization and data sharing between shards through a cross-shard communication protocol; and using zero-knowledge proof technology to encrypt and verify sensitive information in the transaction to ensure the privacy and security of storage nodes and data.

4. The data distributed storage and retrieval method based on multimodal protection according to claim 1, characterized in that, The process of receiving a user's data layer query request, obtaining the corresponding encrypted segmented data through the user's permission vector, and decrypting and reassembling the encrypted segmented data to obtain the query data includes: defining the user's permission vector, biometric vector, and timestamp; parsing the permission vector and combining it with a smart contract to determine whether the user has the right to access the encrypted segmented data; if the user has the right to access the data, obtaining the distributed storage node information of the encrypted segmented data from the blockchain based on the permission vector; obtaining the encrypted segmented data based on the distributed storage node information; decrypting the encrypted segmented data; and concatenating and reassembling the decrypted segmented data according to a sorting rule to obtain the query data.

5. A distributed data storage and retrieval system based on multimodal protection, characterized in that, include: A preprocessing module is used to receive input data and preprocess it to obtain multimodal segmentation data, including: identifying each modality of data and its corresponding index in the input data, wherein the multimodal data includes text data and image data; and extracting feature vectors from the modality data respectively, wherein, for text data, semantic feature vector T is extracted. i Where i represents the index of the text segment, and for image data, the visual feature vector I is extracted. j Where j represents the index of the image region; an attention mechanism is used for feature fusion, and the attention weight of each modality feature is calculated; based on the attention weight, the fused feature vector corresponding to each modality feature is calculated, including: calculating the attention weight of text on the image. ， ,in A learnable attention matrix, The number of image regions, and the fused feature vector. for: Clustering and segmentation based on feature vectors to obtain multimodal segmentation data includes: calculating the similarity matrix S between feature vectors. ,in, It's a bandwidth parameter. and The similarity matrix S, obtained after fusion via an attention mechanism, is normalized to obtain the Laplacian matrix L. The eigenvectors of L are then calculated, and clustering is performed based on these eigenvectors to obtain multimodal segmentation data. An encryption module performs multi-key collaborative encryption on the multimodal segmentation data, including: generating a base key based on a chaotic mapping and a system private key; performing hash processing on the eigenvectors corresponding to the multimodal segmentation data to obtain corresponding hash values; performing an XOR operation between the hash values ​​and the base key to obtain the key corresponding to the multimodal segmentation data; and encrypting the corresponding multimodal segmentation data using the key. A storage module performs distributed storage of the encrypted multimodal segmentation data, including: obtaining configuration information of storage nodes, including storage capacity, network bandwidth, remaining storage space, and node reputation value; and calculating dynamic weights based on the key strength and the configuration information, using the following formula: ,in, It is the remaining storage space of the nth node. It is the total storage capacity of all nodes. It is the network bandwidth of the nth node. It is the maximum network bandwidth among all nodes. It is the reputation value of the nth node. It is a quantized value of the key strength. 、 、 、 These are pre-set weighting coefficients, and Based on the dynamic weights Calculate the overall score corresponding to the storage node. The formula is: ,in, It is a monotonically increasing function, ensuring that nodes with larger dynamic weights also have higher comprehensive scores; the comprehensive scores are sorted, and the storage nodes with higher rankings are selected to store the multimodal segmented data; the signature module is used to sign and broadcast the node information of the distributed storage based on the blockchain; the data reorganization module receives the user's data query request, obtains the corresponding encrypted segmented data through the user's permission vector, and performs decryption and reorganization on the encrypted segmented data to obtain the query data.

6. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1-4.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1-4.

Images