Cross-modal retrieval method, system and equipment for encrypted data and medium

Abstract

The invention belongs to the technical field of network and information security, and particularly relates to a cross-modal retrieval method, system, equipment and medium for encrypted data, a client obtains a multi-modal database, and an encryption index and a retrieval token are sequentially constructed according to the multi-modal database; transmitting the multi-modal database and the encrypted index to a cloud server for storage; transmitting the retrieval token to a cloud server; and the cloud server performs retrieval according to the retrieval token to obtain a retrieval result, and transmits the retrieval result to the client. The application of symmetric searchable encryption in cross-modal retrieval is realized. Different multi-modal data are converted into a public space, semantic relations between different types of data are successfully stored, and the content similarity between the data is quantified. In the retrieval process, encrypted Hamming retrieval is converted into multi-keyword query, so that comparison operation with high calculation cost is effectively reduced, only single-round communication is needed, and the communication overhead is remarkably reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of network and information security, and specifically includes a cross-modal retrieval method, system, device and medium for encrypted data. Background Art

[0002] With the rapid development of the mobile Internet and social networks, the application of multi-modal data has gradually penetrated into various industries and fields, becoming an important part of data elements. Multi-modal data is data information from multiple sources and forms, such as text, images, audio, video, etc. This diversified data form has played an extremely important role in production and life. For example: in the field of e-commerce shopping, the integration of multi-modal data of product pictures, text descriptions, and user reviews improves users' comprehensive understanding of products and enhances the accuracy of recommendation systems; in medical diagnosis, the integration of multi-modal data of medical images, medical records, and biological signals helps doctors diagnose diseases and make decisions more accurately. In addition, multi-modal data can not only perform more accurate analysis by integrating multiple information sources, but also provide richer training data for fields such as artificial intelligence and machine learning.

[0003] In the case of limited user storage and computing resources, more and more users tend to outsource multi-modal data to cloud servers for storage and processing to achieve data scalability and efficiency. Typical application scenarios of cross-modal retrieval in cloud service scenarios are as Figure 1 shown, including three entities: the data owner, the cloud service provider, and the query user. Since the credibility of cloud service providers cannot be fully guaranteed, the data faces security threats such as leakage risks and unauthorized access by malicious users. At the same time, there are still great challenges in achieving efficient cross-modal retrieval in the encrypted state.

[0004] Regarding the research on cross-modal retrieval technology for privacy protection under massive big data, existing work mainly focuses on symmetric searchable encryption for single modality, and has not deeply studied the searchable encryption technology for multi-modal data. Moreover, existing cross-modal retrieval schemes on encrypted data all have significant computational overhead or scalability limitations.

[0005] Symmetric Searchable Encryption (SSE) allows users to securely outsource a database to a cloud server and perform retrievals on the database. Privacy-Preserving Content-Based Image Retrieval (PPCBIR) is a branch of SSE research. PPCBIR extracts unique feature vectors from pictures and securely calculates the similarity of pictures on the encrypted feature vectors, achieving a balance between the privacy protection of image data and the search function. The progress of existing PPCBIR schemes is as follows: 1) Multi-user scenario extension, adopting a new framework to support non-shared keys, multi-owner, and multi-user environments, and enhancing the system flexibility through key isolation and malicious user tracking mechanisms. 2) Feature extraction optimization, using convolutional neural networks to extract image features, constructing a retrieval index based on K-means clustering, and improving the feature representation accuracy using deep learning. Although the above schemes have made progress in single-modal retrieval, cross-modal retrieval still faces the following problems: 1) Semantic gap problem: Existing PPCBIR schemes focus on single-modal matching, lacking the quantification of content similarity across different data types, resulting in limited practical applications. 2) Limitations of multi-modal schemes: Existing methods consider multi-modal data composed of multiple modal data, query different modalities separately during querying, and finally use a merging function to summarize the results. However, this method still belongs to a variant of single-modal retrieval and cannot be directly used in CMR, making it difficult to meet the actual application requirements. 3) Low encryption computing efficiency: Existing Content-Mediated Retrieval (CMR) schemes based on Additive Homomorphic Encryption (HE) propose a privacy-preserving Canonical Correlation Analysis (CCA). Although it has certain theoretical advantages, when dealing with a large-scale database, the large amount of encryption computing brought by HE makes it impractical in real-world applications. In addition, this method requires two non-colluding cloud servers, which not only increases the implementation complexity but also raises the cost. 4) Dependence on trusted environment: To avoid computationally complex encryption primitives, there are systems designed relying on a Trusted Execution Environment (TEE), implementing CMR by securely executing CCA inside the TEE. However, due to the compatibility and deployment complexity issues of the trusted environment, this method affects the applicability of CMR and requires additional work to build the integration with existing systems, thus restricting the practical application of this method.In summary, existing technologies have not yet been able to achieve efficient cross-modal retrieval of massive encrypted databases while ensuring security. There is an urgent need for an innovative solution that takes into account computational efficiency, system compatibility, and cross-modal semantic association capabilities. Summary of the Invention

[0006] The purpose of the present invention is to provide a cross-modal retrieval method, system, device and medium for encrypted data, so as to solve the technical problem in the prior art that it is impossible to achieve efficient cross-modal retrieval of massive encrypted databases while ensuring security.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a cross-modal retrieval method for encrypted data, comprising: The client obtains a multimodal database, constructs an encrypted index and a retrieval token based on the multimodal database, transmits the multimodal database and the encrypted index to a cloud server for storage, and transmits the retrieval token to the cloud server. The cloud server searches based on the search token, obtains the search results, and transmits the search results to the client.

[0008] Preferably, the step of constructing an encrypted index based on the multimodal database specifically includes: Extract the binary hash code of each data in the multimodal database; construct a plaintext retrieval index based on the binary hash code; The plaintext index is encrypted using PRF and PRP to obtain an encrypted index.

[0009] Preferably, the constructing of a plaintext search index based on a binary hash code specifically includes: Binary hash code Split into Sub-code, The subcodes are mapped to the corresponding identifiers, and we get hash tables; The hash tables form the plaintext search index.

[0010] Preferably, the retrieval token is constructed by the following steps: Flip subcode bits, and store all the results in a hash table with the key ; Encrypt each element in the hash table to generate a tuple containing the tuple The hash table constitutes the retrieval token.

[0011] Preferably, the cloud server performs a search based on the search token to obtain a search result, which specifically includes: The cloud server retrieves the corresponding encrypted identifier from the encrypted index according to the retrieval token as a candidate item; Combine the candidate items to obtain the retrieval result.

[0012] In a second aspect of the present invention, a cross-modal retrieval system for encrypted data is provided, including: A client for obtaining a multi-modal database, constructing an encrypted index and a retrieval token in sequence according to the multi-modal database; transmitting the multi-modal database and the encrypted index to the cloud server for storage; transmitting the retrieval token to the cloud server; A cloud server for performing retrieval according to the retrieval token, obtaining the retrieval result, and transmitting the retrieval result to the client.

[0013] Preferably, the client includes: A data owner for constructing an encrypted index according to the multi-modal database and transmitting the multi-modal database and the encrypted index to the cloud server for storage; A retrieval user side for constructing a retrieval token according to the multi-modal database and transmitting the retrieval token to the cloud server.

[0014] Preferably, the data owner constructing the encrypted index according to the multi-modal database specifically includes: Extracting the binary hash code of each data in the multi-modal database; constructing a plaintext retrieval index according to the binary hash code; Encrypting the plaintext index through PRF and PRP to obtain the encrypted index.

[0015] In a third aspect of the present invention, an electronic device is provided, which is characterized by including a processor and a memory, and the processor is used to execute a computer program stored in the memory to implement the cross-modal retrieval method for encrypted data described in any one of the above.

[0016] In a fourth aspect of the present invention, a computer-readable storage medium is provided, which is characterized in that the computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, the cross-modal retrieval method for encrypted data described in any one of the above is implemented.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: Aiming at the data security of a large amount of outsourced multi-modal data, the privacy security of query users, and the efficient query requirements, the present invention constructs an SSE framework for cross-modal retrieval and designs a cross-modal ciphertext retrieval system for a large amount of big data. The advantages are mainly the following three points: (1) The application of symmetric searchable encryption in cross-modal retrieval is realized. By using cross-modal hashing learning to transform different multi-modal data into a common space, the semantic relationships between different types of data are successfully preserved, enabling the quantification of the content similarity between them.

[0018] (2) Small computational overhead and low communication cost. During the retrieval process, the present invention converts encrypted Hamming retrieval into multi-keyword queries, effectively reducing the computationally expensive comparison operations and only requiring a single round of communication, significantly reducing the communication overhead.

[0019] (3) Flexible and easy to implement, more applicable in practical applications. The present invention only uses a single cloud server to ensure security, which gives it obvious advantages in terms of deployment and applicability. It does not rely on additional hardware devices, not only reducing the implementation cost but also the difficulty of integrating with existing systems, thus improving the convenience and operability of its practical application. Description of the Drawings

[0020] The specification drawings forming a part of this application are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 It is a typical application scenario diagram of cross-modal retrieval in the prior art; Figure 2 It is a flowchart of the method of the embodiment of the present invention; Figure 3 It is an example diagram of constructing a plaintext index in the embodiment of the present invention; Figure 4 It is an example diagram of encrypting a plaintext index in the embodiment of the present invention; Figure 5 It is a retrieval example diagram of the embodiment of the present invention; Figure 6 It is a combination example diagram of the embodiment of the present invention; Figure 7 It is a system block diagram of the embodiment of the present invention; Figure 8 It is a structural block diagram of an electronic device in the embodiment of the present invention. Detailed Embodiments

[0021] The present invention will be described in detail below with reference to the drawings and in combination with the embodiments. It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other.

[0022] The following detailed descriptions are all exemplary descriptions, aiming to provide further detailed explanations for the present invention. Unless otherwise specified, all technical terms used in the present invention have the same meaning as commonly understood by those of ordinary skill in the art to which this application belongs. The terms used in the present invention are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention.

[0023] The present invention aims to propose a cross-modal retrieval method for encrypted data, which is specifically applied to the storage and processing of massive multi-modal data, meeting the user's high-efficiency requirements in the storage and processing of multi-modal data, and effectively solving the problems of easy leakage and difficult protection of multi-modal data. Through the application of this encryption technology, it can provide strong guarantee for the security of multi-modal data, thus promoting the implementation of its wider application scenarios. Secure and reliable multi-modal data processing can not only enhance users' confidence in data outsourcing services, but also promote the application of data in more fields, further stimulating the potential of data elements and providing a solid technical foundation for the healthy development of the digital economy. This research result is expected to provide new ideas for the secure management and efficient utilization of multi-modal data, and inject new impetus into digital transformation and economic growth.

[0024] See Figure 2 , this application discloses a cross-modal retrieval method for encrypted data, including: S1: The client obtains a multi-modal database, constructs an encrypted index and a retrieval token in sequence according to the multi-modal database; transmits the multi-modal database and the encrypted index to the cloud server for storage; transmits the retrieval token to the cloud server; S2: The cloud server performs retrieval according to the retrieval token, obtains the retrieval result, and transmits the retrieval result to the client.

[0025] In some embodiments, the construction of the encrypted index according to the multi-modal database specifically includes: Extract the binary hash codes of each data in the multi-modal database; construct a plaintext retrieval index according to the binary hash codes; Encrypt the plaintext index through PRF and PRP to obtain the encrypted index.

[0026] In some embodiments, the construction of the plaintext retrieval index according to the binary hash codes specifically includes: Split the binary hash code into sub-codes, map each sub-code to the corresponding identifier to obtain hash tables; The hash tables form the plaintext retrieval index.

[0027] In some embodiments, the retrieval token is constructed through the following steps: Reverse the bits in the sub-code and store all the results in a hash table with the keyword being ; Encrypt each element in the hash table to generate a binary tuple. A retrieval token is composed of hash tables that contain the binary tuple.

[0028] In some embodiments, the cloud server performs a retrieval based on the retrieval token to obtain a retrieval result, which specifically includes: The cloud server retrieves the corresponding encrypted identifier from the encrypted index based on the retrieval token as a candidate; Combine the candidates to obtain the retrieval result.

[0029] Refer to Figure 7 , this application also discloses a cross-modal retrieval system for encrypted data, including: A client, which is used to obtain a multi-modal database, construct an encrypted index and a retrieval token in sequence according to the multi-modal database; transmit the multi-modal database and the encrypted index to the cloud server for storage; transmit the retrieval token to the cloud server; A cloud server, which is used to perform a retrieval based on the retrieval token to obtain a retrieval result, and transmit the retrieval result to the client.

[0030] In some embodiments, the client includes: A data owner side, which is used to construct an encrypted index according to the multi-modal database, and transmit the multi-modal database and the encrypted index to the cloud server for storage; A retrieval user side, which is used to construct a retrieval token according to the multi-modal database, and transmit the retrieval token to the cloud server.

[0031] In some embodiments, the data owner side also generates a key and transmits it to the retrieval user side, including: For a security parameter , the data owner side randomly generates two -bit keys and , and a key . Set . PRF and PRF respectively use and as their keys. The original data (such as text, image, etc.) is encrypted using . For simplicity, the present invention does not explicitly describe the process of encrypting and decrypting the original data, and only considers the identifiers because they are in one-to-one mapping. The data owner side retains and will Sent to the retrieval client.

[0032] In some embodiments, an encrypted index is constructed based on a multimodal database, given a set of keys , a binary database and the number of groups of binary code , the data owner builds an encrypted retrieval index It consists of the following two steps: Step 1: Build a plaintext search index The data owner first builds a plaintext search index. bit binary hash code The data owner splits it into Subcode , and The order in the database (denoted as ) as the key and append it to the corresponding bucket in each inverted index.

[0033] On the surface, subcodes are mapped to identifiers. It is worth noting that subcodes can be treated as integers during execution. In addition, As the simplest bijective function, the function can map the hash code one-to-one to a compact unique representation. In fact, this function can be replaced by any function that can distinguish and index the database. For ease of understanding, the present invention divides the binary hash code into multiple sub-codes of equal length in sequence.

[0034] When the binary hash code is evenly distributed, the inverted index will also be evenly distributed accordingly. However, existing research has defined a suitable code splitting method for different databases, that is, when the distribution of the database changes, the sub-code generation can be optimized to reduce the impact on performance and improve retrieval efficiency. Please note that this assumes yes multiples of . Figure 3 An example is shown here where the settings and Each hash code is in turn divided into two sub-codes of length 8. Each sub-code is treated as a keyword and mapped to its corresponding identifier (here is the order in the database).

[0035] Step 2: Encrypt the index The data owner encrypts the plaintext index. Each inverted index in is encrypted separately. First, select a PRP and a key For each keyword in the inverted index The data owner uses to permute the identifiers in the corresponding set and concatenates them into a string, denoted as .

[0036] Next, the string is padded to the standard length that is the same as the longest string, and is XOR-encrypted with the output of PRF . Finally, it is stored in the encrypted inverted index, and the retrieval keyword is used as the output of another PRF . This padding operation ensures that all values in the encrypted index have the same length, thus enhancing the privacy and security of the data.

[0037] Figure 3 in is encrypted as Figure 4 in , where it is assumed that . The binary code is converted to a decimal integer for to encrypt, and the identifier is encrypted by and . There are three keywords in , , and the maximum number of candidate items is two, denoted as . The padding function ensures that the length of the values in

[0038] is equal to . In some embodiments, the retrieval token is constructed through the following steps: Given a query binary code , Hamming retrieval radius and , the retrieval client generates an encrypted retrieval token and sends it to the cloud server. The retrieval client divides the code into sub-codes . Next, the retrieval client performs the following operations on each sub-code : Flip bits in , and stores all possible results in a hash table , with the keyword being . For example, when the retrieval radius is , assume that

[0100] is a certain sub-code of the query, denoted as

[0039] The retrieval client pairs with each element in , denoted as . Note that binary codes are treated as decimal numbers when encrypting and directly accessing the retrieval index.

[0040] Continuing the previous example, we get

[0041] Finally, the encrypted retrieval token is defined as . Then, the retrieval client will submit to the cloud server. In fact, is a series of hierarchical sub-tokens. A binary tuple in helps to securely find the value related to in .

[0042] In some embodiments, the cloud server retrieves according to the retrieval token and obtains the retrieval result. Given the retrieval index , the retrieval token , the retrieval radius and , the cloud server performs R -Hamming retrieval on the encrypted index and returns the encrypted result to the retrieval client. It includes the following two steps: Step 1: Prepare candidate items For each inverted index, the cloud server selects from the hash table by different radii , here, the binary tuple is simplified to ; then extract the token in the format of , where , . Then, the cloud server uses this token to find the th encrypted inverted index and calculates . If is not in , then ; otherwise, parse as and merge into the set .

[0043] Since the token in has a Hamming distance ( Indicates the number of sub - codes. There are a total of sub - code partitions, (indicating the number of different bits corresponding to the query - partitioned sub - code). Therefore, it can be concluded that finally contains all identifiers whose corresponding binary codes have with the Hamming distance from the th sub - code of the query. That is, where represents the identifier of (previously discussed), and is the th sub - code of the binary code. Note that is the base candidate set for the next combination step.

[0044] Step 2: Combine candidates For , incrementing from 0 to , the cloud server enumerates all combinations that satisfy . Define as the set of all possible combinations of , denoted as , where is the total number of combinations of , and satisfies . For each , the cloud server calculates the set , and the set . Finally, includes all identifiers whose corresponding binary codes have a Hamming distance of from the query , denoted as . R - The result of Hamming retrieval is the union of all , that is, . With , the server will return the relevant ciphertext to the user. The user can use the key to decrypt this data.

[0045] Following in Figure 3, a simple example is shown in Figure 5. First, the user - generated token is retrieved and sent to the cloud server. By given , the cloud server can obtain . is specified in the light - blue table. For example, , while , so it is omitted.

[0046] Specifically, after receiving , the cloud server extracts the sub-tokens in it for using to access , and parses the value by performing exclusive OR with . is the union of the access results. Then, the cloud server combines to obtain the final result, as shown in Figure 6. For example, , because .

[0047] The result obtained through such intersection and union operations effectively reduces the search space and combines the candidate item filtering strategy, thereby improving the efficiency of the system.

[0048] The SSE framework for cross-modal retrieval enables the cloud server to retrieve encrypted data without knowing useful information such as the data and the queries of the retrieving users.

[0049] In summary, the present invention uses the pre-trained cross-modal hashing model CLIP as a feature extractor to generate binary hash codes of different dimensions from each data point. Subsequently, IMI is used to improve the retrieval efficiency of the binary hash codes. Each hash code will be divided into s sub-codes, used as s keywords, and mapped to the corresponding identifiers. Specifically, the cloud server saves an inverted index set composed of s hash tables, denoted as M = (M1,..., Ms), where each mapping is an inverted index, independently storing the sub-code / identifier set. The data owner encrypts the mapping through PRF (Pseudo-Random Function) and PRP (Pseudo-Random Permutation) to generate an encrypted index. The core idea of Search is to convert the Hamming retrieval with a radius of R into a series of exact Hamming retrievals with a distance of r , where 0 ≤ r ≤ R . In addition, each exact Hamming retrieval is further simplified into a series of exact Hamming retrievals with a distance of sub- r , where the sum of these sub- r is equal to r . When the retrieving user wants to perform CMR, the retrieving user first extracts the hash code v and splits v into s sub-codes. Then, the retrieving user enumerates each sub-code with a radius of RHamming balls, and encrypt them as tokens and send them to the cloud server. Then, the cloud server uses these tokens to retrieve the corresponding encrypted identifiers from the EIMI and combines them and returns them to the retrieving user.

[0050] As Figure 8 shown, the present invention also provides an electronic device 100 for implementing a cross-modal retrieval method for encrypted data; The electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on the at least one processor 102, and at least one communication bus 104.

[0051] The memory 101 can be used to store the computer program 103. The processor 102 realizes the steps of the cross-modal retrieval method for encrypted data by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101.

[0052] The memory 101 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the electronic device 100 (such as audio data, etc.). In addition, the memory 101 can include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0053] The at least one processor 102 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 102 can be a microprocessor or the processor 102 can also be any conventional processor, etc. The processor 102 is the control center of the electronic device 100 and connects various parts of the entire electronic device 100 through various interfaces and lines.

[0054] The memory 101 in the electronic device 100 stores multiple instructions to implement a cross-modal retrieval method for encrypted data, and the processor 102 can execute the multiple instructions to implement: The client obtains a multi-modal database, constructs an encrypted index and a retrieval token in sequence according to the multi-modal database; transmits the multi-modal database and the encrypted index to the cloud server for storage; transmits the retrieval token to the cloud server; The cloud server retrieves according to the retrieval token, obtains a retrieval result, and transmits the retrieval result to the client.

[0055] In some embodiments, if the modules / units integrated in the electronic device 100 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above method embodiments of the present invention, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory and read-only memory (ROM, Read-Only Memory).

[0056] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0057] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementing in the process Figure 1 one process or multiple processes and / or blocks Figure 1means for the functions specified in one or more boxes.

[0058] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to work in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction means that implements the functions specified in one Figure 1 process or more processes and / or boxes Figure 1 means for the functions specified in one or more boxes.

[0059] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus, such that a series of operational steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one Figure 1 process or more processes and / or boxes Figure 1 means for the functions specified in one or more boxes.

[0060] In the description of this specification, the descriptions referring to the terms "one embodiment", "example", "specific example", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications or equivalent replacements can still be made to the specific embodiments of the present invention, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered by the protection scope of the claims of the present invention.

Claims

1. A cross-modal retrieval method for encrypted data, characterized in that It includes: The client obtains a multimodal database and constructs an encrypted index and a retrieval token in sequence according to the multimodal database; Transmits the multimodal database and the encrypted index to the cloud server for storage; Transmits the retrieval token to the cloud server; The cloud server retrieves according to the retrieval token, obtains a retrieval result, and transmits the retrieval result to the client.

2. The cross-modal retrieval method for encrypted data according to claim 1, characterized in that The constructing of the encrypted index according to the multimodal database specifically includes: Extracts the binary hash codes of the data in the multimodal database; constructs a plaintext retrieval index according to the binary hash codes; Encrypts the plaintext index through PRF and PRP to obtain the encrypted index.

3. The cross-modal retrieval method for encrypted data according to claim 2, wherein The constructing of the plaintext retrieval index according to the binary hash codes specifically includes: Split the binary hash code into sub-codes. Each sub-code is mapped to the corresponding identifier, obtaining hash tables; A hash table forms a plaintext retrieval index.

4. The cross-modal retrieval method for encrypted data according to claim 3, wherein The retrieval token is constructed through the following steps: Flip the bits in the subcode and store all the results in a hash table with the key being ; Encrypt each element in the hash table to generate a tuple, and the hash tables form a retrieval token.

5. A cross-modal retrieval method for encrypted data according to claim 1, characterized in that The cloud server retrieves according to the retrieval token, obtains a retrieval result, specifically including: The cloud server retrieves the corresponding encrypted identifiers from the encrypted index according to the retrieval token as candidate items; Combines the candidate items to obtain the retrieval result.

6. A cross-modal retrieval system for encrypted data, characterized in that, It includes: A client, which is used to obtain a multimodal database and construct an encrypted index and a retrieval token in sequence according to the multimodal database; Transmits the multimodal database and the encrypted index to the cloud server for storage; Transmits the retrieval token to the cloud server; A cloud server, which is used to retrieve according to the retrieval token, obtain a retrieval result, and transmit the retrieval result to the client.

7. An encrypted data-oriented cross-modal retrieval system according to claim 6, wherein The client includes: A data owner side, which is used to construct an encrypted index according to the multimodal database and transmit the multimodal database and the encrypted index to the cloud server for storage; A retrieval user side, which is used to construct a retrieval token according to the multimodal database and transmit the retrieval token to the cloud server.

8. The cross-modal retrieval system for encrypted data according to claim 7, wherein The data owner side, the constructing of the encrypted index according to the multimodal database specifically includes: Extracts the binary hash codes of the data in the multimodal database; constructs a plaintext retrieval index according to the binary hash codes; Encrypts the plaintext index through PRF and PRP to obtain the encrypted index.

9. An electronic device, characterized in that, It includes a processor and a memory, and the processor is used to execute a computer program stored in the memory to implement the cross-modal retrieval method for encrypted data as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by the processor, it implements the cross-modal retrieval method for encrypted data as described in any one of claims 1 to 5.

Images