A high-recall text-image cross-modal secret retrieval method supporting feature-level confusion and pattern hiding

Abstract

The application discloses a high-recall-rate text-image cross-modal secret retrieval method supporting feature-level confusion and mode hiding, and belongs to the technical field of computer application. The method comprises the following steps: S1, method initialization; S2, user local confusion feature index information; S3, data owner encrypts local data and uploads; S4, data user sends a query request; S5, cloud server searches data; and S6, data user decrypts the returned result of the cloud server. According to the privacy protection levels of different users, the application can alleviate the mode information leakage problem caused by secret data interaction to different degrees in the data retrieval process, optimizes the local differential privacy algorithm by adopting a vector slicing mode, thereby improving the system security and retrieval utility, ensuring that the cross-modal data still has consistent semantic expression ability in the secret environment, and realizing efficient and usable data retrieval function.

Description

Technical Field

[0001] This invention belongs to the field of computer application technology, specifically relating to a high-recall text-image cross-modal dense-state retrieval method that supports feature-level obfuscation and pattern hiding. Background Technology

[0002] Contrastive Language-Image Pre-Training (CLIP) models demonstrate superior cross-modal joint representation capabilities by bridging the gap between visual and linguistic modalities. The core idea is to leverage large-scale image and text datasets, using contrastive learning to enable joint representation of images and text within the same semantic space, thus providing new insights for document analysis and understanding. However, while implementing this approach, the retrieval process may introduce information leakage, including database leaks and the leakage of pattern information during retrieval interactions.

[0003] Traditional cross-modal encrypted retrieval technology provides effective privacy protection for sensitive data, enabling image and text retrieval tasks within the encrypted domain. Encrypting local data before outsourcing effectively prevents servers from directly accessing stored and query content, thus ensuring data and query privacy and achieving the "usable but not visible" characteristic of data. Currently, it has been extensively researched in single-modal text retrieval tasks and is widely applied in cloud storage, data sharing, healthcare, and finance, helping to improve data security.

[0004] However, while existing cross-modal cryptographic retrieval methods effectively prevent servers from directly accessing stored and query content, ensuring data and query privacy, most schemes inevitably leak some pattern information. Typical forms of leakage include: (a) search patterns, revealing identical search requests in the query sequence; (b) access patterns, revealing the set of document identifiers matching the query; (c) size patterns, reflecting the number of matching results contained in the search results; and (d) ranking patterns, reflecting the semantic similarity between the search results and the query. This seemingly harmless leaked information can be maliciously exploited, leading to leak abuse attacks, recovering query keywords, and even reconstructing database records. Current mainstream methods for mitigating these pattern leaks primarily employ cryptographic-based protection schemes, such as ORAM (Oblivious Random Access Memory), Fully Homomorphic Encryption (FHE), and Function Secret Sharing (FSS). These methods effectively protect against pattern leaks, but they typically involve high computational and communication overhead, making them difficult to implement in large-scale data retrieval scenarios.

[0005] If the above methods are directly applied to cross-modal retrieval scenarios, the text and image content need to be encrypted separately. The encoder then extracts features based on the encrypted content, which easily leads to difficulties in cross-modal semantic alignment, thus reducing retrieval efficiency. Combining Local Differential Privacy (LDP) technology directly with encryption technology can alleviate the problems of poor retrieval and semantic alignment difficulties caused by noise interference while ensuring data confidentiality. Specifically, [the following text is incomplete and requires further context]. Taking differential privacy algorithms as an example, feature-level obfuscation can be applied to the original data: the algorithm outputs a set containing multiple vectors, and even if the original feature vectors are still retained in the obfuscation result, they can be hidden within it. Within a set of candidate vectors, pattern leakage is hidden, and the original true feature vector in the index is replaced with this set of candidate vectors.

[0006] However, in large-scale, high-dimensional cross-modal retrieval tasks, direct use When the method is applied to high-dimensional feature confusion, there are two bottlenecks that are difficult to overcome: feature inflation and high-dimensional semantic drift.

[0007] (1) The feature inflation problem refers to the existing The mechanism requires mapping a single original feature vector to a vector containing... The set of candidate vectors causes the amount of data the data owner needs to process locally, the space overhead for storing index data in the cloud, and the communication overhead between the client and the server to all balloon to their original size. This linear expansion can lead to system I / O bottlenecks in large-scale cross-modal indexing scenarios, severely limiting the scalability of retrieval.

[0008] (2) The high-dimensional semantic drift problem refers to the fact that in a high-dimensional feature space of 512 or 1024 dimensions, if a probability perturbation is directly applied to the high-dimensional vector, the confused vector is very likely to deviate from the original semantic center due to the contradiction between the sampling set size and the original semantic characterization ability. When the sampling scale is too large, the semantic randomness of the output vector increases; when the sampling scale is too small, it cannot provide sufficient noise coverage. This drift phenomenon directly destroys the alignment relationship between image-text vectors in the joint semantic space, leading to The recall rate of the search plummeted.

[0009] In summary, existing dense-state retrieval schemes struggle to achieve an effective balance between pattern information hiding, semantic expression accuracy, and system resource overhead. Summary of the Invention

[0010] To address the challenge of achieving an effective balance between pattern information hiding, semantic representation accuracy, and system resource overhead in existing dense-state retrieval schemes, this invention proposes a high-recall text-image cross-modal dense-state retrieval method that supports feature-level obfuscation and pattern hiding. This method can mitigate pattern information leakage caused by dense-state data interaction to varying degrees during data retrieval, depending on the privacy protection level of different users. It improves system security and retrieval efficiency by optimizing the localized differential privacy algorithm using methods such as vector slicing. Simultaneously, while protecting data confidentiality, this invention effectively avoids... Problems such as feature inflation and high-dimensional semantic drift brought about by the algorithm are addressed to ensure that cross-modal data still has consistent semantic expression capabilities in dense environments, thereby achieving efficient and usable data retrieval functions.

[0011] Technical Solution: To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0012] A high-recall text-image cross-modal dense-state retrieval method supporting feature-level obfuscation and pattern hiding includes the following steps:

[0013] S1, Method Initialization: Generates data encryption keys for the data owner, including index encryption keys and image encryption keys;

[0014] S2. User-local obfuscated feature index information: Extract image feature vectors, and based on differential privacy budget, combine vector slicing with localized differential privacy technology to generate corresponding obfuscated feature vectors for each private feature vector;

[0015] S3. Data owner encrypts local data and uploads it: The data owner uses the index encryption key to uniformly encrypt and obfuscate the feature vector to generate a secure index; uses the image key to encrypt the local image content to obtain the image ciphertext; and uploads the secure index and the image ciphertext together to the cloud server.

[0016] S4. Data user sends query request: The data user applies to the data owner for a data encryption key. The data user uses a publicly available text feature encoder to extract query features from the query text, uses an index encryption key to encrypt the query features to generate a search token, and uploads it to the cloud server.

[0017] S5. Cloud server data search: Data users submit query requests to the cloud server, which then performs a full-domain search.

[0018] S6. The data user decrypts the results returned by the cloud server.

[0019] As a preferred option, the specific implementation details in S1 are as follows:

[0020] Data owner: The producer and owner of privacy data, who can privatize local data, request cloud server storage, and share the data encryption key with data users to complete the authorization;

[0021] Data users: Generate query tokens and request to query data; data users share a key with data owners to obtain authorization and can apply to the cloud server to query data;

[0022] Cloud server: Provides remote data storage services, including security feature indexes and image content storage, and completes user data query requests after receiving search credentials.

[0023] As a preferred option, the specific implementation process in S2 is as follows:

[0024] S201. The data owner uses a publicly available image encoder to extract image feature vectors. ;

[0025] S202, under a given differential privacy budget Below, the user performs perturbation based on the exponential mechanism, resulting in an image feature vector for each index position. Construct the perturbation vector corresponding to the original private vector. ;

[0026] S203. Using the constructed perturbation vector corresponding to the original private vector. Replace the image feature vector at the original index position .

[0027] As a preferred option, the specific implementation process in S202 is as follows:

[0028] Step 1, Subcodebook Construction: Select a large public dataset as the sampling candidate set, and then process the extracted feature vectors. Perform vector quantization to construct a subcodebook ;

[0029] Step 2, Vector Subspace Segmentation: Divide the image feature vectors into subspaces. Divide into M segments to obtain the corresponding vectors of each segment. ;

[0030] Step 3, Obfuscate Subvectors: In the corresponding subcodebook Find the subvector in the middle Most similar subvector ;

[0031] Step 4: Construct the perturbation vector: Concatenate all the confusion sub-vectors to obtain the concatenated vector of all perturbation sub-vectors. Then, normalize it to obtain the perturbation vector corresponding to the original private vector. .

[0032] Preferably, in step 3, the specific implementation details are as follows:

[0033] The data owner calculates the probability of selection of the confusion subvector. Perform an exponential mechanism to obfuscate the selection process and determine the probability of being selected. The specific calculation formula is as follows:

[0034]

[0035] in, Indicates the first The subcodebook of segments; Represents the utility function. Indicates sensitivity; This represents a specific output value in the output space of the exponential mechanism; This represents any traversed element in the output space; Indicates differential privacy budget; Subvector The most similar subvector in the corresponding subcodebook.

[0036] As a preferred option, the specific implementation process in S3 is as follows:

[0037] S301, The data owner uses the index encryption key. The perturbation vector corresponding to the original private vector after confusion Encryption is performed to obtain the ciphertext of the obfuscated image vector. Then, the ciphertext of the generated obfuscated image vectors is aggregated to build a secure index for the local data. ;

[0038] S302, The data owner uses the image key. Image library Each image in Encryption, forming an image ciphertext library ;

[0039] S303. Finally, the obtained secure index and image ciphertext library Upload them together to the cloud server.

[0040] As a preferred option, the specific implementation process in S4 is as follows:

[0041] S401, Data user initiates text query At that time, the feature vector of the query text is extracted using a publicly available text encoder. ;

[0042] S402, The data owner uses the index encryption key. Feature vector of query text Encryption is performed to generate ciphertext for the query vector. And upload it to the cloud server.

[0043] As a preferred option, the specific implementation process in S5 is as follows:

[0044] S501, cloud server for security index The various indexes are traversed; then the cloud server uses the ASPE algorithm to combine the search token with the ciphertext of the obfuscated image vector. Similarity is calculated in the dense state domain, and the calculated similarity value is used as the similarity score between the query ciphertext vector and the current image ciphertext vector. ;

[0045] S502. Finally, sort all similarity scores and locate... The set of images corresponding to the matching index Returned to the data user.

[0046] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0047] (1) Based on CLIP, this invention adds a lightweight encryption algorithm and privacy protection functions for pattern information, supporting the concealment of sensitive information during the query process, significantly mitigating the security risks that may be caused by the leakage of ambiguous information. In addition, the localized differential privacy obfuscation algorithm is optimized through vector segmentation and other methods, thereby improving the retrieval recall rate. While ensuring data confidentiality and supporting efficient data retrieval, it can effectively alleviate the problem of attackers guessing user intent.

[0048] (2) This invention allows data users to perform cross-modal retrieval across the entire domain, effectively avoiding the limitations of traditional methods. This invention addresses the issues of feature dilation and high-dimensional semantic drift in the cross-modal matching process of algorithms, aiming to maintain the semantic expressiveness of multiple data modalities. It allows a semi-honest server to observe obfuscated pattern information, preventing the leakage of true pattern information during data retrieval and improving system security and reliability.

[0049] (3) The method of the present invention supports the concealment of sensitive information during the query process, which significantly alleviates the security risks that may be caused by the leakage of ambiguous information, thereby improving the security and flexibility of the system. At the same time, in cross-modal scenarios, the method can maintain the semantic expressive ability of multiple data modalities, significantly improving the retrieval recall rate. Attached Figure Description

[0050] Figure 1 This is a system model diagram of the present invention;

[0051] Figure 2 A flowchart of the overall solution of the present invention.

[0052] Figure 3 This is a schematic diagram of the obfuscation scheme of the present invention;

[0053] Figure 4 This is a schematic diagram illustrating the search process of the present invention. Detailed Implementation

[0054] The present invention will be further illustrated below with reference to specific embodiments. These embodiments are implemented based on the technical solutions of the present invention, and it should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

[0055] This embodiment provides a high-recall text-image cross-modal dense-state retrieval method supporting feature-level obfuscation and pattern hiding, specifically, a method for optimizing recall based on vector segmentation and feature-level obfuscation within a cross-modal dense-state retrieval protection framework. This method is based on the CLIP model and combines encryption technology with localized differential privacy technology, effectively mitigating... The algorithm addresses the issues of feature inflation and high-dimensional semantic drift; it can simultaneously balance privacy and security, retrieval accuracy and resource consumption, thus adapting to the higher security requirements of complex application scenarios.

[0056] like Figure 1 As shown, the participating entities in this method mainly include data owners, data users, and cloud servers; different entities have different functions and responsibilities.

[0057] Data Owner: The creator and owner of privacy data, who can privatize local data, request cloud server storage, and share the data encryption key with data users to complete authorization. The data owner has the right to decide on the generation, collection, and processing methods of the data, and is responsible for the security and compliance of the data. Local data is encrypted using the data encryption key, and the corresponding encrypted database is uploaded to the cloud server.

[0058] Data user: Generates a query token and requests to query data. The data user shares a key with the data owner to obtain authorization and can apply to the cloud server to query data.

[0059] Cloud servers provide remote data storage services, including secure feature indexes and image content storage, and fulfill user data query requests upon receiving search credentials. Cloud servers are honest and inquisitive; they store data uploaded by data owners and provide secure search services to data users.

[0060] This approach incorporates defenses against potentially honest but curious cloud servers that, while honestly executing user requests, may attempt to snoop on users' private data.

[0061] This example requires the application of a high-recall text-image cross-modal dense state retrieval method that supports feature-level obfuscation and pattern hiding. The relevant algorithm is explained below:

[0062]

[0063] The method of this invention supports finding semantically similar data across different modalities, while also controlling the degree of pattern leakage based on the user's privacy preferences, aiming to construct a secure and efficient pattern hiding mechanism. Specific functions include:

[0064] 1. Cross-data modality retrieval: This invention designs a feature-level protection mechanism to ensure data leakage protection while effectively maintaining the semantic expression capabilities of multiple modalities and completing similarity matching in the same feature space.

[0065] 2. Controlled Pattern Leakage: This invention provides a pattern information obfuscation mechanism that stores a secure feature index with controlled pattern leakage on a cloud server, based on the data owner's privacy preferences. This solution effectively defends against word frequency attacks and other potential attacks, while reducing the possibility of adversaries guessing user intent while protecting data privacy. Furthermore, it ensures system storage and computational efficiency, meeting the actual security requirements under semi-honest server monitoring.

[0066] 3. High-efficiency search execution: This invention supports efficient retrieval of encrypted data. The cloud server performs vector similarity calculations in the encrypted domain, enabling data users to quickly obtain the data they need without decrypting all encrypted content, thus balancing data availability and security.

[0067] 4. High Recall Rate: This invention provides a high recall rate retrieval method. The data owner locally segments the private vector to obtain several sub-vectors and constructs a sampling space library, i.e., a sub-codebook, for each sub-vector. Based on the distance between the concatenated vector of the most similar sub-vector and the original private vector, the obfuscated output vector is determined. The resulting concatenated vector can better preserve the semantic features of the private vector, thus ensuring retrieval performance while providing privacy protection.

[0068] Through the above functions, this invention protects data confidentiality and hides pattern information while maintaining the semantic expressive ability of multiple data modalities, significantly improving the security and practicality of the system, and is applicable to various scenarios such as cloud storage and data sharing.

[0069] This invention combines localized differential privacy technology with vector segmentation to achieve feature-level obfuscation and improve retrieval recall, but the specific implementation is not limited to this method. This application takes the cross-modal retrieval task of text-to-image search as an example, based on... The five algorithms, participating entities, and related technical methods, such as Figure 1 , Figure 2 As shown, the implementation steps of this method include:

[0070] S1, Method Initialization: Generate data encryption keys for all data owners, including index encryption keys and image encryption keys;

[0071] These keys are shared among system users, and data users receive and are assigned corresponding keys by the data owner before accessing the data. Simultaneously, the method initializes, generating an empty encrypted database.

[0072] In this step, a key generation algorithm is used to generate a data key, specifically:

[0073] run Generate a data encryption key for each user, namely the index encryption key. Image key Specifically, index encryption key. Used for encrypting feature vectors, image keys Used to encrypt image content.

[0074] S2. User-local obfuscated feature index information: The data owner uses a publicly available image feature encoder to extract image features and establishes an independent feature index for each image, the content of which is the corresponding feature vector. Under a given differential privacy budget ε, the user adopts a divide-and-conquer strategy, combining vector sharding with localized differential privacy technology (index mechanism). This process uses feature distance relationships as a utility function to guide obfuscation, generating a corresponding obfuscated feature vector for each private feature vector, and finally completing the index replacement.

[0075] Algorithm using obfuscated feature indexing The data owner uses a publicly available image encoder to extract image feature vectors. Given a differential privacy budget Next, the data owner generates a perturbation vector for each feature index that corresponds to the original private vector. After obfuscation, the data owner will change the perturbation vector corresponding to the original private vector. Replace the image feature vector at the original index position .

[0076] like Figure 3 As shown, specifically:

[0077] S201. The data owner uses a publicly available image encoder to extract image feature vectors. ;

[0078] Extract each image using an image encoder. The feature vectors are as follows:

[0079]

[0080] in, Represents the image feature vector. Represents image content, This represents the image encoder in the Clip model.

[0081] S202, under a given differential privacy budget Below, the user performs perturbation based on the exponential mechanism, resulting in an image feature vector for each index position. Construct the perturbation vector corresponding to the original private vector. Both are the same size;

[0082] Step 1, Subcodebook Construction: Select a large public dataset as the sampling candidate set, and then process the extracted feature vectors. Vector quantization is performed to construct a subcodebook. Specifically:

[0083] First, for each feature vector in the sampled candidate set Perform vector segmentation, i.e.:

[0084]

[0085] in, This represents the feature vector extracted from the sampled candidate set; express Subvectors after segmentation; Indicates the number of segments in the segmentation vector; This represents a vector splitting function.

[0086] Then, sample all the candidate sets. Joke Feature Vector Set ,pass Algorithm pair Perform clustering, minimize the sum of squares within each cluster, and finally output... The cluster center, i.e. the th cluster center Segment Subcodebook for:

[0087]

[0088] in, Indicates the first The subcodebook of the segment; Indicates the number of cluster centers; Indicates the cluster centers obtained by the clustering algorithm; This represents a clustering algorithm; Indicates all the first A set of feature vectors for jokes. This indicates the number of segments in the split vector.

[0089] Finally, for The function is executed on each feature segment separately, ultimately generating... An independent subcodebook It can be pre-trained and saved as a separate file, supporting multiple reuses.

[0090] Step 2, Vector Subspace Segmentation: Divide the image feature vectors into subspaces. Cut into Segment, to obtain the corresponding molecular vectors of each segment. Specifically:

[0091]

[0092] in, Represents the image feature vector; Indicates the number of segments in the segmentation vector; This represents a vector splitting function.

[0093] Step 3, Obfuscate Sub-vectors: Employ a divide-and-conquer strategy, obfuscating the sub-codebooks of the corresponding sub-segments. Find the subvector in the middle Most similar subvector ;

[0094] The data owner calculates the probability of selection of the confusion subvector. Performing an exponential mechanism to obfuscate the process, the probability of being selected. The specific calculation formula is as follows:

[0095]

[0096] in, Indicates the first The subcodebook of the segment; Represents the utility function. Indicates sensitivity. This represents a specific output value in the output space of the exponential mechanism, i.e., the target result ultimately selected by the mechanism. It represents any traversal element in the output space and is a traversal variable used to normalize the probability distribution; This refers to differential privacy budgeting. Subvector The most similar subvector in the corresponding subcodebook.

[0097] For each subvector The most similar subvector in the corresponding subcodebook Using cosine distance as the determinant of the utility function, in the first... Segment Subcodebook middle and sub-vectors Most similar subvector Samples that are closer in distance have a higher probability of being used as confusing sub-vectors. Here, the differential privacy perturbation output of the sub-vector is set to... This is the confusion sub-vector.

[0098] Step 4: Construct the perturbation vector: The data owner performs the above operation on all the segmented sub-vectors, and then concatenates all the obfuscated sub-vectors to obtain the concatenated vector of each perturbation sub-vector. Then, normalization is performed to obtain the perturbation vector corresponding to the original private vector. Specifically:

[0099]

[0100]

[0101] in, This represents the obfuscated subvector, i.e., the differential privacy perturbation output of the subvector; This represents the concatenated vector of the various perturbation sub-vectors. This represents the perturbation vector corresponding to the original private vector. This represents a vector concatenation function. This represents the normalization function.

[0102] S203. Using the constructed perturbation vector corresponding to the original private vector. Replace the image feature vector at the original index position .

[0103] S3. Data owner uploads encrypted image: The data owner uses the index encryption key to uniformly encrypt and obfuscate the feature vector, generating a secure index; and uses the image key to encrypt the local image content, obtaining the encrypted image. The data owner uploads both the secure index and the encrypted image to the cloud server.

[0104] Specifically:

[0105] S301, The data owner uses the index encryption key. The perturbation vector corresponding to the original private vector after confusion Encryption is performed to obtain the ciphertext of the obfuscated image vector. Finally, the generated ciphertext feature vectors are aggregated to construct a secure index for the local data. .

[0106] in, The ciphertext representing the obfuscated image vector. Indicates the index encryption key. Indicates a secure index. Indicates encryption algorithm Encryption function for image feature vectors.

[0107] S302, The data owner uses the image key. Image library Each image in Encryption, forming an image ciphertext library .

[0108] in, Represents an image encrypted library, Represents the image key. Represents an image library. This refers to the content of an image in the image library. This represents the encryption function for image content. Encrypted text representing the content of an image.

[0109] S303 Finally, each data owner obtains two encrypted messages locally. Then upload it to the cloud server.

[0110] in, Indicates a secure index. This represents an image encrypted library.

[0111] S4. Data user sends a query request: The data user requests a data encryption key from the data owner, including an index encryption key and an image key. The data user uses a publicly available text feature encoder to extract query features from the query text, encrypts the query features using the index encryption key to generate a search token, and uploads it to the cloud server.

[0112] like Figure 3 As shown, specifically:

[0113] S401, When a data user initiates a text query At that time, the first step is to use a publicly available text encoder to extract its feature representation. .

[0114] in, This represents the feature vector of the query text. This represents the query text submitted by the data user. This represents the text encoder in the Clip model.

[0115] S402, The data owner uses the index encryption key. Feature vector of query text Encryption is performed to generate an encrypted query trap. And upload it to the cloud server.

[0116] in, The ciphertext representing the query vector. Indicates the index encryption key. This represents the feature vector of the query text. Indicates encryption algorithm The encryption function for the query vector.

[0117] S5. Cloud Server Data Search: Data users can query data uploaded by all users across the entire domain, including their own uploaded data. Data users submit query requests to the cloud server, which then performs a domain-wide search.

[0118] The cloud server needs to calculate the similarity between the text features of the user query and each index, and use this similarity value as the publication score (not the actual score) for the current index. Finally, the indexes are sorted according to this publication score, and the result is returned. Match the ciphertext of the image corresponding to the index.

[0119] like Figure 3 As shown, specifically:

[0120] S501, cloud server for security index The various indexes are traversed; then the cloud server uses the ASPE algorithm to match the search token with the perturbation image feature vector to be retrieved on the server side. Similarity calculation in the dense state domain .

[0121] in, Indicates encryption algorithm The function for calculating the inner product of two ciphertext vectors. Ciphertext representing the obfuscated image vector; The ciphertext represents the query vector; the similarity score calculated using the inner product is the similarity value of the corresponding plaintext vector. Then, this similarity value is used as the publication similarity score between the current index and the search token. (Instead of the actual score).

[0122] in, This represents the perturbation image feature vector (plaintext) corresponding to the original private vector. The feature vector (plaintext) representing the query text.

[0123] S502. Finally, sort all the posts by similarity and locate them. The set of images corresponding to the matching index Returned to the data user.

[0124] in, express Match the ciphertext of the image corresponding to the index.

[0125] S6. The data user decrypts the results returned by the cloud server.

[0126] Data users decrypt search results: Data users receive noisy search results from the cloud server and decrypt the returned ciphertext results locally using an image key.

[0127] Data users use the image key to decrypt the returned result and obtain the plaintext image. The decrypted image content retains the original image quality.

[0128] in, This indicates the content of an image in the image library. Represents the image key. express Match the ciphertext of the image corresponding to the index. Indicates from The ciphertext set of the matching results The ciphertext of a single image to be decrypted selected from the image; Indicates to Execute the symmetric decryption function.

[0129] The above embodiments use the cross-modal scenario of text retrieval of images as an example. This invention is also applicable to bidirectional cross-modal retrieval tasks such as image retrieval of text. To achieve text retrieval from an image, simply call the corresponding text encoder or image encoder to extract features according to the actual query modality and the stored modality, while other encryption protection and retrieval processes remain unchanged. Therefore, this invention can simultaneously support both "text-to-image" and "image-to-text" cross-modal retrieval needs. This invention employs a feature-level protection strategy, which is also applicable to multimodal retrieval tasks, such as audio and video retrieval needs. Simply call the encoder of the corresponding target modality to extract features, while other encryption protection and retrieval processes remain unchanged.

[0130] This invention, building upon cross-modal retrieval methods, adds features to ensure the confidentiality of stored data content and conceal the leakage of pattern information. It allows a semi-honest server to observe obfuscated pattern information based on the user's privacy preferences, preventing the leakage of genuine pattern information during data retrieval and improving system security and reliability. During data perturbation, the invention segments the vector and uses a segmented obfuscation method to perturb the original private vector. Based on the parallel combination theorem of differential privacy, it ensures that the output perturbed vector has high semantic expressive power at a high privacy level, thereby effectively improving retrieval recall and retrieval performance.

[0131] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A high-recall text-image cross-modal dense-state retrieval method supporting feature-level obfuscation and pattern hiding, characterized in that, Includes the following steps: S1, Method Initialization: Generates data encryption keys for the data owner, including index encryption keys and image encryption keys; S2. User-local obfuscated feature index information: Extract image feature vectors, and based on differential privacy budget, combine vector slicing with localized differential privacy technology to generate corresponding obfuscated feature vectors for each private feature vector; S3. Data owner encrypts local data and uploads it: The data owner uses the index encryption key to uniformly encrypt and obfuscate the feature vector to generate a secure index; uses the image key to encrypt the local image content to obtain the image ciphertext; and uploads the secure index and the image ciphertext together to the cloud server. S4. Data user sends query request: The data user applies to the data owner for a data encryption key. The data user uses a publicly available text feature encoder to extract query features from the query text, uses an index encryption key to encrypt the query features to generate a search token, and uploads it to the cloud server. S5. Cloud server data search: Data users submit query requests to the cloud server, which then performs a full-domain search. S6. The data user decrypts the results returned by the cloud server.

2. The high-recall text-image cross-modal dense-state retrieval method supporting feature-level obfuscation and pattern hiding according to claim 1, characterized in that, In S1, the specific implementation details are as follows: Data owner: The producer and owner of privacy data, who can privatize local data, request cloud server storage, and share the data encryption key with data users to complete the authorization; Data user: Generates a query token and requests to query data; Data users and data owners share a key to obtain authorization to request data queries from the cloud server; Cloud server: Provides remote data storage services, including security feature indexes and image content storage, and completes user data query requests after receiving search credentials.

3. The high-recall text-image cross-modal dense-state retrieval method supporting feature-level obfuscation and pattern hiding according to claim 1, characterized in that, In S2, the specific implementation process is as follows: S201. The data owner uses a publicly available image encoder to extract image feature vectors. ; S202, under a given differential privacy budget Below, the user performs perturbation based on the exponential mechanism, resulting in an image feature vector for each index position. Construct the perturbation vector corresponding to the original private vector. ; S203. Using the constructed perturbation vector corresponding to the original private vector. Replace the image feature vector at the original index position .

4. The high-recall text-image cross-modal dense-state retrieval method supporting feature-level obfuscation and pattern hiding according to claim 3, characterized in that, In S202, the specific implementation process is as follows: Step 1, Subcodebook Construction: Select a large public dataset as the sampling candidate set, and process the extracted feature vectors. Perform vector quantization to construct a subcodebook ; Step 2, Vector Subspace Segmentation: Divide the image feature vectors into subspaces. Divide into M segments to obtain the corresponding vectors of each segment. ; Step 3, Obfuscate Subvectors: In the corresponding subcodebook Find the subvector in the middle Most similar subvector ; Step 4: Construct the perturbation vector: Concatenate all the confusion sub-vectors to obtain the concatenated vector of all perturbation sub-vectors. Then, normalize it to obtain the perturbation vector corresponding to the original private vector. .

5. The high-recall text-image cross-modal dense-state retrieval method supporting feature-level obfuscation and pattern hiding according to claim 4, characterized in that, In step 3, the specific implementation details are as follows: The data owner calculates the probability of selection of the confusion subvector. Perform an exponential mechanism to obfuscate the selection process and determine the probability of being selected. The specific calculation formula is as follows: ； in, Indicates the first The subcodebook of segments; Represents the utility function. Indicates sensitivity; This represents a specific output value in the output space of the exponential mechanism; This represents any traversed element in the output space; Indicates differential privacy budget; Subvector The most similar subvector in the corresponding subcodebook.

6. The high-recall text-image cross-modal dense-state retrieval method supporting feature-level obfuscation and pattern hiding according to claim 1, characterized in that, In S3, the specific implementation process is as follows: S301, The data owner uses the index encryption key. The perturbation vector corresponding to the original private vector after confusion Encryption is performed to obtain the ciphertext of the obfuscated image vector. Then, the ciphertext of the generated obfuscated image vectors is aggregated to build a secure index for the local data. ; S302, The data owner uses the image key. Image library Each image in Encryption, forming an image ciphertext library ; S303. Finally, the obtained secure index and image ciphertext library Upload them together to the cloud server.

7. The high-recall text-image cross-modal dense-state retrieval method supporting feature-level obfuscation and pattern hiding according to claim 1, characterized in that, In S4, the specific implementation process is as follows: S401, Data user initiates text query At that time, the feature vector of the query text is extracted using a publicly available text encoder. ; S402, The data owner uses the index encryption key. Feature vector of query text Encryption is performed to generate ciphertext for the query vector. And upload it to the cloud server.

8. The high-recall text-image cross-modal dense-state retrieval method supporting feature-level obfuscation and pattern hiding according to claim 1, characterized in that, In S5, the specific implementation process is as follows: S501, cloud server for security index The various indexes are traversed; then the cloud server uses the ASPE algorithm to combine the search token with the ciphertext of the obfuscated image vector. Similarity is calculated in the dense state domain, and the calculated similarity value is used as the similarity score between the query ciphertext vector and the current image ciphertext vector. ; S502. Finally, sort all similarity scores and locate... The set of images corresponding to the matching index Returned to the data user.

Images