Cross-modal data retrieval method based on robust hamming coding

Abstract

The application discloses a cross-modal data retrieval method based on robust hamming coding, and the specific process is as follows: firstly, relevant parameters are generated, and the parameters are sent to a query user and a cloud server CS1; secondly, a data owner acquires hash codes of different modal data, maps the hash codes to a symmetric set, constructs a reversible bloom filter, encrypts original data, and sends the reversible bloom filter and the encrypted original data to the cloud server CS1 and a cloud server CS2; the cloud server CS1 locally stores the reversible bloom filter, and the cloud server CS2 stores the encrypted original data; thirdly, the query user decodes by comparison with query data, obtains a similarity measurement result, and returns relevant retrieval results to the query user in ascending order according to the similarity measurement result; and finally, the query user decrypts a secret key to obtain a final retrieval result. The method solves the problems of low security and robustness in the prior art.

Description

Technical Field

[0001] This invention belongs to the fields of data security and cryptography application technology, specifically relating to a cross-modal data retrieval method based on robust Hamming coding. Background Technology

[0002] Cross-modal retrieval, a current hot topic in multimedia research, can meet the search needs of users in the Internet era across different media data. For example, given an image sample, it can effectively and quickly retrieve related text, video, and other modal data. Compared to traditional single-modal retrieval, the challenge of cross-modal retrieval lies in the heterogeneity of different modal data, making direct comparison difficult and creating a key issue of the heterogeneity gap. Therefore, retrieving similar samples from large-scale heterogeneous datasets has become a challenging task. Hashing methods aim to map high-dimensional raw data to low-dimensional binary hash codes in a unified Hamming space while preserving the semantic relationships in the original feature space. Cross-media retrieval is then achieved by calculating the Hamming distance between the hash codes of the query and the database sample. Due to its small storage space and fast retrieval speed, it has been successfully applied to solve the problem of large-scale multimedia data retrieval. Many cross-modal hashing methods have been proposed to effectively map data from different modalities to a unified common Hamming space. For example, the classic Collective Matrix Factorization Hashing (CMFH) method decomposes the feature matrices of data from different modalities and obtains latent implicit subspaces, thereby comparing the similarity of heterogeneous data. However, traditional cross-modal hashing schemes rely solely on XOR operations between hash codes for similarity measurement. This method has low security and robustness and is not suitable for complex network environments. Summary of the Invention

[0003] The purpose of this invention is to provide a cross-modal data retrieval method based on robust Hamming coding, which solves the problems of low security and robustness in existing methods.

[0004] The technical solution adopted in this invention is a cross-modal data retrieval method based on robust Hamming coding. The specific process is as follows: First, relevant parameters are generated and sent to the query user and cloud server CS1. Second, the data owner obtains the hash codes of different modalities and maps them to symmetric sets. Simultaneously, a reversible Bloom filter is constructed to encrypt the original data. The reversible Bloom filter and the encrypted original data are sent to cloud servers CS1 and CS2. Cloud server CS1 stores the reversible Bloom filter locally, and cloud server CS2 stores the encrypted original data. Third, the query user compares and decodes the data to obtain a similarity measurement result. Based on the similarity measurement result, relevant retrieval results are returned to the query user in ascending order. Finally, the query user decrypts the key to obtain the final retrieval result.

[0005] The invention is further characterized in that,

[0006] The specific steps are as follows:

[0007] Step 1: The data owner selects the security parameter κ, generates the key K for the symmetric encryption algorithm Enc, and randomly selects a set of hash functions {h1,...,h...}. k Set a similarity threshold θ, and construct a k-row, 2θ-column random matrix. The data owner will set the parameters {(h1,...,h...} k The similarity threshold θ is sent to the cloud server CS1 and the query user is sent to the queried user.

[0008] Step 2: Encrypt the original modal data and construct a reversible Bloom filter IBLT, where IBLT = {IBLT} j |j∈[n]} and update it. The data owner sends the updated reversible Bloom filter IBLT to cloud server CS1 and sends the encrypted original modal data to cloud server CS2.

[0009] Step 3: Cloud server CS1 receives the updated reversible Bloom filter IBLT uploaded by the data owner, where IBLT = {IBLT} j |j∈[n]} and stored locally, while the cloud server CS2 stores the encrypted raw data uploaded by the data owner;

[0010] Step 4: The user encodes and maps the multimedia data they want to query into the Hamming space in the same way, and constructs a reversible Bloom filter IBLT′, where IBLT′ = {IBLT′}. j |j∈[n]}, send the query reversible Bloom filter IBLT′ to the cloud server CS1;

[0011] Step 5: Cloud server CS1 compares the similarity between the queried reversible Bloom filter IBLT′ and the updated reversible Bloom filter IBLT, and based on the similarity results, it collaborates with cloud server CS2 to return the search results in ascending order.

[0012] Step 6: The user uses key K to decrypt the encrypted data to obtain the plaintext data.

[0013] The specific process of step 2 is as follows:

[0014] Step 2.1, for each type of modal data, there are n original data points M = {m}. j For any type of modal data, the data owner uses the collaborative matrix factorization method to decompose any modal data m. j The characteristic matrix X j Project onto a unified Hamming representation space and obtain the corresponding binary hash code x. j At the same time, the original data m is encrypted using key K. j Calculate encrypted data c j =Enc(m j ,K), to obtain the encrypted original modal data C={c j |j∈[n]};

[0015] Step 2.2, the data owner obtains any binary hash code x. j =x j1 …x ji …x jl (i∈[l]), where l is the length of the string, and the data owner uses any binary hash code x. j Construct the corresponding symmetric set Y j ;

[0016] The process of constructing a symmetric set is as follows: for hash code x j Each bit x ji Encode sequentially and calculate y. ji ,y ji =2i-x ji , thus obtaining the symmetric set Y ji ={y j1 ,...,y ji ,...,y jl};

[0017] Step 2.3, construct a reversible Bloom filter IBLT, where IBLT = {IBLT} j |j∈[n]};

[0018] Step 2.4, update the reversible Bloom filter IBLT;

[0019] Step 2.5: The data owner sends the updated reversible Bloom filter IBLT to the cloud server CS1, and sends the encrypted original modal data C = {c j |j∈[n]} is sent to the cloud server CS2.

[0020] The specific process of step 2.3 is as follows: First, there exists a sample space for data with different modalities. Where T represents the number of modalities, and n represents the number of samples in each different modality. Represents sample data, Representing sample data The corresponding feature matrix; secondly, for sample data m under any type of modality. j The characteristic matrix X j For j∈[n], the data owner uses the collaborative matrix factorization method to decompose it into a binary semantic space, generating a unified set of binary hash codes {x1,...,x j ,...,x n Finally, for any hash code x... j Encode the data and construct a reversible Bloom filter, denoted as IBLT, where IBLT = {IBLT}. j |j∈[n]}.

[0021] The specific process of step 2.4 is as follows: There exists a matrix It is also equipped with a randomly selected set of hash functions {h1,...,h... α ,...,h k}, each hash function h α (α∈[k]) corresponds to the α-th row of the matrix. In the α-th row of the matrix, choose the hash function h. α Calculate set element Y ji ={y j1 ,y j2 ,...,y jl The hash value of}{h} α (y j1 ), h α (y j2 ),...,h α (y jl )}, and write the obtained hash value into the α-th row h α (y ji The position of the column is updated, and the value of each position in the hash table of the reversible Bloom filter is updated to... The reversible Bloom filter IBLT has completed its update operation.

[0022] The specific process of step 4 is as follows:

[0023] Step 4.1: Using the collaborative matrix factorization method, repeat step 2.1 to project the feature matrix of the query data onto a unified Hamming space to obtain the binary hash code x′. i ;

[0024] Step 4.2: Repeat steps 2.1 to 2.4 to encode the binary hash code and construct the reversible Bloom filter IBLT′ for the query, where IBLT′ = {IBLT′} j |j∈[n]},;

[0025] Step 4.3: The querying user sends the constructed reversible Bloom filter IBLT′ to the cloud server CS1.

[0026] The specific process of step 5 is as follows:

[0027] Step 5.1, cloud server CS1 will obtain IBLT′, IBLT′={IBLT′ j |j∈[n]} and the updated reversible Bloom filter IBLT from step 2, IBLT={IBLT j Subtracting each entry from |j∈[n]} yields a new reversible Bloom filter IBLT″,,IBLT″={IBLT″ j =IBLT j -IBLT′ j |(j∈[1,n])};

[0028] Step 5.2, cloud server CS1 applies the new reversible Bloom filter IBLT″ j By decoding each element one by one, the number ω of the element differences in the original symmetric set can be recovered. j Cloud server CS1 utilizes ω j The size of the two strings is used to compare their similarity.

[0029] Step 5.3: Cloud server CS1 sends the similarity measurement results to cloud server CS2, and cloud server CS2 returns the corresponding search results based on the degree of similarity.

[0030] In step 5.2, when ω j When ω < 2θ, the two data points are considered to meet the preset similarity requirement, that is, their Hamming distance is less than the preset threshold θ; when ω j ≥2θ or ω j If the two data points are equal to ⊥, then the two data points are considered not to meet the search requirements.

[0031] The beneficial effects of this invention are:

[0032] (1) The present invention provides a cross-modal data retrieval method based on robust Hamming coding, which supports secure and efficient multimodal data retrieval. It can construct a unified Hamming space representation method for multimodal data, and the similarity measurement method based on Hamming distance can further improve retrieval efficiency and reduce computational costs.

[0033] (2) The cross-modal data retrieval method based on robust Hamming coding of the present invention uses a robust coding method to map the uniform Hamming space distance to the difference of symmetric sets. The security of the coding ensures that it is difficult for a computationally limited attacker to find a set with a small difference in symmetry from the original symmetric set. Even if a coded set with a small difference in symmetry can be found, the decoded set will either return an incorrect result or fail. Therefore, the security of the present invention is high. Attached Figure Description

[0034] Figure 1 This is a flowchart of the cross-modal data retrieval method based on robust Hamming coding according to the present invention. Detailed Implementation

[0035] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0036] Example 1

[0037] This invention discloses a cross-modal data retrieval method based on robust Hamming coding. The specific process is as follows: First, relevant parameters are generated and sent to the query user and cloud server CS1. Second, the data owner obtains the hash codes of different modalities and maps them to symmetric sets. Simultaneously, a reversible Bloom filter is constructed to encrypt the original data. The reversible Bloom filter and the encrypted original data are sent to cloud servers CS1 and CS2. Cloud server CS1 stores the reversible Bloom filter locally, and cloud server CS2 stores the encrypted original data. Third, the query user compares and decodes the data to obtain a similarity measurement result. Based on the similarity measurement result, relevant retrieval results are returned to the query user in ascending order. Finally, the query user decrypts the key to obtain the final retrieval result.

[0038] Example 2

[0039] This invention relates to a cross-modal data retrieval method based on robust Hamming coding, such as... Figure 1 As shown, please follow these steps:

[0040] Step 1: The data owner selects the security parameter κ, generates the key K for the symmetric encryption algorithm Enc, and randomly selects a set of hash functions {h1,...,h...}. k Set a similarity threshold θ, and construct a k-row, 2θ-column random matrix. The data owner will set the parameters {(h1,...,h...}k The similarity threshold θ is sent to the cloud server CS1 and the query user is sent to the queried user.

[0041] Step 2: Encrypt the original modal data and construct a reversible Bloom filter IBLT, where IBLT = {IBLT} j |j∈[n]}, and update the reversible Bloom filter IBLT. The data owner sends the updated reversible Bloom filter IBLT to cloud server CS1 and sends the encrypted original modal data to cloud server CS2.

[0042] The specific process is as follows:

[0043] Step 2.1, for each type of modal data, there are n original data points M = {m}. j For any type of modal data, the data owner uses the collaborative matrix factorization method to decompose any modal data m. j The characteristic matrix X j Project onto a unified Hamming representation space and obtain the corresponding binary hash code x. j At the same time, the original data m is encrypted using key K. j Calculate encrypted data c j =Enc(m j ,K), to obtain the encrypted original modal data C={c j |j∈[n]};

[0044] Wherein, any modal data m j The characteristic matrix X j Projecting onto a unified Hamming representation space (the data owner uses a collaborative matrix factorization method to obtain the binary hash codes x of different modalities of the data). ji The specific process of obtaining the projection matrices of different types of modal data through training is as follows:

[0045] There are sample spaces for different modal data. Where T represents the number of modalities, and n represents the number of samples in each different modality. Represents sample data, Representing sample data The corresponding feature matrix, assuming d t Let X represent the feature dimension of the t-th modality. Given a string length l, we use the collaborative matrix factorization method to learn the unified hash code representation of samples of different types of modalities. For example, there are feature matrices X of two different types of modalities. 1 With X 2 , and

[0046] X 1 ≈U1V

[0047] X 2 ≈U2V

[0048] in, and Denotes the factorization factor, V∈R l×n Represents potential public semantics;

[0049] Specifically, the objective function for learning cross-modal common semantics is obtained using co-operational matrix factorization: Where λ is the equilibrium parameter, Denotes the square of the F norm of a matrix;

[0050] The objective function of collaborative matrix factorization hashing is:

[0051]

[0052] Where P1 and P2 represent the projection matrices of the two modalities, respectively. γ represents the regularization term, μ represents the error term parameter between the hash code and the feature, and γ represents the regularization parameter.

[0053] During the training phase: Input data X 1 X 2 The parameters λ, μ, γ, hash code length l, and maximum iteration count Count (the number of iterations is set by the user) are used to continuously update U1, U2, P1, P2, and V until the objective function converges or the maximum number of iterations Count is reached. The final output is the hash code x. ji and projection matrices P1 and P2;

[0054] During the query phase: Input the feature matrix to be encoded, the P-value of the corresponding type of modal data projection matrix. β β represents the type of different modal data, where β∈[1,2]. Finally, the hash code x′ of the modal data is output. ji =sign(P β X j ) and the projection matrix P obtained from the training β Send to the inquirer;

[0055] Step 2.2, the data owner obtains any binary hash code x. j =x j1 …x ji …x jl (i∈[l]), where l is the length of the string, and the data owner uses any binary hash code x. j Construct the corresponding symmetric set Y j ;

[0056] The process of constructing a symmetric set is as follows: for hash code x j Each bit x ji Encode sequentially and calculate y. ji ,y ji =2i-x ji , thus obtaining the symmetric set Y ji ={y j1 ,...,y ji ,...,y jl};

[0057] Step 2.3, construct a reversible Bloom filter IBLT, where IBLT = {IBLT} j |j∈[n]};

[0058] First, there exists a sample space for different modal data. Where T represents the number of modalities, and n represents the number of samples in each different modality. Represents sample data, Representing sample data The corresponding feature matrix; secondly, for sample data m under any type of modality. j The characteristic matrix X j For j∈[n], the data owner uses the co-factor matrix method to decompose it into a binary semantic space, generating a unified set of binary hash codes {x1,...,x...}. j ,...,x n Finally, for any hash code x... j Encode the data and construct an Invertible Bloom Lookup Table (IBLT) based on it, denoted as IBLT, where IBLT = {IBLT}. j |j∈[n]};

[0059] Step 2.4, update the reversible Bloom filter IBLT;

[0060] The specific process is as follows:

[0061] Existence matrix It is also equipped with a randomly selected set of hash functions {h1,...,h... α ,...,h k}, each hash function h α (α∈[k]) corresponds to the α-th row of the matrix. In the α-th row of the matrix, choose the hash function h. α Calculate set element Y ji ={y j1 ,y j2 ,...,y jlThe hash value of}{h} α (y j1 ), h α (y j2 ),...,h α (y jl )}, and write the obtained hash value into the α-th row h α (y ji The position of the column, for example: in the first row of the matrix, choose hash function h1 to calculate the set element Y. ji ={y j1 ,y j2 ,...,y jl The hash value of} is used to obtain {h1(y j1 ), h1(y j2 ),...,h1(y jl )}, and write it into the first row of the matrix at the position corresponding to the hash value, and so on, until the data owner sets Y. ji ={y j1 ,y j2 ,...,y jl All elements in} are hashed using k different hash functions h. α After performing k hashes, the values ​​are written to the corresponding positions in the matrix, and the value of each position in the hash table of the reversible Bloom filter is updated accordingly. This allows for the completion of an update operation on the reversible Bloom filter IBLT;

[0062] Step 2.5, the data owner updates the reversible Bloom filter IBLT, IBLT = {IBLT} j |j∈[n]} is sent to the cloud server CS1, and the encrypted original modal data C={c j |j∈[n]} is sent to the cloud server CS2;

[0063] Step 3: Cloud server CS1 receives the updated reversible Bloom filter IBLT uploaded by the data owner, where IBLT = {IBLT} j |j∈[n]}; and store locally, while cloud server CS2 stores the encrypted original data uploaded by the data owner;

[0064] Step 4: The multimedia data information that the user wants to query is encoded and mapped to the Hamming space in the same way, and a reversible Bloom filter IBLT′ is constructed for the query, where IBLT′ = {IBLT′}. j |j∈[n]}, send the query reversible Bloom filter IBLT′ to the cloud server CS1;

[0065] The specific process is as follows:

[0066] Step 4.1: Using the collaborative matrix factorization method, repeat step 2.1 to project the feature matrix of the query data onto a unified Hamming space to obtain the binary hash code x′. i ;

[0067] The querying party uses the collaborative decomposition matrix method to train and obtain the projection matrix P. β The hash code x′ of the queried data can be obtained. ji The specific implementation method is as follows: In the query phase, the collaborative matrix factorization algorithm is used, and the input feature vector X is... j Based on the different modalities of the data, the joint projection matrix P β Based on the different modal types of the data, output the hash code x′ corresponding to the modal data. ji , x′ ji =sign(P β X j );

[0068] Step 4.2: Repeat steps 2.1 to 2.4 to encode the binary hash code and construct the reversible Bloom filter IBLT′ for the query, where IBLT′ = {IBLT′} j |j∈[n]};

[0069] Step 4.3: The querying user sends the constructed reversible Bloom filter IBLT′ to the cloud server CS1;

[0070] Step 5: Cloud server CS1 compares the similarity between the queried reversible Bloom filter IBLT′ and the updated reversible Bloom filter IBLT, and based on the similarity results, it collaborates with cloud server CS2 to return the search results in ascending order.

[0071] The specific process is as follows:

[0072] Step 5.1, cloud server CS1 will obtain IBLT′, IBLT′={IBLT′ j |j∈[n]} and the updated reversible Bloom filter IBLT from step 2, IBLT={IBLT j Subtracting each entry from |j∈[n]} yields a new reversible Bloom filter IBLT″,,IBLT″={IBLT″ j =IBLT j -IBLT′ j |(j∈[1,n])};

[0073] Step 5.2, cloud server CS1 applies the new reversible Bloom filter IBLT″ j By decoding each element one by one, the number ω of the element differences in the original symmetric set can be recovered. j Cloud server CS1 utilizes ωj The size of the two strings is used to compare their similarity.

[0074] The decoding in step 5.2 is achieved by continuously refining the filter from the new reversible Bloom filter IBLT″. j The solution involves subtracting the decoded value from the corresponding cell in the existing matrix, and the values ​​in the matrix need to be iteratively updated after each decoding.

[0075] The specific process is as follows:

[0076] First, subtract the two reversible Bloom filters item by item to obtain a new reversible Bloom filter IBLT″ = {IBLT″}. j =IBLT j -IBLT′ j |(j∈[1,n])};

[0077] From IBLT″ j The search results for (x) met the requirements. ji ,w), α, They are IBLT″ j The number of rows and columns of the matrix, w is the number of rows α. For a given value in a column, find the value of w that meets the requirements, and write the key-value pairs that meet the conditions into an empty set Z:

[0078]

[0079] The obtained (x) ji The corresponding w value in (,w) is written into an empty set Y′:

[0080]

[0081] The recovered values ​​w are extracted from each row of the original matrix, i.e., pruning is performed. The specific method is as follows:

[0082] For each (x) ji For the matrix ,w)∈Z, for the α-th row (α∈[k]):

[0083] H[α,h α (x ji )]:=H[α,h α (x ji )]-w

[0084] Then, return to the pruned matrix and decode using the new matrix, repeating the above decoding steps to continuously search for (x) that meet the requirements. ji Iterative decoding is performed on matrix H until all values ​​in matrix H are 0, i.e., H = (0, w). n ) k×2θReturn the final symmetric set Y", thus obtaining the size ω of set Y". j and using ω j The size of the two strings is used to compare their similarity.

[0085] When ω j When the distance is less than 2θ, the two data are considered to meet the preset similarity requirements, that is, the Hamming distance between the two is less than the preset threshold θ.

[0086] When ω j ≥2θ or ω j If the two data points are equal to or opposite to each other, then the two data points are considered not to meet the search requirements.

[0087] Step 5.3: Cloud server CS1 sends the similarity measurement results to cloud server CS2, and cloud server CS2 returns the corresponding search results based on the degree of similarity.

[0088] Step 6: Query the user's decryption of the encrypted data using key K, m j =Dec(c j K) obtains plaintext data.

[0089] Example 3

[0090] The working principle of this invention is as follows: the data owner uses collaborative matrix factorization to learn a unified binary hash code representation from feature vectors of different modalities, and then measures the similarity of the obtained hash codes of different modalities. To achieve a highly robust and privacy-preserving similarity measurement, the Hamming distance in the binary hash code is mapped to the number of distinct elements in a symmetric set. Then, by comparing the element differences between two sets, the similarity between the two hash codes is determined to be whether it meets a set threshold. If it meets the preset threshold, the corresponding search result is returned.

[0091] The method of this invention utilizes a cross-modal hashing method based on cooperative matrix factorization to achieve efficient cross-media information retrieval under a dual-cloud architecture, thereby reducing computational overhead. The method of this invention also utilizes a robust symmetric set encoding method and a reversible Bloom filter to achieve a highly robust and privacy-preserving Hamming distance similarity measure.

Claims

1. A cross-modal data retrieval method based on robust Hamming coding, characterized in that, The specific process is as follows: First, generate relevant parameters and send them to the querying user and the cloud server. Secondly, the data owner will obtain the hash codes of different modalities of data, map the hash codes to symmetric sets, construct a reversible Bloom filter to encrypt the original data, and send the reversible Bloom filter and the encrypted original data to the cloud server. cloud servers cloud server Local storage reversible Bloom filter, cloud server The system first stores the encrypted original data; then, the querying user decodes the data by comparing it with the query data to obtain a similarity measurement result, and returns relevant search results to the querying user in ascending order based on the similarity measurement result; finally, the querying user decrypts the key to obtain the final search result. Step 1, the data owner selects security parameters. Generate symmetric encryption algorithm key And randomly select a set of hash functions Set a similarity threshold ,structure OK Random matrix of columns The data owner will set parameters Send to the querying user, with similarity threshold Send to cloud server ; Step 2: Encrypt the original modal data and construct a reversible Bloom filter. , And update it; the data owner will update the reversible Bloom filter. Send to cloud server The encrypted raw modal data is sent to the cloud server. ; Step 3, cloud server Receive the updated reversible Bloom filter uploaded by the data owner. , And store locally, cloud server Stores encrypted raw data uploaded by the data owner; Step 4: The multimedia data information that the user wants to query is encoded and mapped to the Hamming space in the same way, and a reversible Bloom filter for the query is constructed. , The reversible Bloom filter for querying Send to cloud server ; Step 5, cloud server Reversible Bloom filter for query With the updated reversible Bloom filter Perform a similarity comparison and, based on the similarity results, coordinate with the cloud server. Return search results in ascending order; Step 6, query the user's key. Decrypt the encrypted data to obtain the plaintext data.

2. The cross-modal data retrieval method based on robust Hamming coding according to claim 1, characterized in that, The specific process of step 2 is as follows: Step 2.1, there exists under each different type of modal data. Original data For any type of modal data, the data owner uses the collaborative matrix factorization method to decompose any modal data. Feature matrix Project onto a unified Hamming representation space and obtain the corresponding binary hash code. At the same time, using the key Encrypt raw data Calculate encrypted data The encrypted original modal data is obtained. ; Step 2.2, the data owner obtains any binary hash code. , l To determine the length of the string, the data owner can use any binary hash code. Construct the corresponding symmetric set ; The process of constructing a symmetric set is as follows: For hash codes... Each bit Encode sequentially and calculate. y ji , , thus obtaining a symmetric set ; Step 2.3, Construct a reversible Bloom filter , ; Step 2.4, update the reversible Bloom filter. ; Step 2.5, the data owner will update the reversible Bloom filter. , Send to cloud server The encrypted original modal data Send to cloud server .

3. The cross-modal data retrieval method based on robust Hamming coding according to claim 2, characterized in that, The specific process of step 2.3 is as follows: First, there exists a sample space for data with different modalities. ,in, Indicates the number of modal types. This represents the number of samples in each different modality. Represents sample data, Representing sample data The corresponding feature matrix; secondly, for sample data under any type of modality. Feature matrix The data owner uses the co-factor matrix method to decompose it into a binary semantic space, generating a unified set of binary hash codes. Finally, for any hash code... Encode the data and construct a reversible Bloom filter, denoted as . , .

4. The cross-modal data retrieval method based on robust Hamming coding according to claim 2, characterized in that, The specific process of step 2.4 is as follows: There exists a matrix It is equipped with a randomly selected set of hash functions. Each hash function The first corresponding matrix row, in the matrix Okay, choose a hash function. Calculate set elements hash value And write the obtained hash value into the first OK The column position is updated, and the value at each position in the hash table of the reversible Bloom filter is updated to... Reversible Bloom filter Update complete.

5. The cross-modal data retrieval method based on robust Hamming coding according to claim 2, characterized in that, The specific process of step 4 is as follows: Step 4.1: Using the collaborative matrix factorization method, repeat step 2.1 to project the feature matrix of the query data onto a unified Hamming space to obtain the binary hash code. ; Step 4.2: Repeat steps 2.1 to 2.4 to encode the binary hash code and construct a reversible Bloom filter for the query. , ; Step 4.3: The query user will construct a reversible Bloom filter for the query. Send to cloud server .

6. The cross-modal data retrieval method based on robust Hamming coding according to claim 1, characterized in that, The specific process of step 5 is as follows: Step 5.1, Cloud Server The result , Compared with the reversible Bloom filter updated in step 2 , Subtracting the items, we obtain the new reversible Bloom filter. , ; Step 5.2, Cloud Server For the new reversible Bloom filter By decoding each element one by one, the number of differences between elements in the original symmetric set can be recovered. cloud server use The size of the two strings is used to compare their similarity. Step 5.3, Cloud Server Send the similarity measurement results to the cloud server. cloud server Return the corresponding search results based on the degree of similarity.

7. The cross-modal data retrieval method based on robust Hamming coding according to claim 6, characterized in that, In step 5.2, when If the two sets of data meet the preset similarity requirement, that is, their Hamming distance is less than the preset threshold, then the two sets of data are considered to meet the preset similarity requirement. ;when or If either of the above conditions is not met, then the two sets of data are considered not to meet the retrieval requirements.

Images