A document processing method, an electronic device, and a storage medium

Abstract

This application discloses a file processing method, an electronic device, and a storage medium. The file processing method includes: determining a first identifier vector for a file based on its position in a file set; generating a first storage location for storing the first identifier vector based on a hash function and the file's feature vector; determining a second identifier vector based on the hash table corresponding to the hash function and the first identifier vector; and storing the second identifier vector in a second storage location obtained by performing a pseudo-random permutation on the first storage location. The second storage location is used to locate the first storage location corresponding to the feature vector of the file to be searched when searching for a file. Thus, based on the second storage location and the position of the file to be searched in the file set, the corresponding file can be found without requiring the user to provide precise keywords for the search, thereby improving file search efficiency.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more particularly to a file processing method, electronic device, and storage medium. Background Technology

[0002] In recent years, with the development and popularization of cloud computing, some cloud service products have begun to store data files and build systems for users on cloud servers. More and more data owners are choosing to store their massive amounts of data, especially multimedia files including audio and video, in the cloud. To protect personal privacy and sensitive data (such as personal profiles, medical records, and family videos), users need to encrypt files locally before uploading them.

[0003] In existing technologies, when users need to search for relevant data files, one approach is to download all ciphertext to their local machine, decrypt it, and then search on the resulting plaintext. However, this incurs significant network and storage overhead, as well as computational costs associated with encryption and decryption. Another approach is Searchable Symmetric Encryption (SSE). Users create a plaintext index, encrypt it, and upload it to a remote server. When a user searches for a keyword, a search trapdoor for that keyword can be generated and submitted to the server. Upon receiving the trapdoor, the server searches the encrypted index and returns the corresponding ciphertext results to the user. Finally, the user decrypts the returned ciphertext. However, existing SSE schemes require precise keyword-based index queries, resulting in inefficient file searching. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a file processing method, an electronic device, and a storage medium.

[0005] The technical solution of this invention is implemented as follows:

[0006] In a first aspect, embodiments of the present invention provide a file processing method, including:

[0007] Determine the first identifier vector of the file based on its position in the file set;

[0008] Based on the hash function and the feature vector of the file, a first storage location is generated for storing the first identifier vector;

[0009] The second identifier vector is determined based on the hash table corresponding to the hash function and the first identifier vector;

[0010] The second identifier vector is stored in a second storage location obtained by performing a pseudo-random permutation on the first storage location; the second storage location is used to locate the first storage location corresponding to the feature vector of the file to be searched when searching for a file, based on the indication of the first identifier vector of the file to be searched.

[0011] Further, determining the second identifier vector based on the hash table corresponding to the hash function and the first identifier vector includes:

[0012] Based on the hash table corresponding to the hash function, determine the adjacent storage locations of the first storage location;

[0013] A second identifier vector is generated by forming a union between the vectors stored in the adjacent storage locations and the first identifier vector.

[0014] Furthermore, the method also includes:

[0015] If no data is stored in the adjacent storage location, then write a vector of all zeros with the same length as the first identifier vector into the adjacent storage location.

[0016] Furthermore, the method also includes:

[0017] Concatenate the identification information corresponding to the file set and the information of the second storage location;

[0018] The concatenated information is calculated using a pseudo-random function, and the second storage location is encrypted based on the calculation result.

[0019] Furthermore, the method also includes:

[0020] If data has already been stored in the first storage location, then the first identifier vector and the already stored data are combined and stored in the first storage location.

[0021] If no data is stored in the first storage location, then the first identifier vector is stored in the first storage location.

[0022] Secondly, embodiments of the present invention provide a file processing method, including:

[0023] The first identifier vector of the file to be searched is determined based on its position in the file set.

[0024] Determine the storage location where the first identifier vector is contained in the stored data;

[0025] The original storage location is determined based on the storage location of the first identifier vector and the inverse function of the pseudo-random permutation;

[0026] The file to be searched is located based on the feature vector corresponding to the original storage location.

[0027] Further, determining the storage location of the first identifier vector within the stored data includes:

[0028] Based on the first identifier vector, a second identifier vector is determined; the second identifier vector is the union of the first identifier vector and other vectors.

[0029] Determine the storage location of the second identifier vector.

[0030] Further, determining the storage location of the second identifier vector includes:

[0031] Determine the encrypted storage location information corresponding to the second identifier vector;

[0032] Based on the identifier information corresponding to the file set and the second identifier vector, the encrypted storage location information is decrypted to obtain the storage location of the second identifier vector.

[0033] Thirdly, embodiments of the present invention provide a document processing apparatus, comprising:

[0034] The first determining unit is configured to determine a first identifier vector of the file based on the file's position in the file set; and to determine a second identifier vector based on the hash table corresponding to the hash function and the first identifier vector.

[0035] The generation unit is configured to generate a first storage location for storing the first identifier vector based on the hash function and the feature vector of the file;

[0036] The storage unit is used to store the second identifier vector into a second storage location obtained by performing a pseudo-random permutation on the first storage location; the second storage location is used to locate the first storage location corresponding to the feature vector of the file to be searched when searching for a file, based on the indication of the first identifier vector of the file to be searched.

[0037] Furthermore, the storage unit is specifically used for:

[0038] Based on the hash table corresponding to the hash function, determine the adjacent storage locations of the first storage location;

[0039] A second identifier vector is generated by forming a union between the vectors stored in the adjacent storage locations and the first identifier vector.

[0040] Furthermore, the device also includes:

[0041] The writing unit is used to write a vector of all zeros with the same length as the first identifier vector into the adjacent storage location if no data is stored in the adjacent storage location.

[0042] Furthermore, the device also includes:

[0043] The splicing unit is used to splice together the identification information corresponding to the file set and the information of the second storage location;

[0044] The encryption unit is used to calculate the concatenated information using a pseudo-random function and encrypt the second storage location based on the calculation result.

[0045] Furthermore, the storage unit is also used for:

[0046] If data has already been stored in the first storage location, then the first identifier vector and the already stored data are combined and stored in the first storage location.

[0047] If no data is stored in the first storage location, then the first identifier vector is stored in the first storage location.

[0048] Fourthly, embodiments of the present invention provide a document processing apparatus, comprising:

[0049] The second determining unit is used to determine the first identifier vector of the file to be searched based on the position of the file to be searched in the file set; determine the storage location in the stored data that contains the first identifier vector; and determine the original storage location based on the storage location of the first identifier vector and the inverse function of the pseudo-random permutation.

[0050] The search unit is used to search for the file to be searched based on the feature vector corresponding to the original storage location.

[0051] Furthermore, the second determining unit is specifically used for:

[0052] Based on the first identifier vector, a second identifier vector is determined; the second identifier vector is the union of the first identifier vector and other vectors.

[0053] Determine the storage location of the second identifier vector.

[0054] Furthermore, the second determining unit is specifically used for:

[0055] Determine the encrypted storage location information corresponding to the second identifier vector;

[0056] Based on the identifier information corresponding to the file set and the second identifier vector, the encrypted storage location information is decrypted to obtain the storage location of the second identifier vector.

[0057] Fifthly, embodiments of the present invention provide an electronic device, the electronic device comprising: a processor and a memory for storing a computer program capable of running on the processor;

[0058] When the processor runs the computer program, it performs the steps of the method described in one or more of the foregoing technical solutions.

[0059] In a sixth aspect, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions; the computer-executable instructions, when executed by a processor, are capable of implementing the methods described in one or more of the foregoing technical solutions.

[0060] The file processing method provided by this invention includes: determining a first identifier vector of a file based on its position in a file set; generating a first storage location for storing the first identifier vector based on a hash function and a feature vector of the file; determining a second identifier vector based on a hash table corresponding to the hash function and the first identifier vector; storing the second identifier vector in a second storage location obtained by performing a pseudo-random permutation on the first storage location; the second storage location is used to locate the first storage location corresponding to the feature vector of the file to be searched when searching for a file. Thus, based on the second storage location, the corresponding first storage location can be found based on the file's position in the file set, and the file to be searched can be located based on the feature vector corresponding to the first storage location, eliminating the need for users to provide precise file keywords for searching, thereby greatly improving file search efficiency. Moreover, based on the calculation process of hash function and pseudo-random permutation, compared with directly recording the correspondence between files and their storage locations, it offers higher privacy and security. Attached Figure Description

[0061] Figure 1 A flowchart illustrating a file processing method provided in an embodiment of the present invention;

[0062] Figure 2 A flowchart illustrating a file processing method provided in an embodiment of the present invention;

[0063] Figure 3 A flowchart illustrating a file processing method provided in an embodiment of the present invention;

[0064] Figure 4 A flowchart illustrating a file processing method provided in an embodiment of the present invention;

[0065] Figure 5 This is a schematic diagram of the structure of a file processing device provided in an embodiment of the present invention;

[0066] Figure 6 This is a schematic diagram of a file processing device provided in an embodiment of the present invention. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0069] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of the invention described herein can be implemented in an order other than that illustrated or described herein.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0071] like Figure 1 As shown, an embodiment of the present invention provides a file processing method, including:

[0072] S110: Determine the first identifier vector of the file based on its position in the file set;

[0073] S120: Generate a first storage location for storing the first identifier vector based on the hash function and the feature vector of the file;

[0074] S130: Determine the second identifier vector based on the hash table corresponding to the hash function and the first identifier vector;

[0075] S140: Store the second identifier vector into a second storage location obtained by performing a pseudo-random permutation on the first storage location; the second storage location is used to locate the first storage location corresponding to the feature vector of the file to be searched when searching for a file, based on the indication of the first identifier vector of the file to be searched.

[0076] In this embodiment of the invention, the file set can be a collection of multiple files that need to be stored for subsequent searching. For example, the files can be multimedia files such as audio and video files, or other types of files such as log data files. For instance, the file set can be a complete video, in which case the files can be multiple frame files formed by dividing the video into frames, or multiple video segment files formed by dividing the video into segments. The position of a file in the file set can represent which file it is in the file set, or its relative position to other files in the file set. For example, when the file set is a video, the position of a file in the file set can be which frame the file is in the video, or which second of the video segment the file is in.

[0077] Here, the length of the first identifier vector can be equal to the number of files in the file set, or it can be represented by 0 and 1. For example, if the file set contains 100 files, the length of the first identifier vector is 100. The first identifier vector of the i-th file in the file set can be a vector where the i-th element is 1 and the remaining 99 elements are 0. Thus, the first identifier vector represents the position of the file in the file set.

[0078] A feature vector is a vector that can characterize the features of file data. For example, for multimedia data files, the feature vector can be a feature vector extracted using a color histogram algorithm. For instance, when the file is an image file, the feature vector can be extracted using OpenCV.

[0079] In one embodiment, for a set of files containing a total number of files #f, the i-th file is f. i where i∈[1, #f]. i The feature vector can be a d-dimensional feature vector p obtained through a feature extraction algorithm. i The first identifier vector can be an inverse file vector (IFV) with a length equal to the number of files in the file set, #f. Step S120 may include: applying a position-sensitive hash function to the feature vector p i The calculation is performed to obtain the generated hash bucket position as the first storage location. For example, the first storage location x = H(p) is calculated based on the position-sensitive hash function H. i )(mod m), where mod represents the modulo function calculation, and m is the total number of hash buckets contained in the hash table corresponding to the position-sensitive hash function H.

[0080] In another embodiment, step S120 may further consist of: selecting M position-sensitive hash functions h1, h2, ..., h MAccording to the formula x j =h j (p i (mod m)+m(j-1) respectively for the eigenvector p i The calculation is performed, where j∈[1,M]. Each hash function generates a hash bucket, so M functions generate a total of M hash buckets, i.e., M first storage locations x1, x2, ..., xn. M Each first storage location stores a first identifier vector. Here, M can be a randomly determined number or a preset number. M functions can be randomly selected from the family of location-sensitive hash functions, or a preset set of M functions can be used directly. In this way, calculations are performed using multiple location-sensitive hash functions, thus forming a distributed storage location generation mechanism and mitigating the lower security of the storage location establishment process caused by calculations using a single hash function.

[0081] In one embodiment, step S130 may include: determining a second identifier vector based on a first identifier vector and vectors stored in other storage locations within the hash table containing the first storage location of the first identifier vector. For example, the first identifier vector is stored in a hash bucket x generated by a position-sensitive hash function H. Then, based on the vectors stored in the hash buckets other than x in the hash table corresponding to H, such as the vectors stored in the neighboring buckets of x in the hash table, and... Together, we determine the second identifier vector.

[0082] In another embodiment, if M first storage locations are generated using M position-sensitive hash functions, then S130 can be: determining M second identifier vectors based on all storage locations in the M hash tables and the first identifier vector. For example, the second identifier vector corresponding to each hash table can be determined based on the storage locations in each hash table and the first identifier vector.

[0083] In one embodiment, a pseudo-random permutation (PRP) is performed on the first storage location x to generate a random location, namely the second storage location X. The second identifier vector is stored in the second storage location. Based on the association between the first and second identifier vectors, when a user needs to search for a file, they only need to use the first identifier vector to find the associated second identifier vector in the second storage location. Furthermore, based on the inverse function of the PRP and the second storage location, the first storage location can be reconstructed. Then, based on the first storage location, the corresponding feature vector is determined, which is the feature vector of the file to be searched. The corresponding file can be directly matched based on the feature vector.

[0084] In another embodiment, if M first storage locations are generated using M location-sensitive hash functions, then for the M first storage locations x1, x2, ..., x... M PRP calculations are performed based on pseudo-random permutation F(x) to obtain M second storage locations X1, X2, ..., X... M , where X M =F(x) M Each of the M second identifier vectors is stored in the second storage location corresponding to the hash table containing the first storage location.

[0085] Thus, based on the second storage location and a pseudo-random permutation with an inverse function, the corresponding first storage location can be found based on the file's position in the file set. Furthermore, the file to be searched can be located using the feature vector corresponding to the first storage location, eliminating the need for users to provide precise file keywords, thereby significantly improving file search efficiency. Moreover, based on the calculation and storage location conversion processes using hash functions and pseudo-random permutations, the storage security of the user's file's first identifier vector and related data is higher compared to directly recording the correspondence between files and their storage locations.

[0086] In some embodiments, S130 may include:

[0087] Based on the hash table corresponding to the hash function, determine the adjacent storage locations of the first storage location;

[0088] A second identifier vector is generated by forming a union between the vectors stored in the adjacent storage locations and the first identifier vector.

[0089] In this embodiment of the invention, the adjacent storage location of the first storage location can be the storage location in the hash table that is closest to the left of the first storage location, and / or the storage location that is closest to the right of the first storage location. For example, if the first storage location is hash bucket x in the hash table, then the adjacent storage location can be a neighboring bucket located to the left and / or right of hash bucket x.

[0090] In one embodiment, since the position-sensitive hash function H can be used to generate first storage locations for multiple files, the storage locations contained in the hash table corresponding to H can be used to store the first identifier vectors of different files. For example, the first identifier vector stored in the first storage location x is a reverse file identifier vector. The vector stored in the adjacent storage location to the left of x is represented as: The vector stored in the adjacent storage location to the right of x is represented as follows: Therefore, the vectors stored in the adjacent storage locations to the left and right of x, along with the first identifier vector, form a union, which is the second identifier vector fid.i for Here, v represents the "OR" operation.

[0091] In another embodiment, if M first storage locations are generated by M location-sensitive hash functions, then the vectors in the adjacent storage locations of the first storage location in each hash table are determined, and the first identifier vector is combined with the vectors in the adjacent storage locations in each hash table to form a union, which is then stored in the second storage location corresponding to the hash table.

[0092] In another embodiment, all storage locations in the hash table can be subjected to pseudo-random permutation and adjacent storage location vector union calculation. That is, pseudo-random permutation is performed for each storage location to generate a corresponding second storage location, and the vector stored in each storage location and the adjacent union of its adjacent storage locations are stored in the second storage location corresponding to that storage location.

[0093] Thus, when two files have high similarity, their feature vectors also have high similarity. Due to the characteristics of position-sensitive hash functions, similar feature vectors, after hashing, result in closer storage locations, and the higher the similarity, the closer the storage locations. Therefore, the files corresponding to the vectors stored in adjacent storage locations in the hash table have the highest similarity to the file corresponding to the current first identifier vector. Based on this, storing the union of the vectors from adjacent storage locations with the first identifier vector in the second storage location can provide similarity search when the user searches based on the first identifier vector, that is, providing the user with the file to be searched and the file with the highest similarity to the file to be searched.

[0094] In some embodiments, the method further includes:

[0095] If no data is stored in the adjacent storage location, then write a vector of all zeros with the same length as the first identifier vector into the adjacent storage location.

[0096] In this embodiment of the invention, since some storage locations in the hash table may be empty, indicating that no data is stored in those locations, in order to suppress calculation errors caused by adjacent empty storage locations during the calculation of the union, a vector of all zeros with the same length as the first identifier vector is stored in all unstored data storage locations in the hash table before the calculation.

[0097] For example, when the first identifier vector When the length is #f, the vector of length #f containing all zeros is stored in all empty hash buckets within the hash table.

[0098] In some embodiments, such as Figure 2 As shown, the method further includes:

[0099] S150: Concatenate the identification information corresponding to the file set and the information of the second storage location;

[0100] S160: Calculate the concatenated information using a pseudo-random function, and encrypt the second storage location based on the calculation result.

[0101] In this embodiment of the invention, since the second storage location can be used to search for files, the encrypted second storage location can be recorded as the index information of the files. The identification information corresponding to the file set can be the identity identification information of each file set, for example, it can be represented by an Index Identity Document (IndexID).

[0102] Because pseudo-random functions (PRFs) differ from pseudo-random permutations in that they do not have an inverse function, decrypting an object encrypted using a PRF requires a second encryption using the same encryption algorithm to obtain the decrypted original object. This further enhances the security of storage location establishment and storage, and significantly reduces the likelihood of the encryption / decryption process being stolen and misused.

[0103] In one embodiment, the second storage location X of the first storage location x is calculated based on pseudo-random permutation. Then, the second storage location is encrypted using a pseudo-random function G(x) according to the formula. The encrypted storage location information I can be obtained. Here, := signifies "assigning a value", The symbol represents the XOR operation, and || represents the string concatenation operation.

[0104] Thus, using the encrypted second storage location as index information to indicate the file to be searched can greatly improve the security of file indexing and the search process. Furthermore, using IndexID to represent the identity of a file set can effectively distinguish the second storage locations between different file sets when multiple file sets exist simultaneously.

[0105] In some embodiments, such as Figure 3 As shown, the method further includes:

[0106] S101: If data has already been stored in the first storage location, then the first identifier vector and the stored data are combined and stored in the first storage location.

[0107] S102: If no data is stored in the first storage location, then the first identifier vector is stored in the first storage location.

[0108] In this embodiment of the invention, although the first storage location is generated by a hash function based on the feature vector of the current file, in practical applications, the hash value of the feature vector of the current file may already be occupied, i.e., a hash collision. This could result in the first storage location being used to store other vectors before the first identifier vector of the current file is stored.

[0109] Therefore, in order to ensure that the first identifier vector of the current file can be effectively stored in the first storage location without affecting or interfering with the previously stored vectors, the first identifier vector and the previously stored vectors are stored together in the first storage location in the form of a union.

[0110] like Figure 4 As shown, an embodiment of the present invention provides a file processing method, including:

[0111] S210: Determine the first identifier vector of the file to be searched based on its position in the file set;

[0112] S220: Determine the storage location where the first identifier vector is contained in the stored data;

[0113] S230: Determine the original storage location based on the storage location of the first identifier vector and the inverse function of the pseudo-random permutation;

[0114] S240: Locate the file to be searched based on the feature vector corresponding to the original storage location.

[0115] In this embodiment of the invention, the position of the file to be searched in the file set can be determined. For example, if the file to be searched is a video clip, the second number of the video clip in the file set, i.e., the complete video, can be determined. Or, if the file to be searched is a frame file, the frame number of the frame in the completed video can be determined.

[0116] Since the second storage location obtained by transformation based on the first storage location stores data after a union operation based on the first identifier vector, searching based on the first identifier vector can match at least one second storage location containing the first identifier vector.

[0117] In one embodiment, since the first storage location is generated by pseudo-random permutation to obtain the second storage location, the storage location obtained when searching for a file is the second storage location. The first storage location, i.e. the original storage location, can be calculated based on the inverse function of pseudo-random permutation.

[0118] In another embodiment, the feature vector is used to perform hash calculation using a position-sensitive hash function H to obtain the original storage location. A one-to-one correspondence exists between the feature vector and the original storage location; once the original storage location is obtained, the corresponding feature vector can be determined. Furthermore, the corresponding file can be matched and queried based on the feature vector.

[0119] In another embodiment, if multiple second storage locations containing the first identifier vector are found based on the first identifier vector, then the original storage locations corresponding to the multiple second storage locations are determined, and the corresponding feature vectors and files are determined respectively. The resulting multiple files contain the file to be searched, and all files other than the file to be searched are files with high similarity to the file to be searched. Thus, similarity searches can be performed according to the user's search needs, allowing the user to select a file that meets their requirements from multiple highly similar files.

[0120] Thus, when a user needs to search for a file, they only need to provide the first identifier vector representing the file's position in the file set to retrieve the file to be searched, without needing to provide precise file keywords, greatly improving search efficiency. Furthermore, similarity search results can be provided, giving users more choices and greater flexibility.

[0121] In some embodiments, S220 may include:

[0122] Based on the first identifier vector, a second identifier vector is determined; the second identifier vector is the union of the first identifier vector and other vectors.

[0123] Determine the storage location of the second identifier vector.

[0124] In this embodiment of the invention, the second storage location stores the union of the first identifier vector and the data stored in the adjacent storage location. Therefore, the storage location containing the first identifier vector can be found based on the first identifier vector.

[0125] In one embodiment, during the generation of the second storage locations, a pseudo-random permutation and union operation are performed on each hash bucket (first storage location) in the hash table. Then, each of the m second storage locations in the hash table is a union with the storage vectors of its adjacent buckets. Therefore, the multiple storage locations found based on the first identifier vector correspond to the first storage location used to store the first identifier vector, and the adjacent storage locations to the left and right of that first storage location.

[0126] Thus, based on the first identifier vector, the first storage location containing the first identifier vector and its adjacent storage locations can be effectively queried. Due to the characteristics of the position-sensitive hash function, the feature vectors corresponding to adjacent storage locations have high similarity. Based on this, one or more files with high similarity to the file to be searched can be obtained, thereby achieving similarity search.

[0127] In some embodiments, determining the storage location of the second identifier vector includes:

[0128] Determine the encrypted storage location information corresponding to the second identifier vector;

[0129] Based on the identifier information corresponding to the file set and the second identifier vector, the encrypted storage location information is decrypted to obtain the storage location of the second identifier vector.

[0130] In this embodiment of the invention, after generating the storage location of the second identifier vector, i.e., the second storage location, the second storage location is encrypted to generate encrypted storage location information. Therefore, when searching for files, the first identifier vector matches one or more encrypted storage location information, which needs to be decrypted to obtain the corresponding second storage location.

[0131] In one embodiment, the encrypted storage location information can be decrypted based on a pseudo-random function and an XOR operation to obtain the second storage location, the identification information of the file set, the second identification vector, etc.

[0132] In another embodiment, the obtained second identifier vector is the union of the first identifier vector and the vectors in adjacent storage locations, and the files corresponding to the vectors in adjacent storage locations have a high similarity to the files corresponding to the first identifier vector. Therefore, one or more identifier vectors contained in the second identifier vector can be recorded, and a separate matching search can be performed on each identifier vector to determine the files corresponding to adjacent storage locations, as well as the files corresponding to the adjacent storage locations of those adjacent storage locations. This allows for a faster and broader similarity search, providing users with more similar files to choose from.

[0133] like Figure 5 As shown, an embodiment of the present invention provides a document processing apparatus, the apparatus comprising:

[0134] The first determining unit 110 is configured to determine a first identifier vector of the file based on the file's position in the file set; and to determine a second identifier vector based on the hash table corresponding to the hash function and the first identifier vector.

[0135] The generation unit 120 is configured to generate a first storage location for storing the first identifier vector based on the hash function and the feature vector of the file;

[0136] Storage unit 130 is used to store the second identifier vector into a second storage location obtained by performing a pseudo-random permutation on the first storage location; the second storage location is used to locate the first storage location corresponding to the feature vector of the file to be searched when searching for a file, according to the indication of the first identifier vector of the file to be searched.

[0137] In some embodiments, the storage unit 130 is specifically used for:

[0138] Based on the hash table corresponding to the hash function, determine the adjacent storage locations of the first storage location;

[0139] A second identifier vector is generated by forming a union between the vectors stored in the adjacent storage locations and the first identifier vector.

[0140] In some embodiments, the apparatus further includes:

[0141] The writing unit is used to write a vector of all zeros with the same length as the first identifier vector into the adjacent storage location if no data is stored in the adjacent storage location.

[0142] In some embodiments, the apparatus further includes:

[0143] The splicing unit is used to splice together the identification information corresponding to the file set and the information of the second storage location;

[0144] The encryption unit is used to calculate the concatenated information using a pseudo-random function and encrypt the second storage location based on the calculation result.

[0145] In some embodiments, the storage unit 130 is further configured to:

[0146] If data has already been stored in the first storage location, then the first identifier vector and the already stored data are combined and stored in the first storage location.

[0147] If no data is stored in the first storage location, then the first identifier vector is stored in the first storage location.

[0148] like Figure 6 As shown, an embodiment of the present invention provides a document processing apparatus, the apparatus comprising:

[0149] The second determining unit 210 is used to determine the first identifier vector of the file to be searched based on the position of the file to be searched in the file set; determine the storage location in the stored data that contains the first identifier vector; and determine the original storage location based on the storage location of the first identifier vector and the inverse function of the pseudo-random permutation.

[0150] The search unit 220 is used to search for the file to be searched based on the feature vector corresponding to the original storage location.

[0151] In some embodiments, the second determining unit 220 is specifically used for:

[0152] Based on the first identifier vector, a second identifier vector is determined; the second identifier vector is the union of the first identifier vector and other vectors.

[0153] Determine the storage location of the second identifier vector.

[0154] In some embodiments, the second determining unit 220 is specifically used for:

[0155] Determine the encrypted storage location information corresponding to the second identifier vector;

[0156] Based on the identifier information corresponding to the file set and the second identifier vector, the encrypted storage location information is decrypted to obtain the storage location of the second identifier vector.

[0157] The following provides a specific example in conjunction with any of the above embodiments:

[0158] This invention provides a technique for performing similarity searches on encrypted multimedia data.

[0159] 1. Pseudo-random permutation (PRP) And pseudo-random functions (PRF)

[0160] Let #f represent the total number of files in the file set, and let f be the multimedia data file. i Let p be the i-th file in the file set. i Represents the multimedia data file f i The d-dimensional feature vector extracted from it. This indicates that the eigenvector p i The corresponding inverse file identifier vector (IFV), if If the i-th bit is 1, it indicates that the i-th file is included; if it is 0, it indicates that the file is not included. Additionally, fid represents a reverse file identifier vector obtained by combining multiple reverse file identifier vectors. For a given vector v, v[i] or v[i] represents the file identifier vector. i It represents its i-th element.

[0161] 2. The steps to build an encrypted search index are as follows:

[0162] Randomly select M position-sensitive hash functions h1, h2, ..., h3 from the family of position-sensitive hash functions H. M These M hash functions will be used to generate M hash tables.

[0163] For each multimedia data file f i , where i∈[1,#f].

[0164] For each multimedia data file, a feature extraction algorithm (such as the color histogram algorithm) is used to obtain the corresponding d-dimensional feature vector p. i And generate a reverse file identifier vector (IFV).

[0165] Calculate the positions of the M hash buckets generated by the position-sensitive hash function, i.e., x1 = g1(p i ), ..., x M =g M (p i ), g j =h j (p i )(mod m)+m(j-1), where j∈[1,M], and m is the number of buckets in each hash table.

[0166] For the position x of each bucket j Where j∈[1,M], the reverse file identifier vector Insert into the bucket according to the following rules. If the position in bucket I[xj] is empty, then directly insert... Insert I[xj]; otherwise, insert the "new" one. And the "old" one that existed in this location before. After performing the OR operation, store it in I[x] j ], denoted as

[0167] Using y1, y2, ..., Y Mm Let represent the position of all buckets, and store a vector of length |fid| consisting entirely of zeros in all empty buckets. For each bucket y... i , i∈[1,Mm].

[0168] use This indicates that it is stored in I[y] i The reverse file identifier vector. Calculate. fid as the "union" reverse file identifier vector i Among them, fid i - and fid i + These represent the reverse file identifier vectors stored in the left and right neighboring buckets, respectively.

[0169] For each bucket y i For i∈[1,Mm], a new corresponding random position is generated through pseudo-random permutation. i∈[1,Mm], and fid i Stored in middle.

[0170] For each bucket in the index i∈[1,Mm], encrypt fid as follows: i : Where || represents the concatenation operation in the strings. IndexID is used to uniquely identify the encrypted similarity index I.

[0171] For i∈[1, #f], encrypt each file using the PCPA-secure symmetric encryption algorithm, i.e. SKE is a symmetric encryption scheme.

[0172] Finally, the obtained (I, c) is uploaded to the remote server, where c = (c1, ..., c2) #f ).

[0173] 3. The above-mentioned encrypted similarity index is constructed based on the entire multimedia data file set f rather than each individual data file. Therefore, the length of the reverse file identifier vector is set to #f.

[0174] In addition to the reverse file identifier vector, a hash table needs to be built, and the hash function H is used to hash each reverse file identifier vector. Mapped to a unique real file identifier (such as a pathname in a file system). For example, given a... Through formula Here, mod#f represents the modulo operation based on the file set length #f. This determines the index where the corresponding entry can be found. By locating the actual file identifier extracted from the corresponding bucket, we can find the corresponding data file. The reason for using the concept of IndexID is that, in practice, if the data owner wants to create an index based on different combinations of multimedia data files, IndexID can be used to distinguish each index individually.

[0175] This invention also provides an electronic device, which includes a processor and a memory for storing a computer program that can run on the processor. When the processor runs the computer program, it performs the steps of the methods described in one or more of the foregoing technical solutions.

[0176] This invention also provides a computer-readable storage medium storing computer-executable instructions. When executed by a processor, the computer-executable instructions can implement the methods described in one or more of the foregoing technical solutions.

[0177] The computer storage medium provided in this embodiment may be a non-instantaneous storage medium.

[0178] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0179] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0180] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing module, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0181] In some cases, where any two of the above technical features do not conflict, they can be combined to form a new method or technical solution.

[0182] In some cases, where any one or two of the above technical features do not conflict, they can be combined to form a new equipment technical solution.

[0183] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0184] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A file processing method, characterized in that, The method includes: The first identifier vector of the file is determined based on the file's position in the file set; the length of the first identifier vector is the same as the number of files in the file set. Based on the position-sensitive hash function and the feature vector of the file, a first storage location is generated for storing the first identifier vector of the file; the feature vector is a vector characterizing the file data features of the file. Based on the hash table corresponding to the location-sensitive hash function, determine the adjacent storage locations of the first storage location; A second identifier vector is generated by forming a union between the vectors stored in the adjacent storage locations and the first identifier vector of the file. The second identifier vector is stored in a second storage location obtained by performing a pseudo-random permutation on the first storage location; the second storage location is used to locate the first storage location corresponding to the feature vector of the file to be searched when searching for a file; The files mentioned include multimedia files.

2. The method according to claim 1, characterized in that, The method further includes: If no data is stored in the adjacent storage location, then write an all-zero vector with the same length as the first identifier vector of the file into the adjacent storage location.

3. The method according to claim 1, characterized in that, The method further includes: Concatenate the identification information corresponding to the file set and the information of the second storage location; The concatenated information is calculated using a pseudo-random function, and the second storage location is encrypted based on the calculation result.

4. The method according to claim 1, characterized in that, The method further includes: If data has already been stored in the first storage location, then the first identifier vector of the file and the already stored data are combined and stored in the first storage location. If no data is stored in the first storage location, then the first identifier vector of the file is stored in the first storage location.

5. A file processing method, characterized in that, The method includes: The first identifier vector of the file to be searched is determined based on its position in the file set; the length of the first identifier vector is the same as the number of files in the file set. A second identifier vector is determined based on the first identifier vector of the file to be searched; the second identifier vector is the union of the first identifier vector of the file to be searched and other vectors. Determine the storage location of the second identifier vector; The original storage location is determined based on the storage location of the first identifier vector of the file to be searched and the inverse function of the pseudo-random permutation; The file to be searched is located based on the feature vector corresponding to the original storage location; the feature vector is a vector characterizing the file data features of the file to be searched. The files mentioned include multimedia files.

6. The method according to claim 5, characterized in that, Determining the storage location of the second identifier vector includes: Determine the encrypted storage location information corresponding to the second identifier vector; Based on the identifier information corresponding to the file set and the second identifier vector, the encrypted storage location information is decrypted to obtain the storage location of the second identifier vector.

7. An electronic device, characterized in that, The electronic device includes: a processor and a memory for storing a computer program capable of running on the processor; wherein, when the processor runs the computer program, it performs the steps of any one of claims 1 to 4, or the file processing method of claim 5 or 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions; when executed by a processor, the computer-executable instructions can implement the file processing method as described in claims 1 to 4, or any one of claims 5 or 6.

Images