Data query method, data query device, data query system and storage medium

Abstract

The invention relates to a data query method, a data query device, a data query system and a storage medium, and the data query method comprises the following steps: under the condition that a data query request is received, extracting a preliminary feature vector in the data query request, and encoding the preliminary feature vector to obtain an encoded feature vector; wherein the data query request is obtained by encryption based on a trusted computing environment; performing inverse rearrangement on the coding feature vector to obtain an inverse rearrangement feature vector; determining a target query vector from a vector database according to the similarity between the inverse rearrangement feature vector and a vector in a preset vector database; obtaining a target access address corresponding to the target query vector from a preset vector and address mapping table; the target access address is accessed after being subjected to hiding processing, and an initial access result is obtained; and returning a target access result corresponding to the target access address in the initial access result to the client. Through the data query method and device, the security of data query is improved.

Description

Technical Field

[0001] The present application relates to the field of data security and privacy protection research, and particularly relates to a data query method, a data query device, a data query system, and a storage medium. Background Art

[0002] Currently, the increasingly widely deployed monitoring networks have collected a vast amount of audio, video, and image data. These data provide great convenience for the review and analysis of the real situations of events concerned in various scenarios such as specific times, specific areas, and specific populations. These multi-modal data have become a huge information database, providing good data resource support for intelligent transportation, urban security, and case investigation, and capable of providing various meaningful target queries, such as suspect tracing, vehicle trajectory restoration, and stolen item investigation. However, in some sensitive review requirements, such as querying specific personnel or vehicle information, it is not desired that the data owner knows the content being queried and the relevant results after the query. To achieve privacy protection for data queries, currently, mainly after determining the cameras in the involved scenarios, the data collected by these cameras at specific times are manually downloaded from the database of the data aggregation service provider for manual analysis. However, there are the following problems with manually downloading target data:

[0003] (1) The privacy protection ability during the query process is poor. Data owners usually easily know the data being reviewed, which may lead to an expanded scope of knowledge or even leakage of picture or video data containing sensitive information.

[0004] (2) The intelligence and efficiency of data analysis are not high. Especially for video analysis, it mostly relies on manual understanding, with a large workload and low efficiency, and it is difficult to meet the timeliness requirements of actual scenarios.

[0005] (3) There is a lack of an online query system. The current query method is relatively primitive. Only after downloading the target video can analysis be carried out, and there are potential hazards in terms of bandwidth limitation and data security for video downloading and offline storage.

[0006] Regarding the problem of poor data query security in the related art, no effective solution has been proposed yet. Summary of the Invention

[0007] In the present embodiment, a data query method, a data query device, a data query system, and a storage medium are provided to solve the problem of poor data query security in the related art.

[0008] In the first aspect, in the present embodiment, a data query method is provided, including:

[0009] Upon receiving a data query request, extract the preliminary feature vector in the data query request, and encode the preliminary feature vector to obtain an encoded feature vector; wherein, the data query request is encrypted based on a trusted computing environment;

[0010] Perform inverse rearrangement on the encoded feature vector to obtain an inverse rearranged feature vector;

[0011] Determine a target query vector from the vector database according to the similarity between the inverse rearranged feature vector and the vectors in the preset vector database;

[0012] Obtain the target access address corresponding to the target query vector from the preset vector and address mapping table;

[0013] Perform access after concealing the target access address to obtain an initial access result;

[0014] Return the target access result corresponding to the target access address in the initial access result to the client.

[0015] In some of the embodiments, before receiving the data query request, it further includes:

[0016] Perform security verification on the trusted computing environment, and receive the data query request when the verification is successful.

[0017] In some of the embodiments, the encoding of the preliminary feature vector includes:

[0018] Perform rearrangement on the preliminary feature vector to obtain a rearranged feature vector;

[0019] Encode the rearranged feature vector to obtain an encoded feature vector.

[0020] In some of the embodiments, the vectors in the vector database include picture vectors, text vectors, audio vectors, and video vectors.

[0021] In some of the embodiments, the determining of the target query vector from the vector database according to the similarity between the inverse rearranged feature vector and the vectors in the preset vector database includes:

[0022] Calculate the similarity between the inverse rearranged encoded feature vector and each vector in the preset vector database to obtain each similarity score;

[0023] Determine the target query vector from the vector database according to the similarity score and the Top - n rule.

[0024] In some of these embodiments, accessing after performing the concealment process on the target access address to obtain an initial access result includes:

[0025] Performing an expansion process on the target access address by using a randomized data access method to obtain a preset number of expanded access addresses;

[0026] Performing an access according to the expanded access addresses to obtain the initial access result.

[0027] In some of these embodiments, returning the target access result corresponding to the target access address in the initial access result to the client includes:

[0028] Obtaining the target access result corresponding to the target access address;

[0029] Encrypting the target access result to obtain an encrypted target access result;

[0030] Returning the encrypted target access result to the client.

[0031] In a second aspect, a data query device is provided in this embodiment, including: an extraction module, an inverse rearrangement module, a similarity comparison module, an access module, and a sending module; where,

[0032] The extraction module is configured to, when receiving a data query request, extract a preliminary feature vector in the data query request and encode the preliminary feature vector to obtain an encoded feature vector; where the data query request is encrypted based on a trusted computing environment;

[0033] The inverse rearrangement module is configured to perform an inverse rearrangement on the encoded feature vector to obtain an inverse rearranged feature vector;

[0034] The similarity comparison module is configured to determine a target query vector from the vector database according to the similarity between the inverse rearranged feature vector and vectors in a preset vector database;

[0035] The access module is configured to obtain a target access address corresponding to the target query vector from a preset vector and address mapping table; perform an access after performing a concealment process on the target access address to obtain an initial access result;

[0036] The sending module is configured to return the target access result corresponding to the target access address in the initial access result to the client.

[0037] In a third aspect, a data query system is provided in this embodiment, including: a client and a server, where,

[0038] The client is used to initiate a data query request, encrypt the data query request based on a trusted computing environment, and is also used to receive a target access result and decrypt the target access result;

[0039] The server includes a trusted computing environment and a rich execution environment. Among them, in the trusted computing environment, the server is used to extract a feature vector from the data query request, encrypt and rearrange the feature vector, inverse rearrange it, calculate similarity, randomly expand access, and return a target access result; in the rich execution environment, the server is used to encode the encrypted and rearranged feature vector.

[0040] In a fourth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, it implements the data query method described in the first aspect above.

[0041] Compared with the related art, in the data query method provided in this embodiment, when a data query request is received, a preliminary feature vector in the data query request is extracted, and the preliminary feature vector is encoded to obtain an encoded feature vector; wherein, the data query request is encrypted based on a trusted computing environment; the encoded feature vector is inverse rearranged to obtain an inverse rearranged feature vector; according to the similarity between the inverse rearranged feature vector and a vector in a preset vector database, a target query vector is determined from the vector database; a target access address corresponding to the target query vector is obtained from a preset vector and address mapping table; after the target access address is obfuscated and accessed, an initial access result is obtained; the target access result corresponding to the target access address in the initial access result is returned to the client, solving the problem of poor data query security and improving the security of data query.

[0042] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more comprehensible. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0044] Figure 1 is a hardware structure block diagram of a terminal of the data query method in this embodiment.

[0045] Figure 2 is a flowchart of the data query method in this embodiment.

[0046] Figure 3It is a flowchart of another data query method of this embodiment.

[0047] Figure 4 It is a structural block diagram of the data query device of this embodiment.

[0048] Figure 5 It is the architecture and query flowchart of the data query system of this embodiment. Detailed implementation manners

[0049] To understand the purpose, technical solution and advantages of this application more clearly, the following describes and explains this application in combination with the accompanying drawings and embodiments.

[0050] Unless otherwise defined, the technical terms or scientific terms involved in this application should have the general meanings understood by those with ordinary skills in the technical field to which this application belongs. In this application, words such as "a", "one", "a kind of", "the", "these" and the like do not represent a limitation in quantity, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device including a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent in these processes, methods, products or devices. The terms "connection", "connection", "coupling" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly connected or indirectly connected. The "multiple" involved in this application means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Usually, the character " / " represents an "or" relationship between the associated objects before and after. The terms "first", "second", "third" and the like involved in this application only distinguish similar objects and do not represent a specific sorting of the objects.

[0051] The method embodiments provided in this embodiment can be executed on a terminal, a computer or a similar computing device. For example, running on a terminal Figure 1 It is a hardware structural block diagram of the terminal of the data query method of this embodiment. As Figure 1 shown, the terminal may include one or more ( Figure 1Only one (not shown) processor 102 and a memory 104 for storing data are shown. Among them, the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 The structure shown is only schematic and does not limit the structure of the above terminal. For example, the terminal may further include more or fewer components than Figure 1 shown in, or have a different configuration from Figure 1 that shown.

[0052] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the data query method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely set relative to the processor 102, and these remote memories can be connected to the terminal through a network. Examples of the above network include, but are not limited to, the Internet, enterprise intranets, local area networks, mobile communication networks, and combinations thereof.

[0053] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one instance, the transmission device 106 includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0054] In this embodiment, a data query method is provided. Figure 2 is a flowchart of the data query method of this embodiment, as Figure 2 shown, and the process includes the following steps:

[0055] Step S201, when a data query request is received, extract the preliminary feature vector in the data query request and encode the preliminary feature vector to obtain an encoded feature vector; wherein, the data query request is encrypted based on a trusted computing environment.

[0056] Specifically, this embodiment supports multi-modal data retrieval. Multi-Modal Retrieval (MMR for short) is an information retrieval method involving multiple media modalities (such as text, image, audio, video, etc.). To ensure the privacy throughout the data query process, different confidentiality treatments are performed on the query requests submitted by the querying party, the query processing process, and the query results. Among them, the data query requests can be text, image, audio, video, etc. After the querying party submits the query request, it is encrypted using the key negotiated by the trusted execution environment (TEE for short) of the client and the server. TEE creates an isolated execution environment at the hardware level to ensure the integrity and confidentiality of the code and data running therein. Compared with the ordinary operating system environment, TEE has a higher security level and can resist various attack means from the outside and the inside. On the client side, TEE can be used to encrypt the query requests input by the user; on the server side, TEE can be used to securely decrypt and process these requests while preventing the server system administrator or other unauthorized users from accessing sensitive data.

[0057] After the TEE environment of the client is started, a pair of asymmetric keys (public key and private key) is generated. The public key is sent to the server side, while the private key is securely stored inside the TEE of the client. After the TEE environment of the server side receives the public key of the client, it generates a pair of asymmetric keys, encrypts the public key of the server side using the public key of the client, and then sends the encrypted public key back to the client. After the client receives the encrypted public key of the server side, it decrypts it using its own private key to obtain the public key of the server side. At this time, both the TEE environments of the client and the server side can use the public keys and private keys of both parties to calculate a shared symmetric key through a key exchange algorithm (such as Diffie-Hellman). This symmetric key will be used for the encryption and decryption operations of subsequent query requests. During the entire key negotiation process, all key generation, exchange, and calculation operations are completed within the TEE environments of the client and the server side, ensuring the security of the keys. Even if an external attacker intercepts the communication data, they cannot obtain valid key information from it because the characteristics of the key exchange algorithm make it impossible for the attacker to deduce the shared symmetric key even if they know the public keys of both parties. After the client completes the key negotiation, it encrypts the query request using the negotiated symmetric key. The encrypted query request is encapsulated in a common network communication protocol (such as HTTP / HTTPS) and sent to the server side through the network. Since the query request has been encrypted, even if it is intercepted by an external attacker during transmission, the attacker cannot directly read the content of the query request because there is no corresponding decryption key. After the server side receives the encrypted query request, it passes it to the TEE environment of the server side. The TEE of the server side decrypts the query request using the previously negotiated symmetric key to obtain the data query request.

[0058] After the server side obtains the data query request, it performs preliminary feature extraction on the data query request. Depending on the type of the data query request, different feature extraction methods can be used. For text queries, feature extraction may include natural language processing techniques such as word segmentation, stop word removal, and stemming, converting the text into feature representations such as word vectors or TF-IDF vectors; for image queries, feature extraction may involve computer vision techniques such as image preprocessing (such as cropping, scaling), edge detection, and color histogram extraction, converting the image into a set of feature vectors; for audio queries, feature extraction may include preprocessing of audio signals (such as noise reduction, sampling rate conversion), Mel spectrogram extraction, and audio feature encoding, converting the audio signal into feature vectors.

[0059] After the initial feature extraction is completed, the TEE environment inputs the extracted features into a multi-modal retrieval model for encoding. A multi-modal retrieval model is a model that can process various types of data (such as text, images, audio, etc.). Its core function is to encode the features of different modalities into a unified feature space for similarity retrieval. Through the multi-modal retrieval model, a complete feature encoding is output, and then the complete feature encoding is returned to the TEE environment to obtain an encoded feature vector.

[0060] Step S202: Perform inverse rearrangement on the encoded feature vector to obtain an inverse rearranged feature vector.

[0061] Specifically, even though data is encrypted and privacy-protected during transmission and processing, attackers may still attempt to infer the content of the original data by analyzing the structure and pattern of the feature vector. This type of attack is called "vector inversion attack". To further enhance the security of the system and prevent attackers from recovering the original data through feature vector inversion, in this embodiment, the inverse rearrangement technique is used to process the encoded feature vector. Inverse rearrangement makes the structure and pattern of the encoded feature vector unpredictable by rearranging or transforming the encoded feature vector, thereby increasing the difficulty of inversion. The specific inverse rearrangement can be achieved by randomly permuting the dimensions of the vector, adding noise, performing linear transformation, etc. Through inverse rearrangement processing, an inverse rearranged feature vector is obtained. This processing makes the structure of the feature vector unpredictable, thus preventing attackers from recovering the original data through vector inversion.

[0062] Step S203: Determine the target query vector from the vector database according to the similarity between the inverse rearranged feature vector and the vectors in the preset vector database.

[0063] Specifically, first set up a vector database in advance. Extract the feature vectors of the original picture or video file through a pre-trained convolutional neural network (CNN) and store these vectors in the vector database. Calculate the similarity between the inverse rearranged feature vector and the vectors in the vector database one by one. The similarity calculation can use methods such as cosine similarity and Euclidean distance. This embodiment does not make specific limitations on this. According to the similarity score, find the vector in the vector database that is most similar to the query vector, and determine these most similar vectors as the target query vectors.

[0064] Step S204: Obtain the target access address corresponding to the target query vector from the preset vector and address mapping table; perform access after concealing the target access address to obtain the initial access result.

[0065] Specifically, the file ID corresponding to the vector is recorded in the vector database, a vector and address mapping table is established, and for the target query vector obtained by similarity calculation, the target access address corresponding to the target query vector is searched in the vector and address mapping table. The target access address is the address where the target file actually needs to be accessed and returned. To prevent the database administrator from obtaining the target file address queried by the querier through the file access log and thus learning the target file data, in this embodiment, before accessing the target access address, the target access address is first anonymized. Specifically, technologies such as proxy servers, VPNs, Tor, or distributed proxy networks, and randomly expanding the access address can be used to hide the user's access behavior and target address and prevent being tracked and analyzed. After anonymization processing, access is performed to obtain the initial access result. Among them, the initial access result includes the target file and non-target files, and the non-target files play a confusing role, making it impossible for the database administrator to directly learn the target file.

[0066] Step S205: Return the target access result corresponding to the target access address in the initial access result to the client.

[0067] Specifically, after the data access ends, other non-target access results are cleared, and only the target access result corresponding to the target access address is retained. These files are the results that the user truly needs, and these target access results are returned to the client.

[0068] Through the above steps S201 to S205, in the case of receiving a data query request, the preliminary feature vector in the data query request is extracted and encoded to obtain an encoded feature vector; among them, the data query request is encrypted based on the trusted computing environment; the encoded feature vector is inversely rearranged to obtain an inversely rearranged feature vector; according to the similarity between the inversely rearranged feature vector and the vectors in the preset vector database, the target query vector is determined from the vector database; the target access address corresponding to the target query vector is obtained from the preset vector and address mapping table; after anonymization processing of the target access address, access is performed to obtain the initial access result; the target access result corresponding to the target access address in the initial access result is returned to the client. Compared with the prior art of manually downloading the target data, in this embodiment, after encrypting the data query request in the TEE environment, extracting the feature vector, and then performing inverse rearrangement processing after encoding, it is prevented from inversely inferring and restoring the original request, increasing the security of data query. Finally, after obtaining the target access address through similarity comparison, the target access address is anonymized, further preventing the database administrator from obtaining the target file address queried by the querier through the file access log and improving the security of data query.

[0069] In some of these embodiments, before receiving the data query request, it further includes:

[0070] Perform a security verification on the trusted computing environment, and receive a data query request in the case of verified security.

[0071] Specifically, the TEE provides a highly secure execution environment that can protect the confidentiality, integrity, and availability of data. However, simply deploying the TEE environment is not enough. It is also necessary to verify whether the TEE environment is truly secure and reliable. Remote attestation is a mechanism for verifying the security of the TEE environment. Through remote attestation, it can be ensured that the TEE environment has not been tampered with and is running on trusted hardware. Before initiating a query request, the client first requests remote attestation from the TEE environment. The client sends an attestation request to the TEE environment through the network, requesting the TEE environment to generate and send an attestation report. The client receives the attestation result from the verification server. If the attestation result indicates that the TEE environment is trusted, the client can continue with subsequent operations; if the attestation result indicates that the TEE environment is not trusted, the client will reject the query request and may take other security measures, such as warning the user or terminating the connection. By performing a security verification on the TEE environment, the security of data queries can be further improved.

[0072] In another embodiment, encoding the preliminary feature vector includes: rearranging the preliminary feature vector to obtain a rearranged feature vector; encoding the rearranged feature vector to obtain an encoded feature vector.

[0073] Specifically, the feature vector contains sufficient information such that an attacker can recover part or all of the content of the original data through reverse engineering attacks. To prevent such attacks, the feature vector needs to be processed so that even if the attacker obtains the feature vector, they cannot reverse engineer to recover the original data. After extracting the preliminary feature vector from the original data (such as images, voice, text, etc.), the preliminary feature vector is rearranged. By rearranging, the structure of the feature vector is disrupted, making the relationship between the various dimensions of the feature vector unpredictable, thereby increasing the difficulty of reverse engineering. The rearrangement method can be random permutation, linear transformation, encryption algorithm, etc. In the TEE environment, the encoded feature vector is randomly inverse rearranged to obtain the inverse rearranged feature vector. This inverse rearrangement can be based on a random seed to ensure that the inverse rearrangement method is unique in each processing process. For example, the various dimensions of the feature vector can be randomly permuted, or a random linear transformation can be performed on the feature vector. Based on the rearranged feature vector, it is further encrypted. Encryption can use a symmetric encryption algorithm (such as AES), and the rearranged feature vector is encrypted using the key generated inside the TEE. In this way, even if the attacker can obtain the encrypted feature vector, they cannot directly decrypt or reverse engineer the original data. The inverse rearranged and encrypted feature vectors are stored and transmitted in the TEE environment. The isolation of the TEE environment ensures that these feature vectors cannot be accessed by the system administrator or other unauthorized users. Even if intercepted during storage or transmission, the attacker cannot recover the original query information through reverse engineering attacks.

[0074] The randomized rearrangement makes the structure and pattern of the feature vector unpredictable, and the attacker cannot find the reverse engineering rule through simple analysis; even if the encrypted feature vector is obtained, it cannot be directly decrypted, further increasing the difficulty of the attack; the isolation of the TEE environment ensures that the processing and storage process of the feature vector will not be interfered by the outside, and even the system administrator cannot access the data inside the TEE. After rearrangement and encryption processing, the final feature vector obtained in the TEE environment is secure and can be used for subsequent data retrieval or other processing tasks. These feature vectors are generated and used in the TEE environment, ensuring the security and privacy of the entire processing process from the original query information to the final feature vector.

[0075] After rearranging the preliminary feature vector, the inverse rearranged feature vector is input into the multimodal retrieval model, and a complete feature encoding is output through the multimodal retrieval model, and then the complete feature encoding is returned to the TEE environment to obtain the encoded feature vector.

[0076] In some of these embodiments, the vectors in the vector database include picture vectors, text vectors, audio vectors, and video vectors.

[0077] Specifically, the vector database is obtained by extracting the feature vectors of the original picture or video file through a pre-trained convolutional neural network (CNN), including specifically picture vectors, text vectors, audio vectors, and video vectors. The file ID corresponding to the vector is recorded in the vector database, supporting multimodal data retrieval.

[0078] In another embodiment, determining a target query vector from the vector database according to the similarity between the inverse-rearranged feature vector and the vectors in the preset vector database includes:

[0079] Calculating the similarity between the encoded feature vector after inverse rearrangement and each vector in the preset vector database to obtain each similarity score; determining the target query vector from the vector database according to the similarity score and the Top-n rule.

[0080] Specifically, calculating the similarity between the feature vector after inverse rearrangement and each vector in the vector database one by one to obtain each similarity score. Similarity calculation usually adopts methods such as cosine similarity and Euclidean distance. According to the similarity scores, all the calculated similarity scores are sorted from high to low, and the Top-n rule is adopted to retain the top n results with the highest similarity scores as the target query vector, where the value of n can be adjusted according to actual needs. For example: n = 3: Return the top 3 results with the highest similarity. n = 5: Return the top 5 results with the highest similarity.

[0081] In some of these embodiments, performing access after concealing the target access address to obtain an initial access result includes:

[0082] Adopting a randomized data access method to perform expansion processing on the target access address to obtain a preset number of expanded access addresses; performing access according to the expanded access addresses to obtain an initial access result.

[0083] Specifically, directly accessing the target file corresponding to the target query vector in the database after obtaining the target query vector may expose the user's query intention. To further ensure the security of data query, randomized data access processing is performed on the target access address. Exemplarily, after obtaining n (5) target query vectors, the corresponding target access address is obtained according to the target query vector, and the target access address is randomly extended. For example, the target access address is set to m (100), so as to obfuscate the target access address. Among them, the number of randomly extended target access addresses can be flexibly configured according to the security level of the query task. For example, for a query task with a high security level, m can be set to 1000, while for a query task with a normal security level, it can be set to 100. The greater the number of random extensions, the better the obfuscation effect. After the extension is completed, the extended target access address is accessed to obtain the initial access result. Since the access request set contains a large number of random addresses, the database administrator only knows that m files have been accessed, but cannot determine the n results that are actually hit. When the access result is returned, only n target access results need to be returned. In this way, it is ensured that the database administrator cannot directly learn the files that the visitor actually needs through the access log, ensuring the confidentiality and security of data query.

[0084] In another embodiment, returning the target access result corresponding to the target access address in the initial access result to the client includes:

[0085] Obtaining the target access result corresponding to the target access address; encrypting the target access result to obtain the encrypted target access result; returning the encrypted target access result to the client.

[0086] Specifically, after obtaining the initial access result, only the target access result corresponding to the target access address in the initial access result is returned to the client, and the remaining access results can be discarded. Since the communication between the client and the server may be subject to security threats such as man-in-the-middle attacks and data leakage. To ensure the confidentiality and integrity of the data, the data is encrypted during the data transmission process, and the encrypted target access result is returned to the client. At the same time, when returning the target access result, it is necessary to ensure that only authorized clients can decrypt and correctly use this data. Thus, ensuring the security of the target access result when it is returned.

[0087] In this embodiment, a data query method is also provided. Figure 3 It is a flowchart of another data query method of this embodiment, as Figure 3 shown, and this process includes the following steps:

[0088] Step S301: Perform a security verification on the trusted computing environment. When the verification shows security, receive a data query request; wherein, the data query request is encrypted based on the trusted computing environment.

[0089] Step S302: Extract the preliminary feature vector from the data query request.

[0090] Step S303: Rearrange the preliminary feature vector to obtain the rearranged feature vector.

[0091] Step S304: Encode the rearranged feature vector to obtain the encoded feature vector.

[0092] Step S305: Perform inverse rearrangement on the encoded feature vector to obtain the inverse-rearranged feature vector.

[0093] Step S306: Calculate the similarity between the inverse-rearranged encoded feature vector and each vector in the preset vector database to obtain each similarity score; according to the similarity score and the Top-n rule, determine the target query vector from the vector database.

[0094] Step S307: Obtain the target access address corresponding to the target query vector from the preset vector and address mapping table.

[0095] Step S308: Use a randomized data access method to perform an expansion process on the target access address to obtain a preset number of expanded access addresses; access according to the expanded access addresses to obtain the initial access result.

[0096] Step S309: Obtain the target access result corresponding to the target access address from the initial access result; encrypt the target access result to obtain the encrypted target access result; return the encrypted target access result to the client.

[0097] Exemplarily, taking the stealth search of image search as an example, it is illustrated that the user submits a picture to search for similar pictures in the file database:

[0098] The TEE environment is constructed using AMD's SEV technology, and the REE environment is constructed using GPU hardware.

[0099] (1) The user submits a picture as a query request through the client. In this example, a photo of a sports shoe is selected as the data query request. Then the client first completes the remote attestation of the TEE environment, verifies the security of the TEE environment, negotiates the encryption key with the TEE after verification, and then encrypts and transmits the picture to the TEE environment using this key.

[0100] (2) The TEE environment will perform preliminary feature extraction and feature encryption and rearrangement operations, submit the rearranged preliminary feature vectors to the REE environment for retrieval model encoding, and after the REE completes the encoding, return the encoded feature vectors. The TEE environment will perform inverse rearrangement on the encoded feature vectors to obtain inverse rearranged feature vectors.

[0101] (3) The TEE environment loads the feature vector database of the picture files in the database, calculates the similarity scores between the inverse rearranged feature vectors and each vector in the feature vector database, and obtains the corresponding access addresses of the top-3 vectors as the target access addresses. Randomly expand the access addresses to 100, and then access 100 files in the file database, but finally only retain the 3 hit files and return them to the client.

[0102] Through the above steps S301 to S309, compared with the prior art of manually downloading target data, in this embodiment, first, in the trusted computing environment (TEE environment), the data query request initiated by the client is encrypted to achieve the concealment of the data query request; second, after extracting the preliminary feature vectors of the data query request, the preliminary feature vectors are encrypted and rearranged and then encoded to obtain the encrypted and rearranged encoded feature vectors, and then the encoded features are inversely rearranged to prevent reverse inference and recovery of the original data query request; finally, according to the inverse rearranged feature vectors, the target query vectors and the target access addresses corresponding to the target query vectors are matched, and through the randomized data access method, the target access addresses are expanded, and accessing the expanded addresses is used to replace directly accessing the target access addresses. After the results are accessed, only the target access results corresponding to the target access addresses are returned to the client, thereby preventing the database administrator from learning the accessed target data through the access logs and improving the security of data query.

[0103] Through the data query method of this embodiment, the entire process of multi-modal data query has the privacy protection ability and can provide the following four aspects of privacy security:

[0104] (1) By encrypting the original query modal data (such as the original text description and pictures of the query) submitted by the data query party, and the encryption key is obtained through key negotiation between the client and the TEE environment, and only the TEE can decrypt it, the concealment of the data query request is achieved.

[0105] (2) By completing the preliminary encoding of the data features in the TEE for the features of the modal data in the original request and performing encryption and rearrangement, the rearranged features are output. The rearranged features conceal the features of the original modal data, and the original modal data cannot be reverse-inferred and recovered through the rearranged features, thus achieving the concealment of the modal data features.

[0106] (3) After the data features encoded by the multi-modal retrieval model are rearranged in the TEE, the target query vector and the target access address corresponding to the target query vector are obtained by calculating the vector similarity, and the randomized data access method is used to achieve the stealth access to the target access address. An attacker, including the database administrator, cannot confirm the hit file through the access log, realizing the stealth of the target access address.

[0107] (4) Only the target access result is retained in the TEE (other access results can be discarded), and the target access result is encrypted and returned to the query client. Only the data query party can decrypt it, realizing the stealth of the query result return.

[0108] In this embodiment, a data query device is also provided. This device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated here. The following terms such as "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function. Although the device described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0109] Figure 4 is the structural block diagram of the data query device in this embodiment. As Figure 4 shown, the device 40 includes: an extraction module 41, an inverse rearrangement module 42, a similarity comparison module 43, an access module 44, and a sending module 45; among them,

[0110] The extraction module 41 is used to extract the preliminary feature vector in the data query request and encode the preliminary feature vector to obtain the encoded feature vector when receiving the data query request; among them, the data query request is encrypted based on the trusted computing environment.

[0111] The inverse rearrangement module 42 is used to perform inverse rearrangement on the encoded feature vector to obtain the inverse rearranged feature vector.

[0112] The similarity comparison module 43 is used to determine the target query vector from the vector database according to the similarity between the inverse rearranged feature vector and the vectors in the preset vector database.

[0113] The access module 44 is used to obtain the target access address corresponding to the target query vector from the preset vector and address mapping table; perform stealth processing on the target access address and then access it to obtain the initial access result.

[0114] The sending module 45 is used to return the target access result corresponding to the target access address in the initial access result to the client.

[0115] It should be noted that the above-mentioned various modules can be functional modules or program modules, and can be implemented either by software or by hardware. For the modules implemented by hardware, the above-mentioned various modules can be located in the same processor; or the above-mentioned various modules can also be located in different processors in any combined form.

[0116] In this embodiment, a data query system is further provided, including: a client and a server, where

[0117] The client is used to initiate a data query request, encrypt the data query request based on a trusted computing environment, and is also used to receive a target access result and decrypt the target access result.

[0118] The server includes a trusted computing environment and a rich execution environment, where, in the trusted computing environment, the server is used to extract feature vectors from the data query request, encrypt and rearrange the feature vectors, inverse rearrangement, similarity calculation, randomly extended access, and return the target access result; in the rich execution environment, the server is used to encode the encrypted and rearranged feature vectors.

[0119] Specifically, Figure 5 is the data query system architecture and query flow chart of this embodiment, as Figure 5 shown, the data query system architecture includes a user side (data query party) and a server side (data owner side). The server side includes a TEE trusted computing environment and an REE rich execution environment. The data query process includes the following steps:

[0120] User side (data query party):

[0121] 1. Original query request (query request encryption);

[0122] The user side first generates an original query request, which contains query conditions for specific data. To protect the user's privacy and the security of the query, the original query request is encrypted.

[0123] Server side (data owner side):

[0124] 2. Preliminary feature extraction;

[0125] After receiving the encrypted query request, the server side first performs preliminary feature extraction on the data in the TEE environment.

[0126] 3. Feature encryption and rearrangement;

[0127] The extracted feature vectors are encrypted and rearranged to further enhance data security and privacy protection.

[0128] 4. Multimodal retrieval model encoding;

[0129] Submit the rearranged feature vectors to the regular REE environment, and use the multi-modal retrieval model to encode the rearranged feature vectors, which involves using the GPU for high-performance computing. Return the encoded feature vectors to the TEE environment.

[0130] 5. Inverse rearrangement;

[0131] Perform inverse rearrangement on the encoded feature vectors to prevent the original modal data from being recovered by inverse inference of the rearranged features.

[0132] 6. Similarity calculation;

[0133] In the TEE environment, calculate the similarity between the inverse-rearranged feature vectors and the vectors in the vector database to determine which vectors in the vector database best match the user's query request. Then, according to the Top-n principle, select the target query vectors.

[0134] 7. Random extended access;

[0135] Obtain the target access address according to the target query vector, perform random extension on the target access address, and access the file database according to the extended address to obtain the initial access result.

[0136] 8. Retain the hit results;

[0137] Retain the truly hit target access results from the initial access results and discard other access results.

[0138] 9. Hit result file (return result encryption);

[0139] Encrypt the hit result file, i.e., the target access result, and return it to the client.

[0140] 10. Decrypt the result;

[0141] After receiving the target access result, the client decrypts the target access result to obtain the target query data.

[0142] In addition, in combination with the data query method provided in the above embodiments, a storage medium can also be provided in this embodiment to implement it. A computer program is stored on the storage medium; when the computer program is executed by a processor, any one of the data query methods in the above embodiments is implemented.

[0143] It should be understood that the specific embodiments described here are only used to explain this application, rather than to limit it. According to the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of this application.

[0144] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0145] Obviously, the accompanying drawings are only some examples or embodiments of this application. For those of ordinary skill in the art, this application can also be applied to other similar situations based on these drawings without creative efforts. Additionally, it can be understood that although the work done during this development process may be complex and time-consuming, for those of ordinary skill in the art, certain design, manufacturing, or production changes based on the technical content disclosed in this application are only routine technical means and should not be regarded as insufficient disclosure of this application.

[0146] The term "embodiment" in this application means that the specific features, structures, or characteristics described in connection with an embodiment may be included in at least one embodiment of this application. The phrase appears in various positions in the specification and does not necessarily mean the same embodiment, nor does it mean being independent or alternative to other embodiments and mutually exclusive. Those of ordinary skill in the art can clearly or implicitly understand that the embodiments described in this application can be combined with other embodiments without conflict.

[0147] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0148] The above embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of patent protection. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A data query method, characterized in that, Including: Upon receiving a data query request, extracting a preliminary feature vector from the data query request and encoding the preliminary feature vector to obtain an encoded feature vector; wherein, the data query request is encrypted based on a trusted computing environment; Performing inverse rearrangement on the encoded feature vector to obtain an inverse rearranged feature vector; Determining a target query vector from the vector database according to the similarity between the inverse rearranged feature vector and vectors in a preset vector database; Obtaining a target access address corresponding to the target query vector from a preset vector and address mapping table; Performing access after concealing the target access address to obtain an initial access result; Returning the target access result corresponding to the target access address in the initial access result to the client.

2. The data query method according to claim 1, wherein Before receiving the data query request, the method further includes: Performing security verification on the trusted computing environment, and upon successful verification, receiving the data query request.

3. The data query method according to claim 1, wherein The encoding of the preliminary feature vector includes: Rearranging the preliminary feature vector to obtain a rearranged feature vector; Encoding the rearranged feature vector to obtain an encoded feature vector.

4. The data query method according to claim 1, wherein The vectors in the vector database include picture vectors, text vectors, audio vectors, and video vectors.

5. The data query method according to claim 1, wherein The determining of the target query vector from the vector database according to the similarity between the inverse rearranged feature vector and vectors in a preset vector database includes: Calculating the similarity between the inverse rearranged encoded feature vector and each vector in the preset vector database to obtain respective similarity scores; Determining the target query vector from the vector database according to the similarity scores and the Top-n rule.

6. The data query method according to claim 1, wherein The performing of access after concealing the target access address to obtain an initial access result includes: Performing expansion processing on the target access address by using a randomized data access method to obtain a preset number of expanded access addresses; Performing access according to the expanded access addresses to obtain the initial access result.

7. The data query method according to claim 6, wherein The returning of the target access result corresponding to the target access address in the initial access result to the client includes: Obtaining the target access result corresponding to the target access address; Encrypting the target access result to obtain an encrypted target access result; Returning the encrypted target access result to the client.

8. A data query device, characterized in that, Including: An extraction module, an inverse rearrangement module, a similarity comparison module, an access module, and a sending module; wherein, The extraction module is configured to, upon receiving a data query request, extract a preliminary feature vector from the data query request and encode the preliminary feature vector to obtain an encoded feature vector; wherein, the data query request is encrypted based on a trusted computing environment; The inverse rearrangement module is configured to perform inverse rearrangement on the encoded feature vector to obtain an inverse rearranged feature vector; The similarity comparison module is configured to determine a target query vector from the vector database according to the similarity between the inverse rearranged feature vector and vectors in a preset vector database; The access module is configured to obtain the target access address corresponding to the target query vector from a preset vector and address mapping table; perform access after performing stealth processing on the target access address to obtain an initial access result; The sending module is configured to return the target access result corresponding to the target access address in the initial access result to the client.

9. A data query system, characterized in that, including: a client and a server, wherein, the client is configured to initiate a data query request, encrypt the data query request based on a trusted computing environment, and is further configured to receive the target access result and decrypt the target access result; the server includes a trusted computing environment and a rich execution environment. In the trusted computing environment, the server is configured to perform feature vector extraction, feature vector encryption rearrangement and inverse rearrangement, similarity calculation, random extended access, and return the target access result on the data query request; in the rich execution environment, the server is configured to encode the encrypted and rearranged feature vectors.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the data query method according to any one of claims 1 to 7.

Images