Retrieval method, device and system and nonvolatile computer readable storage medium
By encrypting the content to be retrieved in the trusted execution environment of the central cluster and searching in multiple data clusters, the problem of degradation of data security in cross-domain search is solved, and high security and consistent data retrieval is achieved.
Patent Information
- Application Number
- CN202311189203.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-14
- Publication Date
- 2025-03-14
AI Technical Summary
In cross-domain retrieval scenarios, data comes from different organizations or departments and is in different clusters, and there are information security risks such as data leakage, resulting in a decrease in data security.
The search method on the central cluster side is adopted to encrypt the search content in the trusted execution environment of the central cluster, and the encrypted content is distributed to multiple data clusters for searching. The search results returned by multiple data clusters are fused in the central cluster to determine the final search results.
Through the use of encrypted and trusted execution environments, the security of data during transmission and processing is improved, the risk of data leakage is reduced, and the security and consistency of data is ensured.
Smart Images
Figure CN119621794A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of communication technology, and in particular to a retrieval method, a retrieval device, a retrieval system, an electronic device, and a non-volatile computer-readable storage medium. Background Art
[0002] With the innovation of business models, more and more companies need to combine their own data with the data of other companies to mine value in order to guide the development and decision-making of the company. Therefore, for cross-domain retrieval scenarios, joint data retrieval is required.
[0003] In related technologies, data in cross-domain retrieval scenarios comes from different organizations or departments, is in different clusters, and is physically isolated from each other. Summary of the invention
[0004] The inventors of the present disclosure have discovered that the above-mentioned related technologies have the following problems: there are information security risks such as data leakage, which leads to a decrease in data security.
[0005] In view of this, the present disclosure proposes a retrieval technology solution that can improve the security of data during the retrieval process.
[0006] According to some embodiments of the present disclosure, a retrieval method on the central cluster side is provided, including: encrypting content to be retrieved in a trusted execution environment of the central cluster; sending the encrypted content to be retrieved to multiple data clusters so as to retrieve the encrypted content to be retrieved in the trusted execution environments of the multiple data clusters; and fusing multiple first retrieval results returned by the multiple data clusters to determine a second retrieval result.
[0007] In some embodiments, in the trusted execution environment created by the central cluster, encrypting the content to be retrieved includes: generating a public key and a private key in the trusted execution environment of the central cluster; using the public key to encrypt the content to be retrieved so that the encrypted content to be retrieved can be decrypted using the private key in the trusted execution environments of multiple data clusters.
[0008] In some embodiments, fusing multiple first search results returned by multiple data clusters to determine a second search result includes: receiving multiple encrypted first search results returned by multiple data clusters, encrypting the multiple first search results using a public key in a trusted execution environment of the multiple data clusters; and decrypting the encrypted multiple first search results using a private key in a trusted execution environment of a central cluster to obtain the multiple first search results.
[0009] In some embodiments, in the trusted execution environment of the central cluster, decrypting the encrypted multiple first search results using a private key includes: authenticating the encrypted multiple first search results; and decrypting the encrypted multiple first search results if the encrypted multiple first search results pass the authentication.
[0010] In some embodiments, multiple first search results are obtained in a trusted execution environment of multiple data clusters in the following manner: searching based on scalar information and vector information of the content to be searched, and vector indexes and scalar indexes of data in the database to obtain multiple first search results.
[0011] In some embodiments, the plurality of first search results are obtained using a secure application programming interface of a trusted execution environment of the plurality of data clusters.
[0012] In some embodiments, scalar indexes and vector indexes are obtained in a trusted execution environment of multiple data clusters in the following manner: based on the scalar information of the data in the database, the data is divided into different data partitions, and the scalar information is generated based on the key information of the data; based on the keywords of the data in the data partition, an inverted index of the data in the data partition is established as a scalar index; and the data in the data partition is vectorized to generate a vector index.
[0013] In some embodiments, the multiple first search results are obtained in the trusted execution environment of the multiple data clusters in the following manner: searching the content to be searched according to the vector information and keywords of the content to be searched to obtain the multiple first search results.
[0014] In some embodiments, the encrypted content to be retrieved is retrieved in a trusted execution environment of multiple data clusters in the following manner: the encrypted content to be retrieved is authenticated; if the encrypted content to be retrieved passes the authentication, the encrypted content to be retrieved is decrypted in the trusted execution environment of multiple data clusters to obtain the content to be retrieved.
[0015] According to other embodiments of the present disclosure, a retrieval method on the data cluster side is provided, including: receiving content to be retrieved encrypted in a trusted execution environment of a central cluster; performing decryption processing in the trusted execution environment of the data cluster to obtain the content to be retrieved; searching the content to be retrieved in the trusted execution environment of the data cluster to obtain a first retrieval result; returning the first retrieval result to the central cluster, so that the central cluster can fuse the first retrieval results returned by multiple data clusters to determine a second retrieval result.
[0016] In some embodiments, the content to be retrieved is encrypted by generating a public key in the trusted execution environment of the central cluster, and decrypted in the trusted execution environment of the data cluster to obtain the content to be retrieved, including: decrypting in the trusted execution environment of the data cluster using a private key, and the private key is generated in the trusted execution environment of the central cluster.
[0017] In some embodiments, returning the first search result to the central cluster includes: encrypting the first search result using a public key in a trusted execution environment of the data cluster; returning the encrypted first search result to the central cluster so that the encrypted multiple first search results can be decrypted using a private key in the trusted execution environment of the central cluster.
[0018] In some embodiments, the encrypted first search result is decrypted in the trusted execution environment of the central cluster in the following manner: authenticating the encrypted first search result; and decrypting the encrypted first search result if the encrypted first search result passes the authentication.
[0019] In some embodiments, in a trusted execution environment of a data cluster, searching for content to be searched includes: searching according to scalar information and vector information of the content to be searched, and vector indexes and scalar indexes of data in a database to obtain a first search result.
[0020] In some embodiments, the first search result is obtained using a secure application programming interface of a trusted execution environment of the data cluster.
[0021] In some embodiments, retrieval based on scalar information and vector information of content to be retrieved, as well as vector indexes and scalar indexes of data in a database includes: dividing the data into different data partitions based on the scalar information of the data in the database, and generating scalar information based on key information of the data; establishing an inverted index of the data in the data partition as a scalar index based on keywords of the data in the data partition; and vectorizing the data in the data partition to generate a vector index.
[0022] In some embodiments, in the trusted execution environment of the data cluster, searching for the content to be searched includes: searching for the content to be searched according to vector information and keywords of the content to be searched to obtain a plurality of first search results.
[0023] In some embodiments, in a trusted execution environment of a data cluster, performing decryption processing includes: authenticating the encrypted content to be retrieved; and when the encrypted content to be retrieved passes the authentication, decrypting the encrypted content to be retrieved to obtain the content to be retrieved.
[0024] According to some further embodiments of the present disclosure, a retrieval system is provided, including: a central cluster for executing the retrieval method on the central cluster side in any of the above embodiments; and multiple data clusters for executing the retrieval method on the data cluster side in any of the above embodiments.
[0025] According to some further embodiments of the present disclosure, a retrieval device on the central cluster side is provided, including: a content distribution module, used to encrypt the content to be retrieved in the trusted execution environment of the central cluster, and send the encrypted content to be retrieved to multiple data clusters, so as to retrieve the encrypted content to be retrieved in the trusted execution environment of the multiple data clusters; a result fusion module, used to fuse multiple first retrieval results returned by the multiple data clusters to determine a second retrieval result.
[0026] In some embodiments, the content distribution module generates a public key and a private key in the trusted execution environment of the central cluster, and uses the public key to encrypt the content to be retrieved, so that the encrypted content to be retrieved can be decrypted using the private key in the trusted execution environment of multiple data clusters.
[0027] In some embodiments, the result fusion module receives multiple encrypted first search results returned by multiple data clusters, and the multiple first search results are encrypted using a public key in a trusted execution environment of the multiple data clusters; the retrieval device also includes an authentication and decryption module, which is used to decrypt the encrypted multiple first search results using a private key in the trusted execution environment of the central cluster to obtain multiple first search results.
[0028] In some embodiments, the authentication and decryption module authenticates the encrypted multiple first search results; if the encrypted multiple first search results pass the authentication, the encrypted multiple first search results are decrypted.
[0029] In some embodiments, multiple first search results are obtained in a trusted execution environment of multiple data clusters in the following manner: searching based on scalar information and vector information of the content to be searched, and vector indexes and scalar indexes of data in the database to obtain multiple first search results.
[0030] In some embodiments, the plurality of first search results are obtained using a secure application programming interface of a trusted execution environment of the plurality of data clusters.
[0031] In some embodiments, scalar indexes and vector indexes are obtained in a trusted execution environment of multiple data clusters in the following manner: based on the scalar information of the data in the database, the data is divided into different data partitions, and the scalar information is generated based on the key information of the data; based on the keywords of the data in the data partition, an inverted index of the data in the data partition is established as a scalar index; and the data in the data partition is vectorized to generate a vector index.
[0032] In some embodiments, the encrypted content to be retrieved is retrieved in a trusted execution environment of multiple data clusters in the following manner: the encrypted content to be retrieved is authenticated; if the encrypted content to be retrieved passes the authentication, the encrypted content to be retrieved is decrypted in the trusted execution environment of multiple data clusters to obtain the content to be retrieved.
[0033] According to some further embodiments of the present disclosure, a retrieval device on the data cluster side is provided, including: an authentication and decryption module, used to receive content to be retrieved encrypted in a trusted execution environment of a central cluster, and perform decryption processing in the trusted execution environment of the data cluster to obtain the content to be retrieved; a retrieval module, used to search the content to be retrieved in the trusted execution environment of the data cluster to obtain a first retrieval result, and return the first retrieval result to the central cluster, so that the central cluster can fuse the first retrieval results returned by multiple data clusters to determine a second retrieval result.
[0034] In some embodiments, the content to be retrieved is encrypted by generating a public key in the trusted execution environment of the central cluster, and the authentication and decryption module performs decryption processing in the trusted execution environment of the data cluster using a private key generated in the trusted execution environment of the central cluster.
[0035] In some embodiments, the retrieval module encrypts the first retrieval result using a public key in the trusted execution environment of the data cluster; and returns the encrypted first retrieval result to the central cluster so that the encrypted multiple first retrieval results can be decrypted using a private key in the trusted execution environment of the central cluster.
[0036] In some embodiments, the encrypted first search result is decrypted in the trusted execution environment of the central cluster in the following manner: authenticating the encrypted first search result; and decrypting the encrypted first search result if the encrypted first search result passes the authentication.
[0037] In some embodiments, the retrieval module performs a search based on the scalar information and vector information of the content to be retrieved, and the vector index and scalar index of the data in the database to obtain a first search result.
[0038] In some embodiments, the first search result is obtained using a secure application programming interface of a trusted execution environment of the data cluster.
[0039] In some embodiments, the retrieval module divides the data into different data partitions based on the scalar information of the data in the database, and the scalar information is generated based on the key information of the data; based on the keywords of the data in the data partition, an inverted index of the data in the data partition is established as a scalar index; and the data in the data partition is vectorized to generate a vector index.
[0040] In some embodiments, the authentication and decryption module authenticates the encrypted content to be retrieved; if the encrypted content to be retrieved passes the authentication, the encrypted content to be retrieved is decrypted to obtain the content to be retrieved.
[0041] According to some further embodiments of the present disclosure, there is provided an electronic device, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the retrieval method in any one of the above embodiments based on instructions stored in the memory device.
[0042] According to some further embodiments of the present disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the retrieval method in any of the above embodiments is implemented.
[0043] In the above embodiment, a trusted execution environment is used to create an isolated execution environment for the central cluster and multiple data clusters. Cross-domain retrieval of data in the isolated execution environment can avoid information security risks such as data leakage, thereby improving data security. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0045] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings:
[0046] Figure 1 A flowchart showing some embodiments of the retrieval method of the present disclosure;
[0047] Figure 2a Schematic diagrams showing some embodiments of the retrieval system of the present disclosure;
[0048] Figure 2b Schematic diagrams showing some embodiments of the retrieval process of the present disclosure;
[0049] Figure 3 Flowcharts showing other embodiments of the retrieval method disclosed herein;
[0050] Figure 4a-4b A block diagram showing some embodiments of the retrieval device of the present disclosure;
[0051] Figure 5 A block diagram showing some embodiments of the electronic device of the present disclosure;
[0052] Figure 6 A block diagram showing some other embodiments of the electronic device of the present disclosure;
[0053] Figure 7 A block diagram showing some embodiments of the retrieval system of the present disclosure. DETAILED DESCRIPTION
[0054] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.
[0055] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.
[0056] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.
[0057] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.
[0058] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.
[0059] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0060] As mentioned above, in the process of data sharing, data security and privacy leakage have always been the focus of enterprise concerns. Therefore, how to conduct reliable data value mining under the condition of ensuring data security and privacy has become a current research trend.
[0061] For example, cross-cluster joint retrieval (such as semantic retrieval, etc.) mainly has the following technical problems.
[0062] (1) Data leakage risk. Cross-cluster federated retrieval involves multiple data sources and storage systems. If not properly protected, sensitive information may be leaked during data transmission or processing, leading to privacy issues and security vulnerabilities.
[0063] (2) Access control and authentication issues. In a cross-cluster environment, different data sources may use different authentication and access control mechanisms. Therefore, it is important to ensure the consistency of authentication and access control between clusters to avoid unauthorized access and information leakage.
[0064] (3) Malicious attacks and tampering. Cross-cluster joint retrieval without proper protection is vulnerable to malicious attacks, such as man-in-the-middle attacks, data tampering, or denial of service attacks.
[0065] In order to avoid data security issues in cross-cluster semantic retrieval, related technologies centralize user data to a centralized server for processing, which may lead to the risk of privacy leakage.
[0066] In response to the above technical problems, the technical solution disclosed in the present invention adopts TEE (Trusted Execution Environment) to perform cross-domain retrieval of data.
[0067] TEE is a secure computing environment that provides hardware and software level protection to protect sensitive data and execution processes from unauthorized access and attacks. It creates an isolated execution environment where code and data in a trusted execution environment can be securely computed and processed in an untrusted environment.
[0068] Cross-domain retrieval technology based on TEE can effectively solve the data security and data consistency problems in cross-domain retrieval. Moreover, by using the hardware security provided by TEE technology to ensure the credibility and security of the retrieval process, the availability and practicality of cross-domain retrieval technology can be improved.
[0069] For example, the following embodiments may be used to implement the technical solution of the present disclosure.
[0070] Figure 1 A flowchart showing some embodiments of the retrieval method of the present disclosure.
[0071] like Figure 1 As shown, in step 110, the content to be retrieved is encrypted in the trusted execution environment of the central cluster.
[0072] For example, the central cluster is responsible for connecting to each sub-cluster (i.e., each data cluster), distributing the user's search requirements to each sub-cluster, receiving the search results of each sub-cluster, and determining the overall search results through fusion processing.
[0073] For example, each data cluster can be located in different places, belong to the same enterprise or unit, and be physically isolated from each other. The data cluster needs to complete registration in the central cluster and obtain authorization to interact with the central cluster. The authorized data cluster receives the search request of the central cluster, performs search within the data cluster, and returns the results to the central cluster after the search is completed.
[0074] In some embodiments, a public key and a private key are generated in a trusted execution environment of a central cluster; the content to be retrieved is encrypted using the public key, so that the encrypted content to be retrieved can be decrypted using the private key in the trusted execution environments of multiple data clusters.
[0075] In some embodiments, the content to be retrieved is encrypted in the central cluster and then distributed to the data cluster after encryption. After receiving the retrieval content distributed by the central cluster, the data cluster performs retrieval within the data cluster.
[0076] For example, create a TEE for semantic retrieval (including the central cluster and the TEEs of each data cluster); generate a symmetric key, including a public key and a private key, in the TEE environment of the central cluster to encrypt the data; and encrypt the content to be retrieved. For example, symmetric encryption algorithms such as AES (Advanced Encryption Standard) can be used for encryption.
[0077] In this way, the generated key only exists within the TEE and cannot be obtained by external programs, ensuring the confidentiality and integrity of the key.
[0078] In step 120, the encrypted content to be searched is sent to multiple data clusters, so that the encrypted content to be searched can be searched in the trusted execution environments of the multiple data clusters.
[0079] For example, the encrypted information to be retrieved is distributed to various data clusters for the next step of semantic retrieval.
[0080] In some embodiments, the encrypted content to be retrieved is authenticated in a trusted execution environment of multiple data clusters; if the encrypted content to be retrieved passes the authentication, the encrypted content to be retrieved is decrypted in the trusted execution environment of multiple data clusters to obtain the content to be retrieved.
[0081] For example, in the semantic retrieval program where the data cluster is located, the encrypted data to be retrieved and the authentication information are transmitted to the TEE environment of the data cluster for authentication and decryption.
[0082] In this way, the authentication process ensures that only authorized entities can access the data in the data cluster, thereby improving data security.
[0083] For example, after receiving the information to be retrieved, the data cluster authenticates the encrypted information to be retrieved to ensure that the data has not been tampered with during transmission. For example, authentication algorithms such as HMAC (Hash-based Message Authentication Code) can be used for authentication. For example, the private key in the symmetric key generated by the content distribution module can be used to decrypt the authenticated data to obtain the decrypted information to be retrieved.
[0084] In some embodiments, in the trusted execution environment of multiple data clusters, a search is performed based on the scalar information and vector information of the content to be searched, and the vector index and scalar index of the data in the database to obtain multiple first search results. For example, the multiple first search results are obtained using a secure application program interface of the trusted execution environment of multiple data clusters.
[0085] In some embodiments, the content to be searched is searched based on the vector information and keywords of the content to be searched to obtain a plurality of first search results.
[0086] In some embodiments, the semantic retrieval program runs in the TEE environment of the data cluster; in this TEE environment, key components of the semantic retrieval system can be run securely.
[0087] For example, data security can be improved through a secure boot process. When starting the TEE, the hardware verifies the integrity and authenticity of the TEE. For example, the verification process can include the hardware verifying the identity of the TEE based on the signature of the TEE and ensuring that the code and data of the TEE have not been tampered with. Only when the verification is passed can the TEE be securely started and begin execution.
[0088] For example, data security can be improved through the security mechanism of TEE. The code and data inside TEE are protected by the security mechanism of TEE, such as memory encryption, code integrity verification, and access control.
[0089] In this way, index construction and query processing are performed in the TEE environment of the data cluster, thereby completing the training and indexing of data in the data cluster to improve retrieval efficiency.
[0090] In some embodiments, in a trusted execution environment of multiple data clusters, data is divided into different data partitions based on scalar information of the data in the database, and the scalar information is generated based on key information of the data; based on keywords of the data in the data partition, an inverted index of the data in the data partition is established as a scalar index; and the data in the data partition is vectorized to generate a vector index.
[0091] For example, a data partition corresponding to the information to be retrieved can be determined based on at least one of the scalar index or the vector index, and the information to be retrieved can be retrieved in the data partition. In this way, the search scope can be narrowed for the information to be retrieved, thereby improving the search efficiency.
[0092] For example, extract key information of data in a data cluster to form scalar information; partition the data in the data cluster based on the scalar information to improve retrieval efficiency; extract keywords from the data in each data partition and use the keywords to build an inverted index; vectorize the partitioned data through a vectorization model to form a vector index.
[0093] For example, a multi-way recall mechanism of vector index and inverted index can be used to quickly perform nearest neighbor search on massive data in the data cluster based on semantics and retrieve the results.
[0094] In some embodiments, the plurality of first search results are obtained using a secure application programming interface of a trusted execution environment of the plurality of data clusters.
[0095] For example, index data and user queries are encrypted and protected within the TEE by using the secure API (Application Programming Interface) provided by the TEE, ensuring that the data is inaccessible outside the TEE.
[0096] These APIs provide access to TEE functions such as encryption, decryption, secure storage, and secure computing. By using secure APIs, the retrieval system can perform operations such as query processing, index building, and result return within the TEE to protect the security of data and operations.
[0097] In step 130 , a plurality of first search results returned by a plurality of data clusters are merged to determine a second search result.
[0098] In some embodiments, multiple encrypted first search results returned by multiple data clusters are received, and the multiple first search results are encrypted using a public key in a trusted execution environment of the multiple data clusters; in a trusted execution environment of a central cluster, the encrypted multiple first search results are decrypted using a private key to obtain the multiple first search results.
[0099] For example, the encrypted multiple first search results are authenticated; if the encrypted multiple first search results pass the authentication, the encrypted multiple first search results are decrypted.
[0100] For example, the central cluster receives the search results returned by multiple data clusters, performs authentication and decryption in the TEE environment of the central cluster, and obtains the decrypted search results.
[0101] This ensures that data cannot be read outside the TEE and prevents man-in-the-middle attacks or unauthorized access.
[0102] For example, the returned first search results can be sorted based on the similarity distance between each first search result and the content to be searched; the first n first search results with the closest distance are selected (the value of n can be set according to actual needs) as the second search results.
[0103] In the above embodiment, through the cross-cluster joint retrieval method based on TEE, massive unstructured data can be retrieved without moving the data, thereby realizing the release of data value. During the retrieval process, the security and privacy of the data are protected, and the retrieval process is monitored and verified to ensure the accuracy and credibility of the calculation results.
[0104] Figure 2a Schematic diagrams showing some embodiments of the retrieval system of the present disclosure.
[0105] like Figure 2a As shown, the central cluster is responsible for connecting to each sub-cluster (i.e., each data cluster), distributing the user's search requirements to each sub-cluster through the central node; receiving the search results of each sub-cluster, and determining the overall search results through fusion processing.
[0106] Each data cluster can be located in different places, can belong to the same enterprise or unit, and be physically isolated from each other. The data cluster needs to complete registration in the central cluster and obtain authorization to interact with the central cluster. The authorized data cluster receives the search requirements of the central cluster, performs searches within the data cluster, and returns the results to the central cluster after the search is completed.
[0107] In some embodiments, the retrieval system mainly includes four modules: a content distribution module, an authentication and decryption module, a retrieval module, and a result fusion module.
[0108] For example, the content distribution module is mainly responsible for: encrypting the content to be searched in the central cluster; distributing the encrypted content to be searched to the data cluster. After the data cluster receives the search content distributed by the central cluster, it searches within the data cluster. The following process may be included.
[0109] Create a TEE for semantic retrieval; generate a symmetric key, including a public key and a private key, in the TEE environment of the central cluster to encrypt data; encrypt the content to be retrieved. For example, symmetric encryption algorithms such as AES can be used for encryption. The generated key only exists inside the TEE and will not be obtained by external programs to ensure the confidentiality and integrity of the key. Distribute the encrypted information to be retrieved to each data cluster for the next step of semantic retrieval.
[0110] For example, the authentication and decryption module is mainly responsible for: in the semantic retrieval program where the data cluster is located, transmitting the encrypted data to be retrieved and the authentication information to the TEE environment of the data cluster for authentication and decryption. In this way, the authentication link ensures that only authorized entities can access the data in the data cluster, thereby improving the security of the data. The following process can be included.
[0111] After receiving the information to be retrieved, the data cluster authenticates the encrypted information to be retrieved (such as through software signature, etc.) to ensure that the data has not been tampered with during transmission. For example, authentication algorithms such as HMAC can be used for authentication. For example, the private key in the symmetric key generated by the content distribution module can be used to decrypt the authenticated data to obtain the decrypted information to be retrieved.
[0112] For example, the semantic retrieval module is mainly responsible for the training and indexing process of data in the data cluster to improve the retrieval efficiency. The semantic retrieval program runs in the TEE environment of the data cluster; in this TEE environment, the key components of the semantic retrieval system can be run safely. For example, the security of data can be improved through the secure startup process. The following process can be included.
[0113] When starting the TEE, the hardware verifies the integrity and authenticity of the TEE. For example, the verification process can include the hardware verifying the identity of the TEE based on the TEE's signature and ensuring that the TEE's code and data have not been tampered with. Only after passing the verification can the TEE be safely started and begin execution. The security of data can be improved through the security mechanism of the TEE. The code and data inside the TEE are protected by the security mechanisms of the TEE, such as memory encryption, code integrity verification, and access control.
[0114] Figure 2b Schematic diagram showing some embodiments of the retrieval process of the present disclosure.
[0115] like Figure 2b As shown, the process of establishing an index may include: extracting key information of the data in the data cluster to form scalar information; partitioning the data in the data cluster based on the scalar information to improve retrieval efficiency; extracting keywords from the data in each data partition and using the keywords to build an inverted index; vectorizing the partitioned data through a vectorization model to form a vector index.
[0116] For example, the retrieval process may include: using a multi-way recall mechanism of vector index and inverted index to quickly perform nearest neighbor retrieval on massive data in the data cluster based on semantics to retrieve results.
[0117] In the above embodiments, index data and user queries are encrypted and protected inside the TEE, which is achieved by using the security API provided by the TEE to ensure that the data is inaccessible outside the TEE. These APIs provide access to TEE functions such as encryption, decryption, secure storage, and secure computing. By using the security API, the retrieval system can perform operations such as query processing, index building, and result return inside the TEE to protect the security of data and operations.
[0118] For example, the result fusion module is mainly responsible for: the central cluster receives the first search results (such as search results 1 and 2) returned by multiple data clusters, performs authentication (such as authentication through software signatures, etc.) and decryption in the TEE environment of the central cluster, and obtains the decrypted search results.
[0119] This ensures that data cannot be read outside the TEE and prevents man-in-the-middle attacks or unauthorized access.
[0120] For example, the returned first search results can be sorted based on the similarity distance between each first search result and the content to be searched; the first n first search results with the closest distance are selected (the value of n can be set according to actual needs) as the second search results.
[0121] In the above embodiment, the cross-cluster semantic retrieval system is protected by TEE to prevent the risk of data leakage caused by plain text transmission or sharing of data in semantic retrieval. In this way, cross-domain retrieval can be performed while protecting user data privacy, realizing joint computing across different domains.
[0122] In addition, the cross-cluster retrieval method based on TEE can store index data and sensitive information inside TEE. In this way, the integrity and confidentiality of the data can be ensured, data can be prevented from being tampered with or leaked, and data protection measures can be increased.
[0123] Moreover, through a series of security mechanisms, including secure API, code verification, access control, etc., verifiability and credibility are provided to monitor and verify the retrieval calculation process to ensure the accuracy and credibility of the calculation results.
[0124] Figure 3 Flowcharts showing other embodiments of the retrieval method of the present disclosure.
[0125] like Figure 3 As shown, in step 310, content to be retrieved that is encrypted in a trusted execution environment of a central cluster is received.
[0126] In step 320, a decryption process is performed in the trusted execution environment of the data cluster to obtain the content to be retrieved.
[0127] In some embodiments, the encrypted content to be retrieved is authenticated; if the encrypted content to be retrieved passes the authentication, the encrypted content to be retrieved is decrypted to obtain the content to be retrieved.
[0128] In some embodiments, the content to be retrieved is encrypted by generating a public key in the trusted execution environment of the central cluster, and decrypted in the trusted execution environment of the data cluster using a private key generated in the trusted execution environment of the central cluster.
[0129] In step 330, the content to be searched is searched in the trusted execution environment of the data cluster to obtain a first search result.
[0130] In some embodiments, a search is performed based on scalar information and vector information of the content to be searched, and a vector index and scalar index of data in a database to obtain a first search result.
[0131] For example, the first search result is obtained using a secure application programming interface of a trusted execution environment of the data cluster.
[0132] In some embodiments, data is divided into different data partitions based on scalar information of the data in a database, and the scalar information is generated based on key information of the data; based on keywords of the data in the data partition, an inverted index of the data in the data partition is established as a scalar index; and the data in the data partition is vectorized to generate a vector index.
[0133] In step 340, the first search result is returned to the central cluster so that the central cluster can fuse the first search results returned by multiple data clusters to determine the second search result.
[0134] In some embodiments, the first search result is encrypted using a public key in the trusted execution environment of the data cluster; the encrypted first search result is returned to the central cluster so that the encrypted multiple first search results can be decrypted using a private key in the trusted execution environment of the central cluster.
[0135] In some embodiments, in the trusted execution environment of the central cluster, the encrypted first search result is authenticated; if the encrypted first search result passes the authentication, the encrypted first search result is decrypted.
[0136] Figure 4a-4b A block diagram showing some embodiments of the retrieval device of the present disclosure.
[0137] like Figure 4aAs shown, the retrieval device 4a on the central cluster side includes: a content distribution module 41a, which is used to encrypt the content to be retrieved in the trusted execution environment of the central cluster, and send the encrypted content to be retrieved to multiple data clusters, so as to retrieve the encrypted content to be retrieved in the trusted execution environment of multiple data clusters; a result fusion module 42a, which is used to fuse multiple first retrieval results returned by multiple data clusters to determine a second retrieval result.
[0138] In some embodiments, the content distribution module 41a generates a public key and a private key in the trusted execution environment of the central cluster, and uses the public key to encrypt the content to be retrieved, so that the encrypted content to be retrieved can be decrypted using the private key in the trusted execution environment of multiple data clusters.
[0139] In some embodiments, the result fusion module 42a receives multiple encrypted first search results returned by multiple data clusters, and the multiple first search results are encrypted using a public key in a trusted execution environment of the multiple data clusters; the retrieval device 4a also includes an authentication and decryption module 43a, which is used to decrypt the encrypted multiple first search results using a private key in the trusted execution environment of the central cluster to obtain multiple first search results.
[0140] In some embodiments, the authentication and decryption module 43a authenticates the encrypted multiple first search results; if the encrypted multiple first search results pass the authentication, the encrypted multiple first search results are decrypted.
[0141] In some embodiments, multiple first search results are obtained in a trusted execution environment of multiple data clusters in the following manner: searching based on scalar information and vector information of the content to be searched, and vector indexes and scalar indexes of data in the database to obtain multiple first search results.
[0142] In some embodiments, the multiple first search results are obtained in the trusted execution environment of the multiple data clusters in the following manner: searching the content to be searched according to the vector information and keywords of the content to be searched to obtain the multiple first search results.
[0143] In some embodiments, the plurality of first search results are obtained using a secure application programming interface of a trusted execution environment of the plurality of data clusters.
[0144] In some embodiments, scalar indexes and vector indexes are obtained in a trusted execution environment of multiple data clusters in the following manner: based on the scalar information of the data in the database, the data is divided into different data partitions, and the scalar information is generated based on the key information of the data; based on the keywords of the data in the data partition, an inverted index of the data in the data partition is established as a scalar index; and the data in the data partition is vectorized to generate a vector index.
[0145] In some embodiments, the encrypted content to be retrieved is retrieved in a trusted execution environment of multiple data clusters in the following manner: the encrypted content to be retrieved is authenticated; if the encrypted content to be retrieved passes the authentication, the encrypted content to be retrieved is decrypted in the trusted execution environment of multiple data clusters to obtain the content to be retrieved.
[0146] like Figure 4b As shown, the retrieval device 4b on the data cluster side includes: an authentication and decryption module 41b, which is used to receive the content to be retrieved encrypted in the trusted execution environment of the central cluster, and perform decryption processing in the trusted execution environment of the data cluster to obtain the content to be retrieved; a retrieval module 42b, which is used to search the content to be retrieved in the trusted execution environment of the data cluster to obtain a first retrieval result, and return the first retrieval result to the central cluster, so that the central cluster can merge the first retrieval results returned by multiple data clusters to determine a second retrieval result.
[0147] In some embodiments, the content to be retrieved is encrypted by generating a public key in the trusted execution environment of the central cluster, and the authentication and decryption module performs decryption processing in the trusted execution environment of the data cluster using a private key generated in the trusted execution environment of the central cluster.
[0148] In some embodiments, the retrieval module 42b encrypts the first retrieval result using a public key in the trusted execution environment of the data cluster; and returns the encrypted first retrieval result to the central cluster so that the encrypted multiple first retrieval results can be decrypted using a private key in the trusted execution environment of the central cluster.
[0149] In some embodiments, the encrypted first search result is decrypted in the trusted execution environment of the central cluster in the following manner: authenticating the encrypted first search result; and decrypting the encrypted first search result if the encrypted first search result passes the authentication.
[0150] In some embodiments, the retrieval module 42b performs retrieval based on the scalar information and vector information of the content to be retrieved, as well as the vector index and scalar index of the data in the database, to obtain a first retrieval result.
[0151] In some embodiments, the first search result is obtained using a secure application programming interface of a trusted execution environment of the data cluster.
[0152] In some embodiments, the retrieval module 42b divides the data into different data partitions according to the scalar information of the data in the database, and the scalar information is generated according to the key information of the data; according to the keywords of the data in the data partition, an inverted index of the data in the data partition is established as a scalar index; and the data in the data partition is vectorized to generate a vector index.
[0153] In some embodiments, the search module 42b searches the content to be searched according to the vector information and keywords of the content to be searched to obtain a plurality of first search results.
[0154] In some embodiments, the authentication and decryption module 41b authenticates the encrypted content to be retrieved; if the encrypted content to be retrieved passes the authentication, the encrypted content to be retrieved is decrypted to obtain the content to be retrieved.
[0155] Figure 5 A block diagram showing some embodiments of the electronic device of the present disclosure.
[0156] like Figure 5 As shown, the electronic device 5 of this embodiment includes: a memory 51 and a processor 52 coupled to the memory 51 , and the processor 52 is configured to execute the retrieval method in any one of the embodiments of the present disclosure based on the instructions stored in the memory 51 .
[0157] The memory 51 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, a database, and other programs.
[0158] Figure 6 A block diagram showing some other embodiments of the electronic device of the present disclosure.
[0159] like Figure 6 As shown, the electronic device 6 of this embodiment includes: a memory 610 and a processor 620 coupled to the memory 610 , and the processor 620 is configured to execute the retrieval method in any one of the aforementioned embodiments based on instructions stored in the memory 610 .
[0160] The memory 610 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs.
[0161] The electronic device 6 may also include an input / output interface 630, a network interface 640, a storage interface 650, etc. These interfaces 630, 640, 650, the memory 610, and the processor 620 may be connected, for example, via a bus 660. The input / output interface 630 provides a connection interface for input / output devices such as a display, a mouse, a keyboard, a touch screen, a microphone, and a speaker. The network interface 640 provides a connection interface for various networked devices. The storage interface 650 provides a connection interface for external storage devices such as an SD card and a USB flash drive.
[0162] Figure 7 A block diagram showing some embodiments of the retrieval system of the present disclosure.
[0163] like Figure 7 As shown, the retrieval system 7 includes: a central cluster 71, which is used to execute the retrieval method on the central cluster side in any of the above embodiments; and multiple data clusters 72, which are used to execute the retrieval method on the data cluster side in any of the above embodiments.
[0164] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media including but not limited to disk storage, CD-ROM, optical storage, etc., which contain computer-usable program code.
[0165] So far, the retrieval method, retrieval device, retrieval system, electronic device and non-volatile computer-readable storage medium according to the present disclosure have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed herein.
[0166] The method and system of the present disclosure may be implemented in many ways. For example, the method and system of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.
[0167] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.
Claims
1. A search method, comprising: In the trusted execution environment of the central cluster, the content to be retrieved is encrypted; Sending the encrypted content to be searched to multiple data clusters, so as to search the encrypted content to be searched in the trusted execution environments of the multiple data clusters; The multiple first search results returned by the multiple data clusters are merged to determine a second search result.
2. The search method according to claim 1, wherein: The encrypting of the content to be retrieved in the trusted execution environment created by the central cluster includes: In the trusted execution environment of the central cluster, generating a public key and a private key; The content to be retrieved is encrypted using the public key, so that the encrypted content to be retrieved is decrypted using the private key in the trusted execution environment of the multiple data clusters.
3. The search method according to claim 2, wherein: The step of fusing the plurality of first search results returned by the plurality of data clusters to determine the second search result includes: receiving the encrypted plurality of first search results returned by the plurality of data clusters, wherein the plurality of first search results are encrypted using the public key in a trusted execution environment of the plurality of data clusters; In the trusted execution environment of the central cluster, the encrypted multiple first search results are decrypted using the private key to obtain the multiple first search results.
4. The search method according to claim 3, wherein: The decrypting the encrypted plurality of first search results by using the private key in the trusted execution environment of the central cluster comprises: authenticating the encrypted plurality of first search results; When the encrypted first search results are authenticated, the encrypted first search results are decrypted.
5. The search method according to any one of claims 1 to 4, wherein: The multiple first search results are obtained in the trusted execution environment of the multiple data clusters in the following manner: A search is performed based on the scalar information and vector information of the content to be searched, and the vector index and scalar index of the data in the database to obtain the multiple first search results.
6. The search method according to claim 5, wherein: The plurality of first search results are obtained using a secure application programming interface of a trusted execution environment of the plurality of data clusters.
7. The search method according to claim 5, wherein: The scalar index and the vector index are obtained in the trusted execution environment of the multiple data clusters in the following manner: Dividing the data into different data partitions according to scalar information of the data in the database, wherein the scalar information is generated according to key information of the data; According to the keywords of the data in the data partition, an inverted index of the data in the data partition is established as the scalar index; Vectorization processing is performed on the data in the data partition to generate the vector index.
8. The search method according to any one of claims 1 to 4, wherein: The multiple first search results are obtained in the trusted execution environment of the multiple data clusters in the following manner: The content to be searched is searched according to the vector information and keywords of the content to be searched to obtain the multiple first search results.
9. The search method according to any one of claims 1 to 4, wherein: The encrypted content to be retrieved is retrieved in the trusted execution environment of the multiple data clusters in the following manner: authenticating the encrypted content to be retrieved; When the encrypted content to be retrieved passes authentication, the encrypted content to be retrieved is decrypted in the trusted execution environment of the plurality of data clusters to obtain the content to be retrieved.
10. A retrieval method comprising: receiving content to be retrieved that is encrypted in a trusted execution environment of a central cluster; In the trusted execution environment of the data cluster, decryption processing is performed to obtain the content to be retrieved; In the trusted execution environment of the data cluster, searching the content to be searched to obtain a first search result; The first search result is returned to the central cluster, so that the central cluster fuses the first search results returned by multiple data clusters to determine a second search result.
11. The search method according to claim 10, wherein: The content to be retrieved is encrypted by generating a public key in the trusted execution environment of the central cluster, The performing of decryption processing in the trusted execution environment of the data cluster to obtain the content to be retrieved includes: In the trusted execution environment of the data cluster, a decryption process is performed using a private key, where the private key is generated in the trusted execution environment of the central cluster.
12. The search method according to claim 11, wherein: The returning the first search result to the central cluster comprises: In the trusted execution environment of the data cluster, encrypting the first search result using the public key; The encrypted first search results are returned to the central cluster, so that the encrypted multiple first search results are decrypted using the private key in the trusted execution environment of the central cluster.
13. The search method according to claim 12, wherein: The encrypted first search result is decrypted in the trusted execution environment of the central cluster in the following manner: authenticating the encrypted first search result; When the encrypted first search result passes the authentication, the encrypted first search result is decrypted.
14. The search method according to any one of claims 10 to 13, wherein: The retrieving the content to be retrieved in the trusted execution environment of the data cluster includes: A search is performed based on the scalar information and vector information of the content to be searched, and the vector index and scalar index of the data in the database to obtain the first search result.
15. The search method according to claim 14, wherein: The first search result is obtained using a secure application programming interface of a trusted execution environment of the data cluster.
16. The search method according to claim 14, wherein: The searching according to the scalar information and vector information of the content to be searched, and the vector index and scalar index of the data in the database includes: Dividing the data into different data partitions according to scalar information of the data in the database, wherein the scalar information is generated according to key information of the data; According to the keywords of the data in the data partition, an inverted index of the data in the data partition is established as the scalar index; Vectorization processing is performed on the data in the data partition to generate the vector index.
17. The search method according to any one of claims 10 to 13, wherein: The retrieving the content to be retrieved in the trusted execution environment of the data cluster includes: The content to be searched is searched according to the vector information and keywords of the content to be searched to obtain the multiple first search results.
18. The search method according to any one of claims 10 to 13, wherein: The decryption process in the trusted execution environment of the data cluster includes: authenticating the encrypted content to be retrieved; When the encrypted content to be retrieved passes the authentication, the encrypted content to be retrieved is decrypted to obtain the content to be retrieved.
19. A retrieval system comprising: A central cluster, used to execute the search method according to any one of claims 1 to 9; A plurality of data clusters are used to execute the retrieval method described in any one of claims 10-18.
20. A retrieval device, comprising: A content distribution module, used for encrypting the content to be searched in the trusted execution environment of the central cluster, and sending the encrypted content to be searched to multiple data clusters, so as to search the encrypted content to be searched in the trusted execution environments of the multiple data clusters; The result fusion module is used to fuse the multiple first search results returned by the multiple data clusters to determine the second search result.
21. A retrieval device, comprising: An authentication and decryption module, used to receive the content to be retrieved encrypted in the trusted execution environment of the central cluster, and perform decryption processing in the trusted execution environment of the data cluster to obtain the content to be retrieved; The retrieval module is used to search the content to be searched in the trusted execution environment of the data cluster to obtain a first search result, and return the first search result to the central cluster so that the central cluster can merge the first search results returned by multiple data clusters to determine a second search result.
22. An electronic device, comprising: Memory; and A processor coupled to the memory, wherein the processor is configured to execute the retrieval method according to any one of claims 1 to 18 based on instructions stored in the memory.
23. A non-volatile computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the retrieval method according to any one of claims 1 to 18 is implemented.