A ciphertext data retrieval method and device based on overlapping clustering and a medium

Abstract

The application discloses a ciphertext data retrieval method and device based on overlapping clustering and a medium. Non-sensitive element information in a target original message is acquired in real time and quantified to generate a current multi-dimensional non-sensitive element feature vector. Overlapping clustering is performed to generate each current fuzzy membership degree and a plaintext clustering center vector. The ciphertext indexes corresponding to each message obtained are distributed and stored in the corresponding clusters to realize hierarchical storage architecture storage. According to a current user query vector, the similarity of each plaintext clustering center vector in the current plaintext clustering center layer is calculated, and a target cluster set is determined by combining a cluster selection method. Retrieval is performed in each ciphertext index subset corresponding to the current ciphertext index layer to generate and feed back a target ciphertext data retrieval result. The problems of high time complexity, low retrieval efficiency, low data security and high storage cost in the prior art are solved, and the efficiency and security of ciphertext data retrieval are improved.

Description

Technical Field

[0001] This invention relates to the fields of information security and encrypted data retrieval technology, and in particular to an encrypted data retrieval method, apparatus and medium based on overlapping clustering. Background Technology

[0002] With the rapid development of cloud computing and big data technologies, data privacy protection has become increasingly prominent. To prevent data leakage, encrypting data and storing it in the cloud or untrusted environments has become the mainstream practice. However, while data encryption ensures security, it also renders traditional plaintext-based indexing and retrieval technologies ineffective. How to quickly and accurately retrieve data items that meet specific conditions from massive amounts of encrypted data without decryption—the encrypted retrieval problem—has become a research hotspot in cryptography and information security. In the field of encrypted retrieval, with the explosive growth of data volume, retrieval efficiency has become a core bottleneck restricting system performance. Especially in real-time systems with high throughput and low latency requirements (such as battlefield situational awareness and real-time risk control in financial transactions), millisecond-level response latency is crucial.

[0003] In the process of developing this invention, the inventors discovered the following shortcomings in existing technologies: Existing technologies perform full-scale traversal retrieval, storing a full encrypted index in the index database. During retrieval, each index record is scanned sequentially, checking for match to the query conditions, and finally returning all matching message numbers. While this approach is logically simple, its time complexity is high. When the data volume reaches millions or even hundreds of millions, a single retrieval can take seconds or even minutes, which cannot meet the business requirements of real-time reporting scenarios demanding sub-second response times. Acceleration using B+ tree single-dimensional indexes only allows efficient retrieval on a single dimension; filtering conditions for other dimensions can only be applied after the result set is returned, potentially leading to an exceptionally large intermediate result set. Retrieval efficiency remains limited by I / O overhead and memory processing capabilities. Pre-filtering retrieval based on hard clustering forces a message into a single cluster if it falls within a fuzzy region between two or more cluster boundaries, resulting in decreased recall and low data security. For parallel retrieval using multiple indexes, index redundancy is extremely severe, requiring significant storage costs, resulting in high computational overhead and poor performance. Summary of the Invention

[0004] This invention provides a method, apparatus, and medium for retrieval of encrypted data based on overlapping clustering, so as to improve the efficiency and security of encrypted data retrieval and reduce storage costs.

[0005] According to one aspect of the present invention, a method for retrieving encrypted data based on overlapping clustering is provided, comprising:

[0006] The non-sensitive metadata in the target original message is acquired in real time, and the non-sensitive metadata is quantified to generate the current multi-dimensional non-sensitive metadata feature vector.

[0007] The non-sensitive metadata includes at least one of the following: service domain, message type, time window coarse-grainedness, and the sender's organization;

[0008] The current multi-dimensional non-sensitive meta-feature vector is clustered using a preset overlapping clustering method to generate the current fuzzy membership degree and plaintext cluster center vector for each message corresponding to each cluster.

[0009] The current fuzzy membership degree is converted into the corresponding current discrete cluster affiliation relationship through a preset dual threshold mechanism, and the ciphertext index corresponding to each message is distributed and stored in the cluster corresponding to the current discrete cluster affiliation relationship, so as to realize the storage of the layered storage architecture consisting of the current plaintext clustering center layer and the current ciphertext index layer.

[0010] In real time, based on the current user query vector, the similarity between the vector and the plaintext cluster center vector corresponding to each cluster in the current plaintext cluster center layer is calculated. Combined with a preset cluster selection method, the target cluster set is determined. Based on the target cluster set, the retrieval is performed in each ciphertext index subset corresponding to the current ciphertext index layer, and the target ciphertext data retrieval result is generated and fed back.

[0011] According to another aspect of the present invention, a ciphertext data retrieval device based on overlapping clustering is provided, comprising:

[0012] The current multi-dimensional non-sensitive element feature vector generation module is used to acquire non-sensitive element information in the target original message in real time, quantify the non-sensitive element information, and generate the current multi-dimensional non-sensitive element feature vector.

[0013] The non-sensitive metadata includes at least one of the following: service domain, message type, time window coarse-grainedness, and the sender's organization;

[0014] The current fuzzy membership degree and plaintext cluster center vector generation module is used to cluster the current multi-dimensional non-sensitive element feature vectors using a preset overlapping clustering method, and generate the current fuzzy membership degree and plaintext cluster center vector for each message corresponding to each cluster.

[0015] The hierarchical storage module is used to convert each current fuzzy membership degree into the corresponding current discrete cluster affiliation relationship through a preset dual threshold mechanism, and distribute and store the ciphertext index corresponding to each message to the cluster corresponding to the current discrete cluster affiliation relationship, so as to realize the hierarchical storage architecture consisting of the current plaintext clustering center layer and the current ciphertext index layer for storage.

[0016] The target encrypted data retrieval result generation and feedback module is used to acquire and calculate the similarity between the target encrypted data retrieval result and the plaintext cluster center vector corresponding to each cluster in the current plaintext cluster center layer, based on the current user query vector, and combine it with a preset cluster selection method to determine the target cluster set. Based on the target cluster set, it performs retrieval in each encrypted index subset corresponding to the current encrypted index layer, generates and feeds back the target encrypted data retrieval result.

[0017] According to another aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the encrypted data retrieval method based on overlapping clustering as described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the encrypted data retrieval method based on overlapping clustering as described in any embodiment of the present invention.

[0019] The technical solution of this invention involves acquiring non-sensitive metadata from the target original message in real time, quantizing the non-sensitive metadata, and generating a current multi-dimensional non-sensitive metadata feature vector. A preset overlapping clustering method is used to cluster the current multi-dimensional non-sensitive metadata feature vector, generating a current fuzzy membership degree and plaintext cluster center vector for each message corresponding to each cluster. A preset dual-threshold mechanism is used to convert each current fuzzy membership degree into a corresponding current discrete cluster affiliation. The acquired ciphertext index corresponding to each message is distributed and stored in the cluster corresponding to the current discrete cluster affiliation, thereby realizing a structure consisting of a current plaintext cluster center layer and a current ciphertext index layer. The system uses a hierarchical storage architecture for storage. It acquires and calculates the similarity between the current user query vector and the plaintext cluster center vector corresponding to each cluster in the current plaintext cluster center layer. Combined with a preset cluster selection method, it determines the target cluster set and performs retrieval in each ciphertext index subset corresponding to the current ciphertext index layer based on the target cluster set. It then generates and returns the target ciphertext data retrieval results. This solves the problems of high time complexity, low retrieval efficiency, low data security, and high storage cost in existing technologies, improving the efficiency and security of ciphertext data retrieval, reducing the time complexity of ciphertext data retrieval, and lowering the cost requirements for data storage.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a ciphertext data retrieval method based on overlapping clustering according to Embodiment 1 of the present invention;

[0023] Figure 2 This is a detailed flowchart of a encrypted data retrieval method based on overlapping clustering according to Embodiment 2 of the present invention;

[0024] Figure 3 This is a schematic diagram of the structure of a ciphertext data retrieval device based on overlapping clustering according to Embodiment 3 of the present invention;

[0025] Figure 4This is a schematic diagram of the structure of an electronic device provided according to Embodiment 4 of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "target," "current," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] It is worth noting that the information collected in the technical solution of this application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with the relevant laws, regulations and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse; if the user chooses to refuse, the process will proceed to the expert decision-making process.

[0029] Example 1

[0030] Figure 1 The flowchart of the encrypted data retrieval method based on overlapping clustering is provided in Embodiment 1 of the present invention. This embodiment is applicable to scenarios in which encrypted data can be retrieved efficiently and accurately through the overlapping clustering mechanism. The method can be executed by an encrypted data retrieval device based on overlapping clustering, which can be implemented in hardware and / or software.

[0031] Correspondingly, such as Figure 1 As shown, the method includes:

[0032] S110. Obtain non-sensitive metadata in the target original message in real time, quantify the non-sensitive metadata, and generate the current multi-dimensional non-sensitive metadata feature vector.

[0033] The non-sensitive metadata includes at least one of the following: service domain, message type, time window coarse-grainedness, and the sending party's organization.

[0034] The encrypted data retrieval system includes a data preprocessing module, an overlapping clustering module, an index building module, a retrieval execution module, and an adaptive maintenance module.

[0035] Specifically, the business domain can be vectorized using one-hot encoding; business domains can include command and control, intelligence analysis, and logistical support, with low sensitivity for category labels. Message types can also be vectorized using one-hot encoding; message types can include instructions, reports, warnings, and queries, with low sensitivity. Coarse-grained time windows can be vectorized using sequential encoding, which can be time buckets divided by day, week, or month, with low sensitivity for time range labels. The sender's organization can be vectorized using one-hot encoding, with low-level sensitivity for organizational identifier labels.

[0036] In this embodiment, the dimensions selected for clustering follow the principle of plaintext storage; the information of these dimensions does not itself constitute a core security risk, that is, it does not contain the specific payload content of the message or high-precision sensitive values, but it can effectively distinguish the categories of messages and provide a good semantic basis for clustering.

[0037] Furthermore, for categorical dimensions (such as business domains), one-hot encoding can be used for quantization; for ordered dimensions (such as security level or coarse-grained time window), sequential encoding can be used. Ultimately, each message corresponding to the target original message can be converted into a multi-dimensional non-sensitive meta-feature vector, where the vector corresponding to the i-th message can be... , where m represents the total number of clustering dimensions; This represents the vector element of the first dimension in the non-sensitive meta-feature vector corresponding to the i-th message. The target original message can be a set of messages, where... The corresponding set of multi-dimensional insensitive meta-feature vectors is Where n represents the number of messages in the target original message.

[0038] S120. Using a preset overlapping clustering method, cluster the current multi-dimensional non-sensitive meta-feature vector to generate the current fuzzy membership degree and plaintext cluster center vector for each message corresponding to each cluster.

[0039] Optionally, the step of clustering the current multi-dimensional non-sensitive meta-feature vector using a preset overlapping clustering method to generate the current fuzzy membership degree and plaintext cluster center vector for each message corresponding to each cluster includes: clustering the current multi-dimensional non-sensitive meta-feature vector using a preset overlapping clustering method, and using the formula... and It is possible to calculate the plaintext cluster center vector and the current fuzzy membership degree for each message corresponding to each cluster; where, p is a fuzziness index used to control the degree of fuzziness in the clustering results, with a value greater than 1. This indicates that under the fuzzy index p, the i-th message The current fuzzy membership degree of cluster j; Indicates message The current multi-dimensional non-sensitive meta-feature vector; n represents the total number of packets in the target original message; Let represent the plaintext cluster center vector of cluster j; C represents the number of clusters. Let represent the Euclidean norm; where the objective function corresponding to the overlapping clustering method is . .

[0040] Continuing from the previous example, the target original message is defined as follows: The corresponding set of multi-dimensional insensitive meta-feature vectors is The preset number of clusters is set to C. The objective function for the overlapping clustering method is... The goal of this algorithm is to minimize the objective function J. Wherein, This indicates that under the fuzziness index p, the message The current fuzzy membership degree of cluster j is a real number in the interval [0,1].

[0041] Furthermore, the constraints of the objective function can be determined as follows: That is, for each message, the sum of its membership degrees to all clusters is 1.

[0042] Accordingly, by solving the above optimization problem using the Lagrange multiplier method, we can obtain the iterative update formulas for the current fuzzy membership degree and the plaintext cluster center vector: and The algorithm iteratively executes the two formulas mentioned above until the change in the objective function J is less than a preset threshold, or the maximum number of iterations is reached. It can calculate, under the fuzzy index p, the plaintext cluster center vector and current fuzzy membership degree corresponding to the i-th message and the corresponding cluster j. Furthermore, it can calculate, under the fuzzy index p, the plaintext cluster center vector and current fuzzy membership degree corresponding to each message and each cluster, and thus, generate a fuzzy membership degree matrix. .

[0043] S130. The current fuzzy membership degree is converted into the corresponding current discrete cluster affiliation relationship through a preset dual threshold mechanism, and the ciphertext index corresponding to each message is distributed and stored in the cluster corresponding to the current discrete cluster affiliation relationship, so as to realize the storage of the layered storage architecture composed of the current plaintext clustering center layer and the current ciphertext index layer.

[0044] Unlike traditional hard clustering, this embodiment of the invention does not set a "hard truncation threshold," allowing any number of clusters. This means that a message can belong to multiple clusters with different strengths, thus providing multiple coverage for boundary samples and solving the missed detection problem of hard clustering.

[0045] Correspondingly, the fuzzy membership matrix provides a continuous probability distribution, but in actual index storage, packets need to be allocated to specific physical clusters. If packets are copied to all... In clusters where storage overhead is uncontrollable, a dual-threshold mechanism is designed to convert fuzzy membership to actual storage affiliation.

[0046] Specifically, the step of converting each current fuzzy membership degree into a corresponding current discrete cluster affiliation through a preset dual-threshold mechanism includes: obtaining the high threshold parameter and low threshold parameter corresponding to the dual-threshold mechanism; sequentially determining the current fuzzy membership degree corresponding to each cluster of each message; if the current fuzzy membership degree is greater than or equal to the high threshold parameter, then determining that the message belongs to the target cluster; if the current fuzzy membership degree is greater than or equal to the low threshold parameter and less than the high threshold parameter, then dynamically determining whether it belongs to the target cluster based on the current system storage capacity and retrieval performance target; if the current fuzzy membership degree is less than the low threshold parameter, then determining whether the message belongs to the target cluster.

[0047] In this embodiment, the high threshold parameter is set to and low threshold parameter are ;satisfy For messages And cluster j, if the current fuzzy membership degree is greater than or equal to , confirm message Belongs to the target cluster j; if the current fuzzy membership degree is greater than or equal to and smaller than When, then a message is sent. You can choose to assign the message to the target cluster j. The system will dynamically decide whether to assign the message to the target cluster j based on the current storage capacity and retrieval performance goals. Add the packets to cluster j. For example, when the overall storage redundancy is below a preset upper limit, the system can prioritize including packets with higher membership degrees into the cluster. If the current fuzzy membership degree is less than... , confirm message Clusters that do not belong to the target cluster j are considered irrelevant to the message and their indexes are not included in the cluster.

[0048] Correspondingly, by adjusting and The system can flexibly balance recall rate and storage overhead. The lower or The higher the value, the greater the probability of the message being copied, resulting in improved recall but increased storage overhead; conversely, the lower the value, the greater the probability of the message being copied.

[0049] Furthermore, the encrypted indexes corresponding to each acquired message are distributed and stored in clusters corresponding to the current discrete cluster affiliation, thus realizing a hierarchical storage architecture consisting of the current plaintext clustering center layer and the current encrypted index layer. This storage architecture is crucial for improving retrieval efficiency and security, explicitly dividing data storage into two layers. The current plaintext clustering center layer stores the centroid vectors of all clusters (which may contain only non-sensitive metadata); its security level is plaintext, and it can be publicly disclosed; its main purpose is to quickly locate the target cluster in the first retrieval stage. The current encrypted index layer stores the encrypted indexes (such as security indexes, message UUIDs, etc.) of each cluster's message; its security level is encrypted, and it is used for precise matching in the second retrieval stage.

[0050] Correspondingly, the plaintext cluster center vectors only contain aggregated metadata such as "business domain = command," and do not contain any specific message content, keywords, values, or any sensitive information that can be used to reconstruct the original message. Therefore, storing the cluster centers in plaintext does not introduce additional security risks, and its security level is equivalent to publicly available data classification and grading standards outside the system.

[0051] In this embodiment, a subset of ciphertext indexes is stored in each cluster j, and the message indexes are encrypted using a secure encryption algorithm (such as AES). The index of each message may be stored multiple times, and there is one index record for each cluster it belongs to.

[0052] S140. In real time, based on the current user query vector, calculate the similarity between the vector and the plaintext cluster center vector corresponding to each cluster in the current plaintext cluster center layer, and combine it with a preset cluster selection method to determine the target cluster set. Then, based on the target cluster set, perform retrieval in each ciphertext index subset corresponding to the current ciphertext index layer, generate and return the target ciphertext data retrieval results.

[0053] In this embodiment, when a user submits a query vector q for the current user, the system first extracts the clustering dimension constraints (such as "business domain = command and control, time window coarse-grained = last week") from the query vector and converts it into a query vector with the same dimension as the cluster center. , where R m It is an m-dimensional real vector space. The transformation rules are consistent with those of the data preprocessing module. Then, the system calculates q with each plaintext cluster center vector (based on the plaintext cluster center vector of cluster j). For example, the similarity between two dimensions (e.g., the similarity function can be selected based on the dimension type). The cosine similarity function is a preferred option, and its calculation formula is as follows: To further optimize the selection of target clusters, a query-aware cluster selection mechanism is proposed, instead of simply selecting the top few clusters with the highest similarity.

[0054] The mechanism includes the following steps: 1. Similarity filtering can filter out those with similarity below a threshold. 1. Clusters that are considered irrelevant to the query. 2. Calculate expected returns: For each remaining cluster (using cluster j as an example), calculate its expected return. : ,in: It represents the total number of messages contained in cluster j; It is an adjustment parameter ( The condition (greater than 0 and less than 1) is used to control the impact of cluster size on revenue. The core idea of ​​this formula is to balance relevance and retrieval cost. A highly relevant cluster, if its size is too large, will have its priority reduced due to high retrieval costs; conversely, a cluster with slightly lower relevance but a smaller size may gain higher revenue because it can complete retrieval faster. 3. Selecting the target cluster: Selecting the expected revenue. The top K clusters are selected as the target cluster set T for the first stage output. K is a preset parameter with a default value of 3, which can be adjusted according to actual recall requirements.

[0055] Further, encrypted retrieval within the target cluster. The system operates only on the target cluster set T selected in the first stage. First, it needs to read the encrypted index: retrieve the subset of encrypted indexes corresponding to the target cluster j belonging to T from storage. Since only the indexes of K clusters are read (K is usually much smaller than C), disk I / O overhead is significantly reduced. Next, ciphertext retrieval needs to be performed: the specific retrieval logic is executed on a subset of the ciphertext index. If the total number of packets in the target cluster... If the number of packets is less than the threshold M (e.g., M=10000), a full traversal is performed, decrypting each index header and matching them one by one. If the total number of packets within the target cluster is still large, a secondary index (such as a B+ tree or inverted index) can be further built within the cluster to speed up the matching process. Finally, the UUIDs (or data item IDs) of all matching packets need to be returned, and the client can retrieve the ciphertext as needed and decrypt it using the key.

[0056] The technical solution of this invention involves acquiring non-sensitive metadata from the target original message in real time, quantizing the non-sensitive metadata, and generating a current multi-dimensional non-sensitive metadata feature vector. A preset overlapping clustering method is used to cluster the current multi-dimensional non-sensitive metadata feature vector, generating a current fuzzy membership degree and plaintext cluster center vector for each message corresponding to each cluster. A preset dual-threshold mechanism is used to convert each current fuzzy membership degree into a corresponding current discrete cluster affiliation. The acquired ciphertext index corresponding to each message is distributed and stored in the cluster corresponding to the current discrete cluster affiliation, thereby realizing a structure consisting of a current plaintext cluster center layer and a current ciphertext index layer. The system uses a hierarchical storage architecture for storage. It acquires and calculates the similarity between the current user query vector and the plaintext cluster center vector corresponding to each cluster in the current plaintext cluster center layer. Combined with a preset cluster selection method, it determines the target cluster set and performs retrieval in each ciphertext index subset corresponding to the current ciphertext index layer based on the target cluster set. It then generates and returns the target ciphertext data retrieval results. This solves the problems of high time complexity, low retrieval efficiency, low data security, and high storage cost in existing technologies, improving the efficiency and security of ciphertext data retrieval, reducing the time complexity of ciphertext data retrieval, and lowering the cost requirements for data storage.

[0057] Example 2

[0058] Figure 2 This document provides a detailed flowchart of a method for retrieving encrypted data based on overlapping clustering, as described in Embodiment 2 of the present invention. This embodiment optimizes upon the above embodiment, and after retrieving data from each encrypted index subset corresponding to the current encrypted index layer based on the target cluster set, generating and returning the target encrypted data retrieval results, it further includes real-time monitoring of the size of each cluster to perform corresponding post-processing operations.

[0059] S210. Obtain non-sensitive metadata in the target original message in real time, quantify the non-sensitive metadata, and generate the current multi-dimensional non-sensitive metadata feature vector.

[0060] The non-sensitive metadata includes at least one of the following: service domain, message type, time window coarse-grainedness, and the sending party's organization.

[0061] S220. Using a preset overlapping clustering method, cluster the current multi-dimensional non-sensitive meta-feature vector to generate the current fuzzy membership degree and plaintext cluster center vector for each message corresponding to each cluster.

[0062] S230. The current fuzzy membership degree is converted into the corresponding current discrete cluster affiliation relationship through a preset dual threshold mechanism, and the ciphertext index corresponding to each message is distributed and stored in the cluster corresponding to the current discrete cluster affiliation relationship, so as to realize the storage of the layered storage architecture composed of the current plaintext clustering center layer and the current ciphertext index layer.

[0063] S240. In real time, based on the current user query vector, calculate the similarity between the vector and the plaintext cluster center vector corresponding to each cluster in the current plaintext cluster center layer, and determine the target cluster set by combining the preset cluster selection method. Then, based on the target cluster set, perform retrieval in each ciphertext index subset corresponding to the current ciphertext index layer, generate and return the target ciphertext data retrieval results.

[0064] S250. Monitor the size of each cluster in real time. When the number of messages in any cluster exceeds the preset message number threshold, the cluster will trigger a cluster split operation.

[0065] In traditional clustering schemes, the number of clusters C is predetermined. However, in a data transmission scenario, data grows dynamically over time. A fixed C would cause the number of messages within each cluster to continuously expand, ultimately degrading the second-stage retrieval to a full traversal. To address this, a three-level adaptive maintenance mechanism was designed. The message count threshold can be a pre-set threshold.

[0066] Optionally, the real-time monitoring of the size of each cluster, and triggering a cluster split operation when the number of messages in any cluster exceeds a preset message number threshold, includes: real-time monitoring of the size of each cluster, marking the cluster as split when the number of messages in any cluster exceeds the preset message number threshold; re-executing the overlapping clustering method on the multi-dimensional non-sensitive meta-feature vectors of all messages corresponding to the cluster, and generating each target cluster sub-cluster; wherein the number of target cluster sub-clusters is greater than or equal to 2; calculating the plaintext cluster center vectors corresponding to each target cluster sub-cluster, and generating a new membership matrix; migrating the ciphertext index of the original cluster to the corresponding target cluster sub-cluster according to the new membership matrix, replacing the original plaintext cluster center vector with the plaintext cluster center vectors corresponding to each target cluster sub-cluster, and updating the global membership matrix.

[0067] In this embodiment, the system monitors the number of messages N for each cluster in real time. When any cluster (e.g., cluster j) Exceeding the preset message count threshold (For example, When there are 10,000 records, the cluster is marked as split; triggering the split operation of the cluster.

[0068] Specifically, the cluster is locked and marked as "splitting," rejecting new write requests but allowing reads. Then, internal re-clustering is performed, re-executing fuzzy C-means clustering on the multi-dimensional non-sensitive meta-feature vectors of all messages within the cluster. The number of target sub-clusters, C', can be set to 2 or 3. Based on the re-clustering results, C' sub-clusters are generated, and their respective plaintext cluster center vectors are calculated. The new membership matrix is ​​then updated. Further, based on the membership degrees corresponding to the new membership matrix, the ciphertext index of the original cluster is migrated to the corresponding target cluster sub-cluster, and the original plaintext cluster center vector is replaced with the plaintext cluster center vector corresponding to each target cluster sub-cluster. The migrated portion is then deleted from the original cluster's message index, and the global membership matrix is ​​updated.

[0069] The advantages of this setup are: by triggering cluster splitting operations through clustering clusters, the accuracy of plaintext cluster center vectors and fuzzy membership degrees can be improved, thereby achieving faster and more accurate retrieval of ciphertext data, enhancing the security of ciphertext data, and reducing the memory usage of data storage.

[0070] S260. When a new message causes the plaintext cluster center vector to drift, an incremental update operation is triggered.

[0071] Optionally, the step of triggering an incremental update operation when a new message causes a center shift in the plaintext cluster center vector includes: when a new message is detected and causes a center shift in the plaintext cluster center vector, calculating the membership vector of the new message by using the multi-dimensional non-sensitive meta-feature vector corresponding to the new message and the plaintext cluster center vector corresponding to each cluster; allocating the new message to at least one target cluster using the dual threshold mechanism; and for each assigned target cluster, using the formula... The plaintext cluster center vector is incrementally updated based on the characteristics of the newly added messages; wherein, This represents the updated plaintext cluster center vector corresponding to cluster j; This represents the plaintext cluster center vector before the update corresponding to cluster j; The learning rate is the step size used to control the center's updates, satisfying the following conditions: ; Indicates a new message The corresponding multi-dimensional insensitive meta-feature vector; This represents the fuzzy membership degree of the newly added message to cluster j.

[0072] In this embodiment, when a new message is added to the system and causes a center shift in the plaintext cluster center vector, there is no need to rebuild the full index; instead, an incremental update is performed. Specifically, the multi-dimensional non-sensitive meta-feature vector corresponding to the new message is calculated, and its membership vector is calculated by comparing it with the plaintext cluster center vectors of all existing clusters to obtain the membership vector of the new message.

[0073] Furthermore, a dual-threshold mechanism is used to assign new packets to one or more target clusters. For each assigned target cluster, the formula is used... The plaintext cluster center vector is incrementally updated based on the characteristics of the new message, that is, the center vector is fine-tuned according to the characteristics of the new message.

[0074] The advantage of this setup is that by performing incremental update operations on new messages, incremental updates can quickly respond to data changes with minimal overhead.

[0075] S270. According to the preset adaptive maintenance cycle, the global re-clustering operation is triggered periodically to maintain the performance of the encrypted data retrieval system.

[0076] Optionally, the step of periodically triggering global re-clustering operations according to a preset adaptive maintenance cycle to maintain the performance of the encrypted data retrieval system includes: reading the latest multi-dimensional non-sensitive element feature vectors of the current full data according to the preset adaptive maintenance cycle; re-triggering the execution of the complete global re-clustering operation based on each latest multi-dimensional non-sensitive element feature vector to generate new fuzzy membership degrees and plaintext cluster center vectors corresponding to each cluster; wherein, during the execution of global re-clustering, the original clustering structure continues to be used to serve users' online query vector retrieval requests; after the global re-clustering is completed, the new clustering structure is atomically switched to the online service structure, and the old clustering structure is discarded.

[0077] In this embodiment, the cumulative incremental updates over a long period may lead to cluster center drift and suboptimal overall structure. Therefore, the system will periodically (e.g., weekly) trigger a global re-clustering in the background.

[0078] Specifically, according to the adaptive maintenance cycle, the background task reads the latest multi-dimensional non-sensitive meta-feature vectors of the current full dataset. Based on these latest multi-dimensional non-sensitive meta-feature vectors, it re-triggers the complete global mean re-clustering operation, generating new fuzzy membership degrees and plaintext cluster center vectors for each cluster. Furthermore, during the re-clustering process, the system continues to use the old clustering structure to serve online retrieval requests, ensuring service continuity. After re-clustering is complete, the new clustering structure is atomically switched to the online service structure, and the old structure is discarded.

[0079] Accordingly, through the three-tiered mechanism of cluster splitting, incremental update, and adaptive maintenance cycle, the size of a single cluster is always controlled within a certain range, regardless of how much data grows. Within this range, the time complexity of the second-stage retrieval is always maintained at O(n). It is a bounded constant, independent of the total data volume n.

[0080] The technical solution of this invention involves acquiring non-sensitive metadata from the target original message in real time, quantizing the non-sensitive metadata, and generating a current multi-dimensional non-sensitive metadata feature vector. An overlapping clustering method is used to cluster the current multi-dimensional non-sensitive metadata feature vector, generating a current fuzzy membership degree and plaintext cluster center vector for each message corresponding to each cluster. A dual-threshold mechanism is used to convert each current fuzzy membership degree into a corresponding current discrete cluster affiliation. The acquired ciphertext index corresponding to each message is distributed and stored in the cluster corresponding to the current discrete cluster affiliation, thereby achieving a hierarchical storage architecture consisting of a current plaintext cluster center layer and a current ciphertext index layer. The solution also involves real-time acquisition and processing based on the current... The system calculates the similarity between the user query vector and the plaintext cluster center vector corresponding to each cluster in the current plaintext cluster center layer. Combined with a cluster selection method, it determines the target cluster set and performs searches within the corresponding ciphertext index subsets of the current ciphertext index layer based on this target cluster set. The system then generates and returns the target ciphertext data retrieval results. This approach reduces retrieval complexity, enables flexible multi-dimensional query support, reduces data storage overhead, and improves data security. Through a three-level adaptive mechanism—cluster splitting, incremental update, and adaptive maintenance cycle—the system can readily handle continuous data growth, ensuring that the size of each cluster remains controlled and avoiding the problem of retrieval performance degradation over time, thus achieving high system scalability.

[0081] A specific example:

[0082] Suppose the system receives the target raw message. First, it parses the header metadata of each message. For example, the message... The metadata includes: "Service Domain = Command and Control", "Message Type = Instruction", "Timestamp = 20260509 10:00:00", and "Sending Unit = A Certain Unit". The preprocessing module vectorizes this information into... One-hot encoding is used for "business domain" and "message type". "Time stamp" is assigned to the "19th week of 2026" time bucket and encoded.

[0083] Furthermore, assume the preset number of clusters C is 20. The clustering module reads the feature vectors X of all n messages. It then initializes 20 cluster centers. The initial settings can be randomly selected. The fuzziness index is set to 2, and the iteration threshold is... The following loop can be executed: Calculate the i-th message under the fuzzy index p according to the membership update formula. For the fuzzy membership degree of cluster j Update the plaintext cluster center vector of cluster j. Furthermore, the plaintext cluster center vector and current fuzzy membership degree corresponding to each cluster can be calculated for each message, and the objective function J can be calculated accordingly. If the change in J is less than... Stop the iteration. After the iteration converges, the fuzzy membership matrix is ​​obtained. and 20 plaintext cluster center vectors, where, It represents the set of n rows and 20 columns of real number matrices.

[0084] Accordingly, when building a hierarchical index, two thresholds can be set, with the higher threshold parameter being... and low threshold parameter are For messages Iterate through its fuzzy membership degrees across 20 clusters. If the fuzzy membership degree of one of the clusters is greater than or equal to 0.6, the message will be sent. The cluster number and its ciphertext index Encrypt(i) are forcibly written to the ciphertext index layer of cluster j. If the fuzzy membership degree of the cluster is greater than 0.2 and less than 0.6, based on the current system storage redundancy (e.g., a target redundancy of 2.5 times), the clusters with higher membership degrees are selected for writing. If the fuzzy membership degree of the cluster is less than 0.2, it is not written. Simultaneously, the calculated 20 plaintext cluster center vectors are stored in plaintext form in a cache or database for retrieval.

[0085] Furthermore, the retrieval execution process involves user query input, such as a user submitting the query "Business Domain = Command and Control, Message Type = Instruction, Time Window Coarse-grained = Week 19 of 2026". The first stage involves plaintext spatial cluster localization. The query conversion module converts the current user query instruction into a query vector q; it calculates the cosine similarity between q and all 20 plaintext cluster centers in the system, obtaining a similarity list [0.95, 0.12, 0.88, …, 0.05]; then, similarity filtering can be performed, setting... Clusters with a similarity score below 0.3 are filtered out, leaving 5 candidate clusters. For these 5 clusters, the current number of packets is queried. Let α equal 0.5, then calculate the expected return for each cluster. For example: Cluster 1: , The expected return is 84.9. After sorting, the three clusters with the highest returns (K=3) are selected as target clusters, let's say cluster 1, cluster 5, and cluster 9. Further, a ciphertext search is performed within the target clusters; that is, the system reads all ciphertext indexes of clusters 1, 5, and 9 from ciphertext storage. Since the total number of packets in these three clusters is 13,000, exceeding the threshold M=10,000, the system constructs a temporary hash table in memory to quickly match the 13,000 indexes; after matching, it returns a list of UUIDs of all matching packets to the user.

[0086] Correspondingly, adaptive cluster splitting is also required. How should the system adaptively split when a cluster becomes too large? We can assume the trigger condition is: the system monitoring thread detects that the number of packets in cluster 1 has reached 12,000, exceeding the preset threshold of 10,000. The specific execution flow is as follows: cluster 1 is marked as "splitting," and newly written packets are temporarily routed to a buffer; a background thread extracts the latest feature vectors of all packets within cluster 1, performs fuzzy mean clustering, and sets the target number of sub-clusters to 2. Assume a new center is generated. and Based on the new clustering results, each message in cluster 1 is reassigned to either sub-cluster 1 or sub-cluster 2; its ciphertext index is deleted from the storage space of cluster 1 and written to the storage spaces of sub-cluster 1 and sub-cluster 2 respectively. In the global cluster center list, use... and Replace the original The membership matrix is ​​updated, and new packets in the buffer are incrementally added to the new sub-cluster. After these operations, sub-cluster 1 and sub-cluster 2 are marked as available, and service is restored. Through this process, the size of a single cluster returns to its original size. The search efficiency was then restored.

[0087] Example 3

[0088] Figure 3 This is a schematic diagram of a encrypted data retrieval device based on overlapping clustering provided in Embodiment 3 of the present invention. The encrypted data retrieval device based on overlapping clustering provided in this embodiment can be implemented by software and / or hardware, and can be configured in a terminal device or server to implement the encrypted data retrieval method based on overlapping clustering in this embodiment of the present invention. Figure 3 As shown, the device includes: a current multi-dimensional non-sensitive meta-feature vector generation module 310, a current fuzzy membership degree and plaintext cluster center vector generation module 320, a hierarchical storage module 330, and a target ciphertext data retrieval result generation and feedback module 340.

[0089] The current multi-dimensional non-sensitive element feature vector generation module 310 is used to acquire non-sensitive element information in the target original message in real time, quantify the non-sensitive element information, and generate the current multi-dimensional non-sensitive element feature vector.

[0090] The non-sensitive metadata includes at least one of the following: service domain, message type, time window coarse-grainedness, and the sending party's organization;

[0091] The current fuzzy membership degree and plaintext cluster center vector generation module 320 is used to cluster the current multi-dimensional non-sensitive element feature vector through a preset overlapping clustering method, and generate the current fuzzy membership degree and plaintext cluster center vector corresponding to each cluster for each message.

[0092] The hierarchical storage module 330 is used to convert each of the current fuzzy membership degrees into the corresponding current discrete cluster affiliation relationship through a preset dual threshold mechanism, and distribute and store the ciphertext index corresponding to each message to the cluster corresponding to the current discrete cluster affiliation relationship, so as to realize the hierarchical storage architecture consisting of the current plaintext clustering center layer and the current ciphertext index layer for storage.

[0093] The target encrypted data retrieval result generation and feedback module 340 is used to acquire and calculate the similarity between the target encrypted data retrieval result and the plaintext cluster center vector corresponding to each cluster in the current plaintext cluster center layer, based on the current user query vector, and determine the target cluster set by combining the preset cluster selection method. Then, it performs retrieval in each encrypted index subset corresponding to the current encrypted index layer based on the target cluster set, and generates and feeds back the target encrypted data retrieval result.

[0094] The technical solution of this invention involves acquiring non-sensitive metadata from the target original message in real time, quantizing the non-sensitive metadata, and generating a current multi-dimensional non-sensitive metadata feature vector. A preset overlapping clustering method is used to cluster the current multi-dimensional non-sensitive metadata feature vector, generating a current fuzzy membership degree and plaintext cluster center vector for each message corresponding to each cluster. A preset dual-threshold mechanism is used to convert each current fuzzy membership degree into a corresponding current discrete cluster affiliation. The acquired ciphertext index corresponding to each message is distributed and stored in the cluster corresponding to the current discrete cluster affiliation, thereby realizing a structure consisting of a current plaintext cluster center layer and a current ciphertext index layer. The system uses a hierarchical storage architecture for storage. It acquires and calculates the similarity between the current user query vector and the plaintext cluster center vector corresponding to each cluster in the current plaintext cluster center layer. Combined with a preset cluster selection method, it determines the target cluster set and performs retrieval in each ciphertext index subset corresponding to the current ciphertext index layer based on the target cluster set. It then generates and returns the target ciphertext data retrieval results. This solves the problems of high time complexity, low retrieval efficiency, low data security, and high storage cost in existing technologies, improving the efficiency and security of ciphertext data retrieval, reducing the time complexity of ciphertext data retrieval, and lowering the cost requirements for data storage.

[0095] Based on the above embodiments, a post-processing module is further included. After the retrieval is performed in each ciphertext index subset corresponding to the current ciphertext index layer according to the target cluster set, and the target ciphertext data retrieval result is generated and fed back, the module may specifically include: a cluster splitting operation unit, which may be used to: monitor the size of each cluster in real time, and trigger a cluster splitting operation when the number of messages in any cluster exceeds a preset message number threshold; an incremental update operation unit, which may be used to: trigger an incremental update operation when the addition of a message causes the plaintext cluster center vector to drift; and a global re-clustering operation unit, which may be used to: periodically trigger a global re-clustering operation according to a preset adaptive maintenance cycle, so as to maintain the performance of the ciphertext data retrieval system.

[0096] Based on the above embodiments, the current fuzzy membership degree and plaintext cluster center vector generation module 320 can be specifically used to: cluster the current multi-dimensional non-sensitive meta-feature vector using a preset overlapping clustering method, and then use the formula... and It is possible to calculate the plaintext cluster center vector and the current fuzzy membership degree for each message corresponding to each cluster; where, p is a fuzziness index used to control the degree of fuzziness in the clustering results, with a value greater than 1. This indicates that under the fuzzy index p, the i-th message The current fuzzy membership degree of cluster j; Indicates message The current multi-dimensional non-sensitive meta-feature vector; n represents the total number of packets in the target original message; Let represent the plaintext cluster center vector of cluster j; C represents the number of clusters. Let represent the Euclidean norm; where the objective function corresponding to the overlapping clustering method is . .

[0097] Based on the above embodiments, the hierarchical storage module 330 can be specifically used to: obtain the high threshold parameter and low threshold parameter corresponding to the dual threshold mechanism; sequentially determine the current fuzzy membership degree corresponding to each cluster of each message; if the current fuzzy membership degree is greater than or equal to the high threshold parameter, then determine that the message belongs to the target cluster; if the current fuzzy membership degree is greater than or equal to the low threshold parameter and less than the high threshold parameter, then dynamically determine whether it belongs to the target cluster based on the current system storage capacity and retrieval performance target; if the current fuzzy membership degree is less than the low threshold parameter, then determine whether the message belongs to the target cluster.

[0098] Based on the above embodiments, the cluster splitting operation unit can be specifically used to: monitor the size of each cluster in real time, and mark the cluster as split when the number of messages in any cluster exceeds a preset message number threshold; re-execute the overlapping clustering method on the multi-dimensional non-sensitive meta-feature vectors of all messages corresponding to the cluster, and generate each target cluster sub-cluster; wherein the number of the target cluster sub-clusters is greater than or equal to 2; calculate the plaintext cluster center vector corresponding to each target cluster sub-cluster, and generate a new membership matrix; migrate the ciphertext index of the original cluster to the corresponding target cluster sub-cluster according to the new membership matrix, replace the original plaintext cluster center vector with the plaintext cluster center vector corresponding to each target cluster sub-cluster, and update the global membership matrix.

[0099] Based on the above embodiments, the incremental update operation unit can be specifically used to: when a new message is detected and the new message causes a center drift in the plaintext cluster center vector, calculate the membership vector of the new message by using the multi-dimensional non-sensitive meta-feature vector corresponding to the new message and the plaintext cluster center vector corresponding to each cluster; allocate the new message to at least one target cluster using the dual threshold mechanism; and for each allocated target cluster, use the formula... The plaintext cluster center vector is incrementally updated based on the characteristics of the newly added messages; wherein, This represents the updated plaintext cluster center vector corresponding to cluster j; This represents the plaintext cluster center vector before the update corresponding to cluster j; The learning rate is the step size used to control the center's updates, satisfying the following conditions: ; Indicates a new message The corresponding multi-dimensional insensitive meta-feature vector; This represents the fuzzy membership degree of the newly added message to cluster j.

[0100] Based on the above embodiments, the global re-clustering operation unit can be specifically used to: read the latest multi-dimensional non-sensitive element feature vectors of the current full message according to a preset adaptive maintenance cycle; re-trigger the execution of the complete global re-clustering operation based on the latest multi-dimensional non-sensitive element feature vectors, and generate new fuzzy membership degrees and plaintext cluster center vectors corresponding to each cluster; wherein, during the execution of global re-clustering, the original clustering structure continues to be used to serve the user's online query vector retrieval request; after the global re-clustering is completed, the new clustering structure is atomically switched to the online service structure, and the old clustering structure is discarded.

[0101] The encrypted data retrieval device based on overlapping clustering provided in this embodiment of the invention can execute the encrypted data retrieval method based on overlapping clustering provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0102] Example 4

[0103] Figure 4 A schematic diagram of an electronic device 10, which can be used to implement Embodiment 4 of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0104] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0105] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0106] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as encrypted data retrieval methods based on overlapping clustering.

[0107] In some embodiments, the encrypted data retrieval method based on overlapping clustering can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the encrypted data retrieval method based on overlapping clustering described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the encrypted data retrieval method based on overlapping clustering by any other suitable means (e.g., by means of firmware).

[0108] The method includes: acquiring non-sensitive metadata in the target original message in real time, quantifying the non-sensitive metadata, and generating a current multi-dimensional non-sensitive metadata feature vector; wherein the non-sensitive metadata includes at least one of the following: service domain, message type, coarse-grained time window, and the sending unit; clustering the current multi-dimensional non-sensitive metadata feature vector using a preset overlapping clustering method to generate a current fuzzy membership degree and plaintext cluster center vector for each message corresponding to each cluster; and converting each current fuzzy membership degree into a corresponding current discrete cluster affiliation using a preset dual-threshold mechanism. The acquired ciphertext indexes corresponding to each message are distributed and stored in clusters corresponding to the current discrete cluster affiliation, thereby realizing a hierarchical storage architecture consisting of the current plaintext clustering center layer and the current ciphertext index layer. In real time, based on the current user query vector, the similarity between the current plaintext clustering center layer and the plaintext clustering center vector corresponding to each cluster is calculated. Combined with a preset cluster selection method, the target cluster set is determined. Based on the target cluster set, a search is performed in each ciphertext index subset corresponding to the current ciphertext index layer, generating and feeding back the target ciphertext data retrieval results.

[0109] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0110] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0111] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0112] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0113] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0114] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0115] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0116] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

[0117] Example 5

[0118] Embodiment 5 of the present invention also provides a computer-readable storage medium, wherein the computer-readable instructions, when executed by a computer processor, are used to perform a method for retrieving encrypted data based on overlapping clustering. The method includes: acquiring non-sensitive metadata in the original target message in real time, quantizing the non-sensitive metadata, and generating a current multi-dimensional non-sensitive metadata feature vector; wherein the non-sensitive metadata includes at least one of the following: service domain, message type, coarse-grained time window, and the sending unit; clustering the current multi-dimensional non-sensitive metadata feature vector using a preset overlapping clustering method, generating a current fuzzy membership degree and plaintext cluster center vector for each message corresponding to each cluster; and using a preset... The dual-threshold mechanism converts each current fuzzy membership degree into a corresponding current discrete cluster affiliation relationship, and distributes and stores the obtained ciphertext indexes corresponding to each message into clusters corresponding to the current discrete cluster affiliation relationship, thereby realizing a hierarchical storage architecture consisting of the current plaintext clustering center layer and the current ciphertext index layer. In real time, based on the current user query vector, similarity is calculated between the current plaintext clustering center layer and the plaintext clustering center vector corresponding to each cluster. Combined with a preset cluster selection method, a target cluster set is determined. Based on the target cluster set, a search is performed in each ciphertext index subset corresponding to the current ciphertext index layer, generating and feeding back the target ciphertext data retrieval results.

[0119] Of course, the computer-executable instructions provided in the embodiments of the present invention, which include a computer-readable storage medium, are not limited to the method operations described above, but can also perform related operations in the encrypted data retrieval method based on overlapping clustering provided in any embodiment of the present invention.

[0120] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0121] It is worth noting that in the embodiments of the above-mentioned encrypted data retrieval method based on overlapping clustering, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0122] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for retrieving encrypted data based on overlapping clustering, characterized in that, include: The non-sensitive metadata in the target original message is acquired in real time, and the non-sensitive metadata is quantified to generate the current multi-dimensional non-sensitive metadata feature vector. The non-sensitive metadata includes at least one of the following: service domain, message type, time window coarse-grainedness, and the sending party's organization; The current multi-dimensional non-sensitive meta-feature vector is clustered using a preset overlapping clustering method to generate the current fuzzy membership degree and plaintext cluster center vector for each message corresponding to each cluster. The current fuzzy membership degree is converted into the corresponding current discrete cluster affiliation relationship through a preset dual threshold mechanism, and the ciphertext index corresponding to each message is distributed and stored in the cluster corresponding to the current discrete cluster affiliation relationship, so as to realize the storage of the layered storage architecture consisting of the current plaintext clustering center layer and the current ciphertext index layer. In real time, based on the current user query vector, the similarity between the vector and the plaintext cluster center vector corresponding to each cluster in the current plaintext cluster center layer is calculated. Combined with a preset cluster selection method, the target cluster set is determined. Based on the target cluster set, the retrieval is performed in each ciphertext index subset corresponding to the current ciphertext index layer, and the target ciphertext data retrieval result is generated and fed back.

2. The method according to claim 1, characterized in that, After performing a search in each ciphertext index subset corresponding to the current ciphertext index layer based on the target cluster set, generating and returning the target ciphertext data retrieval results, the method further includes: The size of each cluster is monitored in real time. When the number of messages in any cluster exceeds the preset message count threshold, the cluster will trigger a cluster split operation. When a new message causes the plaintext cluster center vector to shift, an incremental update operation is triggered. In this system, a global re-clustering operation is periodically triggered according to a preset adaptive maintenance cycle in order to maintain the performance of the encrypted data retrieval system.

3. The method according to claim 1, characterized in that, The step of clustering the current multi-dimensional non-sensitive meta-feature vectors using a preset overlapping clustering method to generate the current fuzzy membership degree and plaintext cluster center vector for each message corresponding to each cluster includes: The current multi-dimensional insensitive meta-feature vectors are clustered using a preset overlapping clustering method, and the clustering is performed using the formula. and It is possible to calculate the plaintext cluster center vector and the current fuzzy membership degree corresponding to each cluster for each message; in, p is a fuzziness index used to control the degree of fuzziness in the clustering results, with a value greater than 1. This indicates that under the fuzzy index p, the i-th message The current fuzzy membership degree of cluster j; Indicates message The current multi-dimensional non-sensitive meta-feature vector; n represents the total number of packets in the target original message; Let represent the plaintext cluster center vector of cluster j; C represents the number of clusters. Denotes the Euclidean norm; Wherein, the objective function corresponding to the overlapping clustering method is .

4. The method according to claim 3, characterized in that, The step of converting each current fuzzy membership degree into a corresponding current discrete cluster affiliation relationship through a preset dual-threshold mechanism includes: Obtain the high threshold parameter and low threshold parameter corresponding to the dual threshold mechanism; The current fuzzy membership degree corresponding to each cluster of each message is determined sequentially. If the current fuzzy membership degree is greater than or equal to the high threshold parameter, the message is determined to belong to the target cluster. If the current fuzzy membership degree is greater than or equal to the low threshold parameter and less than the high threshold parameter, then it is dynamically determined whether to belong to the target cluster based on the current system storage capacity and retrieval performance target. If the current fuzzy membership degree is less than the low threshold parameter, then it is determined that the message does not belong to the target cluster.

5. The method according to claim 2, characterized in that, The real-time monitoring of the size of each cluster, when the number of messages in any cluster exceeds a preset message count threshold, triggers a cluster split operation, including: The size of each cluster is monitored in real time. When the number of messages in any cluster exceeds the preset message count threshold, the cluster is marked as split. The overlapping clustering method is re-executed on the multi-dimensional non-sensitive meta-feature vectors of all messages corresponding to the cluster, and each target cluster sub-cluster is generated; wherein the number of the target cluster sub-clusters is greater than or equal to 2; The plaintext cluster center vectors corresponding to each target cluster sub-cluster are calculated, and a new membership matrix is ​​generated. Based on the new membership matrix, the ciphertext index of the original cluster is migrated to the corresponding target cluster, and the plaintext cluster center vector corresponding to each target cluster is used to replace the original plaintext cluster center vector, and the global membership matrix is ​​updated.

6. The method according to claim 2, characterized in that, When a new message causes a center shift in the plaintext cluster center vector, an incremental update operation is triggered, including: When a new message is detected and the new message causes the plaintext cluster center vector to shift, the membership vector of the new message is obtained by calculating the membership vector of the new message through the multi-dimensional non-sensitive meta-feature vector corresponding to the new message and the plaintext cluster center vector corresponding to each cluster. The new message is assigned to at least one target cluster using the dual threshold mechanism. For each assigned target cluster, the formula is used... The plaintext cluster center vector is incrementally updated based on the characteristics of the newly added message. in, This represents the updated plaintext cluster center vector corresponding to cluster j; This represents the plaintext cluster center vector before the update corresponding to cluster j; The learning rate is the step size used to control the center's updates, satisfying the following conditions: ; Indicates a new message The corresponding multi-dimensional insensitive meta-feature vector; This represents the fuzzy membership degree of the newly added message to cluster j.

7. The method according to claim 2, characterized in that, The step of periodically triggering a global re-clustering operation according to a preset adaptive maintenance cycle to maintain the performance of the encrypted data retrieval system includes: According to the preset adaptive maintenance cycle, read the latest multi-dimensional non-sensitive meta-feature vector of the current full message; Based on the latest multi-dimensional non-sensitive meta-feature vectors, the complete global re-clustering operation is retried to generate new fuzzy membership degrees and plaintext cluster center vectors for each cluster. During the global re-clustering process, the original clustering structure continues to serve users' online query vector retrieval requests. After the global re-clustering is completed, the new clustering structure is atomically switched to the online service structure, and the old clustering structure is discarded.

8. A encrypted data retrieval device based on overlapping clustering, characterized in that, include: The current multi-dimensional non-sensitive element feature vector generation module is used to acquire non-sensitive element information in the target original message in real time, quantify the non-sensitive element information, and generate the current multi-dimensional non-sensitive element feature vector. The non-sensitive metadata includes at least one of the following: service domain, message type, time window coarse-grainedness, and the sending party's organization; The current fuzzy membership degree and plaintext cluster center vector generation module is used to cluster the current multi-dimensional non-sensitive element feature vectors using a preset overlapping clustering method, and generate the current fuzzy membership degree and plaintext cluster center vector for each message corresponding to each cluster. The hierarchical storage module is used to convert each current fuzzy membership degree into the corresponding current discrete cluster affiliation relationship through a preset dual threshold mechanism, and distribute and store the ciphertext index corresponding to each message to the cluster corresponding to the current discrete cluster affiliation relationship, so as to realize the hierarchical storage architecture consisting of the current plaintext clustering center layer and the current ciphertext index layer for storage. The target encrypted data retrieval result generation and feedback module is used to acquire and calculate the similarity between the target encrypted data retrieval result and the plaintext cluster center vector corresponding to each cluster in the current plaintext cluster center layer, based on the current user query vector, and combine it with a preset cluster selection method to determine the target cluster set. Based on the target cluster set, it performs retrieval in each encrypted index subset corresponding to the current encrypted index layer, generates and feeds back the target encrypted data retrieval result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a method for retrieving encrypted data based on overlapping clustering as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute and implement a method for retrieving encrypted data based on overlapping clustering as described in any one of claims 1-7.

Images