A hierarchical isolation retrieval method and system for implicit privacy data

Abstract

The application discloses a kind of implicit privacy data hierarchical isolation retrieval method and system.The method comprises the following steps: step S1: unstructured data intake and atomization dimension reduction;Step S2: the structural reorganization of knowledge unit;Step S3: construct the cosine fusion vector space with metadata constraint;Step S4: dynamic password verification of fusion authorization level Max Level;Step S5: gateway reverse check and physical truncation retrieval.The application mainly solves the technical problems of overstepping, overstepping retrieval and excessive leakage of user implicit common sense data, relationship network data and core credential data caused by cross-domain large language model collaborative work;Realize the fine classification and permission control of privacy data, can carry out dynamic permission control and physical truncation filtering, accurately limit query secret level in cross-domain retrieval, effectively prevent high secret level privacy information from being reverse assembled, so as to improve the advantages of data privacy protection security.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and data privacy security technology, specifically relating to a hierarchical isolation retrieval method and system for implicit privacy data. Background Technology

[0002] In application scenarios based on large language models, especially when cross-domain collaboration is involved (such as cloud models calling local personal data), data privacy protection is a significant challenge.

[0003] Common privacy protection methods in existing technologies include: role-based access control, global API (Application Programming Interface) key authorization, sensitive word masking, named entity masking, and front-end processing such as replacing placeholders in the input. These methods can, to some extent, prevent explicit privacy information (such as ID card number, mobile phone number, and name) from being directly leaked.

[0004] However, the aforementioned existing technologies still have the following common shortcomings in practical applications: First, unstructured free text data (such as users' diaries, notes, memos, etc.) often contains implicit information of different dimensions, such as eating habits, social relationships, financial behavior, and health status, each with varying degrees of privacy sensitivity. Existing solutions based on sensitive word matching or fixed rules struggle to semantically distinguish different knowledge points within the same paragraph (e.g., "Today I took my aunt out to eat; she is lactose intolerant; I also spent over 8,000 yuan on cryptocurrency"). This mixes multiple levels of implicit privacy relationships, such as "dietary preferences" (low-secrecy level), "relatives' whereabouts" (medium-secrecy level), and "financial habits" (high-secrecy level). Traditional mechanisms are simply unable to extract and independently classify these information at the atomic level in terms of semantics and relationships, let alone assign an appropriate PrivacyLevel to each knowledge point, thus failing to achieve deep protection for specific knowledge. This results in coarse-grained privacy protection, leading to either incomplete overall desensitization (high-secrecy information is retained) or excessive masking that results in the loss of information usability.

[0005] Second, in cross-network or cross-domain AI Agent (intelligent agent) invocation scenarios, existing technologies typically only verify the identity of the caller (such as through a one-time password or session credentials) and grant a global static permission. Once the credentials are obtained or the model is subjected to a prompt word injection attack, attackers may be able to gradually probe deeper sensitive information in legitimate sessions. Existing solutions lack the ability to tightly bind fine-grained authorization boundaries to each request.

[0006] Third, in vector-based augmented generation architectures, data is typically converted into high-dimensional vectors and stored in a vector database. Even if sensitive words in the original text are anonymized or replaced, the corresponding vector fragments may still be recalled due to their high semantic similarity. Most existing technologies perform anonymization at the string level, failing to achieve hard isolation based on privacy levels at the underlying vector retrieval layer. Therefore, even if the recalled high-density vector blocks (containing the full picture of the event despite the removal of names) do not contain explicit sensitive words, they can still be pieced together by large language models through contextual reasoning to deduce the user's true privacy information, posing a significant risk of leakage.

[0007] In summary, how to perform fine-grained classification of multi-dimensional privacy in unstructured text in cross-domain large-scale model collaboration scenarios, and to achieve permission-based hard isolation at the retrieval level, is a technical problem that urgently needs to be solved. Summary of the Invention

[0008] In view of the above-mentioned deficiencies of the prior art, the first aspect of the present invention provides a hierarchical isolation retrieval method for implicit privacy data, comprising the following steps: Step S1: Unstructured data ingestion and atomized dimensionality reduction; perform paragraph segmentation and knowledge unit extraction on the unstructured text, forcibly extracting each paragraph into several discrete knowledge units; Step S2: Structured reorganization of knowledge units; the knowledge units are standardized into data entities with multi-dimensional standard structures, which at least include: subjective fact classification, subject, action, object, environmental features, emotional features, judgment confidence level, privacy level, and original paragraph source tracing block; Step S3: Construct a cosine fusion vector space with metadata constraints; integrate the subject, action, object, environmental features and emotional features of the knowledge unit into a dimensionality-reduced feature sentence, map it into a dense vector through a vectorization model, and mount and store the dense vector and its corresponding privacy level as metadata constraints on the high-dimensional vector node of the vector database HNSW (Hierarchical Navigable Small World); Step S4: Dynamic password verification based on the Max Level of the integrated authorization hierarchy; when the local gateway receives an authorized third-party query request, it performs identity and permission verification based on the tamper-proof dynamic password based on the Max Level of the integrated authorization hierarchy; the dynamic password is derived by using the time window stamp, the Max Level of the authorization hierarchy, and the random number buffer as implicit factors; Step S5: Gateway reverse verification and physical truncation retrieval; extract the maximum authorization level (Max Level) allowed by the caller from the dynamic password. The maximum authorization level (Max Level) represents the highest privacy level that the caller is allowed to access in this query; embed the maximum authorization level (Max Level) as a metadata filtering condition into the vector database query statement, physically truncate and filter high-secrecy vector fragments whose privacy level is greater than the maximum authorization level (Max Level), and return only the retrieval results that meet the permission requirements to the caller.

[0009] In the hierarchical isolation retrieval method for implicit privacy data as described above, optionally, the knowledge unit extraction in step S1 adopts a three-level fault-tolerant degradation design: The first level is direct extraction, which attempts to obtain structured data directly from the output of the semantic parsing engine; The second level is Markdown (Lightweight Markup Language) code block addressing and stripping. When the output contains Markdown code block tags, the JSON content within the code block is located and extracted using regular expressions. The third level is delimiter cutting and stripping. When the above two levels fail, the valid JSON area is forcibly extracted by locating specific square brackets or curly braces delimiters.

[0010] In the hierarchical isolation retrieval method for implicit privacy data as described above, optionally, the dimensionality reduction feature sentence in step S3 combines subject, action, object, environmental features and emotional features into continuous text through specific splicing characters. The continuous text retains the original semantic relationship while reducing the data dimensionality. The Privacy Level uses a numerical scale identifier. When modifying the level rules, the changes only need to be made by overwriting the metadata value of the corresponding node through the system API, without needing to recalculate the vector representation.

[0011] In the hierarchical isolation retrieval method for implicit privacy data as described above, optionally, the dynamic password derivation process in step S4 includes: Construct a binary message body, which includes: an 8-byte timestamp, a 4-byte unsigned integer Max Level, and a 4-byte random number buffer; The local private key is used to perform a hash calculation on the binary message body to obtain a Token (a one-time password token, i.e., the original token value obtained after hash calculation). The token is truncated and its dimensions reduced to output a 6-digit numeric string and an accompanying random signature code. The authentication status is then strongly bound to the authorization level (Max Level).

[0012] In the hierarchical isolation retrieval method for implicit privacy data as described above, optionally, the physical truncation filtering in step S5 is performed before the HNSW topological space metric or cosine similarity comparison. The underlying vector database directly uses the attached metadata constraints to filter and exclude high-density vector fragments with a privacy level greater than the authorization level (Max Level), so that the high-density vector fragments cannot enter the recall scoring pool.

[0013] In the hierarchical isolation retrieval method for implicit privacy data described above, the method may optionally be applied to cross-domain collaborative scenarios: The local client acts as the data holder, while the cloud-based third party acts as the caller. The local client grants restricted query permissions to the cloud-based third party by generating a dynamic password carrying the Max Level of authorization. The cloud-based third party initiates a natural language query carrying the dynamic password. The local gateway determines the query authorization level (Max Level) by verifying the dynamic password and performs physical truncation during the vector retrieval stage to ensure that the cloud-based third party can only obtain the de-identified conclusions within the authorized scope and cannot reverse-engineer high-secret privacy information through semantic association.

[0014] In the hierarchical isolation retrieval method for implicit privacy data as described above, optionally, in step S1, multi-level regular expressions are used to read unformatted user-stored documents locally, and paragraphs are segmented by blank lines. The semantic parsing engine adopts a locally deployed small-parameter model, setting the core parameter temperature (temperature parameter, which is the core parameter used to control randomness when generating large language models; the higher the temperature value, the more random and diverse the output; the lower the value (approaching 0), the more the model tends to select the output with the highest probability, and the more certain and conservative the result) to approach 0.1 to fix the determinism, and limiting the redundancy of ultra-high word boundary as inference slots; In step S2, the extracted knowledge unit is instantiated into a KnowledgeUnit data model. The privacy level is based on a digital scale, where 1 represents public information and 10 represents top secret credentials. In step S3, PersistentClient is used to perform hard disk data persistence processing and supports lossless erasure and rewriting of metadata values ​​mounted on vector nodes.

[0015] To achieve the above objectives, a second aspect of the present invention provides a hierarchical isolation retrieval system for implicit privacy data, wherein implementing the hierarchical isolation retrieval method for implicit privacy data as described in any of the embodiments of the first aspect includes: The inbound pipeline extraction module is configured to read unformatted user-retained local documents using multi-level regular expressions, perform paragraph segmentation and knowledge unit extraction based on blank lines, forcibly extract each paragraph into several discrete knowledge units, and use a three-level fault tolerance and degradation strategy to process the output characters. The knowledge unit structuring module is configured to standardize the knowledge unit into a data entity with a multi-dimensional standard structure. The multi-dimensional standard structure includes at least: subjective fact classification, subject, action, object, environmental features, emotional features, judgment confidence level, privacy level, and original paragraph tracing block. The multidimensional vector storage engine module is configured to integrate the subject, action, object, environmental features and emotional features of the knowledge unit into a dimensionality-reduced feature sentence, map it into a dense vector through a feature embedding engine, and mount and store the dense vector and its corresponding privacy level as metadata constraints on the HNSW high-dimensional vector node of the vector database. The secure access modifier password gateway module is configured to perform identity and permission verification based on a tamper-proof dynamic password with a maximum authorization level when an authorized third-party query request is received. The dynamic password is derived by using a time window stamp, the maximum authorization level, and a random number buffer as implicit factors, and then assembling a structure and performing SHA256 hash calculation. The Restricted RAG (Retrieval-Augmented Generation) question-answering retrieval generation core module is configured to extract the maximum authorization level (Max Level) allowed by the caller from the dynamic password, embed the maximum authorization level (Max Level) as a metadata filtering condition into the vector database query statement, physically truncate and filter high-secret vector fragments whose privacy level (PrivacyLevel) is greater than the maximum authorization level (Max Level), and return only the retrieval results that meet the permission requirements to the caller.

[0016] To achieve the above objectives, a third aspect of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when running the program, implements the hierarchical isolation retrieval method for implicit privacy data as described in any of the embodiments of the first aspect above.

[0017] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions or a computer program, which, when executed by a processor, implements the hierarchical isolation retrieval method for implicit privacy data as described in any of the first aspects above.

[0018] The hierarchical isolation and retrieval method and system for implicit privacy data provided by this invention have the following beneficial effects: (1) Achieve atomic-level deconstruction and fine-grained classification of implicit privacy data: By using a local large model, multi-dimensional knowledge in unstructured text is extracted into knowledge units with privacy level, which solves the problem that multi-level implicit privacy in long text diaries and random data is difficult to deconstruct, and realizes atomic control of privacy information.

[0019] (2) Combining dynamic access control with physical truncation filtering: By incorporating the authorization level Max Level and random number factor into the tamper-proof dynamic password, the query access level is locked cryptographically in a single dialogue session; at the same time, hard isolation filtering is performed based on privacy level metadata at the vector retrieval layer to physically prevent the recall of data that exceeds the level.

[0020] (3) In cross-domain retrieval, the query security level is precisely restricted, which effectively prevents high-security privacy information from being reverse-engineered and significantly improves the security of data privacy protection.

[0021] (4) It supports dynamic adjustment of the privacy level without re-vectorization, reducing system computing power and operation and maintenance costs; and it does not depend on a specific platform carrier, and has good versatility and deployability.

[0022] In summary, this application primarily addresses the technical challenges of unauthorized and excessive disclosure of users' implicit common sense data, relationship network data, and core credential data during cross-domain large language model collaborative work (such as cloud applications accessing local personal data). These challenges arise from the lack of fine-grained dynamic authorization mechanisms and the difficulty in dissecting unstructured, subjective data for privacy purposes. Furthermore, this application achieves fine-grained classification and access control of privacy data, enabling dynamic access control and physical truncation filtering. It precisely limits the confidentiality level of queries in cross-domain searches, effectively preventing the reverse engineering of highly confidential information, thereby enhancing data privacy protection and security. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the concept, specific structure and technical effects of the present invention will be further explained below in conjunction with the accompanying drawings, so as to fully understand the purpose, features and effects of the present invention.

[0024] Figure 1 This is a flowchart illustrating an embodiment of a hierarchical isolation and retrieval method for implicit privacy data provided by the present invention. Detailed Implementation

[0025] In this document, to make the technical means, inventive features, achieved objectives and effects of the invention readily understandable, the invention is further described below with reference to specific illustrations. However, the invention is not limited to the embodiments described below.

[0026] In traditional large-scale model applications, access mechanisms based on RBAC (Role-Based Access Control) or global single API Key authorization, as well as existing privacy protection schemes such as privacy content masking and segmentation reasoning, have the problem of being unable to perform atomic-level extraction, hierarchical classification, dynamic permission verification, and physical truncation retrieval of implicit privacy data in unstructured text. Existing technologies heavily rely on sensitive word replacement processing, failing to achieve atomic-level deconstruction of unstructured content with mixed multi-level implicit relationship privacy at the semantic and relational dimensions. This results in the inability to independently identify and classify privacy elements of different security levels, such as dietary preferences, relatives' whereabouts, and financial habits. Furthermore, cross-network agent calls lack a cryptographic mechanism for dynamically transmitting the maximum level of access authorization from the client to the data end, limiting authentication boundaries to static authentication and failing to dynamically constrain access depth within a single conversation. In addition, at the vector database retrieval level, existing solutions only perform desensitization mapping at the string level, without implementing physical isolation before underlying vector retrieval. This allows high-security data blocks to still be hit by cosine similarity retrieval, resulting in a persistent risk of privacy leakage. Consequently, the system is significantly constrained in key performance indicators such as data security, privacy protection capabilities, and cross-domain collaborative reliability, failing to meet the deep privacy protection needs of complex personal data structure scenarios.

[0027] For example, in a scenario where a user stores unstructured diaries using a personal knowledge base application, the diary content might include phrases like, "Today I took my aunt out to eat, but she's lactose intolerant; I also spent over 8,000 yuan on cryptocurrency." Current technology cannot atomically extract and independently classify the information related to "lactose intolerance" (low-level confidentiality), "aunt's whereabouts" (medium-level confidentiality), and "cryptocurrency purchase" (high-level confidentiality); it can only perform overall desensitization. When a third-party AI agent calls local data via API, the black-box nature of the cloud-based large model makes it vulnerable to prompt injection attacks. Existing authentication mechanisms cannot dynamically limit access depth, potentially leading to the leakage of high-level confidential content. Furthermore, during vector database retrieval, even if sensitive words are replaced, the overall event features may still be recalled by cosine similarity through the diffusion effect of the model's attention mechanism, causing high-level data blocks to be extracted, creating a risk of privacy information patchwork. Specifically, in this scenario, the system cannot achieve deep protection of specific knowledge, significantly increasing the risk of user data exposure.

[0028] If the aforementioned issues are not addressed, implicit privacy data will continue to face the technical risks of reverse engineering and piecing together during cross-domain large-scale model agent collaboration. The system's inability to physically truncate data at the vector retrieval level means that high-security data blocks will always be susceptible to recall, leading to a continuous accumulation of privacy leak risks and ultimately impacting the security of the entire data ecosystem and the foundation of user trust. As a preferred implementation method, the persistent existence of this problem will hinder secure applications in scenarios with complex personal data structures, necessitating a mechanism to achieve atomic processing, dynamic permission binding, and underlying physical isolation.

[0029] In response, this application proposes a collaborative innovation method and system for large model entity atomic extraction and classification, hierarchical dynamic encryption passwords, and vector base metadata interception.

[0030] First, unstructured text is extracted using a local inference model and transformed into a multi-dimensional standard structure including privacy levels through rules, and metadata constraints are constructed accordingly. Second, the traditional TOTP (Time-Based One-Time Password) algorithm, which is limited by identity verification, is fundamentally modified. It innovatively integrates the authorization level (Max Level) and random number response anti-collision slots (Nonce) as implicit encryption factors into dynamic password generation. Finally, when a caller requests this password, the local gateway decomposes the allowed access level (i.e., the authorization level (Max Level)) through reverse verification. At the vector database stage, the authorization level (Max Level) is directly used as a metadata filtering condition for HNSW metadata-level filtering and truncation. This completely eliminates the possibility of recalling non-compliant, overstepping data from both physical and algorithmic levels, thereby achieving highly sensitive desensitization and restricted supply of conclusions exposed in the cloud.

[0031] As described above, this application first proposes a hierarchical isolation retrieval method for implicit privacy data, such as... Figure 1 As shown, the specific steps may include the following: Step S1: Unstructured data ingestion and atomized dimensionality reduction refers to the process of importing raw, unformatted text data (such as diaries, notes, etc.) into the system and decomposing it into the smallest, independent knowledge units with clear semantics through a series of processes. This involves segmenting the unstructured text into paragraphs and extracting knowledge units, forcibly extracting each paragraph into several discrete knowledge units.

[0032] It should be noted that the term "ingestion" is similar in meaning to data collection and data access, but with a different emphasis. In this application, ingestion specifically refers to the pre-entry stage of absorbing external heterogeneous source data into the pipeline, emphasizing the directional semantics of ingestion from the outside in. In this step, the program in the pipeline extraction module segments long texts (such as diaries and memos) by blank lines, and forcibly extracts the segmented text into several discrete KnowledgeUnits using a local natural language model (such as the Qwen parameter-level model) with preset strict JSON example prompts. A KnowledgeUnit is the smallest piece of information with independent semantics extracted from unstructured text. Each KnowledgeUnit can represent a specific fact, event, or opinion, and in this application, it is assigned specific attributes and a Privacy Level. This logic utilizes a three-level fault tolerance and degradation strategy (such as direct extraction, Markdown code block addressing stripping, and delimiter cutting stripping, which will be explained in detail later) to ensure the robustness of data extraction.

[0033] Step S2: Structured reorganization of knowledge units; standardize knowledge units into data entities with multi-dimensional standard structures. The multi-dimensional standard structure includes at least: subjective fact classification, subject, action, object, environmental features, emotional features, judgment confidence level, privacy level (e.g., 1 represents public information, 10 represents top secret credentials), and original paragraph source chunk, which is used to comprehensively describe the semantic content, privacy attributes and sources of the knowledge unit.

[0034] Step S3: Construct a cosine fusion vector space with metadata constraints; integrate the subject, action, object, environmental features, and emotional features of knowledge units into a dimensionality-reduced feature sentence (e.g., "Grandpa sends users to school (childhood, recurring)"). The dense vector group generated by the vectorization model based on this transcoding is stored in a vector database (e.g., ChromaDB). The vectorization model maps it to dense vectors, and the dense vectors and their corresponding privacy levels in the multidimensional standard structure are mounted and stored as metadata constraints (i.e., associated with the dense vectors) on the HNSW high-dimensional vector nodes of the vector database.

[0035] In this step, semantic information (such as subject, action, object, environmental features, and emotional features) in the multi-dimensional standard structure of knowledge units is mapped to a high-dimensional vector space, and cosine similarity is used for measurement and retrieval. Knowledge units with more similar semantics are closer in the cosine fusion vector space. HNSW high-dimensional vector nodes refer to the smallest units used to store dense vectors in the structure of hierarchical navigable small world graph index.

[0036] Step S4: Dynamic password verification based on the Max Level of the integrated authorization hierarchy; When the local gateway receives an authorized third-party query request, it performs identity and permission verification based on the tamper-proof dynamic password based on the Max Level of the integrated authorization hierarchy; The dynamic password is derived by using the time window stamp, the Max Level of the authorization hierarchy, and the random number buffer as implicit factors.

[0037] In this step, implicit factors refer to key parameters used to derive dynamic passwords that are not directly exposed in the plaintext of the password. These factors are encoded into the dynamic password through cryptographic algorithms to enhance its security and dynamism.

[0038] Step S5: Gateway Reverse Verification and Physical Truncation Retrieval; The Max Level of authorization allowed by the caller is extracted from the dynamic password. The Max Level represents the highest privacy level that the caller is allowed to access in this query. Subsequently, the Max Level is embedded as a metadata filtering condition into the vector database query statement. High-security vector fragments with a Privacy Level greater than the Max Level are physically truncated and filtered, returning only the retrieval results that meet the permission requirements to the caller.

[0039] In this step, when the large model of the outer network segment initiates a natural language request to query sensitive information using this dynamic password (containing serial code + random code), the system API gateway performs reverse collision verification to extract the authorization level Max Level carried by this dynamic password.

[0040] Furthermore, in an optional embodiment, the knowledge unit extraction in step S1 employs a three-level fault-tolerant degradation design: The first level is direct extraction, which attempts to obtain structured data directly from the output of the semantic parsing engine. This level is the most efficient extraction method, and its purpose is to directly parse and extract data when the output of the semantic parsing engine conforms to the expected standard structured data format. For example, if the semantic parsing engine directly outputs a standard JSON object or XML document, the system can directly extract the data using the corresponding parsing library (such as a JSON parser or XML parser). Another implementation approach is that if the output is a predefined data structure (such as a Python dictionary or Java object), type conversion and field access can be performed directly. This direct extraction method avoids complex pattern matching and error handling, thereby improving processing speed and efficiency.

[0041] The second level is Markdown block addressing and stripping. When Markdown block tags are detected in the output, regular expressions are used to locate and extract the JSON content within the block. This level handles situations where the semantic parsing engine output contains non-standard but recognizable structured data blocks, especially when the output is embedded in Markdown formatted blocks. Its function is to locate and extract the structured content contained within Markdown block tags (e.g., 'json ... '). Implementation can include using regular expressions to match the start and end tags of the Markdown block, then extracting the text content from the matched block, and attempting to parse it into JSON or other structured formats. For example, regular expressions can be used to locate and extract JSON content.

[0042] The third level is delimiter-based truncation. When the first two levels fail, it forces the extraction of valid JSON regions by locating specific square brackets or curly braces. This level is the final fault-tolerance mechanism, used to handle the extreme case where the first two levels fail—that is, when the output is neither in a standard structured format nor encapsulated in Markdown code blocks, but may still contain recognizable structured data fragments. Its function is to force the extraction of regions that may contain valid structured data by locating specific delimiters (such as square brackets '[]' or curly braces '{}'). Implementation methods may include iterating through the output text, finding the first left delimiter and the last right delimiter, then extracting the content between these two delimiters and attempting to parse it. For example, one could find the first '{' and the last '}', or the first '[' and the last ']', and then extract the string in between for further JSON parsing attempts. While this method may not be as precise as the first two levels, it provides a means of forcibly recovering data when other methods fail.

[0043] This design aims to improve the robustness and reliability of knowledge unit extraction, ensuring that the system can still extract as much valid structured data as possible even if the semantic parsing engine's output contains formatting errors or inconsistencies. Its core idea is to provide multi-level extraction strategies. When a higher-level strategy fails, it automatically tries the next higher-level strategy, down to the lowest-level strategy, thereby minimizing the risk of data loss and processing interruptions. This can be understood as an adaptive data parsing mechanism capable of handling varying degrees of output formatting irregularities.

[0044] With this design, this method can effectively handle the diversity and uncertainty of the output of the semantic parsing engine, ensuring the stable acquisition of knowledge units, thus providing a reliable data foundation for the subsequent structured recombination of knowledge units, and avoiding the interruption of the entire privacy data classification and isolation retrieval process due to failed data extraction. Therefore, this method can more reliably process complex personal unstructured data, effectively avoiding the breakage of the privacy information processing chain caused by data parsing errors, thereby enhancing the effectiveness of privacy protection.

[0045] Furthermore, in an optional embodiment, the dimensionality-reduced feature sentence in step S3 combines the subject, action, object, environmental feature, and emotional feature into a continuous text through a specific splicing symbol. While retaining the original semantic relationship, the continuous text reduces the data dimension; the privacy level is marked with a numerical scale, and when modifying the grading rule, it only needs to cover the metadata value of the corresponding node through the system API to take effect, without recalculating the vector representation.

[0046] Specifically, in this solution, the specific splicing symbol aims to effectively integrate discrete semantic elements to form a text representation that can express the complete semantics and has a compact structure. The specific splicing symbol can be a predefined special character sequence, such as "[SEP] ", "|", "##", or a conjunction in natural language, such as "of", "and", "in". Through this combination method, it can ensure that when multiple semantic components are fused into a single text, their internal logical relationship and semantic reference are clearly maintained, avoiding semantic ambiguity or loss caused by loose structure during the dimensionality reduction process. The generation of this continuous text provides a more contextually relevant input for the subsequent vectorization model, thereby enhancing the ability of the dense vector to capture the original semantics. The continuous text retains the original semantic relationship while reducing the data dimension. The core of this feature lies in achieving a balance between information compression and semantic fidelity. Retaining the original semantic relationship means that the combined text can accurately reflect the logical connection and the overall picture of the event among the subject, action, object, and environmental feature, for example, who did what, to whom, and under what background. Reducing the data dimension means that by integrating multiple discrete semantic elements into a shorter and more compact text sequence, the number of independent information units to be processed is reduced, thereby reducing the computational complexity and storage overhead of the subsequent vectorization process. This can be achieved by removing redundant information, refining the expression, or using the inherent structure of the language for efficient encoding. For example, combining "Zhang San runs in the park" directly into "Zhang San_runs_park" instead of processing the five independent units of "Zhang San", "in", "park", "in", and "runs" separately.

[0047] Furthermore, the Privacy Level uses a numerical scale to provide a quantifiable and standardized representation of the sensitivity of privacy data. The numerical scale can be an integer, such as 1, 2, 3...10, or a floating-point number, and its range and granularity can be defined according to actual needs. For example, a smaller number represents a lower Privacy Level (e.g., public information), while a larger number represents a higher Privacy Level (e.g., top-secret information). This digital representation has the advantages of being easy to compare, sort, and perform logical judgments, enabling the system to efficiently manage and filter data with different Privacy Levels. Simultaneously, modifying the level rules only requires overwriting the corresponding node's metadata value via the system API to take effect; this feature aims to provide an efficient and flexible privacy level management mechanism. In this scenario, by calling the system API, the metadata value associated with a specific vector node in the vector database can be directly modified, i.e., its Privacy Level identifier can be updated. This operation is characterized by strong atomicity, high efficiency, and no need to directly manipulate the underlying data structure. It can be implemented through the database's UPDATE statement or through a specific metadata management interface provided by the vector database. One of the key advantages of the aforementioned efficient management mechanism is that it eliminates the need to recompile vector representations. Vector representations are numerical vectors mapped from the original text using complex models, and their computation is typically resource-intensive and time-consuming. By storing the privacy level as independent metadata separate from the vector itself, and updating only the metadata when the privacy level is modified, the time-consuming vectorization process can be avoided. This means that even if the privacy level of a knowledge unit needs to be adjusted, its corresponding dense vector remains unchanged, significantly saving computational resources, reducing system latency, and improving the system's dynamic responsiveness.

[0048] As a specific implementation method, when generating dimensionality-reduced feature sentences, assume a knowledge unit contains "Subject: Li Si", "Action: Diagnosis", "Object: Diabetes", and "Environmental Feature: At the Community Hospital". The system can use "[SEP]" as a specific concatenation character to combine these semantic elements into continuous text, such as: "Li Si [SEP] Diagnosed [SEP] Diabetes [SEP] At the Community Hospital". After this continuous text is input into the vectorization model, it can generate a dense vector that represents the complete event "Li Si was diagnosed with diabetes at the community hospital" and is more compact than the original discrete element set. Regarding privacy level management, numerical scale identifiers can be set, for example, 1 represents "public information", 3 represents "internal information", 5 represents "sensitive information", 7 represents "confidential information", and 10 represents "top secret credentials". When a knowledge unit (e.g., a record about "Li Si's diagnosis of diabetes") is initially marked as Privacy Level 5 (sensitive information), but needs to be upgraded to Privacy Level 7 (confidential information) due to a change in user authorization, the system administrator or authorized program can call a preset system API, such as 'update_metadata(knowledge_unit_id, 'privacy_level', 7)'. This API will directly retrieve the vector node corresponding to the knowledge unit from the vector database and update its attached Privacy Level metadata value from 5 to 7. During this process, the dense vector of the knowledge unit itself does not need to be recalculated, thus achieving dynamic and efficient adjustment of the Privacy Level.

[0049] Through the above technical solution, this application effectively solves the problem of semantic integrity loss during the generation of dimensionality reduction feature sentences, ensuring that the core semantics and relationships of knowledge units are accurately preserved while reducing data dimensionality. This improves the ability of subsequent vectorization models to capture implicit privacy data semantics and enhances retrieval accuracy. Simultaneously, by employing a digitally scaled privacy level and a mechanism where the system API overwrites metadata values, the flexibility and efficiency of privacy level management are significantly improved. This avoids frequent, time-consuming, and labor-intensive vector recalculations due to privacy policy adjustments, greatly reducing the computational burden and latency of system operation. This enables the system to respond more agilely to changes in privacy policies, ensuring the dynamism and maintainability of privacy boundaries during hierarchical isolation retrieval, thereby improving the practicality and response speed of the entire retrieval method while ensuring data security.

[0050] In an optional embodiment, in step S1, multi-level regular expressions can be used to read unformatted user-retained local documents, and paragraphs can be segmented by blank lines. The semantic parsing engine adopts a locally deployed small parameter model, sets the core parameter temperature to close to 0.1 to fix determinism, and limits the redundancy of ultra-high word boundary as inference slots.

[0051] Specifically, this approach utilizes multi-level regular expressions to read unformatted, locally stored user documents and segments them by blank lines, aiming to efficiently and accurately extract meaningful paragraphs from various unstructured texts. This can be achieved by defining a series of regular expression patterns; for example, first identifying and processing common document structure elements such as headings, list items, or code blocks, and then using consecutive blank lines as the primary paragraph separators. Another implementation method is to combine natural language processing libraries for initial text preprocessing, such as removing unnecessary whitespace or special characters, and then using a custom set of regular expression rules to adaptively segment paragraphs for different document types (such as plain text and Markdown files), ensuring that even poorly formatted documents can be effectively parsed.

[0052] Semantic parsing engines can employ locally deployed, low-parameter models. By setting the core parameter `temperature` close to 0.1 to maintain determinism and limiting excessively high lexical boundary redundancy as inference slots, the aim is to improve the accuracy and stability of knowledge unit extraction while reducing reliance on external services. The locally deployed low-parameter model can be a pre-trained language model that has undergone knowledge distillation or quantization, such as a lightweight Transformer model with far fewer parameters than large cloud-based models, enabling efficient operation on local devices. Setting the core parameter `temperature` close to 0.1 means the model will tend to select the output with the highest probability during generation or classification, thereby reducing the randomness and uncertainty of the results and ensuring consistency in parsing results. Limiting excessively high lexical boundary redundancy as inference slots, for example, by setting the maximum number of input lexical units to be much larger than the actual paragraph length, ensures that the model has sufficient contextual information for inference when processing long paragraphs, avoiding information loss or semantic comprehension bias caused by truncation.

[0053] Optionally, in step S2, the extracted knowledge units can be instantiated as a KnowledgeUnit data model. The Privacy Level is based on a numerical scale, where 1 represents public information and 10 represents top-secret credentials. The KnowledgeUnit data model can be a class definition in an object-oriented programming language, containing fields such as subjective fact classification, subject, action, object, environmental characteristics, judgment confidence level, Privacy Level, and original paragraph tracing block, and providing corresponding constructors and access methods. Another implementation approach is to define a JSON Schema (a specification for defining JSON data structures) or Protocol Buffer message format, forcing all extracted knowledge units to conform to this structure, thereby ensuring data consistency and interoperability, and facilitating transfer and storage between different modules.

[0054] This can be achieved by defining an integer field in the KnowledgeUnit data model to store the Privacy Level, and establishing a mapping table or enumeration type to map numbers 1 to 10 to specific privacy level descriptions. For example, Level 1 represents public information, i.e., content that can be accessed publicly without restriction; Level 2 represents internal information, for use only within the organization or within a limited scope; Level 3 represents general privacy, such as low-sensitivity information like user interests; Level 4 represents restricted information, such as some non-critical contact information; Level 5 represents confidential information, including relatively sensitive personal data such as health status and spending habits; Level 6 represents sensitive privacy, such as specific income and expenditure records or detailed family member information; Level 7 represents highly sensitive information, such as precise location history or medical diagnosis details; Level 8 represents confidential credentials, such as partial account numbers or token fragments; Level 9 represents critical secrets, approaching the highest level of confidentiality; and Level 10 represents top-secret credentials, the disclosure of which would have serious consequences, such as complete login credentials or payment passwords. Internally, the system can implement corresponding access control, storage isolation, and retrieval filtering strategies based on the above numerical scale to achieve refined privacy hierarchical management.

[0055] Optionally, in step S3, PersistentClient is used to perform hard disk data persistence processing and supports lossless erasure and rewriting of metadata values ​​mounted on vector nodes. PersistentClient can be a module that encapsulates underlying file system operations; for example, it can write the processed knowledge units and their vector representations in encrypted form to a specific directory on a local solid-state drive (SSD) or hard disk drive (HDD), and periodically verify the data to ensure integrity. Another implementation approach is that PersistentClient can integrate with an embedded database (such as SQLite or RocksDB), storing data in a local database file and utilizing the database's transaction mechanisms and logging capabilities to guarantee data atomicity, consistency, isolation, and durability. In the vector database, each vector can be associated with a set of metadata. Lossless erasure and rewriting means that the associated metadata can be directly updated without recalculating or restoring the vector itself. This can be achieved through the API provided by the vector database, for example, by calling the 'update_metadata(vector_id, new_metadata_fields)' function to modify the privacy level or other attributes of a specific vector. Another approach is to store metadata and vector data separately. For example, vectors are stored in a vector database, while metadata is stored in a separate key-value store (such as Redis or LevelDB), linked by vector IDs. When metadata needs to be modified, only the corresponding record in the key-value store needs to be updated, and the system merges the two during the query.

[0056] The following example illustrates this. Suppose a user has a locally stored unstructured diary document containing entries such as "Today I took my aunt out to dinner, but she is lactose intolerant; I also spent over 8,000 yuan on cryptocurrency."

[0057] In step S1, the system first uses multi-level regular expressions to parse the document. For example, it identifies "I took my aunt out to eat today, she's lactose intolerant" as one paragraph and "I spent over 8,000 yuan to buy cryptocurrency" as another paragraph. Then, the locally deployed small-parameter semantic parsing engine extracts knowledge units from these paragraphs. For example, it extracts knowledge points such as "aunt" and "lactose intolerant" from the first paragraph, and "over 8,000 yuan" and "cryptocurrency" from the second paragraph. During this process, because the core parameter temperature is set close to 0.1, the model stably outputs these knowledge points, and the extremely high lexical boundary redundancy ensures that even long sentences can be fully understood.

[0058] In step S2, these extracted knowledge units are instantiated into KnowledgeUnit data models. For example, the knowledge unit "Auntie is lactose intolerant" might be assigned a Privacy Level 2 (internal information), while the knowledge unit "spent over 8,000 yuan to buy cryptocurrency" might be assigned a Privacy Level 9 (highly confidential financial information). These numerical scales clearly indicate the privacy sensitivity of different knowledge units.

[0059] In step S3, these instantiated KnowledgeUnit data models and their corresponding vector representations are permanently stored on the local hard drive via PersistentClient, for example, in an encrypted local database file. If the user subsequently decides to disclose the information about "Aunt's lactose intolerance," the system can directly update the metadata value mounted on the corresponding vector node via API calls. In this embodiment, the PrivacyLevel of the knowledge unit is changed from 2 to 1 without recalculating or restoring the vector representation of the knowledge unit.

[0060] Through the above technical solution, this application effectively solves the problems of low efficiency, insufficient accuracy, vague definition of privacy level, insecure data storage, and inconvenience in modifying metadata in traditional methods when processing unstructured implicit privacy data.

[0061] Furthermore, in optional embodiments, such as Figure 1 As shown, the dynamic password derivation process in step S4 may include the following steps: First, a binary message body is constructed, designed to carry the core parameters required for dynamic password generation. This message body can be constructed using a fixed-length byte array or a structure. The binary message body contains an 8-byte timestamp, a 4-byte unsigned integer Max Level, and a 4-byte random number buffer. The 8-byte timestamp provides the password's timeliness, effectively preventing replay attacks; it can be represented as the current Unix timestamp or a time-synchronized window value. The 4-byte unsigned integer Max Level specifies the depth of access permissions for the user or caller, and can use predefined integer values ​​(e.g., 1-10, representing different privacy levels) or bitmasks to represent various permission combinations. The 4-byte random number buffer aims to increase the randomness and unpredictability of the password to prevent collisions and guessing; it can be generated using a cryptographically secure pseudo-random number generator (CSPRNG) or provided by a hardware random number generator (TRNG).

[0062] Next, the binary message body is hashed using the HMAC-SHA256 algorithm with the local private key to obtain the token. The local private key serves as both an authentication and message integrity protection tool in this process. It can be securely stored in a hardware security module (such as a TPM or HSM) or stored in encrypted form on the local file system. The HMAC-SHA256 algorithm, based on the RFC 2104 standard, uses SHA-256 as the underlying hash function to generate a hash value with a key, ensuring message integrity and authentication. The resulting token is the raw form of a dynamic password, containing authentication and authorization information.

[0063] Subsequently, the token undergoes truncation and dimensionality reduction processing, outputting a 6-digit numeric string and an accompanying random signature code. The authentication status is then strongly bound to the authorization level (Max Level). Truncation and dimensionality reduction aims to simplify the token's representation, making it easier for transmission and user input. For example, it can extract a portion of the token's hash value or map it to a specified range using modulo operations. The 6-digit numeric string provides a user-friendly short code, for example, by converting a portion of the token into a decimal number. The random signature code provides additional verification information, further enhancing security; it can be another part of the token or derived from the token using a specific algorithm.

[0064] The following is a concrete example. Suppose we need to generate a dynamic password that allows access to data with a Privacy Level of 5, and that the password is valid for the next 30 seconds. First, the system constructs a binary message body. For example, the last 8 bytes of the current Unix timestamp are used as a timestamp, the integer value 5 is used as a 4-byte unsigned integer Max Level, and a cryptographically secure pseudo-random number generator generates a 4-byte random number as a random number buffer. These three parts of data are concatenated in a predetermined order to form a 16-byte binary message body. Then, using a pre-configured 256-bit local private key, the 16-byte message body is hashed using the HMAC-SHA256 algorithm to obtain a 32-byte token. To facilitate user input and transmission, the system can truncate and reduce the dimensionality of this 32-byte token. For example, the first 3 bytes of the token are converted to decimal to obtain a 6-digit numeric string (such as "123456"), and the next 4 bytes of the token are used as an accompanying random signature code. Ultimately, this "123456" numeric string and its random signature code represent a dynamic password with access permissions at Max Level 5 within a specific time window.

[0065] Through the above technical solution, this application effectively solves the problem that existing password mechanisms cannot dynamically bind the authorization level MaxLevel to lock the query depth. By directly embedding the authorization level MaxLevel into the dynamic password generation process and using a cryptographic hash algorithm for strong binding, the password itself contains a clear access permission boundary. This fundamentally prevents unauthorized access caused by methods such as prompt injection attacks, because even if an attacker tricks a large model into generating a query containing high-secret information, the gateway, when verifying the dynamic password, will physically truncate and filter the search results according to the authorization level MaxLevel bound in the password, ensuring that high-secret vector fragments cannot enter the recall scoring pool, thereby protecting the secure isolation of implicit privacy data. At the same time, the introduction of time window stamps further enhances the timeliness and security of the password, effectively resisting the risk of replay attacks.

[0066] Furthermore, in an optional embodiment, the physical truncation filtering in step S5 is performed before the HNSW topology space metric or cosine similarity comparison. The underlying vector database directly uses the mounted metadata constraints to filter and exclude high-density vector fragments with a Privacy Level greater than the Authorization Level Max Level, so that the high-density vector fragments cannot enter the recall scoring pool.

[0067] The Max Level, the authorization level parsed from the dynamic password, is the upper limit of query permissions granted to the caller. This value is actively set by the data holder (such as the local client) when generating the dynamic password, and it can be less than or equal to the Privacy Level of certain highly confidential data in the system.

[0068] In other words, when a data holder wants to restrict third parties in the cloud to access only low- or medium-security information, and not allow access to high-security information, the intended authorization level (Max Level) will be set to a small number (e.g., 3) when generating a dynamic password. The database may store high-security data (such as financial records) with a Privacy Level of 8. In this case, the parsed authorization level (Max Level 3) is less than the Privacy Level of 8 for this high-security data, thus triggering physical truncation and filtering out all data entities with a Privacy Level > 3 in the multi-dimensional standard structure.

[0069] In this embodiment, physical truncation filtering refers to completely removing data fragments that do not meet specific requirements from subsequent processing in the early stages of the data processing flow through hard rules or conditions, rather than simply marking or soft filtering. This filtering can be implemented during data loading or index building by directly preventing certain data from being written to the storage or index structure based on preset metadata conditions; or before query execution, by physically blocking data blocks that do not meet the conditions at the data access level through the database engine's internal mechanisms, preventing them from being read or used in computation.

[0070] HNSW (Higher Nearest Neighbors) topological space metric is an efficient Approximate Nearest Neighbor (ANN) search algorithm used to quickly find the vector most similar to a query vector in a high-dimensional vector space. It accelerates the search process by constructing a multi-layered graph structure, where each layer is a small world graph. Upper layers have sparse connections but few hops, while lower layers have dense connections but many hops. HNSW can be used to calculate the distances between the query vector and the vectors at each node in the HNSW graph and traverse the edges to progressively approach the nearest neighbor; or it can utilize the HNSW index structure to search from the top layer down the graph to find the K vectors with the smallest distance to the query vector.

[0071] Cosine similarity is a metric that measures the directional similarity between two non-zero vectors. Its value ranges from -1 to 1 and is commonly used to assess the semantic relevance between a query vector and knowledge unit vectors stored in a database. Cosine similarity can be obtained by calculating the dot product of two vectors and dividing by the product of their respective moduli; alternatively, in a vector database, a built-in similarity calculation function can be used to directly calculate the cosine similarity between the query vector and candidate vectors, and then sort the results.

[0072] Vector databases directly utilize mounted metadata constraints for filtering. This means that when performing query operations, the vector database can directly access and utilize non-vector data (i.e., metadata) stored in association with each vector data point, and perform efficient filtering operations based on this metadata, without needing to retrieve the vector data and perform external processing. This can be achieved by storing metadata as an additional attribute along with the vectors in the same data structure or closely related storage area, allowing direct filtering through metadata indexes during queries; or by the vector database's query optimizer recognizing metadata filtering conditions in the query statement and applying these conditions directly to the underlying data access layer before or during the vector search algorithm, thereby physically excluding vectors that do not meet the conditions.

[0073] Excluding high-secrecy vector fragments with a Privacy Level greater than the Max Level of authorization means identifying and removing all vector data fragments whose Privacy Level is higher than a preset Max Level of authorization. This can be done by comparing the Privacy Level of each vector fragment with the Max Level of the current query request in the data filtering logic; if the Privacy Level > Max Level, the fragment is excluded. Alternatively, it can be done by configuring database access control policies or query filters to set a threshold on the data retrieval path; any data with a Privacy Level exceeding this threshold will be automatically blocked.

[0074] Preventing high-density vector fragments from entering the recall scoring pool refers to ensuring, through the aforementioned physical truncation filtering mechanism, that vector fragments identified as high-density cannot be included in the initial recall result set (i.e., the recall scoring pool) of vector retrieval under any circumstances, thus completely preventing them from participating in subsequent similarity calculations, ranking, and return. This can be achieved by incorporating metadata filtering as a pre-processing step in the vector database's query execution plan, completing data screening before the vector index search begins, ensuring the search algorithm runs only on a subset of data that meets permission requirements; or by modifying the underlying recall logic of the vector database to force a check on the privacy level of each candidate vector when generating the candidate recall set, and immediately discarding vectors that do not meet permission requirements.

[0075] The following example illustrates this. Suppose a local client acts as the data holder, and a cloud-based third party acts as the caller, needing to query unstructured personal diary data stored locally. First, the local client generates a dynamic password carrying the authorization level (Max Level) using the method described above. For example, this password instructs the queryer to only access data with a Privacy Level of 1 to 5 (i.e., an authorization level of Max Level 5). The cloud-based third party then initiates a natural language query carrying this dynamic password, such as "How is my aunt's health lately?". When the local gateway receives this query request, it first verifies the dynamic password and deciphers the authorization level (Max Level) to be 5. Subsequently, during vector retrieval, the vector database initiates physical truncation filtering. Specifically, before the HNSW index performs topological space measurement or performs cosine similarity comparison between the query vector and the stored vector, the underlying query engine of the vector database checks the Privacy Level metadata attached to each knowledge unit vector. For example, if a knowledge unit vector has a Privacy Level of 8 (representing "top secret credentials," higher than Max Level 5), that vector fragment will be directly excluded by the underlying engine, will not enter the HNSW search path, will not participate in cosine similarity calculations, and will not be included in the recall scoring pool. Only vector fragments with a Privacy Level of 5 or less will be allowed to participate in subsequent similarity calculations and recall processes. In this way, even if a query request may semantically relate to high-secrecy privacy data (such as "aunt's financial situation," with a Privacy Level of 8), this high-secrecy data is completely isolated at the underlying level due to the early execution of physical truncation filtering, making it impossible to recall. This ensures that third parties in the cloud can only obtain anonymized conclusions within the authorized scope, such as information about the aunt's dietary preferences (Privacy Level 2) or general health status (Privacy Level 4), and cannot reverse-engineer her high-secrecy financial information through semantic association.

[0076] Through the above technical solution, this application effectively solves the problem in traditional methods where high-density privacy data may still be recalled and enter the scoring pool after vector similarity measurement or recall, thus posing a risk of leakage. By advancing the execution of physical truncation filtering to before HNSW topological space measurement or cosine similarity comparison, it ensures that high-density vector fragments that do not meet the permission requirements are thoroughly physically excluded in the early stage of the vector recall process, i.e., before the data enters the similarity calculation and scoring pool. This low-level, upfront filtering mechanism allows the vector database to directly utilize the attached metadata constraints for efficient screening without additional parsing or conversion steps, thereby improving the efficiency and accuracy of filtering. By strictly excluding high-density vector fragments with a Privacy Level greater than the Max Level of authorization, this application fundamentally eliminates the risk of high-density privacy data being accidentally recalled, participating in subsequent processing, or being reverse-engineered to reveal core privacy information through the reasoning capabilities of large language models. This not only enhances the security of data retrieval, but also provides a more robust technical guarantee for the hierarchical isolation and retrieval of implicit privacy data in cross-domain collaborative scenarios, ensuring that data holders have refined control over their own privacy data.

[0077] Furthermore, this application proposes applying this method to cross-domain collaboration scenarios, where data processing and querying occur between different network domains or trust domains. For example, the local client might be a personal device or an internal enterprise server, while the cloud-based third party is an AI service or data analytics platform deployed in a public cloud environment. This scenario is characterized by the separation of data ownership and processing rights, placing higher demands on data security and privacy protection. As one possible implementation, cross-domain collaboration scenarios can be constructed using technologies such as VPN tunnels, API gateways, or federated learning frameworks. Another implementation approach is to establish a secure communication channel between the local client and the cloud-based third party, and utilize encryption protocols to ensure the confidentiality and integrity of data transmission.

[0078] In this embodiment, the local client acts as the data holder, and the cloud-based third party acts as the caller. Their responsibilities include preprocessing and storing the data, and controlling access to the data according to authorization policies. As one possible implementation, the local client could be a user's personal computer, smartphone, or locally deployed server. Alternatively, the local client could be an edge computing device responsible for initial data processing and storage. The cloud-based third party, acting as the caller, is the entity that initiates a data query request and expects to obtain results within the authorized scope. It typically does not directly hold the original data but interacts with the data holder through an interface. As one possible implementation, the cloud-based third party could be an AI assistant, a data analysis platform, or an enterprise application. Another implementation could be a research institution or partner that needs to access specific types of data for analysis.

[0079] The local client grants restricted query access to third parties in the cloud by generating a dynamic password carrying the maximum authorization level (Max Level). This dynamic password is an authentication credential with time-limited and one-time characteristics, embedding the highest level of privacy information the caller is allowed to access. The generation of this password can be based on various encryption algorithms and random number generation mechanisms to ensure its unpredictability and tamper-proof nature. One possible implementation is an encrypted JSON Web Token (JWT) containing information such as the maximum authorization level and expiration time. Another implementation is a short string generated based on the HMAC algorithm, with the maximum authorization level information embedded through a specific encoding method. Restricted query access means that third parties in the cloud can only query data within the privacy level defined by the dynamic password; requests exceeding this range will be rejected.

[0080] Cloud-based third parties can initiate natural language queries carrying dynamic passwords. This query method improves user experience, allowing non-technical personnel to easily retrieve data. One possible implementation is to send the natural language query via an HTTP POST request, with the dynamic password included in the request header or body. Another approach is for the cloud-based third party to encapsulate the natural language query and dynamic password using a specific SDK or API and send it to the local gateway.

[0081] The local gateway determines the maximum authorization level (Max Level) for queries by verifying dynamic passwords and performs physical truncation during the vector retrieval phase. This ensures that third parties in the cloud can only obtain anonymized results within the authorized scope and cannot reconstruct high-secret privacy information through semantic association. The local gateway is a security component deployed on or near the local client side, responsible for receiving and processing query requests from third parties in the cloud. Verifying dynamic passwords includes verifying their validity (e.g., whether they have expired or been tampered with) and parsing the maximum authorization level information contained within them. One possible implementation is that the local gateway can use a preset key to decrypt and verify the dynamic password and extract the maximum authorization level. Another implementation is that the local gateway can interact with an authentication service, which is responsible for verifying the dynamic password and parsing the maximum authorization level. Physical truncation refers to pre-filtering and excluding high-secret vector fragments that do not meet the permission requirements based on the maximum authorization level determined in the dynamic password before performing similarity retrieval in the vector database. This means that this high-density data is physically prevented from entering the recall scoring pool, thus fundamentally avoiding the possibility of it being retrieved. One possible implementation is physical truncation, which can be implemented at the query optimizer level of the vector database, using the maximum authorization level as a pre-filter condition. Another implementation is to create independent index partitions for data with different privacy levels during the index building phase of the vector database, and only activate partitions within the authorized scope during queries.

[0082] The following example illustrates this. Suppose a user (local client) has a large number of personal diaries and notes, containing low-secret dietary preferences, medium-secret information about relatives' whereabouts, and high-secret financial habits. This user wants to allow a cloud-based AI assistant (third-party cloud provider) to access their dietary preferences and relatives' whereabouts, but absolutely not their financial information. The user first generates a dynamic password on the local client, embedding a "medium-secret" authorization level (Max Level). When the cloud-based AI assistant needs to query the user's health and dietary advice, it sends a natural language query to the local gateway, carrying this dynamic password, such as "What are my recent dietary habits? What health advice do you have?" Upon receiving the request, the local gateway first verifies the dynamic password, confirming its validity and resolving the contained "medium-secret" authorization level (Max Level). Subsequently, during vector retrieval, the local gateway uses this "medium-level confidentiality" as a metadata filtering condition. It physically truncates all vector fragments in the vector database with a privacy level higher than "medium-level confidentiality" (e.g., all financial data marked "high-level confidentiality"), preventing them from entering the recall scoring pool. Ultimately, the vector database only returns low- and medium-level confidentiality data related to dietary preferences and relatives' whereabouts to the cloud-based AI assistant, while all high-level confidentiality financial information is completely isolated. Even with its powerful semantic reasoning capabilities, the AI ​​assistant cannot infer the user's financial habits from the returned results.

[0083] Through the above technical solution, this application effectively solves the problems in cross-domain collaboration scenarios, such as the inability of traditional methods to dynamically transmit access authorization limits and the possibility of third parties in the cloud reconstructing high-secret privacy information through semantic association. This method achieves fine-grained and dynamic control of query permissions by embedding the authorization level (Max Level) into the dynamic password, ensuring clear permission boundaries for each query. Simultaneously, a physical truncation mechanism is introduced in the vector retrieval stage to hard-isolate high-secret privacy data from the data source, completely eliminating the risk of retrieval and leakage. Thus, while ensuring data availability, it significantly improves the security and isolation of implicit privacy data in cross-domain environments.

[0084] To achieve the above objectives, this application proposes a hierarchical isolation retrieval system for implicit privacy data, which implements the aforementioned hierarchical isolation retrieval method for implicit privacy data, and may specifically include: The inbound pipeline extraction module (e.g., the ingestion.py unit) is configured to read unformatted user-retained local documents using multi-level regular expressions, perform paragraph segmentation and knowledge unit extraction based on blank lines, force each paragraph to be extracted into several discrete knowledge units, and use a three-level fault tolerance and degradation strategy to process the output characters.

[0085] In a specific embodiment, this module is responsible for reading unformatted user-retained local documents (TXT / MD) using multi-level regular expressions, logically slicing and recombining them according to predetermined rules, and driving a large model with built-in local small parameters (such as Qwen3.5-27B, setting the core parameter temperature close to 0.1 to fix determinism, limiting its ultra-high word boundary redundancy to be used for inference slots). It translates human language into a regular, standard structured JSON string through customized strong-format demonstration pre-prompts; and includes a three-level disaster recovery regular expression salvage and degradation strategy to handle the output characters of the runaway model.

[0086] The knowledge unit structuring module is configured to standardize knowledge units into data entities with multi-dimensional standard structures. The multi-dimensional standard structure includes at least: subjective fact classification, subject, action, object, environmental features, emotional features, judgment confidence level, privacy level, and original paragraph tracing block.

[0087] The multidimensional vector storage engine module (vector_store.py unit) is configured to integrate the subject, action, object, environmental features and sentiment features of knowledge units into dimensionality-reduced feature sentences, map them into dense vectors through the feature embedding engine, and mount and store the dense vectors and their corresponding privacy levels as metadata constraints on the HNSW high-dimensional vector nodes of the vector database.

[0088] In a specific embodiment, this module is responsible for receiving the extracted multi-dimensional standard structure (including sentiment features, objects, and judgment confidence levels), and combining them using a specific feature value concatenation algorithm to form the optimal summary-state word string for vector generation model delivery. Its underlying implementation utilizes PersistentClient for hard disk data digitization and solidification. It supports lossless erasure and rewriting of metadata values ​​mounted on it with extremely low maintenance overhead.

[0089] The secure access modifier password gateway module (which may include the totp.py verification unit and the FastAPI interception unit) is configured to perform identity and permission verification based on the tamper-proof dynamic password with the Max Level of the fusion authorization hierarchy when an authorized third-party query request is received. The dynamic password is derived by using the time window stamp, the Max Level of the authorization hierarchy, and the random number buffer as implicit factors, and then assembling a structure and performing SHA256 hash calculation.

[0090] In a specific embodiment, this module manages request issuance and interception as an independent component. The core modification standard TOTP is to add a Nonce to avoid high-frequency repetition, and to output a randomized signed coded credential string using an irreversible variant verification equation.

[0091] The core module for generating restricted RAG question-answering retrieval is configured to extract the maximum authorization level (Max Level) that the caller is allowed to access from the dynamic password, embed the maximum authorization level (Max Level) as a metadata filtering condition into the vector database query statement, physically truncate and filter high-secrecy vector fragments whose privacy level is greater than the maximum authorization level (Max Level), and only return the search results that meet the permission requirements to the caller.

[0092] Other specific implementation methods have been explained in detail above and will not be elaborated further here.

[0093] Traditional technologies primarily rely on lexical substitution based on sensitive words or front-end semantic analysis for desensitization. However, unstructured text, such as "The cost is approximately 8,000 yuan, and this money needs to be withdrawn from the investment account," which mixes financial information and daily activities, cannot be easily handled through simple sensitive word identification. This application forcibly decomposes long texts into discrete knowledge units through unstructured data ingestion and atomic dimensionality reduction. Furthermore, it assigns a multi-dimensional standard structure to each knowledge unit through structured recombination, including a clearly defined privacy level. This atomic-level extraction and hierarchical classification capability enables the system to identify and manage implicit privacy that is difficult to reach with traditional methods, solving the problem of unstructured implicit privacy deconstruction that existing technologies cannot handle.

[0094] Furthermore, in cross-domain collaborative scenarios, existing solutions have shortcomings in authentication boundaries, lacking a cryptographic medium for dynamically transmitting access authorization limits from the client to the data end. This application addresses this by integrating dynamic password verification with an authorization level (Max Level). It utilizes a time window stamp, the authorization level (Max Level), and a random number buffer as implicit factors to derive a tamper-proof dynamic password. This dynamic password can pinpoint the depth to which the caller can query within a single session, effectively solving the problems of single-line defense and limited authentication boundaries in existing technologies.

[0095] More importantly, this application implements physical truncation capabilities at the retrieval level. Existing methods use desensitization mapping at the string level, and when recalling data from high-dimensional vector databases, high-density data blocks may still be recalled due to the diffusion effect of the model's attention mechanism. However, this application's local gateway extracts the authorization level (Max Level) from the dynamic password and uses it as a metadata filtering condition. Before performing HNSW topology space measurement or cosine similarity comparison in the vector database, it physically truncates and filters high-density vector fragments with a Privacy Level greater than the authorization level (Max Level). This means that high-density data is physically isolated and cannot enter the recall scoring pool. This mechanism of hard isolation before underlying vector recall fundamentally eliminates the risk of leaking core user privacy by using the powerful reasoning capabilities of large language models to piece together and reverse-engineer it, significantly improving data security.

[0096] To achieve the above objectives, the present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor runs the program, it can implement the steps of the hierarchical isolation retrieval method for implicit privacy data as described in any of the foregoing embodiments.

[0097] Processors and memory can be configured separately or integrated together, for example, integrated on a system-on-chip (SOC) in a terminal device.

[0098] To achieve the above objectives, the present invention also provides a computer-readable storage medium storing computer-executable instructions or computer programs, which, when processed and executed, implement the hierarchical isolation and retrieval method for implicit privacy data as described above.

[0099] The computer-readable storage medium is, for example, memory. Memory can be volatile or non-volatile, or it can include both volatile and non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DRRAM).

[0100] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0101] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A hierarchical isolation and retrieval method for implicit privacy data, characterized in that, Includes the following steps: Step S1: Unstructured data ingestion and atomized dimensionality reduction; perform paragraph segmentation and knowledge unit extraction on the unstructured text, forcibly extracting each paragraph into several discrete knowledge units; Step S2: Structured reorganization of knowledge units; the knowledge units are standardized into data entities with multi-dimensional standard structures, which at least include: subjective fact classification, subject, action, object, environmental features, emotional features, judgment confidence level, privacy level, and original paragraph source tracing block; Step S3: Construct a cosine fusion vector space with metadata constraints; integrate the subject, action, object, environment features and emotion features in the multi-dimensional standard structure into a dimension-reduced feature sentence, map it into a dense vector through a vectorization model, and store the privacy level in the multi-dimensional standard structure as a metadata constraint associated with the dense vector on the HNSW high-dimensional vector node of the vector database; The Privacy Level is a security classification label assigned to each knowledge unit, indicating the sensitivity of the data it contains. Step S4: Dynamic password verification based on the Max Level of the integrated authorization hierarchy; when the local gateway receives an authorized third-party query request, it performs identity and permission verification based on the tamper-proof dynamic password based on the Max Level of the integrated authorization hierarchy; the dynamic password is derived by using the time window stamp, the Max Level of the authorization hierarchy, and the random number buffer as implicit factors; Step S5: Gateway reverse verification and physical truncation retrieval; extract the maximum authorization level (Max Level) allowed by the caller from the dynamic password. The maximum authorization level (Max Level) represents the highest privacy level that the caller is allowed to access in this query; embed the maximum authorization level (Max Level) as a metadata filtering condition into the vector database query statement, physically truncate and filter high-secrecy vector fragments whose privacy level is greater than the maximum authorization level (Max Level), and return only the retrieval results that meet the permission requirements to the caller.

2. The hierarchical isolation and retrieval method for implicit privacy data according to claim 1, characterized in that, The knowledge unit extraction in step S1 adopts a three-level fault-tolerant degradation design: The first level is direct extraction, which attempts to obtain structured data directly from the output of the semantic parsing engine; The second level is Markdown code block addressing and stripping. When the output contains Markdown code block tags, the JSON content within the code block is located and extracted using regular expressions. The third level is delimiter cutting and stripping. When the above two levels fail, the valid JSON area is forcibly extracted by locating specific square brackets or curly braces delimiters.

3. The hierarchical isolation and retrieval method for implicit privacy data according to claim 1, characterized in that, In step S3, the dimensionality reduction feature sentences combine subject, action, object, environmental features and emotional features into continuous text through specific splicing characters. The continuous text retains the original semantic relationship while reducing the data dimensionality. The Privacy Level uses a numerical scale identifier. When modifying the level rules, the changes only need to be made by overwriting the metadata value of the corresponding node through the system API, without needing to recalculate the vector representation.

4. The hierarchical isolation and retrieval method for implicit privacy data according to claim 1, characterized in that, The dynamic password derivation process in step S4 includes: Construct a binary message body, which includes: an 8-byte timestamp, a 4-byte unsigned integer Max Level, and a 4-byte random number buffer; The token is obtained by hashing the binary message body using the local private key. The token is truncated and its dimensions reduced to output a 6-digit numeric string and an accompanying random signature code. The authentication status is then strongly bound to the authorization level (Max Level).

5. The hierarchical isolation and retrieval method for implicit privacy data according to claim 4, characterized in that, The physical truncation filtering in step S5 is performed before the HNSW topology space metric or cosine similarity comparison. The vector database directly uses the attached metadata constraints to filter and exclude high-density vector fragments with a privacy level greater than the authorization level (Max Level), so that the high-density vector fragments cannot enter the recall scoring pool.

6. The hierarchical isolation and retrieval method for implicit privacy data according to claim 1, characterized in that, The method is applied to cross-domain collaboration scenarios: The local client acts as the data holder, while the cloud-based third party acts as the caller. The local client grants restricted query permissions to the cloud-based third party by generating a dynamic password carrying the Max Level of authorization. The cloud-based third party initiates a natural language query carrying the dynamic password. The local gateway determines the query authorization level (Max Level) by verifying the dynamic password and performs physical truncation during the vector retrieval stage to ensure that the cloud-based third party can only obtain the de-identified conclusions within the authorized scope and cannot reverse-engineer high-secret privacy information through semantic association.

7. The hierarchical isolation and retrieval method for implicit privacy data according to claim 2, characterized in that, In step S1, multi-level regular expressions are used to read unformatted user-retained local documents and segment them into paragraphs based on blank lines. The semantic parsing engine adopts a locally deployed small-parameter model, sets the core parameter temperature to close to 0.1 to fix determinism, and limits the redundancy of ultra-high word boundary as inference slots. In step S2, the extracted knowledge unit is instantiated into a KnowledgeUnit data model. The privacy level is based on a digital scale identifier, where 1 represents public information and 10 represents top secret credentials. In step S3, PersistentClient is used to perform hard disk data persistence processing and supports lossless erasure and rewriting of metadata values ​​mounted on vector nodes.

8. A hierarchical isolation and retrieval system for implicit privacy data, characterized in that, The hierarchical isolation retrieval method for implicit privacy data as described in any one of claims 1 to 7 includes: The inbound pipeline extraction module is configured to read unformatted user-retained local documents using multi-level regular expressions, perform paragraph segmentation and knowledge unit extraction based on blank lines, forcibly extract each paragraph into several discrete knowledge units, and use a three-level fault tolerance and degradation strategy to process the output characters. The knowledge unit structuring module is configured to standardize the knowledge unit into a data entity with a multi-dimensional standard structure. The multi-dimensional standard structure includes at least: subjective fact classification, subject, action, object, environmental features, emotional features, judgment confidence level, privacy level, and original paragraph tracing block. The multidimensional vector storage engine module is configured to integrate the subject, action, object, environmental features and emotional features of the knowledge unit into a dimensionality-reduced feature sentence, map it into a dense vector through a feature embedding engine, and mount and store the dense vector and its corresponding privacy level as metadata constraints on the HNSW high-dimensional vector node of the vector database. The secure access modifier password gateway module is configured to perform identity and permission verification based on a tamper-proof dynamic password with a maximum authorization level when an authorized third-party query request is received. The dynamic password is derived by using a time window stamp, the maximum authorization level, and a random number buffer as implicit factors, and then assembling a structure and performing SHA256 hash calculation. The restricted RAG question-answering retrieval generation core module is configured to extract the maximum authorization level (Max Level) that the caller is allowed to access from the dynamic password, embed the maximum authorization level (Max Level) as a metadata filtering condition into the vector database query statement, physically truncate and filter high-secret vector fragments whose privacy level is greater than the maximum authorization level (Max Level), and return only the retrieval results that meet the permission requirements to the caller.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor runs the program, it implements the hierarchical isolation retrieval method for implicit privacy data as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions or computer programs, which, when processed and executed, implement the hierarchical isolation retrieval method for implicit privacy data as described in any one of claims 1 to 7.

Images