A system for privacy-preserving LLM inference through data obfuscation in cloud-based usage workflows

Abstract

A system for privacy-friendly LLM inference, comprising: an input catcher unit, a deterministic semantic obfuscation engine, an obfuscation-sensitive prompt encoder, a cloud LLM interface, a local result generation and recovery layer, and a data protection auditor and policy enforcer; where the deterministic semantic obfuscation engine replaces sensitive terms with domain-specific placeholders using a context-sensitive rule base or NER-based marking; and where the prompt encoder embeds metadata that allows deterministic recovery of the original content after inference.

Description

FIELD OF THE INVENTION

[0001] The present invention relates to the field of privacy-enhancing technologies in cloud computing and artificial intelligence. In particular, it relates to a system that enables privacy-preserving inference using large language models (LLMs) by introducing structured data obfuscation prior to cloud-based processing. BACKGROUND OF THE INVENTION

[0002] Large language models (LLMs) offer powerful natural language processing capabilities and are increasingly being used in cloud-based services. However, the transfer of raw user data to such models raises significant privacy concerns, particularly in sensitive areas such as healthcare, finance, and legal services. Existing approaches such as encryption, secure multi-party computation (MPC), or differential privacy are associated with performance degradation, high complexity, or a loss of usability. Therefore, there is a need for a lightweight, real-time compatible system that ensures privacy without compromising performance. SUMMARY OF THE INVENTION

[0003] The present invention presents a sophisticated and secure system that enables privacy-preserving inference using large language models (LLMs)—especially when such inference occurs in cloud-based environments. At its core, the system addresses the fundamental challenge of protecting sensitive user data—such as personal identifiers, medical terms, financial data, or legal information—while simultaneously leveraging the powerful capabilities of natural language processing provided by external LLMs.

[0004] This is achieved by implementing a pre-processing and post-processing framework that semantically obfuscates sensitive content before transmission and restores the original context after receiving the model response.

[0005] A central element of this system is the Deterministic Semantic Obfuscation Engine (DSOE), which uses techniques such as Named Entity Recognition (NER), pattern recognition, or user-defined rules to identify sensitive tokens in natural language input. These tokens are deterministically replaced with contextually appropriate placeholders—such as [NAME], [DISEASE], or [ACCOUNT_NUMBER]—ensuring semantic consistency in the modified input text. The transformed input is then processed by the Obfuscation-Aware Prompt Encoder (OAPE), which embeds a reversible transformation map either in the prompt metadata or in a parallel internal memory structure. This encoding ensures that each placeholder is traceable and that the obfuscation process can be precisely reversed on the client side.

[0006] The obfuscated prompt is securely transmitted over encrypted channels to a cloud LLM interface, where it is processed by an external or third-party language model without direct access to the original sensitive content. Once the cloud-based LLM returns its response based on the obfuscated prompt, the Local Result Generation and Recovery (LRRL) layer comes into play. This module leverages the previously stored metadata to reintegrate the original sensitive tokens into the generated response. This restores both semantic accuracy and personal meaning without compromising user data during transmission or inference.

[0007] To complement these core components, the system integrates a Privacy Audit and Policy Enforcer (PAPE). This module enables fine-grained control over the obfuscation and remediation processes. The Enforcer applies custom privacy policies, maintains a comprehensive audit trail for compliance purposes, and supports context-aware, real-time decision-making.

[0008] By integrating these modules into a seamless pipeline, the invention provides a powerful framework for privacy-aware, real-time LLM interactions. It enables users and organizations to leverage powerful cloud-based AI tools without exposing confidential information—making it particularly valuable in areas such as healthcare, finance, legal services, and customer service, where data protection is critical. DETAILED DESCRIPTION OF THE INVENTION

[0009] A system for privacy-preserving LLM inference through structured data obfuscation is disclosed. The system modifies sensitive user data using a deterministic semantic obfuscation engine before submitting it to a cloud-based LLM. A prompt encoder annotates the input text with reversible metadata for transformation, while a restoration layer reconstructs the final result using local mappings. The system ensures privacy without compromising the accuracy or utility of the inference and offers configurable privacy settings, real-time capability, and full logging. System components Input Capture Unit

[0010] The input catcher unit forms the system's input interface. It receives natural language input submitted by the user, which may contain sensitive information such as names, medical conditions, financial identifiers, case numbers, or organizational references. This component interfaces directly with the user-facing application and forwards the raw data to subsequent processing units. To preserve context and ensure high-quality inference, the input is retained in its complete, natural form until obfuscation is performed. The input catcher unit can also support functions such as secure input validation, language detection, and simple preprocessing such as spell checking or format normalization to ensure consistency before the data protection transformation begins. Deterministic Semantic Obfuscation Engine (DSOE)

[0011] The DSOE represents a central component of the system, responsible for detecting and masking sensitive entities within the input data. It uses natural language processing (NLP) techniques, such as named entity recognition (NER) and part-of-speech tagging, to identify potentially sensitive tokens. Alternatively, it can apply custom rules or domain-specific keyword libraries. After identifying sensitive elements, the DSOE replaces them with contextual placeholders such as [NAME], [DATE], or [DISEASE_A]. These placeholders not only serve to anonymize but are also chosen to remain semantically compatible with the language patterns expected by the target LLM.The DSOE ensures that the obfuscated prompt remains grammatically correct and semantically logical to avoid misinterpretation or degradation of the response quality by the language model. Obfuscation Sensitive Prompt Encoder (OAPE)

[0012] After obfuscation, the Obfuscation-Aware Prompt Encoder (OAPE) adds a so-called "transformation fingerprint" to the modified input text. This fingerprint contains a secure mapping of the original sensitive tokens to their obfuscated counterparts and can be either embedded as metadata or stored in a local reference database. The fingerprint enables deterministic and reversible back-transformation after inference is complete, allowing sensitive terms to be accurately recovered. Crucially, this fingerprint is not transmitted to the cloud or the LLM provider, thus maintaining the confidentiality of the original data. In addition, the OAPE verifies the syntactic structure of the prompt and prepares it for secure transmission to the LLM interface. Cloud LLM interface

[0013] The Cloud LLM interface is responsible for communication between the privacy-preserving system and the cloud-based language model. It securely transmits the obfuscated prompt over encrypted HTTPS channels, thus protecting it from eavesdropping attempts during transmission. The interface can integrate authentication mechanisms and session management to interact with external APIs (e.g., from OpenAI, Google, or AWS). Crucially, the language model receives only the transformed version of the input and has no access to the original sensitive data. The interface stores the transformation fingerprint generated by the OAPE and links it to the active session to enable subsequent reverse transformation on the client side.

[0014] Local Result Restoration Layer (LRRL) Once the cloud-based LLM returns its obfuscated response, the Local Result Restoration Layer (LRRL) takes over further processing. This module decodes the received content by replacing the placeholder tokens with the original sensitive terms using the stored transformation fingerprint. In addition to the replacement, the LRRL performs semantic validation to ensure that the reconstructed response remains grammatically and contextually correct. The module is also capable of handling rewrites or structural changes to the LLM and intelligently places the restored tokens to preserve meaning and coherence.This post-processing step is essential to achieve a fully understandable and data-sensitive result that can be meaningfully interpreted by the user. Data Protection Auditors and Policy Enforcers (PAPE)

[0015] The Privacy Audit and Policy Enforcer (PAPE) ensures that all system processes comply with user-defined data protection policies and regulatory requirements. Users can define rules such as full redaction, generalization, or partial anonymization of sensitive terms – depending on the application domain, such as legal or healthcare. PAPE continuously monitors the entire transformation and inference pipeline and records detailed logs of every obfuscation, inference request, and recovery. These logs form an auditable audit trail that supports both internal audits and external regulatory requirements (e.g., GDPR, HIPAA). The module also detects irregularities in the handling of sensitive data and enforces access controls as needed. Functional process

[0016] The functional operation of the system can be illustrated using a typical use case. Suppose a user enters the following command: "Summarize John Doe's insurance fraud case."

[0017] After receiving this input, the input catcher unit forwards the text to the DSOE, which identifies the sensitive tokens "John Doe" and "insurance fraud." These are replaced with placeholders, resulting in the following obfuscated prompt: "Summarize [NAME]'s legal case for [CASE TYPE]."

[0018] The OAPE then creates a transformation map, e.g., [NAME] → John Doe, [CASE TYPE] → Insurance Fraud, and stores it locally while preparing the obfuscated prompt for secure transmission.

[0019] The Cloud LLM interface then transmits this sanitized input to the remote LLM. Suppose the LLM responds with: "The individual in question was involved in a case of [CASE TYPE] that raised serious concerns."

[0020] This response is passed to the LRRL, which reconstructs the original terms using the stored fingerprint to produce the final output: "John Doe was involved in an insurance fraud case that raised serious concerns."

[0021] The entire transaction—including the original input, placeholder assignment, and final output—is logged by the PAPE for audit and compliance purposes.

Claims

[1] A system for privacy-preserving LLM inference, comprising: an input catcher unit, a deterministic semantic obfuscation engine, an obfuscation-sensitive prompt encoder, a cloud LLM interface, a local result generation and recovery layer, and a data protection auditor and policy enforcer; where the deterministic semantic obfuscation engine replaces sensitive terms with domain-specific placeholders using a context-sensitive rule base or NER-based marking; and where the prompt encoder embeds metadata that allows deterministic recovery of the original content after inference. [2] The system of claim 1, wherein the local result generation and recovery layer decodes the LLM response using stored metadata and ensures the semantic coherence of the recovered output. [3] The system of claim 1, wherein the privacy auditor and policy enforcer enables configurable privacy profiles and logs all transformation operations for traceability. [4] The system of claim 1, wherein the obfuscation-sensitive prompt encoder generates transformation-specific fingerprints that are unique for each query session. [5] The system of claim 1, wherein the system supports real-time inference while ensuring that sensitive data in its original form never leaves the local environment.