Machine learning models with redacted input data

The hybrid AI framework addresses computational and confidentiality issues by enabling local data redaction and remote processing, ensuring efficient and secure AI inference on resource-constrained devices.

WO2026235681A1PCT designated stage Publication Date: 2026-11-12HEWLETT PACKARD DEVELOPMENT COMPANY LP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/US2025/027749
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2025-05-05
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Existing artificial intelligence (AI) systems face challenges in deploying high-parameter-count models on resource-constrained devices due to computational limitations, and cloud-based infrastructure introduces data confidentiality and integrity issues.

Method used

A hybrid AI framework enables collaborative inference between a local and remote ML model, where the local device redacts sensitive data and transmits redacted input to the remote model for processing, with the local model evaluating and refining the remote responses to maintain privacy and efficiency.

Benefits of technology

This approach preserves data confidentiality while leveraging remote computational power for high-quality inference, improving efficiency and reducing the computational burden on local devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure US2025027749_12112026_PF_FP_ABST
    Figure US2025027749_12112026_PF_FP_ABST
Patent Text Reader

Abstract

A computing device includes: a local ML model; a memory to store a set of input data received from a client; and a processor. The processor is to receive the set of input data; remove a subset of data from the set of input data to generate redacted input data; transmit the redacted input data to a remote ML model; obtain a set of draft responses generated based on the redacted input data from the remote ML model; evaluate the set of draft responses using the local ML model to select an individual draft response according to a likelihood that the local ML model would generate the individual response as a response to the set of input data; and output the individual response as the response to the set of input data to the client.
Need to check novelty before this filing date? Find Prior Art

Description

86379992 PATENTMACHINE LEARNING MODELS WITH REDACTED INPUT DATABACKGROUND

[0001] Artificial intelligence (Al) models have become widely adopted in computing systems (e.g., Al systems) as a means for assisting with tasks such as generating responses, analyzing complex datasets, or making predictions based on user input. The structure and operation of these Al models may vary depending on the intended application or deployment environment. For example, an environment may include a generative Al service, where a user provides prompts and receives responses generated by the Al model. The design and configuration of the Al model may differ depending on the specific use case, including factors such as the type of data being processed and the demanded level of accuracy and reliability.BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Various features will now be described with reference to the following drawings. Throughout the drawings, reference numbers may be re-used to indicate correspondence between referenced elements. The drawings are provided to illustrate examples described herein and are not intended to limit the scope of the disclosure.

[0003] FIG. 1 is a block diagram of an example of a computing device (e.g., local computing device) in accordance with the present disclosure;

[0004] FIG. 2 is a block diagram of an example of a computing device (e.g., remote computing device) in accordance with the present disclosure;

[0005] FIG. 3 is a block diagram of a computing device of FIG. 1, depicting interactions between a processor and a machine-readable storage medium;

[0006] FIG. 4 is a block diagram of a computing device of FIG. 2, depicting interactions between a processor and a machine-readable storage medium;

[0007] FIG. 5 is an interaction diagram between the local computing device (computing device of FIG. 1) and the remote computing device (computing device of FIG. 2) in accordance with the present disclosure;

[0008] FIG. 6 is a flow diagram depicting an example method implemented by the computing device of FIG. 1 in accordance with the present disclosure; and

[0009] FIG. 7 is a flow diagram depicting an example method implemented by the computing device of FIG. 2 in accordance with the present disclosure.DETAILED DESCRIPTION

[0010] Artificial Intelligence (“Al”) systems have advanced rapidly in recent years, enabling a wide range of applications. For example, Al systems implementing models based on deep learning architectures, such as large language models (LLMs), convolutional neural networks (CNNs), transformer-based models, and similar frameworks, have shown significant advancements across domains including natural language understanding, computer vision, speech recognition, autonomous systems, and medical diagnostics. As disclosed herein, an Al system can generally refer to a computing environment designed to perform tasks that typically require human intelligence, such as reasoning, perception, learning, and decision-making, by executing one or more Al models. These systems combine hardware and software components to enable the deployment, management, and execution of Al models in response to specific inputs or application demands. An Al model, as disclosed herein, generally refers to a computational structure, such as a machine learning or deep learning model that has been trained on data to perform a defined task, including classification, prediction, or natural language generation. These Al models, referred to as “deep learning” or “ML” models, can rely on complex architectures that demand substantial computational resources, such as processing large datasets. For the purpose of description, an Al model is also referred to herein as a “machine learning (ML)” model.

[0011] ML models often involve billions of parameters and require extensive matrix multiplications, attention mechanisms, and multi-stage data transformations to produce accurate outputs. As such, Al workloads are often computationally intensive and demanding substantial computing resources. In many deployment scenarios, such as on personal computing devices such as smartphones, laptops, wearable devices (e.g., smart glasses), and embedded Internet on Thing (loT) platforms, the available computing, memory, and storage are constrained due to factors such as power limitations, thermal constraints, and physical form factors. These constraints might preclude the deployment of high-parameter-count modelsdirectly on the device, which can lead to lower inference quality that may be insufficient for meaningful tasks. Furthermore, even if large models could be deployed, their inference speed may be too slow to support real-time task processing. Even higher-end computing devices such as desktops or workstations are often not equipped with the hardware necessary to perform large-scale Al inference efficiently or within acceptable latency bounds.

[0012] To overcome limitations of local computing capacity, many Al deployments leverage cloud-based infrastructure. In a cloud-based infrastructure, a computing device transmits input data to a remote data center that hosts large-scale computational resources. The remote data center can then execute inferences or training tasks and return an output to the computing device.

[0013] However, cloud-based infrastructure may introduce significant challenges related to data confidentiality, integrity, and system trust. The transmission of input data, including, for example, sensitive, personal, or proprietary information such as health records, biometric profiles, confidential legal documents, or corporate source code to a third-party cloud provider may inherently increase an attack surface. A malicious actor might compromise the data or exploit vulnerabilities within the cloud infrastructure to access or tamper with private data (e.g., confidential data).

[0014] To address these deficiencies, aspects of present disclosure provide a hybrid Al framework that can realize the performance benefits of remote computing while maintaining privacy-preserving characteristics of local computation. This disclosure introduces systems and methods that enable collaborative inference between a local ML model and a remote ML model. More specifically, in some examples of the present disclosure, a local computing device identifies and redacts sensitive components of input data to generate redacted input data. The redacted input data can then be transmitted by the local computing device to a remote ML model hosted on computer environment (e.g., a cloud datacenter) with greater computational resources relative to the local computing device. The remote ML model can process the redacted input data and return multiple candidate responses. In some cases, the candidate response can be a full response. In some examples, the candidate response can be response portion, such as phrases or partial responses, rather than complete response. Upon receipt, the local computing device can use a local ML model (which may be simpler or more complex than the remote ML model to determine which of the candidate responses would belikely to be output by the local ML model when prompted with the input data, to un-redact the candidate response as necessary, and to output a selected candidate response. In some cases, the remote ML model may be configured to use smaller models to generate faster draft responses or larger models to provide higher-quality drafts that are more likely to be accepted and better accommodate redacted inputs.

[0015] This approach can ensure that sensitive information does not leave the local environment in its unprotected form, thereby preserving data confidentiality. At the same time, the system can leverage the computational power of remote ML models to maintain inference quality and performance, mitigating the limitations of the local computing device. For example, the local computing device acts as both a security gatekeeper and a refinement engine, while the remote system serves as a high-throughput draft generator. Notably, operations ascribed herein to a local ML model (such as determining which of multiple candidate response provided by a remote ML model would be likely to be output by the local ML model if prompted with the input data) may be computationally more efficient than generating a response directly using the local ML model. For example, a local ML model may be capable of validating multiple tokens of a candidate response in a single forward pass of the local ML model, whereas the local ML model might otherwise require a full forward pass to generate each token. Accordingly, the hybrid local / remote architecture disclosed herein can operate significantly more efficiently than a purely local implementation. Examples of the present disclosure are therefore directed to an improvement in computer-related functionality — enabling high efficiency generation of content from input data including sensitive information without transmission of the sensitive information to a remote computing device and without relying solely on a local ML model to generate the content. Moreover, examples of the present disclosure provide for improvements to technical problems, such as the high resource requirements of generative Al models, the limited computational resources of local computing devices, and confidentiality or privacy risks associated with remote transmission of sensitive information. These technical problems are addressed by the technical solutions and improvements disclosed herein, including the use of a hybrid local / remote architecture to remove a need to transmit sensitive information to a remote computing device while enabling a local ML model to determine content that would have been generated by thelocal ML model according to input data, without requiring that the local ML model directly generate such content.

[0016] Aspects of the present disclosure relate to a computing device, comprising: a local ML model; a memory to store a set of input data received from a client; and a processor. The processor is to receive the set of input data; remove a subset of data from the set of input data to generate redacted input data; transmit the redacted input data to a remote ML model; obtain a set of draft responses generated based on the redacted input data from the remote ML model; evaluate the set of draft responses using the local ML model to select an individual draft response according to a likelihood that the local ML model would generate the individual response as a response to the set of input data; and output the individual response as the response to the set of input data to the client.

[0017] Aspects of the present disclosure relate to a non-transitory computer readable medium having program instructions that when executed cause a hardware processor of a computing device to: obtain a set of redacted input data from a client computing system, the set of redacted input data comprising input data modified to replace a subset of confidential information with redaction placeholders; prompt a generative artificial intelligence ML model with the set of redacted input data to generate a draft response to the set of the redacted input data, wherein the draft response corresponds to a predicted response to a prompt of the generative artificial intelligence ML model with the input data in un-redacted form; and transmit the draft response to the client computing system.

[0018] Aspects of the present disclosure relate to a non-transitory computer readable medium having program instructions that when executed cause a hardware processor of a computing device to: receive a set of input data as input to the computing device from a client; identify a subset of data from the set of input data as sensitive information; remove the identified subset of data from the set of input data to generate redacted input data; transmit the redacted input data to a remote ML model; obtain a plurality of responses generated based on the redacted input data from the remote ML model; evaluate the plurality of responses using a local ML model to select an individual response from the plurality of responses according to a likelihood that the local ML model would generate the individual response as a response to the set of input data; and output the individual response as the response to the set of input data to the client.

[0019] FIG. 1 illustrates a block diagram of a computing device 100, according to examples disclosed herein. The computing device 100 can refer to a device that can provide a user with access to its computing resources, for example, the computing device 100 can include such as a desktop, laptop, tablet computer, wearable computer, server, personal digital assistant (PDA), hybrid PDA / mobile phone, mobile phone, smartphone, and the like. The present disclosure does not limit the type of the computing device 100. As illustrated in FIG. 1, the computing device 100 can include a main processor 110, a memory 120, a non-volatile storage device 130, and an interface 140. For the purpose of description, the computing device 100 (and computing device 300 shown in FIG. 3) can be referred to as a local computing device.

[0020] The main processor 110 may be any type of processor that can provide dedicated computing resources for executing instructions retrieved from the memory 120. For example, the main processor 110 may include a central processing unit (CPU), a tensor processing unit (TPU), a neural processing unit (NPU), or any combination thereof.

[0021] In some examples, the main processor 110 includes or is associated with one or more graphics processing units (GPUs) (not shown in FIG. 1) that are connected to the main processor 110 (e.g., a hosting processing unit in these examples). In these examples, the computing device 100 may function as an Al accelerator, where the main processor 110 includes multiple GPUs configured to perform parallel computations based on instructions associated with, for example, a local ML model 122 stored in memory 120. The GPUs may be interconnected and managed by the main processor 110, where the main processor 110 can function as a host processor, which coordinates tasks including memory allocation, data scheduling, and data transfers between the memory 120 and the GPUs. This configuration can allow the computing device 100 to perform high-throughput, low-latency inference or training operations suitable for Al workloads.

[0022] As used herein, a central processing unit (CPU) can refer to a processing component that performs the processing of data by executing instructions, such as performing basic arithmetic, logic control, and input / output operations in accordance with the instructions. The CPU can have various architectures that dictate how the CPU processes data, executes instructions and communicates with other parts of the computer system. However, the present disclosure does not limit the CPU architecture. As used herein, a tensor processing unit (TPU) can generally refer to a processing unit (e g., a type of application-specific integrated circuit)specifically designed for accelerating Al workloads, such as handling computational requirements of executing or processing Al tasks. The TPU can include, without limiting, matrix multiplication units configured to perform matrix multiplications in accordance with the tasks of Al workload, memory configured to support data transfer demanded for Al workloads, and the like. As used herein, a neural processing unit (NPU) can generally refer to a processing unit specifically designed for accelerating processing Al workloads that involve neural networks. For example, the neural network can generally refer to a network having a plurality of nodes and layers, where each node (organized in specific layer(s)) processes data to perform the tasks (e.g., included in Al workload), such as data patter reorganization, data classification, output predictions, and the like. The NPU is designed to perform specific types of mathematical operations used in the neural network. The NPU can include a plurality of processing cores configured to execute multiple operations in the neural network parallelly. As used herein, a graphics processing unit (GPU) can refer to a processing unit designed to accelerate graphics rendering. The GPU can include a plurality of cores configured to perform parallel processing. The GPU can have various architectures based on its operation, such as parallel processing. In addition, the GPU can be implemented as a stand-alone processing unit or integrated with other processing units, such as the CPU. The present disclosure does not limit the types of GPU architecture and implementation of the GPU.

[0023] The memory 120 may include any type of memory capable of storing a local ML model 122 and its associated data. In some examples, the memory 120 is a volatile memory, such as dynamic random access memory (DRAM), which stores the parameters of the local ML model 122 and intermediate data used during computation by the main processor 110. These parameters can include, without limitation, weights, and biases. Weights are numerical values that represent the strength of the connections between neurons or nodes in different layers of the model. In a neural network, each input is multiplied by its corresponding weight, and the result influences the activation of the connected node. Biases are scalar values added to the weighted sum of inputs before applying an activation function, allowing the model to shift activation thresholds and capture more complex data patterns. The exact nature and role of these parameters depend on the specific type of ML model.

[0024] The local ML model 122 stored in memory 120 may be generated using various ML algorithms. These may include, but are not limited to, supervised and unsupervisedlearning techniques such as regression algorithms (e.g., Ordinary Least Squares Regression), instance-based algorithms (e.g., Learning Vector Quantization), decision tree algorithms (e.g., classification and regression trees), Bayesian algorithms (e.g., Naive Bayes), clustering algorithms (e.g., k-means clustering), association rule learning algorithms (e.g., Apriori), artificial neural networks (e g., Perceptron), deep learning models (e.g., Deep Boltzmann Machines), dimensionality reduction algorithms (e.g., Principal Component Analysis), and ensemble methods (e.g., Stacked Generalization). In one example, the local ML model 122 is a transformer-based model, such as an encoder-only model, a decoder-only model, or an encoder / decoder model. The present disclosure does not impose any limitation on the type or structure of the local ML model 122 that may be stored and executed by the computing device 100.

[0025] The non-volatile storage device 130 can include any type of data storage device capable of storing data, such as, for example, a flash memory, an electrically erasable programmable memory, and any type of non-volatile memory.

[0026] The interface 140 can include any type of interface that can obtain data. For example, the interface 140 can provide wireless or wired interface to communicatively couple with another computing device and / or cloud infrastructure (e.g., data center). The interface 140 can implement any commercially available wired and wireless communication protocols, and present disclosure does not limit the types of protocol. The interface 140 can also provide connection with peripheral devices, such as output devices (e.g., a display, audio output, haptic outputs, etc.) and input devices (e.g., keyboard, mouse, microphone, camera, etc.).

[0027] In some examples, the computing device 100 can obtain inputs (e.g., input query) from a user and provide response(s) by utilizing the local ML model 122 in accordance with aspects of present disclosure.

[0028] Illustratively, the computing device 100 can obtain input data from a user via the interface 140 such that the user can provide the input by utilizing peripheral devices connected to the interface 140. In some cases, the user can provide the input data via the wired or wireless connections provided by the interface 140.

[0029] In the context of the present disclosure, input data refers to any data provided to the computing device 100 (e.g., the local ML model 122) for the purpose of generating a response, performing inference, or executing a specific task. The input data mayinclude, but is not limited to, natural language text, structured or unstructured documents, files, images, numerical values, metadata, sensor readings, or encoded representations of content. In some examples, the input data may be user-provided content, such as a query, prompt, or instruction, which initiates the generation of a ML model output. The input data may also include contextual information or historical data relevant to the task. In some implementations, the input data may include sensitive or confidential information.

[0030] In some examples, the main processor 110 can receive the input data provided via the interface 140. Upon receiving the input data, the main processor 110 can process the received input data. For example, the main processor 110 can redact the input data. Illustratively, the main processor 110 can identify a subset of data from the input data and redact the input data by removing the subset of data. For example, the subset of data can include any types of confidential information, such as personal identification identifiers, personal confidential information, personal health information, or a combination thereof. In some cases, the local ML model 122 is trained to identify the subset of data from the input data and replace or remove the identified subset of data with a replaced mark (or symbol). For example, if the input data is provided as a query related to medical condition of a patient, having name, data of birth, medication, and weight data, the main processor 110 (e.g., by utilizing the local ML model 122) can identify the name, data of birth, and weight data and replace these data with general placeholders. In other cases, the main processor 110 can redact input data independent of the local ML model 122, such as by application of a distinct ML model or non-machine-learning based approaches (e.g., regular expressions, rules, or the like).

[0031] In some instances, during redaction, the main processor 110 may assign a unique placeholder value to each subset of data and replace the subset of data with the corresponding placeholder values. For example, the computing device 100 may locally maintain a dataset of confidential information, with each entry associated with a unique placeholder value. In some cases, the unique placeholder value is stored as unique token. The main processor 110 can identify the subset of data by comparing the input data against this dataset. The non-volatile storage device 130 may store the dataset of confidential information and the corresponding placeholder values (e.g., tokens), allowing the main processor 110 to accurately map the subset of data (e.g., redacted data) to its unique tokens.

[0032] In some cases, the main processor 110 can transmit the redacted input data to a remote computing device (e.g., computing device 200 illustrated in FIG. 2) such as a remote ML model 222 of the computing device 200. In these cases, the computing device 200 can generate a set of draft responses in accordance with examples disclosed with respect to FIG. 2, and the computing device 100 can obtain the set of draft responses via the interface 140. The set of drafts can include various sequences of response drafts generated from the remote computing device 200 by utilizing the remote ML model 222. In some examples, the remote ML model 222 may be more complex than the local ML model 122. For instance, the remote ML model 222 may perform deeper learning by utilizing a greater number of layers and nodes. This increased complexity can enable the remote ML model 222 to generate higher-quality draft responses based on redacted input data, which can then be further evaluated or completed by the local ML model 122. In one example, the remote ML model 222 and the local ML model 122 share an architecture but vary in the number of parameters (e.g., with the local ML model 122 having fewer parameters). For example, the remote ML model 222 and the local ML model 122 may be two models in the same family of models, such as an 80 billion parameter model and an 8 billion parameter model, respectively. In some cases, the remote ML model 222 may be configured to use smaller models to generate faster drafts or larger models to provide higher-quality drafts that are more likely to be accepted and better accommodate redacted inputs.

[0033] In some examples, the main processor 110 receives a set of draft responses and evaluates the draft responses to select a draft response from the set of draft responses. Each individual draft response may include a specific sequence of tokens, where the sequence represents a proposed response generated by a remote ML model. Each token in the sequence may correspond to a word, a portion of a word, a punctuation mark, a space, a symbol, or similar language element. In some cases, the draft response may include placeholder tokens that represent redacted or removed portions of sensitive input data (e.g., the subset of data identified from the input data). In one example, the main processor 110 reverts redaction prior to evaluation by replacing placeholder tokens with corresponding pieces of sensitive information. The non-volatile storage device 130 may store a mapping information of redacted data subsets and their associated unique tokens (e.g., as generated during redaction), enabling accurate reconstruction of the original response, using this mapping, the processor 110 cansystematically traverses the draft responses, detects placeholder tokens inserted during redaction, and replaces each placeholder with the corresponding subset data based on the mapping information. In some implementations, the main processor 110 can concurrently evaluate two or more draft responses (e.g., sequences of tokens) in parallel to improve evaluation efficiency.

[0034] In some examples, the main processor 110 evaluates the set of draft responses using a local ML model to determine a likelihood that the local ML model generates each draft response given the original input data. Based on this evaluation, the main processor 110 selects the draft response based on the likelihood. In cases where the draft responses contain redacted tokens, the main processor 110 may perform an un-redaction step (e.g., prior to evaluation by the local ML model) by reinserting the removed subset of input data into the corresponding positions within the draft responses. For example, if a sensitive term such as a patent inventor's name “John” was redacted and replaced by a unique token (e.g., “1234”), the main processor 110 may retrieve the corresponding value from a non-volatile storage device 130. The non-volatile storage device 130 may store a mapping of redacted data subsets and their associated unique tokens (e.g., as generated during redaction), enabling accurate reconstruction of the original response. In some cases, the un-redaction step can be performed in prior to evaluating the draft response.

[0035] In some implementations, the evaluation of draft responses is performed at the token level through evaluation process. Evaluation may proceed iteratively with respect to token sequence, each of which represents multiple tokens generated by the remote ML model 222. Specifically, the local ML model 122 may evaluate, in parallel, each sequence of tokens to determine a probability that the local ML model 122 would have generated the sequence of tokens based on the prior context (e.g., the input data and any previous tokens, such as previously approved tokens). Notably, evaluation of each sequence of tokens may need only a single forward pass through the local ML model 122, whereas independent generation of tokens by the local ML model may require a single pass per token. Accordingly, evaluation of sequence of tokens may be significantly more computationally efficient than independent generation of tokens. The acceptance of a token sequence may be based on a predefined probability threshold (e.g., approval rate), which may vary in strictness depending on system configuration. In some cases, the threshold may be dynamically adjusted based on theperformance capabilities or available computing resources of the main processor 110. For instance, a computing device 100 with a high-performance main processor 110 may enforce stricter evaluation criteria, whereas a device with limited computational resources may adopt more relaxed criteria to improve throughput.

[0036] For example, if a token sequence in a draft response satisfies the selected evaluation criteria, the token sequence is accepted, and the evaluation process continues to the next token sequence using an updated context that includes the accepted token sequence. In one example, token sequences are evaluated iteratively, and thus may be partially accepted. For example, each token in a token sequence may be evaluated and a probability assigned for the token by the remote ML model 222 may be compared to the probability assigned for the token by the local ML model 122. If the token satisfies acceptance criteria, evaluation may proceed to a next token. However, if a token fails to meet the criteria such that it does not match the expected prediction of the local ML model 122, the evaluation process is halted at that point. The unverified portion of the draft, beginning from the mismatched token, is discarded. The local ML model may then resume generation from the last verified token within the partially accepted or wholly rejected token batch. Notably, while token batches may be evaluated iteratively, such iteration may still only require a single forward pass through the local ML model 122 (to generate multiple probability distributions used during evaluation), thus preserving computational efficiency of this approach. In some cases, multiple tokens in a token sequence can be evaluated concurrently.

[0037] In various examples, the main processor 110 can iteratively evaluate the token sequences, and the frequency of iteration can be determined based on the approval rate. The frequency of iteration may also be influenced by the size of each token sequence. The token sequence size can be dynamically adjusted based on factors such as the complexity of the draft response or outcomes from prior evaluation attempts. Larger token sequence enable the system to evaluate more content per iteration, thereby reducing the number of forward passes required through the local ML model 122. However, when a token in a larger sequence fails to meet the acceptance criteria, the system may need to discard a larger portion of the draft and regenerate from an earlier context, potentially increasing the overall frequency of iteration. In contrast, smaller token sequence allow the main processor 110 to detect failureswith greater precision and reduce the amount of discarded content, which may help control and stabilize the rate of regeneration during the evaluation process.

[0038] FIG. 2 illustrates a block diagram of a computing device 200, according to examples disclosed herein. The computing device 200 may refer to a remote computing infrastructure or a computing node hosted in a data center that provides computing resources for local computing device or distributed computing devices such that the local computing device generally refer to the computing device 100 illustrated in FIG. 1. For example, the computing device 200 may be implemented as a server, rack-mounted system, or part of a larger distributed cluster configured to support remote processing tasks, including but not limited to artificial intelligence (Al) workloads, ML inference, data analytics, and high-performance computing operations. The present disclosure does not limit the form or architecture of the computing device 200. As illustrated in FIG. 2, the computing device 200 can include a main processor 210, a memory 220, and an interface 230. For the purpose of description, the computing device 200 (and computing device 400 shown in FIG. 4) can be referred to as remote computing device. While a single computing device 200 is shown in FIG.2, in some instances a remote ML model 222 may be implemented across multiple computing devices 200.

[0039] The main processor 210 may be any type of processor that provides dedicated computing resources for executing instructions retrieved from the memory 220. For example, the main processor 210 may include any one of a CPU, a TPU, and an NPU. In some cases, the main processor 210 can include multiple processing units of the CPUs, TPUs, and NPUs, and the present disclosure does not limit types and numbers of the processing units. In some implementations, the computing device 200 can include multiple GPUs (not shown in FIG. 2) connected with the main processor 210 (e.g., function as a host processor in this example). In such configurations, the computing device 200 may function as a ML accelerator node, where the main processor 210 includes multiple GPUs configured to perform concurrent, high-throughput computations. These computations may be based on instructions associated with a remote ML model 222 stored in memory 220. The GPUs may be managed by a host processor, such as a CPU, which coordinates tasks including memory management, data scheduling, and data transfers between memory 220 and the main processor 210 (e.g., multiple GPUs). This architecture can enable the computing device 200 to perform scalable inferenceor training tasks suitable for deployment in cloud-based or on-premise data center environments.

[0040] The memory 220 may include any type of memory capable of storing a remote ML model 222 and its associated data. In some examples, the memory 220 is a nonvolatile memory, which stores the parameters of the remote ML model 222 and intermediate data used during computation by the main processor 210. These parameters can include, without limitation, weights, and biases. In some implementations, these parameters represent weights and bias used for the remote ML model 222. For example, the main processor 210 can prompt the remote ML model 222 with a set of redacted input data received from the computing device 100 (e.g., the local computing device). In these examples, the remote ML model 222 and its parameters can be designed to generate draft responses based on the redacted input data. For instance, the remote ML model 222 can (e g., via executing the model by the main processor 210) process the redacted input data through a series of computational layers to produce a sequence of output tokens such that each layer of the remote ML model 222 can include a set of learned parameters, the weights and biases, which define how the model transforms input data at each stage of computation.

[0041] In some examples, where the remote ML model 222 implements a neural network model, having multiple layers and nodes in each layer, the weights can represent numerical values that represent the strength of connections between nodes in adjacent layers of the neural network. For example, during draft generation, each token in the input sequence is embedded into a vector space and propagated forward through the network layers. At each node (e.g., neuron) in a given layer, the input values are multiplied by the corresponding weights associated with that node’s incoming connections. These weighted values are summed and adjusted by an additional parameter known as bias. The bias serves as a constant term added to the weighted sum and enables the model to shift activation thresholds, improving its ability to model complex patterns in the data. The result is then passed through an activation function (e.g., softmax in output layers) to produce a probability distribution over the model's vocabulary. The model selects one or more tokens with the predicted likelihood to form a portion of the draft response. This process is repeated iteratively — taken by the token where the previously generated tokens are fed back into the model as context for generating the next token. As such, the sequence of output tokens forming the draft is a direct result of applyingthe model’s weights and biases to the input and intermediate activations across the model’s layers. Accordingly, the content and structure of the generated draft response are governed by the learned weights and biases, which encode the statistical relationships and semantic patterns present in the training data. These parameters enable the ML model to generalize from past data and generate coherent, contextually relevant responses based on a given input prompt. Such neural network model implemented in the remote ML model 222 is illustrated as examples without limiting the types of remote neural network model 222. The present disclosure does not limit the types of the remote neural network model 222, and any suitable supervised and unsupervised learning techniques and their algorithms can be implemented.

[0042] The interface 230 can include any type of interface that can obtain data from the computing device 100 (e.g., local computing device). For example, the interface 230 can provide wireless or wired interface communicatively couple with the computing device 100. The interface 230 can implement any commercially available wired and wireless communication protocols, and present disclosure does not limit the types of protocol.

[0043] In some examples, the computing device 200 can obtain inputs from the computing device 100 via the interface 230. The inputs can include a set of redacted input data from the computing device 100. For example, the computing device 100 can generate the set of redacted input data by redacting the input data (e.g., the input received from a customer at the computing device 100) such that the input data modified to replace the subset of confidential information with redaction placeholders.

[0044] The main processor 210, upon obtaining the set of redacted input data, can prompt the remote ML model 222 with the set of redacted input data to generate a draft response to the set of the redacted input data.

[0045] In some examples, generating a draft response include executing the remote ML model 222 such that the remote ML model 222 can be a generative ML model. In these examples, the main processor 210, by executing the remote ML model 222, can generate the draft response based on a given set of redacted input data. For example, the redacted input data can be processed by the remote ML model 222 as a prompt and the generated draft responses are generated a sequence of output tokens. In some cases, the draft response can correspond to a predicted response to the prompt. In some examples, the main processor 210, by executing the remote ML model 222, can generate two or more draft responses. In one instance, eachdraft response is generated iteratively as a set of token sequence, each token sequence including multiple next tokens of the draft response.

[0046] In some examples, the computing device 200 can obtain a secondary input representing size of a token sequence (e.g., number of tokens) from the computing device 100. For example, the size of the token sequence and the computing resource of the computing device 100 are proportionally related such that a larger size of token sequence may demand higher computing powers of the computing device 200. In some cases, the main processor 210 can iteratively prompt the generative artificial intelligence matching learning model to generate each token sequence of the draft response in increments without exceeding the draft response length (e.g., token sequence limit in the draft response) and transmit each token sequence of the generated draft response to the computing device 100 after each iteration. In one example, probability distributions (e.g., logits) for each token within a token sequence is further transmitted from the remote ML model 222 to the local computing device. As one illustrative example, the main processor 110 (by executing instruction generated from the local ML model 122) executes a single forward pass over the token sequence to generate a probability distribution p Jocal(t) for each potential tokens t. Simultaneously, or in a preceding step, the main processor 210 (e.g., at the remote computing device by executing instruction generated from the remote ML model 222) generates its own distribution p remote (t) for the same tokens in the draft response. Upon receiving the remote ML model’s proposed token t draft, the system compares p local (t draft) with p remote(t draft) to determine whether t remote should be accepted. If t remote is also the most likely token predicted by the local ML model 122 (e.g., direct match) such that it has the highest probability among all tokens in the local ML model’s distribution, acceptance occurs automatically. If t remote is not the top-ranked token locally (e.g., probabilistic acceptance) a probabilistic acceptance criterion is applied. For example, the system calculates a ratio of p local (t draft) to p remote(t draft) and caps this ratio at Isuch that the token is accepted with probability min(l, p local (t draft) / p draftft draft)). For example, if the local ML model’s probability for t draft is 0.9 while the remote ML model’s probability is 1.0, the token is accepted with 90% probability. Conversely, if the remote ML model’s probability is lower (e.g., 0.7 versus 0.9 locally), the ratio exceeds 1, so the system accepts the token with 100% probability. This mechanism provides a balanced approach that prioritizes agreement with the local ML model’s top choice but allows forcontrolled acceptance of tokens that differ slightly — thereby preserving efficiency by leveraging a single forward pass yet ensuring alignment between the local and remote ML model outputs.

[0047] In some examples, the set of redacted input data further includes the placeholders (e.g., unique tokens) assigned during the redaction performed by the local computing device (e.g., the computing device 100 shown in FIG. 1). For example, in the local computing device (the computing device 100 shown in FIG. 1), the non-volatile storage device 130 may store a mapping of these redacted subset of data to their associated unique tokens (e.g., as generated during redaction by replacing the subset of data), thereby enabling accurate un-redaction of the draft responses generated from the remote computing device (e.g., computing device 200 shown in FIG. 2).

[0048] In some examples, the computing device 200 can receive feedback from the computing device 100 regarding completeness of previously generated draft responses and store the feedback for future remote ML model 222 refinement. In other examples, where the computing device 200 generates a plurality of draft responses, the computing device 200 can receive feedback for each draft response of the plurality of responses and stores feedback for future remote ML model 222 refinement.

[0049] In some examples, the main processor 210 can generate a notification if the set of redacted input data includes un-redacted confidential information and request confirmation from the computing device 100.

[0050] In some examples, where the computing device 200 generates a plurality of draft responses, the computing device 200 can select a subset of the plurality of draft responses based on one or more predefined selection criteria. The selection criteria may include, for example, a relevance score that quantifies the similarity between each draft response and one or more previously validated responses. The relevance score may be determined by using semantic similarity metrics, vector embedding comparisons, or statistical models trained to evaluate alignment with accepted outputs. Based on the relevance scores and other criteria, the processor identifies and selects the contextually appropriate or likely accurate draft responses from the plurality of generated candidates. The selected subset is then transmitted to the computing device 100 for display, review, or further processing. This selection process enhances system efficiency by reducing the volume of output transmitted and improvesresponse quality by prioritizing drafts with higher relevance to known-valid patterns or user preferences.

[0051] FIG. 3 illustrates a block diagram of a computing device 300, depicting interactions between a processor 310 and a machine-readable storage medium 350. For purposes of illustration, the processor 310 may be the same as, or functionally similar to, the main processor 110 described with reference to FIG. 1. Additionally, the computing device 300 can refer to the local computing device, as disclosed herein. Furthermore, the remote computing device refers to the computing device 200 shown in FIG. 2 and computing device 400 shown in FIG. 4.

[0052] In some examples, the machine-readable storage medium 350 can be a nonvolatile storage medium storing various instructions executable by the processor 310 to perform the examples as disclosed herein.

[0053] In some examples, the processor 310 executes an input data request instruction 352. Upon execution of the input data request instruction 352, the processor 310 may initiate a request for a user to provide input data. For instance, executing the instruction may result in the display of a graphical user interface or the execution of an application capable of receiving the input data from the user. The present disclosure does not limit the type of interface or application used to receive such input data.

[0054] In some examples, the processor 310 executes an input data processing instruction 354. When executed by the processor 310, the input data processing instruction 354 enables the identification of a subset of data within the input data, which may include sensitive or confidential information. The identified subset of the data may comprise, for example, personally identifiable information, private health information, or other confidential content. For instance, if the input data includes a user query related to a medical condition and contains a patient’s name, date of birth, prescribed medications, and body weight, the processor 310 can identify the name, date of birth, and weight as confidential data forming the subset. This identification may be performed using various techniques. For example, the processor 310 may use a machine learning model (e.g., the local ML model 122 shown in FIG. 1) such that the local ML model 122 is trained to recognize patterns associated with confidential or sensitive information (e.g., private information and personal identifiable information). In other implementations, the processor 310 may apply a rules-based approach that uses predefinedpatterns (e.g., regular expressions or keyword dictionaries) to detect data fields related to the confidential and sensitive data, such as names or numerical values associated with birth dates or weight measurements, and the like. Alternatively, a distinct ML model (e.g., different from the local ML model 122 can be used for general inference) can be implemented specifically for detecting and classifying confidential or sensitive information within the input data.

[0055] In some implementations, the processor 310 executes an input data redaction instruction 356. The processor 310, upon executing the input data redaction instruction 356, may redact the identified subset of confidential data. Redaction may be performed by either removing the subset of data or replacing it with placeholder tokens or markers. For example, in the case where the input query includes personal health data, the processor 310 may replace the name, date of birth, and weight with unique placeholder tokens, thereby generating a redacted version of the input data suitable for further processing. In for illustrations, the processor 310, by executing the input data redaction instruction 356, may assign a unique placeholder value to each subset of data and replace the subset of data with the corresponding placeholder values. For example, the computing device 300 may locally maintain a dataset of confidential information, with each entry associated with a unique placeholder value. In some cases, the unique placeholder value is stored as unique token such that the machine-readable storage medium 350 (or external non-volatile storage device in communication with the storage medium 350) may store the dataset of confidential information and the corresponding placeholder values (e.g., tokens), allowing the processor 310 to accurately map the subset of data (e.g., redacted data) to its unique tokens

[0056] In some examples, the processor 310 executes a redacted input data transmission instruction 358. Upon execution, the processor 310 transmits the redacted input data to a remote computing device.

[0057] In some implementations, the processor 310 executes an evaluation instruction 360. The processor 310, when executing the evaluation instruction 360, can evaluate a set of draft responses received from the remote computing device. Each draft response may comprise a sequence of tokens, where each token corresponds to a word, subword, punctuation mark, space, symbol, or other language element. In some examples, the draft responses may contain placeholder tokens (the assigned tokens in placement of the subset of data during the redaction) corresponding to the redacted content of the original input data.

[0058] In scenarios where the draft responses contain placeholder tokens, the processor 310 may perform the evaluation by un-redacting (e.g., by reinserting the redacted data into the appropriate positions) prior to evaluation of the draft responses. For example, the processor 310 can un-redact the draft responses by replacing the redaction placeholders with the corresponding subsets of confidential information based on the mapping which associates each unique placeholder token with its original subset of data (e.g., redacted data). In some cases, using this mapping, the processor 310 can systematically traverses the verified draft response, detects placeholder tokens inserted during redaction, and replaces each placeholder with the corresponding confidential data. Thus, the un-redacted draft responses can accurately reflect the original, un-redacted input while maintaining data security throughout the earlier stages of generating draft responses by the remote computing device 200 shown in FIG. 2. In some cases, the placeholder tokens included in the draft responses (redacted draft responses) preserve positional and type metadata, the un-redaction can be performed in contextually accurate locations within the draft response, allowing reintegration of the confidential information. In some cases, the processor 310 may identify and replace placeholder tokens generated by the remote ML model during draft generation.

[0059] In some examples, the processor 310 evaluates the set of draft responses using to determine the likelihood that the local ML model 122 (shown in FIG. 1) would generate each draft response. The processor 310 may select the draft response with the likelihood.

[0060] In some implementations, the evaluation may proceed iteratively with respect to token sequence, each of which represents multiple tokens generated by the remote computing device. Specifically, the processor 310 may evaluate each sequence and determine a probability that the local ML model 122 would have generated the sequence of tokens based on the prior context (e.g., the input data and any previous tokens, such as previously approved tokens). Notably, evaluation of each sequence of tokens may require only a single forward pass through the local ML model 122, whereas independent generation of tokens by the local ML model 122 may require a single pass per token. Accordingly, evaluation of tokens may be significantly more computationally efficient than independent generation of tokens. The acceptance of a token sequence may be based on a predefined probability threshold (e.g., approval rate), which may vary in strictness depending on system configuration. In some cases,the threshold may be dynamically adjusted based on the performance capabilities or available computing resources of the processor 310.

[0061] If a token sequence in a draft response satisfies the selected evaluation criteria, the token sequence is accepted, and the evaluation process continues to the next token sequence using an updated context that includes the accepted token sequence. In one example, token sequence are evaluated iteratively, and thus may be partially accepted. For example, each token in a token sequence may be evaluated and a probability assigned for the token by the remote computing device (e.g., the remote ML model 222) may be compared to the probability assigned for the token by the local ML model 122. If the token satisfies acceptance criteria, evaluation may proceed to a next token. However, if a token fails to meet the criteria such that it does not match the expected prediction of the local ML model 122, the evaluation process is halted at that point. The unverified portion of the draft, beginning from the mismatched token, can be discarded. The local ML model 122 may then resume generation from the last verified token within the partially accepted or wholly rejected token sequence. Notably, while token sequence may be evaluated iteratively, such iteration may still only require a single forward pass through the local ML model 122 (to generate multiple probability distributions used during evaluation), thus preserving computational efficiency of this approach.

[0062] In various examples, the main processor 310 can iteratively evaluate the token sequence, and the frequency of iteration can be determined based on the approval rate. The approval rate may reflect how many tokens within a token sequence meet the predefined acceptance criteria. The frequency of iteration may also be influenced by the size of each token sequence. For example, larger token sequence can enable evaluation of more content per iteration, for example, reducing the number of forward passes. However, if the token sequence in the larger sequence fails to meet acceptance criteria, the main processor 310 may discard a more substantial portion of the draft response (e.g., corresponding to the larger token sequence), potentially increasing the overall iteration frequency. In another example, smaller token sequence can allow the main processor 310 to detect failures with greater granularity, reduce discarded content, and help maintain a more stable regeneration rate during the evaluation process.

[0063] In some illustrations, the local computing device 300 can dynamically adjust token sequence size based on factors such as the complexity of the draft response and / orthe computational constraints of the processor 310. In implementations involving the remote computing device (e.g., the computing device 200 in FIG. 2 or 400 in FIG. 4) for generating draft responses, the local computing device 300 may transmit control signals to the remote computing device to modify the sequence size of subsequent token sequence. In some cases, if the approval rate drops below a threshold, the local computing device 300 may transmit the signal to the remote computing device to reduce the sequence size to improve the evaluation precision and minimize the discard rates. Alternatively, if the approval rate is higher, the local computing device 300 may transmit signal to the remote computing device to increase the sequence size. In some cases, the processor 310 can transmit (e.g., by executing the evaluation instruction 360) these control signals to the remote computing device via the interface 140 shown in FIG. 1. In some cases, the local computing device 300 may transmit control signals to the remote computing device to modify the sequence size of subsequent token sequence. In some cases, the local computing device 300 may also request the generation of additional candidate draft responses.

[0064] In some examples, the processor 310 executes a selected response output instruction 362. Upon execution, the processor 310 outputs the selected response to the user, for example via a graphical interface, communication channel, or external device, depending on the implementation environment.

[0065] FIG. 4 illustrates a block diagram of a computing device 400, depicting interactions between a processor 410 and a machine-readable storage medium 450. For purposes of illustration, the processor 410 may be the same as, or functionally similar to, the main processor 210 described with reference to FIG. 2. Additionally, the computing device 400 can refer to the remote computing device, as disclosed herein. Furthermore, the local computing device refers to the computing device 100 shown in FIG. 1 and computing device 300 shown in FIG. 3.

[0066] In some examples, the machine-readable storage medium 450 is a nonvolatile storage medium configured to store the instructions 452 and 454, and the processor 410 can execute the instructions 452 and 454..

[0067] In some implementations, the processor 410 executes a draft response generation instruction 452 to generate a draft response based on a given set of redacted input data received from a local computing device (e.g., computing device 100). In some examples,executing the draft response generation instruction can enable utilization of a distinct ML model, such as the remote ML model 222 shown in FIG. 2. For example, the redacted input data is processed by the remote ML model 222 as a prompt, and in response, the remote ML model 222 generates one or more draft responses comprising sequences of output tokens. Each draft response corresponds to a predicted response that the remote ML model 222 would generate based on the redacted prompt.

[0068] To initiate generation, the prompt is tokenized into a sequence of discrete units referred to as tokens, where each token may represent a word, subword fragment, punctuation mark, whitespace character, or other language element. The tokenized prompt is input into the remote ML model 222, which includes multiple layers of computational units. These units process the tokens by applying learned weights and biases. At each step, the model produces a probability distribution over a vocabulary of possible next tokens and selects one or more tokens based on that distribution using either deterministic decoding or stochastic sampling. In certain examples, redaction may apply to non-textual inputs before tokenization. For instance, in the case of a prompt that includes an image, facial features of individuals may be blurred, masked, or replaced prior to tokenization to protect personal identity information. The redacted image, along with any accompanying text, is then processed by the remote ML model 222 to generate outputs (e.g., redacted non-textual inputs). In some scenarios that a prompt that includes an image containing text (e.g., through optical character recognition (OCR)), sensitive textual information identified within the image may be blurred, masked, or replaced prior to tokenization to protect confidential or personal information. The redacted image, along with any accompanying text, is then processed by the remote ML model 222 to generate draft responses.

[0069] This token generation process can be performed iteratively, with each newly generated token appended to the existing prompt to serve as context for predicting the next token. This iterative loop continues until a termination condition is met, such as reaching a predefined maximum draft response size, generating an end-of-sequence token, or completing a semantically coherent response. For example, the computing device 400 may transmit tokens in sequence to the local computing device. Each token can be associated with a unique positional identifier to maintain the generation order and context integrity. As disclosed with reference to FIG. 3 (e.g., the processor 310 by executing the evaluation instruction 360), thelocal computing device can evaluate each token (or sequence of tokens) against evaluation criteria and sends a control signal back to the remote computing device (e.g., with indication of the position of the last approved token). For example, if the remote computing device transmits tokens 1-10 and the local computing device approves only tokens 1-5, it responds with a signal identifying “5” as the last approved token. In response, the remote computing device (e.g., the processor 410 by executing the draft response generation instruction 452) discards tokens 6-10 and resumes generation using the context up to and including token 5. This iterative loop can continue until a termination condition is met, such as reaching a predefined maximum token sequence size, generating an end-of-sequence token. Additionally, the local computing device can influence the behavior of the remote computing device in generating the draft responses, for example, by specifying the token sequence size (e.g., how many tokens should be generated) and transmitted before the remote computing device pauses to await approval. For instance, after repeated discards, the local computing device may transmit a control signal to the remote system indicating to reduce the token sequence size. Conversely, if tokens are consistently accepted, the local computing device may allow larger sequence to increase the token sequence size. After generating one or more draft responses by executing the draft response generation instruction 452, the processor 410 may further execute a draft response transmission instruction 454. This instruction causes the processor 410 to transmit the generated draft responses to the local computing device (e.g., computing device 100) for further processing, such as evaluation, as illustrated in FIGs. 1 and 3.

[0070] In some examples, the generated draft response may represent a redacted version of a predicted output. In such cases, the remote ML model 222 can generate a coherent approximation of what it would have produced had the original (un-redacted) input data provided. The redacted input may include placeholders, and the remote ML model 222 may be trained or fine-tuned to produce meaningful outputs despite missing sensitive content (e.g., the redacted data).

[0071] In some examples, the processor 410 by executing the draft response generation instruction 452 (e.g., by further utilizing the remote ML model 222) can generate multiple draft responses in parallel. As disclosed herein, the parallel generation of the draft responses can refer to generating the draft responses during overlapping time intervals, for example, across separate cores of the processor 410. For example, the redacted input (receivedfrom the local computing device) can be “<NAME_SA> complains of mild joint pain. Provide dosage guidance for <NAME_SA>, born <DOB_SA>, currently taking <MED_A> and <MED B>, with a body -weight of <WT SA> kg,” where placeholders protect the patient’s name, date of birth, medications, and body weight. Upon receiving this redacted input, the remote ML model 222 may parallelly generate several drafts: “In this age, <MED_A> or <MED_B> is recommended 20mg or 30mg, respectively;” “In this weight, <MED_A> or <MED_B> is recommended lOmg or 30mg, respectively;” and “For the person taking the <MED_A> and <MED_B>, <MED_A> or <MED_B> are recommended 25mg or 35mg, respectively.” The computing device 400 may transmit these parallelly generated draft responses to the local computing device, where the local computing device evaluates each draft response (e.g., the computing device 300 by executing the evaluation instruction 360 shown in FIG. 3).

[0072] One might recommend a standard weight-based dosage and routine monitoring of liver enzymes; another might suggest a different initial dose with instructions to adjust therapy if creatinine clearance falls; a third could highlight contraindications tied to hepatic impairment while proposing alternative regimens. The local device validates each draft for completeness, safety, and semantic accuracy — verifying, for example, that dosage ranges match established clinical guidelines — then either adopts the best-qualified draft or synthesizes a final response by merging approved segments from the various drafts. Generating alternative outputs in parallel in this way preserves response diversity, robustness, and throughput even when personal medical details are withheld from the remote ML model.

[0073] In some implementations, the remote ML model 222 may receive an additional input representing a draft response length from the local computing device (e.g., computing device 100). The draft response length may reflect the local computing device’s computing capacity, as longer sequences may require increased memory or processing power. Based on the draft response length, the processor 410 can iteratively prompt the generative ML model to generate portions of the draft response in incremental segments that do not exceed the specified limit. After each generation iteration, the processor 410 may transmit the corresponding portion of the draft response back to the local computing device (e.g., computing device 100).

[0074] In one example, probability distributions (e.g., logits) for each token within a token sequence is further transmitted from the remote ML model 222 to the local computing device.

[0075] In some examples, the remote ML model 222 is further configured to receive feedback from the local computing device (e g., computing device 100) regarding the completeness or quality of a specific draft response. This feedback can be stored in association with the draft response and used to refine or update the remote ML model 222 over time.

[0076] In some cases, the processor 410 may generate a notification if the received set of redacted input data is determined to include un-redacted confidential information. Upon detection, the processor 410 may prompt the local computing device (e.g., computing device 100) to confirm whether the un-redacted content was intentionally included replaced with redacted placeholders.

[0077] FIG. 5 illustrates an interaction diagram between the local computing device 500 (e.g., computing devices 100 and 300, as illustrated in FIGs. 1 and 3, respectively) and the remote computing device 550 (e.g., computing devices 200 and 400, as illustrated in FIGs. 2 and 4, respectively). For the brevity of the description, FIG. 5 is described with reference to FIGs. 1-4.

[0078] At block 502, the local computing device 500 redacts input data. For example, the local computing device 500 can receive the input data from a user and identify a subset of data within the input data, which may include sensitive or confidential information. The identified subset of data may comprise, for example, personally identifiable information, private health information, or other confidential content. For example, the local computing device 500 may redact the identified subset of confidential data by either removing the subset of data or replacing it with placeholder tokens or markers. For example, the local computing device 500 performs the redaction as described with respect to FIG. 3, for example, the processor 310 executing the input data processing instruction 354.

[0079] At block 504, the local computing device 500 transmits the redacted input data to the remote computing device 550.

[0080] At block 506, the remote computing device 550 can generate draft responses by processing the redacted input data. For example, the remote ML model 222 can process the redacted input data as a prompt, and in response, the model generates one or more draftresponses comprising sequences of output tokens. Each draft response corresponds to a predicted response that the model would generate based on the redacted prompt. For example, the remote computing device 550 generate draft responses as described with respect to FIG. 4, for example, the processor 410 executing the draft response generation instruction 452.

[0081] At block 508, the remote computing device 550 transmits the draft responses to the local computing device 500.

[0082] At block 510, the local computing device 500 evaluates the draft responses. For example, the local computing device 500 performs the un-redaction and evaluation of the draft responses as described with respect to FIG. 3, for example, the processor 310 executing the evaluation instruction 360.

[0083] In some examples, after selecting draft responses based on the evaluation results performed at block 510, the local computing device 500 can initiate a re-request to the remote computing device 550 to generate an additional set of draft responses. In these examples, the local computing device 500 can again perform the evaluation process as described in block 510.

[0084] n some cases, after selecting draft responses based on the evaluation results performed at block 510, the local computing device 500 can further redact the input data 502 before transmitting it to the remote computing device 550 for generating new draft responses. For example, if the initially generated draft responses 506 include confidential information (e.g., inferred through processing), the local computing device 500 can identify such confidential information and re-redact the input data 502 to remove the corresponding confidential content prior to retransmission to the remote computing device 550.

[0085] FIG. 6 presents a flowchart of a draft response output process 600 in accordance with some examples. The draft response output process 600 can be implemented by a local computing device (e.g., the computing devices 100 and 300, as illustrated in FIGs.1 and 3, respectively). For the brevity of the description, FIG. 6 is described with reference to FIGs. 1 and 3.

[0086] At block 610, the local computing device receives a set of input data from a user. For instance, the local computing device may provide a graphical user interface or a specific application designed to receive the set of input data from the user. For example, the user may utilize peripheral devices (e.g., keyboard and mouse) to input the set of input datavia the graphical user interface or the application, and the present disclosure does not limit the types of the graphical user interface and the application.

[0087] At block 620, the local computing device processes the set of input data. For example, the local computing device processes the input data by performing the redaction as described with respect to FIG. 3, for example, the processor 310 executing the input data processing instruction 354.

[0088] At block 630, the local computing device redacts the set of input data. For example, the computing device 100 may redact the identified subset of confidential data. Redaction may be performed by either removing the subset of data or replacing it with placeholder tokens or markers. For example, in the case where the input query includes personal health data, the local computing device may replace the name, date of birth, and weight with unique placeholder tokens, thereby generating a redacted version of the input data suitable for further processing. For example, the local computing device performs the redaction as described with respect to FIG. 3, for example, the processor 310 executing the input data redaction instruction 356.

[0089] At block 640, the local computing device transmits the set of redacted input data to a remote computing device (e.g., computing devices 200 and 400 shown in FIGs. 2 and 4).

[0090] At decision block 650, the local computing device determines whether the local computing device received the draft response(s) transmitted from the remote computing device. Upon receiving the draft response(s), the process 600 proceeds to block 660.

[0091] At block 660, the local computing device evaluate the draft response(s). For example, the local computing device performs the un-redaction and evaluation of the draft responses as described with respect to FIG. 3, for example, the processor 310 executing the evaluation instruction 360.

[0092] At block 670, the local computing device outputs a selected response selected based on the evaluation results. For example, the local computing device outputs the selected response to the user, such as via a graphical interface, communication channel, or external device, depending on the implementation environment.

[0093] The draft response output process 600 can be ended at block 670. Although the operations of the process 600 are described in a can be particular order, it should beunderstood that the process 600 is not limited as such. Operations of the process 600 may be performed in an alternative order, serially, or at least in parallel. Further, some operations may be repeated.

[0094] FIG. 7 presents a flowchart of a draft response generation process 700 in accordance with some examples. The draft response generation process 700 can be implemented by a remote computing device (e.g., the computing devices 200 and 400, as illustrated in FIGs. 2 and 4, respectively). For the brevity of the description, FIG. 7 is described with reference to FIGs. 2 and 4.

[0095] At block 710, the remote computing device receives a set of redacted input data. For example, the set of redacted input data is transmitted from the local computing device, as illustrated in FIG. 6 (e.g., block 640).

[0096] At block 720, the remote computing device generates draft response(s) by processing the set of redacted input data. For example, the remote computing device generate draft responses as described with respect to FIG. 4, for example, the processor 410 executing the draft response generation instruction 452.

[0097] At block 730, the remote computing device transmits the generated draft response(s) to the local computing device.

[0098] The draft response generation process 700 can be ended at block 740. Although the operations of the process 700 are described in a particular order, it should be understood that the process 700 is not limited as such. Operations of the process 700 may be performed in an alternative order, serially, or at least in parallel. Further, some operations may be repeated.

[0099] It is to be understood that not necessarily all objects or advantages may be achieved in accordance with any example described herein. Thus, some examples may be configured to operate in a manner that achieves or optimizes one advantage or group of advantages as taught herein without necessarily achieving other objects or advantages as may be taught or suggested herein.

[0100] All of the processes described herein may be embodied in, and fully automated via, software code modules, including specific computer-executable instructions, which are executed by a computing system. The computing system may include at least one computer or processor. The code modules may be stored in any type of non-transitorycomputer-readable medium or other computer storage device. Some or all the methods may be embodied in specialized computer hardware.

[0101] Many other variations than those described herein will be apparent from this disclosure. For example, depending on the example, some acts, events, or functions of any of the algorithms described herein can be performed in a different sequence, can be added, merged, or left out altogether (e.g., not all described acts or events are necessary for the practice of algorithms). Moreover, in some examples, acts or events can be performed concurrently, e.g., through multi -threaded processing, interrupt processing, or multiple processors or processor cores or on other parallel architectures, rather than sequentially. In addition, different tasks or processes can be performed by different machines and / or computing systems that can function together.

[0102] The various example logical blocks, components and modules described in connection with the examples disclosed herein can be implemented or performed by a machine, such as a processing unit or processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A processor can be a microprocessor, but in the alternative, the processor can be a controller, microcontroller, or state machine, combinations of the same, or the like. A processor can include electrical circuitry configured to process computer-executable instructions. In another example, a processor includes an FPGA or other programmable device that performs logic operations without processing computer-executable instructions. A processor can also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, at least one microprocessor in conjunction with a DSP core, or any other such configuration. Although described herein primarily with respect to digital technology, a processor may also include primarily analog components. A computing environment can include any type of computer system, including, but not limited to, a computer system based on a microprocessor, a mainframe computer, a digital signal processor, a portable computing device, a device controller, or a computational engine within an appliance, to name a few.

[0103] Conditional language such as, among others, “can,” “could,” “might,” or “may,” unless specifically stated otherwise, are otherwise understood within the context as used in general to convey that some examples include, while other examples do not include, some features, elements, and / or blocks. Thus, such conditional language is not generally intended to imply that features, elements, and / or blocks are in any way required for any example or that any example necessarily includes logic for deciding, with or without user input or prompting, whether these features, elements, and / or blocks are included or are to be performed in any example.

[0104] Disjunctive language such as the phrase “at least one of X, Y, or Z,” unless specifically stated otherwise, is otherwise understood with the context as used in general to present that an item, term, etc., may be either X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Thus, such disjunctive language is not generally intended to, and should not, imply that some examples require at least one of X, at least one of Y, or at least one of Z to each be present.

[0105] Any process descriptions, elements or blocks in the flow diagrams described herein and / or depicted in the attached figures should be understood as potentially representing modules, segments, or portions of code which include executable instructions for implementing specific logical functions or elements in the process. Alternate implementations are included within the scope of the examples described herein in which elements or functions may be deleted, executed out of order from that shown, or discussed, including substantially concurrently or in reverse order, depending on the functionality involved.

[0106] Unless otherwise explicitly stated, articles such as “a” or “an” should generally be interpreted to include one or more described items. Accordingly, phrases such as “a device configured to” are intended to include one or more recited devices. Such one or more recited devices can also be collectively configured to carry out the stated recitations. For example, “a processor configured to carry out recitations A, B, and C” can include a first processor configured to carry out recitation A working in conjunction with a second processor configured to carry out recitations B and C.

Claims

WHAT IS CLAIMED IS:

1. An apparatus comprising:a local machine learning (ML) model;a memory to store a set of input data received from a client; and a processor to:receive the set of input data;remove a subset of data from the set of input data to generate redacted input data;transmit the redacted input data to a remote ML model;obtain a set of draft responses generated based on the redacted input data from the remote ML model;evaluate the set of draft responses using the local ML model to select an individual draft response according to a likelihood that the local ML model would generate the individual response as a response to the set of input data; andoutput the individual response as the response to the set of input data to the client.

2. The apparatus of Claim 1, wherein a particular response of the set of responses comprises a plurality of tokens, and wherein the processor is to evaluate the plurality of tokens partly concurrently.

3. The apparatus of Claim 1, wherein the subset of data comprises personal identification identifiers, personal confidential information, personal health information, or a combination thereof.

4. The apparatus of Claim 1, wherein the processor is to remove the subset of data to result in the redacted input data by adding placeholders into the redacted input data in place of the subset of data.

5. The apparatus of Claim 4, wherein the processor is to evaluate the set of responses by un-redacting the set of responses by replacing the placeholders with a portion of the subset of data.

6. The apparatus of Claim 1, wherein the processor is to evaluate the set of responses iteratively, and wherein a frequency of iteration is determined based on an approval rate of the set of responses.

7. A non-transitory machine-readable storage medium comprising instructions executable by a processor of a computing device to cause the computing device to:obtain a set of redacted input data from a client computing system, the set of redacted input data comprising input data modified to replace a subset of confidential information with redaction placeholders;prompt a generative artificial intelligence machine learning (ML) model with the set of redacted input data to generate a draft response to the set of the redacted input data, wherein the draft response corresponds to a predicted response to a prompt of the generative artificial intelligence ML model with the input data in un-redacted form; andtransmit the draft response to the client computing system.

8. The non-transitory machine-readable storage medium of claim 7, wherein the instructions, when executed by the processor, further cause the processor to:obtain from the client computing system a draft response length for incremental generation of the draft response;iteratively prompt generative artificial intelligence ML model to generate each portion of individual of the draft response in increments without exceeding the token size limit; andtransmit each incremental portion of the generated draft response to the client computing system after each iteration.

9. The non-transitory machine-readable storage medium of claim 7, wherein the set of redacted input data further includes metadata identifying positions and types of confidential information replaced by the redaction placeholders.

10. The non-transitory machine-readable storage medium of claim 7, wherein the instructions, when executed by the processor, further cause the processor to receive feedback from the client computing system regarding completeness of the draft response and to store the feedback for future model refinement.

11. The non-transitory machine-readable storage medium of claim 7, wherein the instructions, when executed by the processor, further cause the processor to generate a notification if the set of redacted input data includes un-redacted confidential information and to request confirmation from the client computing system.

12. A non-transitory machine-readable storage medium comprising instructions executable by a processing resource of a computing device to cause the computing device to:receive a set of input data as input to the computing device from a client; identify a subset of data from the set of input data as sensitive information;remove the identified subset of data from the set of input data to generate redacted input data;transmit the redacted input data to a remote machine learning (ML) model; obtain a plurality of responses generated based on the redacted input data from the remote ML model;evaluate the plurality of responses using a local ML model to select an individual response from the plurality of responses according to a likelihood that the local ML model would generate the individual response as a response to the set of input data; andoutput the individual response as the response to the set of input data to the client.

13. The non-transitory machine-readable storage medium of claim 12, wherein the subset of data comprises personal identification identifiers, personal confidential information, personal health information, or a combination thereof14. The non-transitory machine-readable storage medium of claim 12, wherein the instructions cause the computing device to evaluate the plurality of responses iteratively, and wherein a frequency of iteration is determined based on an approval rate of the plurality of responses.

15. The non-transitory machine-readable storage medium of claim 12, wherein the instructions, when executed by the processing resource, further cause the processing resource to remove the subset of data to result in the redacted input data by adding placeholders into the redacted input data in place of the subset of data.