A privacy data-aware adaptive scheduling optimization method and system for RAG large model inference service

Abstract

The application discloses a privacy data perception adaptive scheduling optimization method and system for RAG large model inference service. The method constructs a module-level privacy data perception RAG inference engine, including four stages of retrieval, encoding, segmentation and aggregation. In the retrieval stage, the correlation retrieval of privacy data perception is performed to obtain private paragraphs and public paragraphs with privacy attribute identification. In the encoding stage, according to the privacy attribute of the input paragraph, the encoder task is divided into private subtasks and public subtasks, which are calculated in the secure domain TEE and the non-secure domain REE respectively, to obtain the context vector set of each paragraph after encoding, which is used as the hidden state input of the decoder task. In the segmentation stage, according to the precision delay knob value set by the user, the context vector set of the TEE / REE decoder subtask is dynamically selected, and the TEE / REE candidate answer is generated accordingly. In the aggregation stage, the conditional probability election algorithm is used to aggregate the TEE / REE candidate answers and generate the final answer, realizing the adaptive balance of precision and efficiency.

Description

Technical Field

[0001] This application relates to the field of privacy-preserving machine learning, and in particular to a privacy-aware adaptive scheduling optimization method and system for RAG large model inference services. Background Technology

[0002] Retrieval-Augmented Generation (RAG) is an innovative approach that combines information retrieval with natural language generation, aiming to enhance information acquisition and generation capabilities in natural language processing tasks. Traditional generative models are often limited by insufficient information when dealing with problems requiring rich background knowledge. RAG, however, introduces external knowledge bases and retrieves relevant paragraphs in real-time during the generation process, incorporating more factual, timely, and accurate external information into the generated content, thereby enhancing the model's reliability and expressiveness. This method is particularly suitable for tasks such as open-domain question answering (QA), information extraction, and dialogue generation. By utilizing relevant information from external paragraphs, RAG can generate more accurate and factually based answers or text, providing robust and interpretable support for various natural language processing applications.

[0003] With the development of RAG technology, user privacy and security issues have gradually attracted attention. During RAG inference, the model needs to access both user-input prompts and retrieved external paragraphs to generate results. These paragraphs in the private knowledge base often contain sensitive or private information. During model inference computation, this data is typically loaded, computed, or cached in plaintext in an untrusted operating environment, easily leading to privacy leaks. Therefore, how to achieve privacy protection during the inference computation phase and complete efficient retrieval and generation without exposing sensitive data has become a key issue in RAG system design.

[0004] Trusted Execution Environment (TEE)-based methods leverage the runtime isolation provided by trusted hardware to achieve efficient privacy-preserving inference to a certain extent. However, as the parameter scale of large RAG models continues to increase, the model weights far exceed the capacity of TEEs, resulting in high inference latency for existing TEE inference schemes. Furthermore, the privacy protection characteristics of RAG tasks differ from traditional machine learning; their main privacy risks originate from segments in the retrieval phase, especially segments with private sources, and existing privacy protection research in machine learning has not considered the specific privacy risks of RAG. Therefore, this invention proposes a privacy data-aware adaptive scheduling optimization method and system for large RAG model inference services. First, by analyzing whether each sub-module of the RAG generation model involves privacy content, a module-level privacy data-aware RAG inference engine divides the inference task into private and public sub-tasks. Further, considering the different impacts of each sub-task on model accuracy and inference latency, a RAG accuracy-latency adaptive scheduling method is provided to meet the differentiated needs of accuracy-first or latency-first approaches. Summary of the Invention

[0005] To address the limitations of existing technologies, this embodiment proposes a privacy-aware adaptive scheduling optimization method and system for RAG large model inference services. This method divides the inference task into private and public subtasks based on a module-level privacy-aware RAG inference engine. Furthermore, considering the different impacts of each subtask on model accuracy and inference latency, a precision-latency adaptive scheduling mechanism is proposed to achieve flexible adaptation to different application scenarios: providing low-latency secure inference in real-time priority scenarios and high-precision secure inference in response quality priority scenarios, thus balancing privacy protection and system performance.

[0006] This method provides a privacy-aware adaptive scheduling optimization approach for RAG large-model inference services. It constructs a module-level privacy-aware RAG inference engine based on confidential computing technology, enabling private and public data to be computed in the secure domain (TEE) and the non-secure domain (REE), respectively. The method includes four stages: retrieval, encoding, segmentation, and aggregation. S1. In the retrieval phase, a privacy-aware retrieval is performed to obtain private and public paragraphs with privacy attribute identifiers. S2. In the encoding stage, a privacy-aware module offloading rule is adopted. Based on the privacy attributes of the input paragraph, the encoder task is divided into private subtasks and public subtasks. These are calculated in the secure domain TEE and the non-secure domain REE respectively to obtain the set of context vectors after encoding each paragraph, which serves as the hidden state input for the decoder task. S3, the segmentation stage employs a precision-delay adaptive scheduling mechanism, based on the user-defined precision-delay knob value. Dynamically select the set of context vectors for the TEE / REE decoder subtasks and generate TEE / REE candidate answers accordingly; S4. In the aggregation stage, a conditional probability election algorithm is used to aggregate TEE / REE candidate answers and generate the final answer, achieving an adaptive balance between model inference accuracy and computational efficiency.

[0007] Preferably, in the retrieval stage described in step S1, a privacy-aware retrieval is performed to obtain private and public paragraphs with privacy attribute identifiers, specifically including: Relevant private and public paragraphs are obtained from the corresponding knowledge base using relevance retrieval methods; Among them, private paragraphs are retrieved locally by the user and uploaded to the cloud in encrypted form; The relevance retrieval method includes dense retrieval algorithm and sparse retrieval algorithm; The corresponding knowledge bases include privately sourced knowledge bases that are independently trained and built by users locally and stored locally; and publicly sourced knowledge bases that include public knowledge bases built by service providers in the cloud and Internet knowledge bases.

[0008] Preferably, during the encoding stage, a privacy-aware module offloading rule is adopted. Based on the privacy attributes of the input segment, the encoder task is divided into private subtasks and public subtasks, which are computed in the secure domain (TEE) and the non-secure domain (REE), respectively. The resulting context vector set of the TEE / REE decoder subtask specifically includes: S21, the encoder tasks are respectively connected to user query prompts. Related paragraphs obtained in the retrieval phase Privacy-aware module unloading rules serve as input to the encoder module. This involves private paragraph input. The task is classified as a private subtask, which involves inputting public paragraphs. These are classified as public subtasks; S22. The encoder's private and public subtasks are computed in the secure domain (TEE) and the non-secure domain (REE), respectively, to obtain the set of context vectors after encoding each segment. The private subtasks obtain the set... Publicly disclose the subtasks to obtain the set .

[0009] Preferably, the encoder task is divided into private subtasks and public subtasks, and computed in the secure domain TEE and the non-secure domain REE respectively, specifically including: Private subtasks are deployed in the server-side TEE, and the client and server agree on a key. Subtask input in encrypted form The plaintext is passed in the form of a variable; in the TEE, the decrypted plaintext is obtained. And calculate the result. ; When performing linear multiplication calculations, the computation is accelerated by a one-time mask encryption method and passed to the REE (Transmission Equipment). The TEE (Transmission Equipment) ensures the input security during this calculation process. and results Not visible to server administrators; When TEE memory space is insufficient, the calculation results are encrypted. Cached in REE; Public subtasks are deployed in the REE and delivered to the acceleration hardware for computation in plaintext. Specifically, during the linear multiplication calculation, a one-time mask encryption is used to accelerate the calculation via REE. At that time, the input is encrypted with a one-time mask. Send to REE, where REE acceleration hardware performs the calculations. The result is then returned to the TEE, which uses the mask m to reconstruct the calculation. ,in Calculate in advance during the offline phase.

[0010] Preferably, in the segmentation stage, a precision-delay adaptive scheduling mechanism is adopted, based on the precision-delay knob value set by the user. The process involves dynamically selecting the context vector set for the TEE / REE decoder subtask and generating corresponding TEE / REE candidate answers, including the following steps: S31. Based on the user-specified precision-delay knob value. Soft constraints are applied to dynamically select the context vector set for the TEE / REE decoder subtask based on the ratio of private to public paragraphs. and ; wherein, the user-specified precision-delay knob value This is used to adjust the trade-off between accuracy and latency in RAG generation tasks; the smaller the knob value, the more the system tends to increase generation accuracy; the larger the knob value, the more the system tends to reduce latency. S32. Based on the proportion of private paragraphs Percentage of publicly available paragraphs Specify that the final answer depends only on the private paragraph influence factor. Relying solely on publicly available paragraph impact factors Simultaneously dependent on the influence factors of both source paragraphs Normalization coefficient ; S33. The REE decoder computation does not increase the TEE computational load. It selects the context output of all encoder public subtasks, and the REE decoder context vector set. ; S34. TEE decoding calculation needs to reduce TEE load for paragraph retrieval relevance score. Recalculate the private context score Public context score Based on the user-specified precision-delay knob value Select paragraph scores from all context vector sets. Greater than the knob value The context, the set of context vectors of the TEE decoder ; S35. Compute the decoder subtask in the secure domain TEE and the non-secure domain REE respectively, and obtain the TEE / REE candidate answers. .

[0011] Preferably, in the aggregation stage, a conditional probability election algorithm is used to aggregate TEE / REE candidate answers and generate the final answer, achieving an adaptive balance between accuracy and efficiency. Specifically, this includes: based on the paragraph scores... Decoder network parameters The probability of generating candidate answers The final answer is obtained by aggregation. ; Among them, the decoder network parameters This refers to the weight parameters corresponding to the decoder neural network; The probability of generating candidate answers This refers to the given decoder network parameters With decoder subtask context When the model outputs the probability of generating candidate answers, it outputs the probability of generating the candidate answer.

[0012] This application also provides a privacy-aware adaptive scheduling optimization system for RAG large model inference services, which constructs a module-level privacy-aware RAG inference engine. The inference engine is based on confidential computing technology, enabling private data and public data to be computed in the secure domain TEE and the non-secure domain REE, respectively. The inference engine includes a retrieval module, an encoding module, a segmentation module, and an aggregation module; wherein; The retrieval module performs privacy-aware retrieval to obtain private and public paragraphs with privacy attribute identifiers; The encoding module adopts a privacy-aware module offloading rule. Based on the privacy attributes of the input segment, the encoder task is divided into private subtasks and public subtasks, which are computed in the secure domain TEE and the non-secure domain REE respectively, to obtain the context vector set of the TEE / REE decoder subtask. The segmentation module employs a precision-delay adaptive scheduling mechanism. Based on the precision-delay knob value set by the user, it dynamically selects the context vector set of the TEE / REE decoder subtask and generates TEE / REE candidate answers accordingly. The aggregation module uses a conditional probability election algorithm to aggregate TEE / REE candidate answers and generate the final answer, achieving an adaptive balance between model inference accuracy and computational efficiency.

[0013] This application provides a computing device, including a processor and a memory; the memory is used to store computer execution... The processor executes the computer execution instructions to cause the computing device to perform the above-described method.

[0014] This application provides a computer-readable storage medium, the computer-readable storage medium including a computer program. The instructions, when executed by a computing device, cause the computing device to perform the methods described above.

[0015] Experiments demonstrate that, by adopting this method, the RAG inference engine, designed for both privacy protection and precision-latency differentiation, exhibits significantly improved computational efficiency compared to the original system, particularly in latency-priority scenarios, achieving a 10-fold speedup. These experimental results demonstrate that this invention, while ensuring data security, achieves improved computational efficiency based on a module-level privacy-aware RAG inference engine, further satisfying the differentiated needs of precision-priority and latency-priority scenarios. This application features high security and high execution efficiency, making it widely applicable in various production and daily life scenarios, effectively protecting the security of personal data and servers. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the privacy-aware adaptive scheduling optimization method for RAG large model inference services according to the present invention. Detailed Implementation

[0017] This invention discloses a privacy-aware adaptive scheduling optimization method and system for RAG large model inference services. This application features high security and high execution efficiency, and can be widely applied in various production and life scenarios, effectively protecting the security of personal data and servers. The specific embodiments of this application are described in detail below with reference to the accompanying drawings. It should be noted that the embodiments shown in this document are merely a typical application of this application, and the technical solutions involved in this application should be protected until those skilled in the art can propose better improvements.

[0018] In one embodiment, such as Figure 1 The figure illustrates the privacy-aware adaptive scheduling optimization method for RAG large-model inference services according to the present invention. It constructs a module-level privacy-aware RAG inference engine, which is based on confidential computing technology, enabling private and public data to be computed in the secure domain (TEE) and the non-secure domain (REE), respectively. The method includes four stages: retrieval, encoding, segmentation, and aggregation, as shown in the figure. S1. In the retrieval phase, a privacy-aware retrieval is performed to obtain private and public paragraphs with privacy attribute identifiers. S2. In the encoding stage, a privacy-aware module offloading rule is adopted. Based on the privacy attributes of the input paragraph, the encoder task is divided into private subtasks and public subtasks. These are calculated in the secure domain TEE and the non-secure domain REE respectively to obtain the set of context vectors after encoding each paragraph, which serves as the hidden state input for the decoder task. S3. In the segmentation stage, a precision-delay adaptive scheduling mechanism is adopted. Based on the precision-delay knob value set by the user, the context vector set of the TEE / REE decoder subtask is dynamically selected, and TEE / REE candidate answers are generated accordingly. S4. In the aggregation stage, a conditional probability election algorithm is used to aggregate TEE / REE candidate answers and generate the final answer, achieving an adaptive balance between model inference accuracy and computational efficiency.

[0019] The retrieval phase, along with the encoding phase, forms the core of RAG technology. Unlike current RAG technologies, this invention distinguishes the retrieval source of segments during the retrieval phase. Specifically, private source segments are specified and provided by the user. By clearly defining the source of the retrieved segments, the encoding phase can divide the network submodules into private and public subtasks at the network module level based on whether the computation of the RAG network submodule involves private segments. Since the public subtasks do not involve private segments, they can be offloaded from the TEE to the REE, reducing the computational load on the TEE and leveraging the REE's acceleration hardware for efficient computation. Through this method, the invention ultimately improves the overall system performance.

[0020] Specifically, during the retrieval phase, a privacy-aware retrieval is performed to obtain private and public paragraphs with privacy attribute identifiers, including: Relevant private and public paragraphs are obtained from the corresponding knowledge base using relevance retrieval methods; Among them, private paragraphs are retrieved locally by the user and uploaded to the cloud in encrypted form; The relevance retrieval method includes dense retrieval algorithm and sparse retrieval algorithm; The corresponding knowledge bases include privately sourced knowledge bases that are independently trained and built by users locally and stored locally; and publicly sourced knowledge bases that include public knowledge bases built by service providers in the cloud and Internet knowledge bases.

[0021] Specifically, during the encoding stage, a privacy-aware module offloading rule is adopted. Based on the privacy attributes of the input segment, the encoder task is divided into private subtasks and public subtasks, which are computed in the secure domain (TEE) and the non-secure domain (REE), respectively. The resulting context vector set for the TEE / REE decoder subtask includes: S21, the encoder tasks are respectively connected to user query prompts. Related paragraphs obtained in the retrieval phase Privacy-aware module unloading rules serve as input to the encoder module. This involves private paragraph input. The task is classified as a private subtask, which involves inputting public paragraphs. These are classified as public subtasks; S22. The encoder's private and public subtasks are computed in the secure domain (TEE) and the non-secure domain (REE), respectively, to obtain the set of context vectors after encoding each segment. The private subtasks obtain the set... Publicly disclose the subtasks to obtain the set .

[0022] Specifically, the encoder task is divided into private subtasks and public subtasks, and computed separately in the secure domain TEE and the non-secure domain REE, including: Private subtasks are deployed in the server-side TEE, and the client and server agree on a key. Subtask input in encrypted form The plaintext is passed in the form of a variable; in the TEE, the decrypted plaintext is obtained. And calculate the result. ; When performing linear multiplication calculations, the computation is accelerated by a one-time mask encryption method and passed to the REE (Transmission Equipment). The TEE (Transmission Equipment) ensures the input security during this calculation process. and results Not visible to server administrators; When TEE memory space is insufficient, the calculation results are encrypted. Cached in REE; Public subtasks are deployed in the REE and delivered to the acceleration hardware for computation in plaintext. Specifically, during the linear multiplication calculation, a one-time mask encryption is used to accelerate the calculation via REE. At that time, the input is encrypted with a one-time mask. Send to REE, where REE acceleration hardware performs the calculations. The result is then returned to the TEE, which uses the mask m to reconstruct the calculation. ,in Calculate in advance during the offline phase.

[0023] Specifically, during the segmentation phase, a precision-delay adaptive scheduling mechanism is employed, based on the precision-delay knob value set by the user. The process involves dynamically selecting the context vector set for the TEE / REE decoder subtask and generating private and public candidate answers accordingly, including the following steps: S31. Based on the user-specified precision-delay knob value. Soft constraints are applied to dynamically select the context vector set for the TEE / REE decoder subtask based on the ratio of private to public paragraphs. and This method can satisfy the differentiated requirements of prioritizing accuracy or delay. The user-specified precision-delay knob value This knob value serves as an adjustment parameter, used to balance the accuracy and latency of the RAG generation task. A smaller knob value tends to increase generation accuracy; a larger knob value tends to decrease latency. S32. Based on the proportion of private paragraphs Percentage of publicly available paragraphs Specify that the final answer depends only on the private paragraph influence factor. Relying solely on publicly available paragraph impact factors Simultaneously dependent on the influence factors of both source paragraphs Normalization coefficient ; S33. Since the REE decoder computation does not increase the TEE computational load, the context output of all encoder public subtasks is selected, and the REE decoder context vector set is used. ; S34. TEE decoding calculation needs to reduce TEE load for paragraph retrieval relevance score. Recalculate the private context score Public context score Based on the user-specified precision-delay knob value Select paragraph scores from all context vector paragraphs. Greater than the knob value The context, the set of context vectors of the TEE decoder ; S35. Compute the decoder subtask in the secure domain TEE and the non-secure domain REE respectively, and obtain the TEE / REE candidate answers. .

[0024] Specifically, the aggregation stage employs a conditional probability election algorithm to aggregate TEE / REE candidate answers and generate the final answer, achieving an adaptive balance between accuracy and efficiency. This includes the following steps: Based on the paragraph score Decoder network parameters The probability of generating candidate answers The final answer is obtained by aggregation. The decoder network parameters This refers to the weight parameters corresponding to the decoder neural network; The probability of generating candidate answers This refers to the given decoder network parameters With decoder subtask context When the model outputs the probability of generating candidate answers, it outputs the probability of generating the candidate answer.

[0025] This application provides a privacy-aware adaptive scheduling optimization system for RAG large model inference services. It constructs a module-level privacy-aware RAG inference engine, which is based on confidential computing technology, enabling private data and public data to be computed in the secure domain (TEE) and the non-secure domain (REE), respectively. The inference engine includes a retrieval module, an encoding module, a segmentation module, and an aggregation module. The retrieval module performs privacy-aware retrieval to obtain private and public paragraphs with privacy attribute identifiers; The encoding module adopts a privacy-aware module offloading rule. Based on the privacy attributes of the input paragraph, the encoder task is divided into private subtasks and public subtasks, which are computed in the secure domain TEE and the non-secure domain REE respectively, to obtain the set of context vectors after encoding each paragraph, which serves as the hidden state input for the decoder task. The segmentation module employs a precision-delay adaptive scheduling mechanism. Based on the precision-delay knob value set by the user, it dynamically selects the context vector set of the TEE / REE decoder subtask and generates TEE / REE candidate answers accordingly. The aggregation module uses a conditional probability election algorithm to aggregate TEE / REE candidate answers and generate the final answer, achieving an adaptive balance between model inference accuracy and computational efficiency.

[0026] This application provides a non-transitory computer-readable storage medium that stores computer instructions that cause the computer to execute the methods provided in the above-described method embodiments, such as setting up the algorithm and related programs developed in this invention and using this application in any field of privacy-preserving machine learning.

[0027] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0028] Finally, it should be noted that although the embodiments and designs in this application are mainly illustrated with specific examples and the software involved is in the form of a system, all components involved in this system are still protected by patents, but are not limited to modules such as resource monitors and task schedulers and their code. Therefore, any modifications, improvements, or uses to the implementation of this invention should be covered in the claims and specification of this invention.

Claims

1. A privacy-aware adaptive scheduling optimization method for RAG large model inference services, characterized in that, A module-level privacy-aware RAG inference engine is constructed. This inference engine is based on confidential computing technology, enabling private and public data to be computed in the secure domain (TEE) and the non-secure domain (REE), respectively. The method includes four stages: retrieval, encoding, segmentation, and aggregation. S1. In the retrieval phase, a privacy-aware retrieval is performed to obtain private and public paragraphs with privacy attribute identifiers. S2. In the encoding stage, a privacy-aware module offloading rule is adopted. Based on the privacy attributes of the input paragraph, the encoder task is divided into private subtasks and public subtasks. These are calculated in the secure domain TEE and the non-secure domain REE respectively to obtain the set of context vectors after encoding each paragraph, which serves as the hidden state input for the decoder task. S3, the segmentation stage employs a precision-delay adaptive scheduling mechanism, based on the user-defined precision-delay knob value. Dynamically select the set of context vectors for the TEE / REE decoder subtasks and generate TEE / REE candidate answers accordingly; S4. In the aggregation stage, a conditional probability election algorithm is used to aggregate TEE / REE candidate answers and generate the final answer, achieving an adaptive balance between model inference accuracy and computational efficiency.

2. The method according to claim 1, characterized in that, In the retrieval phase described in step S1, a privacy-aware retrieval is performed to obtain private and public paragraphs with privacy attribute identifiers, specifically including: Relevant private and public paragraphs are obtained from the corresponding knowledge base using relevance retrieval methods; Among them, private paragraphs are retrieved locally by the user and uploaded to the cloud in encrypted form; The relevance retrieval method includes dense retrieval algorithm and sparse retrieval algorithm; The corresponding knowledge bases include privately sourced knowledge bases that are independently trained and built by users locally and stored locally; and publicly sourced knowledge bases that include public knowledge bases built by service providers in the cloud and Internet knowledge bases.

3. The method according to claim 1, characterized in that, In the encoding stage described in step S2, a privacy-aware module offloading rule is adopted. Based on the privacy attributes of the input segments, the encoder task is divided into private subtasks and public subtasks, which are computed in the secure domain (TEE) and the non-secure domain (REE) respectively, to obtain the set of encoded context vectors for each segment. These vectors serve as the hidden state input for the decoder task. Specifically, this includes: S21, the encoder tasks are respectively connected to user query prompts. Related paragraphs obtained in the retrieval phase Privacy-aware module unloading rules serve as input to the encoder module. This involves private paragraph input. The task is classified as a private subtask, which involves inputting public paragraphs. These are classified as public subtasks; S22. The encoder's private and public subtasks are computed in the secure domain (TEE) and the non-secure domain (REE), respectively, to obtain the set of context vectors after encoding each segment. The private subtasks obtain the set... Publicly disclose the subtasks to obtain the set .

4. The method according to claim 1, characterized in that, The encoder task is divided into private subtasks and public subtasks, and computed separately in the secure domain TEE and the non-secure domain REE, specifically including: Private subtasks are deployed in the server-side TEE, and the client and server agree on a key. Subtask input in encrypted form The plaintext is passed in the form of a variable; in the TEE, the decrypted plaintext is obtained. And calculate the result. ; When performing linear multiplication calculations, the computation is accelerated by a one-time mask encryption method and passed to the REE (Transmission Equipment). The TEE (Transmission Equipment) ensures the input security during this calculation process. and results Invisible to server administrators; When TEE memory space is insufficient, the calculation results are encrypted. Cached in REE; Public subtasks are deployed in the REE and delivered to the acceleration hardware for computation in plaintext. Specifically, during the linear multiplication calculation, a one-time mask encryption is used to accelerate the calculation via REE. At that time, the input is encrypted with a one-time mask. Send to REE, where REE acceleration hardware performs the calculations. The result is then returned to the TEE, which uses the mask m to reconstruct the calculation. ,in Calculate in advance during the offline phase.

5. The method according to claim 1, characterized in that, In the segmentation stage described in step S3, a precision-delay adaptive scheduling mechanism is adopted, based on the precision-delay knob value set by the user. The system dynamically selects the context vector set for the TEE / REE decoder subtask and generates corresponding TEE / REE candidate answers, specifically including: S31. Based on the user-specified precision-delay knob value. Soft constraints are applied to dynamically select the context vector set for the TEE / REE decoder subtask based on the ratio of private to public paragraphs. and ; wherein, the user-specified precision-delay knob value This is used to adjust the trade-off between accuracy and latency in RAG generation tasks; the smaller the knob value, the more the system tends to increase generation accuracy; the larger the knob value, the more the system tends to reduce latency. S32. Based on the proportion of private paragraphs Percentage of publicly available paragraphs Specify that the final answer depends only on the private paragraph influence factor. Relying solely on publicly available paragraph impact factors Simultaneously dependent on the influence factors of both source paragraphs Normalization coefficient ; S33. The REE decoder computation does not increase the TEE computational load. It selects the context vectors output by all encoder public subtasks, forming the REE decoder context vector set. ; S34. TEE decoding calculation needs to reduce TEE load for paragraph retrieval relevance score. Recalculate the private context score Public context score Based on the user-specified precision-delay knob value Select paragraph scores from all context vector sets. Greater than the knob value Context vectors, TEE decoder context vector set ; S35. Compute the decoder subtasks in the secure domain TEE and the non-secure domain REE respectively, and obtain the TEE / REE candidate answers. .

6. The method according to claim 1, characterized in that, Step 4, the aggregation stage, employs a conditional probability election algorithm to aggregate TEE / REE candidate answers and generate the final answer, achieving an adaptive balance between accuracy and efficiency. Specifically, this includes: Based on the paragraph score Decoder network parameters The probability of generating candidate answers The final answer is obtained by aggregation. ; Among them, the decoder network parameters This refers to the weight parameters corresponding to the decoder neural network; The probability of generating candidate answers This refers to the given decoder network parameters With decoder subtask context vector set When the model outputs the probability of generating candidate answers, it outputs the probability of generating the candidate answer.

7. A privacy-aware adaptive scheduling and optimization system for RAG large model inference services, characterized in that, A module-level privacy-aware RAG inference engine is constructed. The inference engine is based on confidential computing technology, which enables private data and public data to be computed in the secure domain TEE and the non-secure domain REE, respectively. It includes a retrieval module, an encoding module, a segmentation module, and an aggregation module. in; The retrieval module performs privacy-aware retrieval to obtain private and public paragraphs with privacy attribute identifiers; The encoding module adopts a privacy-aware module offloading rule. Based on the privacy attributes of the input segment, the encoder task is divided into private subtasks and public subtasks, which are computed in the secure domain TEE and the non-secure domain REE respectively, to obtain the context vector set of the TEE / REE decoder subtask. The segmentation module employs a precision-delay adaptive scheduling mechanism. Based on the precision-delay knob value set by the user, it dynamically selects the context vector set of the TEE / REE decoder subtask and generates TEE / REE candidate answers accordingly. The aggregation module uses a conditional probability election algorithm to aggregate TEE / REE candidate answers and generate the final answer, achieving an adaptive balance between model inference accuracy and computational efficiency.

8. A computing device, characterized in that, It includes a processor and a memory; the memory is used to store computer execution instructions; the processor is used to execute the computer execution instructions to cause the computing device to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer program instructions that, when executed by a computing device, cause the computing device to perform the method as described in any one of claims 1-7.

Images