An end-cloud collaborative large model secret state operation method and system

Abstract

The application discloses an end-cloud collaborative large model secret state operation method and system. The method applies an end-cloud collaborative trusted execution environment, which includes an end-side secure access layer and a cloud-side secret calculation layer deploying a TEE cluster. The method includes that a terminal device negotiates a session key with the cloud-side secret calculation layer through the end-side secure access layer, and sends first ciphertext generated based on the session key to the TEE cluster; receives second ciphertext formed by the TEE cluster based on the first ciphertext, and decrypts the second ciphertext to obtain an interaction result. The application uses a two-level architecture of an end-side secure access layer and a cloud-side secret calculation layer for large model secret state operation, and realizes privacy protection of cloud large model services. Meanwhile, the application uses a trusted execution environment constructed by a TEE and a heterogeneous AI extended TEE, so that the large model secret state operation can run on multi-architecture servers and multiple types of GPUs, and meets the needs of diversified deployment scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data security and confidential computing, and particularly relates to an end-cloud collaborative large model secret state running method and system. BACKGROUND

[0002] With the popularization of AI technology and end-cloud collaborative architecture, the demand for data cross-domain sharing and AI computing in the "terminal-cloud" has significantly increased, but data privacy leakage, hardware adaptation limitation, and insufficient full-link security coverage have become key issues.

[0003] In order to realize the cross-domain sharing and AI computing of data in the "terminal-cloud", the existing methods generally adopt an end-cloud privacy protection mode of exclusive hardware and an end-cloud privacy protection mode of specific technology stack. The end-cloud privacy protection mode of exclusive hardware realizes cloud AI service privacy protection through customized hardware, which needs to be deeply bound to exclusive hardware of a specific ecosystem and cannot be flexibly adapted to multiple different types of terminals. The end-cloud privacy protection mode of specific technology stack realizes cloud AI service privacy protection through specific hardware technology stacks (such as specific architecture CPUs and high-end GPUs), which has insufficient adaptability to multiple architecture hardware and is difficult to meet diversified deployment scenarios.

[0004] Therefore, the existing technology still needs to be improved and enhanced. SUMMARY

[0005] The technical problem to be solved by the present application is to provide an end-cloud collaborative large model secret state running method and system to solve the problems of the prior art.

[0006] In order to solve the above technical problems, the first aspect of the present application provides an end-cloud collaborative large model secret state running method, which applies an end-cloud collaboration trusted execution environment, the end-cloud collaboration trusted execution environment includes an end-side secure access layer and a cloud-side secret calculation layer, and the cloud-side secret calculation layer is deployed with a TEE cluster; the method comprises the following steps:

[0007] A terminal device negotiates a session key with the cloud-side secret calculation layer through the end-side secure access layer, wherein the session key is synchronized to the TEE cluster;

[0008] The terminal device sends first ciphertext generated based on the session key to the TEE cluster through the end-side secure access layer;

[0009] The terminal device receives second ciphertext formed by the TEE cluster based on the first ciphertext, and decrypts the second ciphertext to obtain an interaction result, wherein the formation process of the second ciphertext is as follows:

[0010] The TEE cluster decrypts the first ciphertext based on the session key to obtain plaintext interaction data;

[0011] The TEE cluster calls a ciphertext knowledge base to obtain related data corresponding to the plaintext interaction data, and / or determines a large model running result corresponding to the plaintext interaction data through a large model deployed in the TEE cluster to obtain an interaction result.

[0012] The TEE cluster encrypts the interaction result by using the session key to obtain a second ciphertext.

[0013] The end-cloud collaborative large model ciphertext running method, wherein the TEE cluster comprises a cloud-side TEE and a heterogeneous extended TEE, and one or more operations of adding the heterogeneous extended TEE, deleting the heterogeneous extended TEE and modifying the heterogeneous extended TEE can be performed on the TEE cluster.

[0014] The end-cloud collaborative large model ciphertext running method, wherein the end-side secure access layer comprises a hardware security module and / or a software encryption component, so that the end-side secure access layer is adapted to multiple types of terminal devices; when the end-side secure access layer comprises the hardware security module and the software encryption component, a key formed by the software encryption component can be hosted to the hardware security module.

[0015] The end-cloud collaborative large model ciphertext running method, wherein the terminal device sends, to the TEE cluster through the end-side secure access layer, a first ciphertext generated based on the session key, and the method further comprises:

[0016] The terminal device obtains an environment measurement value of the TEE cluster through the end-side secure access layer and cloud-side cryptographic layer, and verifies a cloud-side TEE environment based on the environment measurement value.

[0017] When the cloud-side TEE environment is verified, the terminal device generates a first ciphertext based on the session key, and distributes the first ciphertext to the TEE cluster through the end-side secure access layer.

[0018] The end-cloud collaborative large model ciphertext running method, wherein the first ciphertext is obtained by encrypting interaction data based on a session key, and before the interaction data is encrypted based on the session key, the method further comprises:

[0019] The terminal device receives interaction data of a user and performs lightweight preprocessing on the interaction data to reduce data transmission quantity, wherein the preprocessed interaction data is saved in the terminal device in the form of a ciphertext.

[0020] The end-cloud collaborative large model ciphertext running method, wherein before the terminal device sends, to the TEE cluster through the end-side secure access layer, a first ciphertext generated based on the session key, the method further comprises:

[0021] The terminal device detects an operation mode in which the terminal device is located;

[0022] When the operation mode is a privacy protection mode, the terminal device sends, through an end-side secure access layer, first ciphertext generated based on the session key to the TEE cluster;

[0023] When the operation mode is a non-privacy protection mode, the terminal device sends, through the end-side secure access layer, plaintext interaction data for generating the first ciphertext to the TEE cluster.

[0024] The end-cloud collaborative large model ciphertext operation method, wherein the terminal device and the cloud-side cryptographic calculation layer negotiate to generate the session key through the end-side secure access layer and the cloud-side cryptographic calculation layer specifically includes:

[0025] The terminal device sends device information to the cloud-side cryptographic calculation layer through the end-side secure access layer to apply for a security credential;

[0026] The terminal device sends, through the end-side secure access layer, a key negotiation request carrying the security credential to the cloud-side cryptographic calculation layer, so that the cloud-side cryptographic calculation layer negotiates with the end-side secure access layer to generate the session key based on the key negotiation request, and synchronizes the session key to the TEE cluster.

[0027] The end-cloud collaborative large model ciphertext operation method, wherein the method further includes:

[0028] The cloud-side cryptographic calculation layer receives a large model and / or a knowledge base uploaded by a development end, and stores an encrypted ciphertext knowledge base and a ciphertext large model after encrypting the large model and / or the knowledge base, wherein the encryption adopts a development end key issued by the cloud-side cryptographic calculation layer to the development end;

[0029] The cloud-side cryptographic calculation layer deploys a model service image in the TEE cluster based on a deployment request sent by the development end, to call the ciphertext knowledge base and the ciphertext large model through the model service image, wherein the model service image contains an image digital signature issued by the cloud-side cryptographic calculation layer to the development end.

[0030] The end-cloud collaborative large model ciphertext operation method, wherein the cloud-side cryptographic calculation layer deploys a model service image in the TEE cluster based on a deployment request sent by the development end, to call the ciphertext knowledge base and the ciphertext large model through the model service image specifically includes:

[0031] The cloud-side cryptographic calculation layer deploys a model service image in the TEE cluster based on a deployment request sent by the development end;

[0032] The cloud-side cryptographic calculation layer receives a calling application sent by the development end and carrying a deployment credential, and verifies the calling application through a security management platform;

[0033] The cloud-side encryption layer calls the encrypted large model through the model service image after the application verification is passed, and decrypts the called encrypted large model and deploys it in the TEE cluster.

[0034] The second aspect of the application provides an end-cloud collaborative large model encrypted operation system, which comprises an application end-cloud collaborative trusted execution environment, the application end-cloud collaborative trusted execution environment comprising an end-side secure access layer and a cloud-side encryption layer, and the cloud-side encryption layer being deployed with a TEE cluster;

[0035] The end-side secure access layer is configured to negotiate a session key with the cloud-side encryption layer, send first ciphertext generated based on the session key to the TEE cluster, receive second ciphertext formed by the TEE cluster based on the first ciphertext, and decrypt the second ciphertext to obtain an interaction result.

[0036] The cloud-side encryption layer is configured to negotiate a session key with the end-side secure access layer and synchronize the session key to the TEE cluster, wherein the TEE cluster is configured to decrypt the first ciphertext based on the session key to obtain plaintext interaction data, call an encrypted knowledge base to obtain relevant data corresponding to the plaintext interaction data, and / or determine a large model operation result corresponding to the plaintext interaction data by a large model deployed in the TEE cluster to obtain an interaction result, and encrypt the interaction result by using the session key to obtain the second ciphertext.

[0037] Advantages:

[0038] 1. The TEE cluster is formed by adapting the TEE and the heterogeneous extended TEE, so that the end-cloud collaborative trusted execution environment supports multi-architecture servers and multiple types of GPUs, and different hardware can be adapted without modifying the application code, thereby reducing the cost of hardware binding dependence and hardware replacement and migration.

[0039] 2. The interaction data is not exposed in plaintext in the whole process of the end side (encrypted storage + encrypted transmission) - cloud side (plaintext processing in TEE + encrypted storage), and the TEE and the heterogeneous extended TEE eliminate the side channel attack surface through hardware-level isolation, thereby eliminating the risk of data theft from the root.

[0040] 3. The combination of end-side lightweight encrypted preprocessing and cloud-side efficient encrypted computation reduces the data transmission pressure between the end and the cloud and the computational load of the cloud, and meets the demand of low-latency scenarios.

[0041] 4. Based on the containerized deployment architecture, the existing large model can be directly deployed to the TEE cluster without modifying the core code, thereby realizing zero-code / low-code adaptation of the large model and data sharing process, and reducing the application migration and deployment cost. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0043] Figure 1 The principle block diagram of the end-cloud collaborative large model secret state running system provided by the embodiments of the present application.

[0044] Figure 2 The flowchart of the end-cloud collaborative large model secret state running method provided by the embodiments of the present application.

[0045] Figure 3 The schematic diagram of one example of the end-cloud collaborative large model secret state running method provided by the embodiments of the present application. DETAILED DESCRIPTION

[0046] The embodiments of the present application provide an end-cloud collaborative large model secret state running method and system. In order to make the purpose, technical solutions and effects of the present application more clear and explicit, the following will further describe the present application in detail with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0047] Those skilled in the art can understand that, unless specifically stated otherwise, the singular forms "a", "an" and "the" as used herein can also include plural forms. It should be further understood that the phrase "comprising" used in the specification of the present application means that the features, integers, steps, operations, elements and / or components exist, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we say an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be intermediate elements. In addition, "connected" or "coupled" used herein can include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any single unit and all combinations of the associated listed items.

[0048] Those skilled in the art can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as that generally understood by those skilled in the art to which the present application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood as having meanings consistent with those in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as such.

[0049] It should be understood that the sequence numbers and sizes of the steps in the embodiments do not mean the order of execution, and the execution order of the processes is determined by their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0050] Through research, it is found that with the popularization of AI technology and end-cloud collaborative architecture, the demand for data sharing and AI computing across domains in "terminal-cloud" has increased significantly, but problems such as data privacy leakage, limited hardware adaptation, and insufficient security coverage of the entire link have become key issues. However, the following problems exist in the existing cross-domain sharing and AI computing process of "terminal-cloud":

[0051] 1. Insufficient hardware compatibility: existing solutions generally rely on specific hardware or technology stacks from a particular manufacturer, which cannot flexibly adapt to multi-architecture servers (such as x86 and ARM architectures) and multiple types of GPUs, and do not meet the needs of "reducing hardware binding dependencies and supporting diversified deployment", which also increases the cost of hardware replacement and migration;

[0052] 2. Poor end-side access collaboration: existing solutions have limitations in supporting the end-side, or only adapt to a single type of terminal, or only provide general software encryption capabilities, lacking adaptation to "multi-form hardware security modules" (such as storage media with built-in security chips and hardware encryption cards), resulting in different brands and types of terminals (such as mobile phones, PCs, and car machines) unable to achieve unified key management and identity authentication, and the end-cloud data sharing link is prone to break due to terminal adaptation problems;

[0053] 3. Security risks are not completely avoided: on the one hand, traditional TEE technology generally has side-channel attack risks, and attackers can steal sensitive data in TEE through information leakage at the hardware level; on the other hand, some solutions only cover the security of cloud-side computing, and the data transmission link between the end and the cloud may still have the risk of exposing plaintext, and lack integrated secure hosting capabilities for AI models and knowledge bases, with high risk of leakage of model assets and knowledge data;

[0054] 4. High migration and deployment costs: the adaptation threshold of existing solutions is high, such as process-level TEEs that require extensive modifications to existing application code for compatibility, and some solutions that support container deployment also require binding to specific technology stacks, resulting in the need for re-development when migrating existing AI applications (such as customized LLM inference services and personalized RAG retrieval services) to the solution, which is low in deployment efficiency and high in cost.

[0055] To solve the above problems, in the embodiments of the present application, a two-level architecture is formed based on an end-side secure access layer and a cloud-side encryption layer, and a TEE cluster is used to build an end-cloud collaborative trusted execution environment. An end-cloud collaborative large model encryption running method implemented by the end-cloud collaborative trusted execution environment is provided. The method includes: a terminal device negotiates a session key with the cloud-side encryption layer through the end-side secure access layer; the terminal device sends first ciphertext generated based on the session key to the TEE cluster through the end-side secure access layer; and the terminal device receives second ciphertext formed by the TEE cluster based on the first ciphertext, and decrypts the second ciphertext to obtain an interaction result. In one aspect, the terminal device is connected to the cloud-side encryption layer through the end-side secure access layer to realize end-cloud collaboration, so that multiple types of terminal devices can access the cloud-side encryption layer, and the problem of broken end-cloud data sharing link caused by different types of terminal devices is avoided. In another aspect, the trusted execution environment is built by using the TEE and the heterogeneous AI extended TE, so that the large model encryption running can run on multiple architecture servers and multiple types of GPUs, and the needs of diversified deployment scenarios are met. At the same time, the data is in a ciphertext state in the full link between the terminal device and the cloud side, and the security of the private data is further improved.

[0056] The application content will be further described through the description of the embodiments in combination with the drawings.

[0057] The embodiments provide an end-cloud collaborative large model encryption running method. The method uses an end-cloud collaborative trusted execution environment, as shown in Figure 1 The end-cloud collaborative trusted execution environment includes an end-side secure access layer and a cloud-side encryption layer, and the end-side secure access layer and the cloud-side encryption layer communicate with each other to form a two-level architecture of “end-side secure access layer-cloud-side encryption layer”. The end-cloud data sharing link is established between the end-side secure access layer and the cloud-side encryption layer, and the terminal device accesses the end-side secure access layer to use the end-cloud data sharing link established between the end-side secure access layer and the cloud-side encryption layer for encryption data sharing, so as to request the large model deployed on the cloud-side encryption layer to perform calculation. The cloud-side encryption layer is deployed with a TEE cluster, and the large model is run by the TEE cluster to respond to the request sent by the terminal device through the end-cloud data sharing link.

[0058] In an embodiment, the end-side secure access layer can include a multi-form secure access module, each of the multi-form secure access module can implement end-side secure access to establish an end-cloud data sharing link with the cloud-side cryptographic layer. The multi-form secure access module can include a hardware security module (such as a storage medium with a built-in security chip, a hardware encryption card, etc.) and / or a software encryption component (such as an encryption software development kit, etc.). The present application supports the use of the same end-cloud data sharing link by multiple types of terminals (such as mobile terminals, unmanned aerial vehicles, PCs, intelligent networked vehicles, etc.) through the multi-form secure access module, avoiding the problem of end-cloud data sharing link breakage caused by terminal device type replacement.

[0059] The end-side secure access layer can be used to generate and store keys, data preprocessing and encryption and decryption, and identity authentication, etc. When generating and storing keys, the end-side secure access layer can determine the key generation method and storage method according to the module form of the security access module forming the end-side secure access layer. Specifically, when the module form is a hardware security module, the key generation method is to generate an encryption key (such as a symmetric key / asymmetric key pair, etc.) through chip-level security isolation, and the storage method is to store in the security module that generates the encryption key; when the module form is a software encryption component (such as an encryption software development kit, etc.), the key generation method is to generate an encryption key through a cryptography algorithm, and when the end-side secure access layer includes a hardware security module and a software encryption component, the storage method of the key formed by the software encryption component can be to store in the hardware security module to ensure the security of the encryption key. The present application generates an encryption key through the end-side secure access layer, which can encrypt and store the interaction data received by the terminal device, avoiding the leakage of interaction data transmission.

[0060] When the end-side secure access layer performs data preprocessing and encryption and decryption, it will perform lightweight preprocessing (such as format conversion, compression) on the interaction data (such as user Query, medical images, industrial data) received by the terminal device, and encrypt the processed interaction data to avoid interaction data leakage during transmission. At the same time, the end-side secure access layer will negotiate a session key with the cloud-side cryptographic layer, and use the session key to encrypt the decrypted interaction data to obtain first ciphertext, and send the first ciphertext to the cloud-side cryptographic layer, so that the cloud-side cryptographic layer processes the first ciphertext data through the TEE-deployed large model; the end-side secure access layer will also decrypt the second ciphertext fed back by the cloud-side cryptographic layer based on the interaction data to obtain the interaction result corresponding to the interaction data.

[0061] The end-side security access layer can initiate an identity verification request to the cloud side through the CA certificate / hardware fingerprint of the hardware security module, the digital signature of the software encryption component, and the like during identity authentication, to ensure that the identity of the terminal device is trusted. At the same time, the identity of the accessed terminal device can also be authenticated, to further improve the security of the shared data between the end and the cloud.

[0062] In one embodiment, the cloud-side encryption layer is deployed with a TEE cluster including a cloud-side TEE and a heterogeneous extended TEE, and a large model is run through the cloud-side TEE and the heterogeneous extended TEE for large model calculation. The cloud-side TEE and the heterogeneous extended TEE in the TEE cluster can be different architecture servers and / or different types of computing components, which are adapted through a hardware adaptation interface (for example, a standardized hardware adaptation interface, etc.). Moreover, the TEE cluster can be horizontally extended, horizontally reduced, or partially replaced, that is, new heterogeneous extended TEEs are added to the TEE cluster, existing heterogeneous extended TEEs in the TEE cluster are deleted, and / or existing heterogeneous extended TEEs in the TEE cluster are replaced with new heterogeneous extended TEEs, and the like. In this way, when the TEE cluster cannot meet the computing requirements, the heterogeneous extended TEEs can be added to meet the running requirements of the large model, or to improve the parallel running efficiency of the large model, thereby improving the response speed of the large model; when the computing power of the TEE cluster is excessive, the heterogeneous extended TEEs can be reduced to avoid waste of hardware resources; and when a heterogeneous extended TEE in the TEE cluster fails, the failed heterogeneous extended TEE can be quickly and timely updated without modifying the application code, thereby improving the hardware adaptation degree. The heterogeneous extended TEE can be a heterogeneous AI extended TEE, such as a domestic security creation hardware, and the like.

[0063] Further, the cloud-side encryption layer further includes a supporting trusted key management service (TKS), a remote attestation service (RAS), a ciphertext model service module, and a ciphertext knowledge base module.

[0064] The trusted key management service (TKS) is used for key management, that is, session key negotiation with the end-side security access layer, and synchronization of the negotiated session key to the TEE cluster, to ensure consistency of the encryption and decryption keys used by the end-side security access layer and the cloud-side encryption layer. Meanwhile, the trusted key management service can also generate a global key for the TEE cluster in the cloud-side encryption layer, to establish cross-domain key synchronization with the end-side security access layer through a standardized key negotiation protocol, to ensure consistency of the encryption and decryption keys of the cloud-side encryption layer and the end-side security access layer.

[0065] The remote attestation service (RAS) is used to verify the cloud-side cryptographic layer based on the environment measurement value of the cloud-side cryptographic layer when the terminal device interacts with the cloud-side cryptographic layer through the end-side secure access layer, to determine that the running environment of the cloud-side cryptographic layer has not been tampered with, and further improve the security of end-to-cloud data sharing.

[0066] The ciphertext model service module is used for ciphertext calculation and storage, that is, the TEE cluster receives the first ciphertext of the end-side secure access layer, decrypts the first ciphertext into plaintext interaction data in the TEE cluster and performs calculation (such as LLM model inference and knowledge base retrieval), and stores the calculation result into the storage module of the cloud-side cryptographic layer after encryption, so that the end-to-cloud shared data is in no plaintext state on the whole connection, further improving the security of cloud shared data. At the same time, the ciphertext model service module stores the encrypted large model into the storage module of the cloud-side cryptographic layer to ensure the security of the large model itself. The ciphertext knowledge base module is used for encrypting the knowledge base to store the knowledge base in the form of a ciphertext knowledge base in the storage module of the cloud-side cryptographic layer, and the retrieval process based on the knowledge base is performed in the TEE cluster, ensuring the security of knowledge data.

[0067] As can be seen, the knowledge base, the large model, the interaction data transmitted by the terminal device to the cloud-side cryptographic layer, and the interaction result formed by the large model based on the interaction data are all stored in the form of ciphertext in the storage module of the cloud-side cryptographic layer, and are in plaintext state only in the TEE cluster. That is, the large model, the knowledge base, the interaction data transmitted by the terminal device to the cloud-side cryptographic layer, and the interaction result formed by the large model based on the interaction data are in ciphertext state in other storage locations of the cloud-side cryptographic layer except in the TEE cluster, so that the end-to-cloud shared data is in no plaintext state on the whole connection, further improving the security of cloud shared data.

[0068] Based on the above end-to-cloud collaboration trusted execution environment, an embodiment of the present application provides an end-to-cloud collaborative large model ciphertext running method, as shown in Figure 2 and Figure 3 The method comprises the following steps:

[0069] S10, the terminal device generates a session key through negotiation with the cloud-side cryptographic layer through the end-side secure access layer.

[0070] Specifically, the end-side security access layer can be deployed in the terminal device or externally to the terminal device. When the end-side security access layer is deployed in the terminal device, the terminal device can directly invoke the end-side security access layer to generate a session key through negotiation with the end-side security access layer. When the end-side security access layer is externally deployed to the terminal device, the terminal device can access the end-side security access layer to generate a session key through negotiation with the end-side security access layer. When the end-side security access layer is externally deployed to the terminal device, the terminal device can access the end-side security access layer, and the end-side security access layer can perform identity authentication on the terminal device, for example, identity authentication based on a device identifier of the terminal device or identity authentication based on account information sent by the terminal device.

[0071] The terminal device can apply for a certificate / key through the end-side security access layer (such as a hardware / software security module), and the cloud-side cryptographic layer returns the certificate and encryption / decryption key based on the request of the end-side security access layer. The session key is obtained through negotiation between the terminal device and the cloud-side cryptographic layer through the end-side security access layer, and is used to encrypt the transmission data between the end-side security access layer and the cloud-side cryptographic layer. After the cloud-side cryptographic layer negotiates the session key with the end-side security access layer, the cloud-side cryptographic layer synchronizes the session key to the TEE cluster, so that the TEE cluster can decrypt the first ciphertext encrypted based on the session key transmitted by the end-side security access layer based on the session key.

[0072] It should be noted that after the cloud-side cryptographic layer synchronizes the session key to the TEE cluster, the cloud-side cryptographic layer can delete the session key so that only the TEE cluster knows the session key, thereby avoiding the session key stored in the storage space other than the TEE cluster in the cloud-side cryptographic layer from being stolen, and thus avoiding the end-cloud shared data from being stolen. Of course, in actual application, the session key can also be encrypted and stored in the storage space other than the TEE cluster in the cloud-side cryptographic layer.

[0073] In one embodiment, the terminal device generates a session key through negotiation with the cloud-side cryptographic layer through the end-side security access layer specifically includes:

[0074] The terminal device sends device information to the cloud-side cryptographic layer through the end-side security access layer to apply for a security credential;

[0075] The terminal device sends a key negotiation request carrying a security credential to the cloud-side cryptographic layer through the end-side security access layer, so that the cloud-side cryptographic layer generates a session key through negotiation with the end-side security access layer based on the key negotiation request, and synchronizes the session key to the TEE cluster.

[0076] Specifically, the security credential is the basis for the terminal device to negotiate the session key with the cloud-side encryption layer, and the device information is used as the basis for the terminal device to apply for the security credential, wherein the device information can include the device model, the end-side security access layer identifier, and the type of the scene where the terminal device is located, etc. After receiving the device information, the cloud-side encryption layer can verify the terminal device based on the device information through the cloud-side security management platform, wherein the verification can include device legality verification, scene adaptability verification, and device type adaptability verification. The device legality verification is to verify the security of the terminal device based on the end-side security access layer identifier. The scene adaptability verification is to verify the adaptability of the scene where the terminal device is located to the applicable scene of the large model based on the type of the scene where the terminal device is located. The device type adaptability verification is to verify the applicability of the scene where the terminal device is located to the applicable scene of the large model based on the device model.

[0077] It should be noted that when the verification of the terminal device based on the device information through the cloud-side security management platform is passed, the security credential is generated for the terminal device, and the security credential is fed back to the terminal device through the end-side security access layer. Otherwise, when the verification of the terminal device based on the device information through the cloud-side security management platform fails, the security credential can not be generated for the terminal device, or the terminal device is fed back with different feedback information to prompt the reason for the failure of the verification, and the number of times of the verification is recorded. When the number of times of the verification failure within a preset time reaches a preset number threshold, a warning information is sent to the end-side security access layer.

[0078] Further, after the terminal device obtains the security credential, the terminal device sends a key negotiation request carrying the security credential to the cloud-side encryption layer, and the cloud-side encryption layer (such as the TKS module in the cloud-side encryption layer) and the end-side security access layer negotiate to generate a session key SK through a key exchange protocol (such as an ECC protocol, etc.), and the session key is used for encrypting and decrypting the data shared by the end and the cloud. At the same time, the cloud-side encryption layer synchronizes the session key SK to the TEE cluster, so that the cloud-side TEE and the heterogeneous extended TEE in the TEE cluster are aware of the session key.

[0079] In one embodiment, before the terminal device sends the device information to the cloud-side encryption layer through the end-side security access layer to apply for the security credential, the identity of the terminal device can also be verified. Specifically, after a user initiates a service login including account information to the large model service backend through the terminal device (such as a mobile phone, a PC, a vehicle-mounted terminal, a UAV control system, etc.), the large model service backend verifies the validity of the service login based on the account information. If the validity verification is passed, a key string token is fed back to the terminal device, which is used as the identity credential of the terminal device to access the large model. The account information can include the account name and the account password, or the terminal device identity identifier (such as the hardware fingerprint, the certificate information, etc.).

[0080] S20, the terminal device sends, to the TEE cluster through the end-side secure access layer, first ciphertext generated based on the session key.

[0081] Specifically, the first ciphertext is encrypted based on the session key on interaction data that needs to be sent to the cloud-side encryption layer, that is, after obtaining the session key, the terminal device receives the interaction data input by the user, and encrypts the interaction data using the session key to obtain the first ciphertext data. In addition, after receiving the interaction data, the terminal device can first preprocess the interaction data, and then encrypt the preprocessed interaction data. Based on this, before encrypting the interaction data based on the session key, the method further comprises:

[0082] The terminal device receives the interaction data of the user, and performs lightweight preprocessing on the interaction data to reduce the data transmission amount, wherein the preprocessed interaction data is stored in the terminal device in the form of ciphertext.

[0083] Specifically, the interaction data can be obtained by keyboard input, voice quality, device sensor collection, and external device sending, etc., and can be text data, image data, audio data, and video data, etc. For example, the interaction data is text data input by the user to the terminal data, and the content of the text data can be "analyze my physical examination report", "generate aircraft patrol data anomaly analysis result", "output vehicle-mounted system fault diagnosis suggestion", etc.

[0084] The lightweight preprocessing is used to reduce the data amount of the interaction data to reduce the data transmission amount, wherein the lightweight preprocessing can include compression processing, format conversion processing, abnormal data deletion processing, etc. In addition, after preprocessing the interaction data, the preprocessed interaction data can be encrypted using the session key to obtain the first ciphertext, and the preprocessed interaction data can also be encrypted using the encryption key stored in the terminal device to obtain encrypted interaction data, and the interaction data is stored in the storage space of the terminal device, so that the interaction data is stored in the terminal device in the form of ciphertext, and the end-side storage and the end-side to cloud-side transmission are both in the ciphertext state.

[0085] In one embodiment, the terminal device sends, to the TEE cluster through the end-side secure access layer, first ciphertext generated based on the session key specifically comprises:

[0086] The terminal device obtains the environment measurement value of the TEE cluster through the end-side secure access layer, and verifies the cloud-side TEE environment based on the environment measurement value.

[0087] When the cloud TEE environment is verified, the terminal device generates first ciphertext based on the session key, and distributes the first ciphertext to the TEE cluster through the terminal-side secure access layer.

[0088] Specifically, the environment metric value is used to reflect the running environment state of the TEE cluster, and the cloud TEE environment can be verified through the environment metric value to determine whether the cloud TEE environment is tampered. To this end, the terminal device can send the terminal device to the cloud-side cryptographic layer before sending the first ciphertext to the cloud-side cryptographic layer. Specifically, the terminal device can send an environment metric value acquisition request carrying a security credential to the cloud-side cryptographic layer, the service gateway in the cloud-side cryptographic layer calls a remote attestation service (RAS), and the remote attestation service (RAS) sends the environment metric value to the terminal device. The terminal device verifies the environment metric value through the remote server, such as comparing the environment metric value with a preset baseline to obtain a cloud TEE environment verification result. When the verification result is that the cloud TEE environment is verified, the terminal device generates first ciphertext based on the session key, and distributes the first ciphertext to the TEE cluster through the terminal-side secure access layer; otherwise, when the verification result is that the cloud TEE environment is not verified, the cloud-side cryptographic layer can prompt that the cloud TEE environment is abnormal.

[0089] It should be noted that the terminal device can first generate first ciphertext based on the session key, and then verify the cloud TEE environment of the cloud-side cryptographic layer; or the terminal device generates first ciphertext based on the session key, and configures the first ciphertext in the environment metric value acquisition request when sending the environment metric value acquisition request to the cloud-side cryptographic layer. The cloud-side cryptographic layer verifies the cloud TEE environment through the remote attestation service (RAS) after receiving the environment metric value acquisition request, and distributes the first ciphertext to the TEE cluster when the verification is passed. In the verification process, the first ciphertext can be stored in the storage space of the cloud-side cryptographic layer, which can reduce the communication frequency between the terminal device and the cloud-side cryptographic layer, and verify the cloud TEE environment of the cloud-side cryptographic layer without the terminal device.

[0090] In one embodiment, the shared data between the terminal device and the cloud can be privacy data or non-privacy data, and the non-privacy data can not be encrypted or decrypted. Therefore, before the terminal device sends the first ciphertext generated based on the session key to the TEE cluster through the terminal-side secure access layer, the method further comprises:

[0091] The terminal device detects its own running mode;

[0092] when the operation mode is the privacy protection mode, the terminal device sends, to the TEE cluster through the end-side secure access layer, first ciphertext generated based on the session key;

[0093] when the operation mode is the non-privacy protection mode, the terminal device sends, to the TEE cluster through the end-side secure access layer, plaintext interaction data used to generate the first ciphertext.

[0094] Specifically, the operation mode includes a privacy protection mode and a non-privacy protection mode, the privacy protection mode indicating that shared data between the terminal device and the cloud needs to be protected, and the non-privacy protection mode indicating that shared data between the terminal device and the cloud does not need to be protected. To this end, when the operation mode is the privacy protection mode, the terminal device encrypts the interaction data by using the session key to form the first ciphertext, and stores and transmits the interaction data in the form of the ciphertext; when the operation mode is the non-privacy protection mode, the terminal device can directly store and transmit the interaction data, that is, transmit and store the interaction data in the form of plaintext. The application sets the privacy protection mode and the non-privacy protection mode, which can protect the privacy data on the one hand, and adapt to scenarios with higher real-time requirements (such as vehicle emergency instruction response and unmanned aerial vehicle real-time inspection feedback) on the other hand.

[0095] S30, the terminal device receives second ciphertext formed by the TEE cluster based on the first ciphertext, and decrypts the second ciphertext to obtain an interaction result.

[0096] Specifically, the second ciphertext is obtained by the TEE cluster responding to the first ciphertext by using a large model and / or a knowledge base, that is, the second ciphertext is in the form of ciphertext of an interaction result of the interaction data corresponding to the first ciphertext running through the large model and / or the knowledge base, that is, the interaction result is encrypted by using the session key.

[0097] In one embodiment, the process of forming the second ciphertext is as follows:

[0098] the TEE cluster decrypts the first ciphertext based on the session key to obtain plaintext interaction data;

[0099] the TEE cluster calls a confidential knowledge base to obtain related data corresponding to the plaintext interaction data, and / or determines a large model running result corresponding to the plaintext interaction data by using a large model deployed in the TEE cluster, to obtain an interaction result;

[0100] the TEE cluster encrypts the interaction result by using the session key to obtain the second ciphertext.

[0101] Specifically, the TEE cluster synchronizes the session key, so that after receiving the first ciphertext, the TEE cluster can decrypt the first ciphertext using the session key to obtain the plaintext interaction data, that is, the interaction data is in plaintext state in the TEE cluster. Then, the cloud-side TEE in the TEE cluster can generate a running prompt based on the plaintext interaction data (for example, supplement the context in combination with the terminal scenario, such as supplementing the vehicle model parameter in the vehicle scenario, supplementing the inspection area information in the aircraft scenario, etc.), and then determine the interaction result corresponding to the interaction data by using the large model deployed in the heterogeneous extended TEE and / or the ciphertext-stored confidential knowledge base in the form of ciphertext, for example, calling the confidential knowledge base, and decrypting the knowledge data in the confidential knowledge base in the TEE cluster to plaintext state, and then comparing the interaction data with the knowledge data in plaintext state to obtain the relevant data corresponding to the plaintext interaction data; or inputting the running prompt into the large model deployed in the TEE cluster, and outputting the large model running result corresponding to the plaintext interaction data through the large model.

[0102] It should be noted that the confidential knowledge base in the embodiments of the present application is an RAG (retrieval augmented generation) knowledge base for a large model, which can be stored in the form of ciphertext vector and retrieved in a closed loop in the TEE to improve the security of the RAG knowledge base.

[0103] Further, after obtaining the interaction result, the TEE cluster encrypts the interaction result using the session key to obtain the second ciphertext, and feeds back the second ciphertext to the terminal device, so that the terminal device obtains the second ciphertext corresponding to the first ciphertext. Wherein, after obtaining the second ciphertext, the terminal device can decrypt the second ciphertext using the session key, and display the decrypted interaction result in the form of terminal device type adaptation display. For example, the terminal device is a mobile terminal type or a desktop type, which can display the interaction result in the form of text and image; the terminal device is a vehicle terminal type, which can display the interaction result through voice and / or central control screen; the terminal device is an unmanned device type, which can store or upload the associated platform of the unmanned device in the form of instruction report.

[0104] It should be noted that after obtaining the interaction result through the TEE cluster, the interaction result is encrypted in the TEE cluster to obtain the second ciphertext, and the second ciphertext exists in other positions in the cloud-side encryption layer except the TEE cluster, that is, for the cloud-side encryption layer, the interaction data and the interaction result are in a secret state except in the TEE cluster. In addition, when the terminal device needs to store the interaction result, the interaction result will be stored in the form of ciphertext.

[0105] In an embodiment, the knowledge base in the cloud-side encryption layer and the large model deployed in the TEE cluster are both pre-deployed in the cloud-side encryption layer, and the knowledge base and the large model are stored in the cloud-side encryption layer in the form of ciphertext, avoiding leakage of the knowledge base and the large model and protecting the security of the knowledge base and the large model. The deployment process of the knowledge base and the large model can be as follows:

[0106] The cloud-side encryption layer receives the large model and / or the knowledge base uploaded by the development end, and stores the encrypted knowledge base and the encrypted large model after encrypting the large model and / or the knowledge base.

[0107] The cloud-side encryption layer deploys a model service image in the TEE cluster based on the deployment request sent by the development end, so as to call the encrypted knowledge base and the encrypted large model through the model service image.

[0108] Specifically, the large model and / or the knowledge base are uploaded to the cloud-side encryption layer by the development end, and before the development end uploads the large model and / or the knowledge base to the cloud-side encryption layer, the cloud-side encryption layer will perform identity verification on the development end, and issue a development end key and an image data signature to the development end after the identity verification is passed. For example, the development end will send identity qualification certificates, such as account information of the development end, device identifier of the development end, etc., to the cloud-side encryption layer; the cloud-side encryption layer will perform identity verification on the development end based on the identity qualification certificates, and if the identity verification is passed, the development end key will be sent to the development end, so that the development end uploads the large model and / or the knowledge base, such as the RAG knowledge base, based on the development end key.

[0109] Further, in order to avoid leakage of the uploaded large model and / or knowledge base, the cloud-side encryption layer will encrypt the large model and / or knowledge base (for example, encrypt the large model and / or knowledge base into a ciphertext vector, etc.) after receiving the large model and / or knowledge base, so that the large model and / or knowledge base are stored in the cloud-side encryption layer in the form of ciphertext, that is, the encrypted knowledge base and the encrypted large model stored in the cloud-side encryption layer, which can manage the large model and / or knowledge base in the encrypted state and reduce the risk of leakage of model assets and knowledge data. The knowledge base is the RAG knowledge base, which is encrypted by the cloud-side encryption layer using the development end key issued by the cloud-side encryption layer.

[0110] Further, since the large model and / or the knowledge base are stored in the cloud-side encryption layer in the form of ciphertext, the TEE cluster in the cloud-side encryption layer needs to call the large model and / or the knowledge base. Thus, after storing the ciphertext knowledge base and the ciphertext large model in the cloud-side encryption layer, a model service image can be deployed in the TEE cluster to form a model service, and the ciphertext knowledge base is called through the model service, and the ciphertext large model is deployed in the TEE cluster in plaintext. The model service image is uploaded by the development end, and the cloud-side encryption layer can encrypt the cloud-side encryption layer, for example, adding an image digital signature to the model service image; then when the cloud-side encryption layer deploys the model service image in the TEE cluster based on the deployment request sent by the development end, the image digital signature of the development end is verified, and when the image digital signature of the development end is consistent with the image digital signature in the model service image, the model service image is deployed in the TEE cluster.

[0111] In one embodiment, the cloud-side encryption layer deploys the model service image in the TEE cluster based on the deployment request sent by the development end, to call the ciphertext knowledge base and the ciphertext large model through the model service image, specifically comprising:

[0112] The cloud-side encryption layer deploys the model service image in the TEE cluster based on the deployment request sent by the development end.

[0113] The cloud-side encryption layer receives the calling application sent by the development end carrying the deployment credentials, and verifies the calling application through the security management platform.

[0114] The cloud-side encryption layer calls the ciphertext large model through the model service image after the calling application is verified, and deploys the called ciphertext large model in the TEE cluster after decryption.

[0115] Specifically, deploying the model service image in the TEE cluster means deploying the model service image in the cloud-side TEE and the heterogeneous extended TEE in the TEE cluster, and generating a standardized model calling interface, so as to call the ciphertext large model to the TEE cluster through the standardized model calling interface, and the TEE cluster decrypts the ciphertext large model to deploy the decrypted large model. The calling application is sent by the development end to the cloud-side encryption layer, which carries the deployment credentials (such as the development end key) of the development end, and the cloud-side encryption layer verifies the development end based on the deployment credentials, and issues a calling certificate after verification, so that the development end calls the ciphertext large model and the ciphertext knowledge base (such as the ciphertext RAG knowledge base) based on the calling certificate, ensures the legitimacy of the development end, and makes it impossible for the attack end and the cloud administrator to obtain the ciphertext large model and the ciphertext knowledge base stored in ciphertext, thereby improving the security of the RAG knowledge base and the large model.

[0116] The application embodiment first performs identity verification on the development end, receives the knowledge base and large model uploaded by the development end after the identity verification is passed, stores the knowledge base and large model in the form of ciphertext, then the development end re-sends the deployment application, and verifies the development end based on the development end key generated during the identity verification, so that the development end is double-verified, the security of the development end is improved, and the reliability of the large model deployed in the TEE cluster is improved.

[0117] Based on the above end-cloud collaborative large model secret state running method, the embodiment provides an end-cloud collaborative large model secret state running system, as shown in Figure 1 The system includes an application end-cloud collaboration trusted execution environment, the application end-cloud collaboration trusted execution environment includes an end-side secure access layer 100 and a cloud-side secret algorithm layer 200, and the cloud-side secret algorithm layer 200 is deployed with a TEE cluster.

[0118] The end-side secure access layer is configured to negotiate a production session key with the cloud-side secret algorithm layer, send a first ciphertext generated based on the session key to the TEE cluster, receive a second ciphertext formed by the TEE cluster based on the first ciphertext, and decrypt the second ciphertext to obtain an interaction result.

[0119] The cloud-side secret algorithm layer is configured to negotiate a production session key with the end-side secure access layer, and synchronize the session key to the TEE cluster, wherein the TEE cluster is configured to decrypt the first ciphertext based on the session key to obtain plaintext interaction data, call a secret knowledge base to obtain related data corresponding to the plaintext interaction data, and / or determine a large model running result corresponding to the plaintext interaction data through a large model deployed in the TEE cluster to obtain an interaction result, and encrypt the interaction result using the session key to obtain the second ciphertext.

[0120] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for dense-state operation of a large-scale model with edge-cloud collaboration, characterized in that, An application-side cloud collaborative trusted execution environment, comprising an endpoint-side secure access layer and a cloud-side secure computing layer, wherein the cloud-side secure computing layer is deployed with a TEE cluster; the method includes: The terminal device negotiates and generates a session key with the cloud-side secure access layer through the terminal-side secure access layer, wherein the session key is synchronized to the TEE cluster; The terminal device sends a first ciphertext generated based on the session key to the TEE cluster through the end-side secure access layer; The terminal device receives the second ciphertext formed by the TEE cluster based on the first ciphertext, and decrypts the second ciphertext to obtain the interaction result. The formation process of the second ciphertext is as follows: The TEE cluster decrypts the first ciphertext based on the session key to obtain plaintext interactive data; The TEE cluster calls the encrypted knowledge base to obtain relevant data corresponding to the plaintext interaction data, and / or determines the running result of the large model corresponding to the plaintext interaction data through the large model deployed in the TEE cluster, so as to obtain the interaction result; The TEE cluster uses the session key to encrypt the interaction result to obtain a second ciphertext.

2. The method for dense-state operation of a large-scale model with end-to-cloud collaboration according to claim 1, characterized in that, The TEE cluster includes cloud-side TEE and heterogeneous extended TEE, and one or more operations such as adding heterogeneous extended TEE, deleting heterogeneous extended TEE, and modifying heterogeneous extended TEE can be performed on the TEE cluster.

3. The method for dense-state operation of a large-scale model with edge-cloud collaboration according to claim 1, characterized in that, The endpoint security access layer includes a hardware security module and / or a software encryption component, enabling the endpoint security access layer to adapt to various types of terminal devices. When the endpoint security access layer includes a hardware security module and a software encryption component, the key generated by the software encryption component can be entrusted to the hardware security module.

4. The method for dense-state operation of a large-scale model with edge-cloud collaboration according to claim 1, characterized in that, The terminal device sends the first ciphertext generated based on the session key to the TEE cluster through the edge-side secure access layer, specifically including: The terminal device obtains the environmental metric value of the TEE cluster from the cloud-side secure access layer through the terminal side secure access layer, and verifies the cloud-side TEE environment based on the environmental metric value; When the cloud-based TEE environment verification is successful, the terminal device generates a first ciphertext based on the session key and distributes the first ciphertext to the TEE cluster through the edge-side security access layer.

5. The method for dense-state operation of a large-scale model with end-to-cloud collaboration according to claim 1 or 4, characterized in that, The first ciphertext is obtained by encrypting the interaction data based on the session key, and before encrypting the interaction data based on the session key, the method further includes: The terminal device receives user interaction data and performs lightweight preprocessing on the interaction data to reduce the amount of data transmission. The preprocessed interaction data is stored in encrypted form in the terminal device.

6. The method for dense operation of a large-scale model in edge-cloud collaboration according to claim 1, characterized in that, Before the terminal device sends the first ciphertext generated based on the session key to the TEE cluster through the end-side secure access layer, the method further includes: The terminal device detects its own operating mode; When the operating mode is privacy protection mode, the terminal device sends the first ciphertext generated based on the session key to the TEE cluster through the end-side security access layer; When the operating mode is non-privacy protection mode, the terminal device sends plaintext interactive data for generating the first ciphertext to the TEE cluster through the end-side security access layer.

7. The method for dense-state operation of a large-scale model with edge-cloud collaboration according to claim 1, characterized in that, The terminal device negotiates and generates a session key with the cloud-side secure access layer through the terminal-side secure access layer, specifically including: Terminal devices send device information to the cloud-side secure computing layer through the edge-side secure access layer to request security credentials; The terminal device sends a key negotiation request carrying security credentials to the cloud-side secure computing layer through the end-side secure access layer, so that the cloud-side secure computing layer can negotiate and generate a session key with the end-side secure access layer based on the key negotiation request, and synchronize the session key to the TEE cluster.

8. The method for dense-state operation of a large-scale model with edge-cloud collaboration according to claim 1, characterized in that, The method further includes: The cloud-side encrypted computing layer receives the large model and / or knowledge base uploaded by the development end, and stores the encrypted knowledge base and encrypted large model after encrypting the large model and / or knowledge base. The encryption uses the development end key issued by the cloud-side encrypted computing layer to the development end. The cloud-side secure computing layer deploys a model service image on the TEE cluster based on the deployment request sent by the development end, so as to call the secure knowledge base and secure large model through the model service image. The model service image contains the image digital signature issued by the cloud-side secure computing layer to the development end.

9. The method for dense-state operation of a large-scale model with edge-cloud collaboration according to claim 8, characterized in that, The cloud-side secure computing layer deploys a model service image on the TEE cluster based on a deployment request sent by the development end. Specifically, this involves calling the secure knowledge base and the secure large model through the model service image. The cloud-side secure computing layer deploys model service images on the TEE cluster based on deployment requests sent by the development end; The cloud-side secure computing layer receives a call request carrying deployment credentials from the development end and verifies the call request through the security management platform; After the cloud-side secure computing layer verifies the call request, it calls the secure large model through the model service image, and then decrypts the called secure large model and deploys it on the TEE cluster.

10. A large-scale, dense-state operation system for edge-cloud collaboration, characterized in that, The system includes an application-side cloud collaborative trusted execution environment, which includes an edge-side secure access layer and a cloud-side secure computing layer. The cloud-side secure computing layer is deployed with a TEE cluster. The endpoint security access layer is used to negotiate and generate a session key with the cloud-side secure computing layer, send a first ciphertext generated based on the session key to the TEE cluster, receive a second ciphertext formed by the TEE cluster based on the first ciphertext, and decrypt the second ciphertext to obtain the interaction result. The cloud-side secure computing layer is used to negotiate and generate a session key with the edge-side secure access layer, and synchronize the session key to the TEE cluster. The TEE cluster is used to decrypt the first ciphertext based on the session key to obtain plaintext interaction data; call the encrypted knowledge base to obtain relevant data corresponding to the plaintext interaction data, and / or determine the running result of the large model corresponding to the plaintext interaction data through the large model deployed in the TEE cluster to obtain the interaction result; and encrypt the interaction result using the session key to obtain the second ciphertext.

Images