Network architecture for artificial intelligence model protection

Abstract

Remote (e.g., cloud, edge cloud) computing environments may be improved by configuring client devices to function as extending the cloud computing environment, such as by coordinating execution and processing of data on the client devices. Secure content or privacy-sensitive content is confidentially protected when in a cloud computing environment or client device, and may be moved to the client device and decrypted to a client local container for various user productivity handling workloads, reducing or eliminating privacy issues. This solution also provides improved security for sharing and use of pre-trained AI models on client devices, such as by extending attestation and provisioning capabilities from a cloud computing environment into clients and by utilizing cloud computing techniques on clients that protect cloud assets. This provides an improved ability to protect the trained AI model and avoid exposing the AI model training process on the client device.

Description

Technical Field

[0001] Embodiments described herein relate to cloud computing resource management. Background Art

[0002] Software and other electronic services are increasingly being executed in cloud computing environments (e.g., in the "cloud"). However, this method of use has many challenges, such as higher costs for cloud service providers (CSPs), high network latency for software execution, exclusion of offline use, user privacy or confidentiality issues, and other challenges. In a privacy example, when private or confidential content is created online using a cloud tool (e.g., Office 365), functions such as autofill, voice to text, and grammar checking are all running in the cloud. When private or confidential content is created online, information may be inadvertently disclosed publicly when the original confidential content is sent to the cloud over the network and analyzed for grammar, spell checking, or other analysis or suggestions. This may expose the service provider to an increased risk of violating privacy regulations (such as the General Data Protection Regulation (GDPR)). BRIEF DESCRIPTION OF THE DRAWINGS

[0003] In the accompanying drawings, which are not necessarily drawn to scale, the same reference numerals may describe similar components in different views. The same numerals with different letter suffixes may represent different instances of similar components. Some embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which:

[0004] Figure 1 is a block diagram illustrating a first cloud computing environment according to an embodiment.

[0005] Figure 2 is a block diagram illustrating a second cloud computing environment according to an embodiment.

[0006] Figure 3 is a block diagram illustrating a secure AI model deployment method and device according to an embodiment.

[0007] Figure 4 is a block diagram illustrating local accelerator provisioning according to an embodiment.

[0008] Figure 5 is a flow chart illustrating a method for cloud computing provisioning of trained artificial intelligence models within an edge / cloud computing environment according to an embodiment.

[0009] Fig. 6A and 6B An overview of example components within a computing device in an edge computing system according to an embodiment is provided.

[0010] Figure 7 is a block diagram showing an overview of a configuration for edge computing according to an embodiment.

[0011] Figure 8 The operational layers between endpoints, edge cloud, and cloud computing environments according to an embodiment are described.

[0012] Fig. 9 Example methods for networking and services in edge computing systems according to embodiments are described.

[0013] Fig.10 An example software distribution platform for distributing software according to an embodiment is described.

[0014] Fig.11 Depicted is an example of an infrastructure processing unit (IPU) according to an embodiment. DETAILED DESCRIPTION

[0015] Technical solutions to technical problems faced with cloud computing environments may include utilizing one or more client devices (e.g., user devices) for hosting and executing cloud services. Client devices may be configured to behave like an extension of a cloud computing environment. In an example solution, cloud services may be moved from a remote computing environment to a more local computing environment without losing the benefits of remote processing provided by a cloud computing or edge computing environment. Cloud services may be moved from a cloud server to a client device, from a cloud server to an edge computing system, or from an edge computing system to a client device. This may be used to provide a variety of cloud services, such as local voice assistance, speech recognition, natural language processing, local AI or artifact removal for video conferencing collaboration, multi-camera stitching, generative AI for creative assistance, and other cloud services.

[0016] By moving cloud services to client devices, end users can avoid sending their secure or privacy-sensitive content to the cloud computing environment for processing. Instead, secure content or privacy-sensitive content is confidentially protected when in the cloud computing environment or client device, and can be moved to the client device and decrypted to a client-local container for various user productivity processing workloads, thereby reducing or eliminating privacy issues. This solution can also improve response time, such as by reducing network latency, working in offline mode, and reducing cloud computing environment data center costs.

[0017] This solution also provides improved security for the sharing and use of pre-trained AI models. IP protection for cloud computing environment providers in a zero-trust computing environment can expose pre-trained AI models to client devices. If run on an untrusted client environment, the AI ​​model IP may be extracted, reverse engineered, or debugged. The AI ​​model can be executed in a cloud service provider (CSP) server with limited exposure to tenants (such as client users). The solution described in this article protects CSP assets (such as pre-trained AI models) on client systems by extending proof and supply capabilities from the cloud computing environment to the client and by utilizing cloud computing technologies that protect CSP assets on the client. This provides improved capabilities to protect trained AI models and avoid exposing the AI ​​model training process on client devices.

[0018] These technical solutions may include device proofs that can be used to protect CSP assets on client devices. This proof may allow cloud service providers (e.g., CSPs, owners of AI models) to have confidence in trusting cloud devices before deploying AI models to client devices. These technical solutions may include AI workload isolation that can be used to isolate AI workloads from each other and isolate AI workloads from the rest of the workloads of client system applications. This isolation can be used to prevent malicious applications in client devices from accessing local AI workloads and date or code assets. Using proofs, the system can allow CSPs to deploy (e.g., provision, instantiate, configure) encrypted AI models from remote servers (e.g., cloud servers, edge computing systems) to cloud devices (e.g., local devices). Using workload isolation, local AI reasoning engines (e.g., in an isolated, protected environment) can securely decrypt AI models and use those models to perform AI tasks locally.

[0019] These technical solutions may include local AI model confidentiality solutions. Such confidentiality solutions can be used to protect AI workload pre-trained models in client devices, which are applicable to both data in memory (e.g., DRAM) and static data using standard encryption algorithms. The confidentiality solution may include a local sealing mechanism with proof of proof to derive a wrapping key for protecting a key encryption key (KEK) at a local storage device for offline next use.

[0020] These technical solutions provide various benefits to end users and CSPs. One benefit includes improved security for cloud providers and their IP asset protection when deploying AI models in untrusted client devices. This improved security can reduce server costs because the reduced AI workload frees up cloud server resources that can be provided. By providing cloud providers with the ability to deploy AI workloads to client devices, these solutions improve the end-user experience (such as by providing offline use and improved network performance (e.g., reduced network throughput requirements)). This can also be used to reduce privacy issues by processing content locally in protected AI workloads on local client devices.

[0021] In the following description, for the purpose of explanation, numerous specific details are set forth in order to provide a thorough understanding of some example embodiments. However, it will be apparent to one skilled in the art that the present disclosure may be implemented without these specific details.

[0022] Figure 1 1 is a block diagram illustrating a first cloud computing environment 100 according to an embodiment. The cloud computing environment 100 includes a cloud computing device 105, such as a cloud server, a cloud processor, an edge computing system, or other cloud computing device. The cloud computing device 105 can be used to execute a first artificial intelligence (AI) model 110 and a second AI model 115. The first AI model 110 can receive input from a first client device 120 and provide intermediate data to the second AI model 115. The second AI model 115 can provide output data to the second client device 125. In another example, the second AI model 115 can provide output data back to the first client device 120, such as to provide feedback to the first client device 120 or to improve one or more processes being executed on the first client device 120.

[0023] In an example, the first client device 120 includes an audio / video (A / V) capture device, such as a webcam and a microphone, that can be used during a video conference. The first client device 120 provides A / V data to the cloud computing device 105. The first AI model 110 within the cloud computing device 105 can process the A / V data, which can include processing the A / V data to provide noise suppression, object segmentation, background replacement (e.g., background blur), or other A / V processing. The first AI model 110 can provide its processed A / V output data to the second AI model 115 to prepare data for the second client device 125, such as language translation, resolution upscaling, or other device-specific processing for the second client device 125. The cloud computing device 105 can provide this processed A / V data to the second client device 125 to generate audio and display video.

[0024] Figure 2 2 is a block diagram illustrating a second cloud computing environment 200 according to an embodiment. The cloud computing environment 200 includes a cloud computing device 205, a first client device 220 configured to execute a first AI model 210, and a second client device 225 configured to execute a second AI model 215. Figure 1 , where the cloud computing device 105 can execute both the first AI model 110 and the second AI model 115. The client devices 220, 225 may include mobile devices, user equipment (UE), autonomous vehicle platforms, information service kiosks, personal computers, or other client devices.

[0025] The first client device 220 may generate data locally for input into the first AI model 210. In an example, the first client device 220 includes an A / V capture device, and the first AI model 210 within the first client device 220 may provide local processing of the A / V data, such as processing the A / V data to provide noise suppression, object segmentation, background replacement, or other local A / V processing using the first AI model 210. The first client device 220 may provide the data generated by the first AI model 210 to the second client device 225 via the cloud computing device 205 for use in the second AI model 215. The second client device 225 may receive the data and provide local processing in the second AI model 215, such as language translation, resolution upscaling, or other processing dedicated to the hardware or software on the second client device 225. In an example, the second client device 225 includes a remote electronic device, and the second AI model 215 may be used to provide real-time audible language translation and resolution upscaling to produce audio and display video on the second client device 225.

[0026] The cloud computing environment 200 can be configured for secure deployment of the first AI model 210 and the second AI model 215. This can include secure deployment of an encrypted copy of the AI ​​model (e.g., an AI workload) from the cloud computing environment 200 to each of the first client device 220 and the second client device 225. After deployment, the AI ​​model is loaded, decrypted, and executed in an isolated (e.g., protected) environment in each of the first client device 220 and the second client device 225. Figure 3 Such a secure deployment is further shown and described.

[0027] Figure 33 is a block diagram illustrating a secure AI model deployment method and apparatus 300 according to an embodiment. A cloud edge device 325 may be used for secure provisioning of one or more pre-trained AI models from the cloud edge device 325 to a client device 315. Although the cloud edge device 325 is shown as an example remote cloud computing device and the client device 315 is shown as an example local computing device, the method and apparatus 300 may also be used to provide secure AI model deployment between a cloud server and an edge computing system or between an edge computing system and a client device. The cloud edge device 325 may include a physical or logical processing device that may be located within a cloud computing environment, may be located within one or more edge nodes along the edge of a network, or may be decomposed (e.g., distributed) between one or more distributed units (DUs) located on a network (e.g., between and including the cloud and the far edge). Similarly, the client device 315 may include a physical or logical processing device that is local to a user or remote from the user but controlled by a client, such as an electronic user device, a mobile device, an endpoint device, an endpoint node, or other user device.

[0028] At step 310, the attestation key may be provisioned (e.g., provided) from the attestation verifier 305 to the client device 315. The attestation verifier 305 may include an attestation verification node (e.g., a cloud edge device), an attestation verification software as a service (SAAS), an attestation verification device, or another attestation verification logical or physical provisioning entity. Figure 3 The proof verifier 305 is shown separate from the cloud edge device 325 and separate from the client device 315, but the proof verifier 305 can be physically or logically within the cloud edge device 325 or within the client device 315. In an example, the proof verifier 305 generates a proof key and supplies the key to the client device 315 at the time of manufacturing or deployment. The proof verifier 305 can generate a proof key that is specific to the client device 315, such as using a device identification number or other uniquely identifiable characteristic to generate the proof key. The proof verifier 305 can include a closed system in which only one party has control over the ability to generate or supply a proof key. The proof verifier 305 can include an open proof system in which multiple parties can have the ability to generate or supply a proof key, such as to share a user-developed AI model between users.

[0029] The attestation key may include a symmetric key or an attestation key pair to provide an asymmetric key. The client device 315 may include trusted hardware, and the key may be stored in a secure storage device within the trusted hardware. The attestation key may be stored so that it can only be accessed by a trusted runtime environment such as a trusted execution environment (TEE), which may include a static confidentiality protected memory (e.g., secure DRAM or secure flash storage device) stored on this client device 315. The client device 315 may include integrity protection, such as a hardware root of trust (HWROT). The provisioning at step 310 may be performed once for each client device 315, such as provisioning in a factory during manufacturing or provisioning on site during initial configuration. The key supplied in step 310 may be changed using a key replacement process.

[0030] At step 320, a user application on the client device 315 may request access to the cloud service before installing and using the cloud service on the client device 315. The cloud service request may include a request for the cloud edge device 325 to deploy the AI ​​model to the client device 315 for use locally at the client device 315.

[0031] At step 330 , the cloud edge device 325 may request remote attestation in response to receiving the cloud service request. The cloud edge device 325 needs to attest the client device 315 before deploying the AI ​​model, so the cloud edge device 325 sends a remote attestation request from the cloud edge device 325 to the client device 315 .

[0032] At step 340, the trusted runtime in the client device 315 can respond to the attestation request. This response can include providing attestation evidence, such as device integrity or measurement quotes signed by an attestation key. The client device can include a remote attestation process architecture that can be used to generate this attestation response. The trusted runtime in the client device 315 can generate a unique device key pair and create an attestation certificate chain for this public key. The client device can include a device identifier combination engine (DICE) attestation architecture that can be used to generate a unique device key pair and create an attestation certificate chain. The private portion of the generated unique device key can be retained inside the trusted runtime environment and can be used in step 390 to unlock the Advanced Encryption Standard (AES) decryption key. The client device 315 can then send those generated attestation materials back to the cloud edge device 325 for attestation.

[0033] At step 350 , cloud edge device 325 may perform attestation verification. This attestation verification may be done with the help of attestation verifier 305 .

[0034] At step 360 , upon successfully attesting the client device 315 , the cloud edge device 325 can extract the unique public key. The unique public key can be extracted from the attestation certificate chain created in step 340 .

[0035] At step 370, the cloud edge device 325 may generate a strong random symmetric key (e.g., a 256+ bit AES key). The cloud edge device 325 may use this key to encrypt one or more requested pre-trained AI models. The cloud edge device 325 may wrap the symmetric encryption key with the cloud device unique public key retrieved in step 360. By wrapping the symmetric encryption key with a unique public key, only the authenticated client device 315 can decrypt the key because only that client device 315 has the private portion of that unique device key pair.

[0036] At step 380, the cloud edge device 325 may send the wrapped symmetric encryption key and the encrypted AI model to the client device 315. Because the symmetric encryption key is wrapped and the AI ​​model is encrypted, they may be sent over untrusted network traffic without exposing either the symmetric encryption key or the AI ​​model.

[0037] At step 390, the client device 315 may receive the wrapped symmetric encryption key and unwrap the symmetric encryption key (e.g., decrypt the symmetric key) using the private portion of the unique device key pair generated in step 340. After unwrap the symmetric encryption key, the client device 315 may use the unwrap key to decrypt the AI ​​model. Once the AI ​​models are decrypted in the memory of the protected environment of the client device 315, they may be loaded and executed in this trusted runtime environment.

[0038] To use the AI ​​model offline, the trusted runtime in the client device 315 can save the encrypted AI model into a local storage device for later use. There is no need to save the encrypted AI models in a secure storage device because their encryption protects the static AI model. The decryption key can also be saved in an internal local secure storage device to keep it securely stored statically, such as in a secure storage device provided by a trusted runtime (e.g., TEE), such as using a software guard extension (SGX) sealing mechanism for sealing. Encryption can be implemented so that the encryption key is only accessible to the trusted runtime by sealing the key wrap encryption key (KEK) in the trusted runtime, where the KEK can be derived using the trusted runtime root key using the proof evidence value as an input to the key generation function. If the runtime environment changes (e.g., due to a deliberate malicious event, due to an accidental event), the trusted runtime may be prevented from regenerating the KEK and will not be able to decrypt the AI ​​model the next time it is used, which provides further protection for the AI ​​model.

[0039] When the AI ​​model is deployed to the client device 315, the device-unique key pair generated in step 340 and the private portion of that key pair may not need to be securely stored. After deployment, the key pair can be permanently discarded by securely erasing it from DRAM or storage to prevent future attacks on the AI ​​model.

[0040] The secure AI model deployment method and apparatus 300 provides the ability to securely provision one or more AI models into a client device 315. This provides improved security by allowing a risk-based approach that is conditional on the certification of the client device 315. This provides the ability to protect the AI ​​model in transit, such as by encrypting and securely deploying the AI ​​model. This also enables the AI ​​model to be protected at runtime and at rest when the AI ​​model is within a trusted runtime execution environment on the client device 315.

[0041] By securely provisioning and deploying the AI ​​model to the client device 315, one or more applications running on the client device 315 can rely on the AI ​​analysis results even when the client device 315 is offline. This can provide an improved user experience and improved privacy for the user and for any user-generated content.

[0042] The steps shown in the secure AI model deployment method and apparatus 300 may be repeated, such as to provide further synchronization between the local AI model and the updated model in the cloud edge device 325. Similarly, one or more AI models may continue to be trained at the client device 315, which provides the ability to reflect user-specific behaviors in the model as a way to further improve the user experience, such as using collaborative machine learning (e.g., federated learning) to train AI models across multiple cloud servers or client devices without exchanging AI models.

[0043] The secure AI model deployment method and device 300 may include various security capabilities that support attestation and create a trusted runtime environment within the client device 315. The trusted runtime may use a trusted boot of a trusted runtime environment that supports attestation such as DICE or SGX as described above. Attestation may include enhanced privacy ID (EPID) provisioning, trusted platform module (TPM) key attestation, trusted domain extension (TDX) provisioning, DICE, or other provisioning. A trusted runtime environment may be provided by a virtualization-based trusted environment (e.g., Hyper-V-based virtualization-based security (VBS), secure operating system (OS) Trusty), trusted execution technology (TXT) hardware extension, software guard extension (SGX), TDX, system management mode (SMM) (e.g., ring-2) operating mode, or other trusted environment.

[0044] The trusted runtime can be used to isolate and protect security-sensitive AI workloads when used at the client device 315. Privacy-sensitive user data can be fed into the AI ​​model in the trusted runtime without exposing the user data to snooping attacks in the cloud computing environment. Local acceleration hardware (such as GPU, visual processing unit (VPU), Gaussian and neural accelerator (GNA)) can be used to offload local but protected operations. The trust equivalence of local acceleration hardware can be evaluated using the Security Protocol and Data Model (SPDM) architecture, the TDX architecture, or another trustworthy verification architecture.

[0045] A secure communication path can be created between a trusted runtime and other trusted storage devices. A graphics processor unit (GPU) or other acceleration hardware (e.g., protected Xe path (PXP), SPDM, TDX) can be used to execute AI model workloads inside a proven, trusted, isolated runtime environment. The inference engine can be loaded as part of the boot process and can be supplied by a cloud edge device 325 or another cloud computing environment. The AI ​​model can be securely supplied to the trusted runtime, and then loaded and executed within the trusted runtime on the client device 315. The encrypted AI model is securely supplied to the trusted runtime using an encryption key that only a valid trusted runtime can use to decrypt the AI ​​model.

[0046] Similarly, the AI ​​model can be encrypted using a key that is accessible only to the trusted runtime by sealing the KEK to the trusted runtime. This can be achieved by deriving the trusted runtime key using the proof evidence value as the input to the key generation function. To enable provisioning, a public KEK can be provided to the cloud edge device 325 to wrap the AI ​​model or to wrap the symmetric key used to encrypt the AI ​​model. When the trusted runtime environment changes due to intentional maliciousness or accidental events, the trusted runtime may not be able to regenerate the KEK or decrypt the AI ​​model the next time it is used. This KEK protection is maintained even during offline operating mode, because the AI ​​model remains encrypted when in the local storage device and can only be decrypted when the current runtime environment matches the original runtime environment.

[0047] The trusted runtime can be updated through a maintenance process of migrating or re-provisioning AI models. This migration can include measuring the new trusted runtime environment before it is fully installed and executed, and then using the current trusted runtime to seed the KEK key generation function with the new measurements. CUR ) is still known, the seed value can be used to generate a new KEK (KEK NEW ). KEK CURCan be used to decrypt stored AI models or symmetric keys. KEK NEW It can then be used to re-encrypt the AI ​​model or symmetric key.

[0048] Multiple trusted environments (e.g., multiple SGX enclaves, multiple TDX guests) can be launched or executed in parallel to serve multiple clients or AI models. The use of multiple trusted environments can improve security and performance by isolating AI workloads from each other.

[0049] Memory encryption can be used to protect AI models at runtime (e.g., in DRAM or memory) and at rest in storage devices, such as using full memory encryption (TME), multi-key TME (MKTME), SGX built-in memory encryption, or other memory encryption. Secure storage devices can use replay protected memory block (RPMB) storage devices to provide anti-replay protection whenever needed, such as using RPMB in universal flash storage devices (UFS) or non-volatile express memory (NVMe) solid-state drive (SSD) storage devices.

[0050] Figure 4 4 is a block diagram illustrating an accelerator supply 400 according to an embodiment. The cloud edge device 410 may be used to securely supply one or more pre-trained AI models from the cloud edge device 410 to the client device 420. The client device 420 may store the AI ​​model in a secure device environment 425. The client device 420 may pass the AI ​​model to a secure accelerator environment 445 in a local accelerator 440 for storage. The local accelerator may include a graphics processing unit (GPU), a visual processing unit (VPU), a Gaussian neural accelerator (GNA), or other accelerator architectures. The secure device environment 425 may include a first security architecture dedicated to the client device 420, and the secure accelerator environment 445 may include a second security architecture dedicated to the local accelerator 440. In an example, the secure device environment 425 may employ an encryption and decryption method used by a general-purpose central processing unit (CPU) of the client device 420, and the secure accelerator environment 445 may employ an encryption and decryption method dedicated to an accelerator architecture (such as a GPU, a VPU, a GNA, or other accelerator architecture). A sealed DICE root of trust may be implemented to prevent any client device 420 or local accelerator 440 from accessing the AI ​​model or other resources without proper certification. The AI ​​model may be sealed using a TPM or DPE (DICE Protection Module Environment) to implement sealing or unsealing of the AI ​​model.

[0051] When an AI model is requested, the local accelerator 440 can decrypt and run the AI ​​model. This accelerator supply 400 of the AI ​​model to the local accelerator 440 can be similar to the process for supplying the AI ​​model to a trusted runtime environment. The accelerator supply 400 may include a certification of the hardware of the local accelerator 440 for providing an isolated AI runtime and determining that the acceleration hardware environment is acceptable. The hardware of the local accelerator 440 can then be evaluated to verify its design or isolation properties. Upon verification, the AI ​​model can be supplied to a trusted environment within the local accelerator 440.

[0052] One or more security policies 430 may be used to improve security at the client device 420 or at the local accelerator 440. The security policy 430 may be implemented and configured by the CSP, and may be used to determine the minimum required strength of the cryptographic algorithm, key size, mode, entropy source, or other security policy requirements. The security policy 430 may be implemented by the local trusted device environment so that the client device 420 or at the local accelerator 440 behaves according to the security policy 430. The security policy 430 may be implemented using random numbers or other data generated using an entropy function executed at the cloud edge device 410 or at the client device 420.

[0053] The attestations described herein may be used to evaluate and reduce the risks associated with distributing AI models to client devices 420 or at local accelerators 440. Similarly, federated learning may include a control plane to coordinate training and synchronize trained models across multiple federated nodes. Trusted paths (e.g., TLS, SPDM) such as trusted paths between a central controller in a cloud edge device 410 and any client device 420 or local accelerator 440 may be used to improve AI model security. A local training controller may be used to coordinate training between client devices 420 or local accelerators 440. A local controller may contribute a locally trained AI model to a cloud or peer node to benefit from peer training, which may cascade the AI ​​model or AI model data into a local accelerator 440. Security policy 430 may be used to determine password strength, identity security, configuration security of attestations, or other security policy requirements to be used by one or more of a secure device environment 425 or a secure accelerator environment 445.

[0054] Figure 5is a flow chart illustrating a method 500 for cloud computing provisioning of trained artificial intelligence (AI) models. The method 500 includes receiving 510 a deployment request for an AI model from a client device. The method 500 includes sending 520 a remote attestation request to the client device in response to the deployment request. The method 500 includes receiving 530 a remote attestation response from the client device. The method 500 includes verifying 540 the remote attestation response. The method 500 includes generating 550 an encrypted model based on the AI ​​model in response to verifying the remote attestation response. The method 500 includes sending 560 the encrypted model to a first trusted environment within the client device.

[0055] The method 500 may further include sending 570 the encryption model to a second trusted environment associated with the accelerator within the client device. The first trusted environment may include a first security architecture specific to the client device, and the second trusted environment may include a second architecture specific to the accelerator and different from the first security architecture.

[0056] The method 500 may further include generating a symmetric key to encrypt the AI ​​model in response to verifying the remote attestation response, wherein the encrypted model is further generated based on the symmetric key. The method 500 may further include generating a wrapping key based on the symmetric key. The method 500 may further include sending the wrapping key to the client device.

[0057] The method 500 may further include provisioning an attestation key to the client computing device prior to receiving the deployment request. Provisioning the attestation key to the client computing device may include storing the attestation key in a confidentiality protected client memory on the client device. The confidentiality protected client memory may include at least one of a confidentiality protected dynamic random access memory and a confidentiality protected flash memory device. Receiving the remote attestation response may include receiving a device offer signed by the attestation key.

[0058] The method 500 may further include receiving a unique device key pair and an attestation certificate chain of an attestation key generated at the client device. The method 500 may further include extracting a public key from the attestation certificate chain after verifying the remote attestation response. Generating a wrapping key is further based on the public key. Verifying the remote attestation response may include sending a remote attestation response to an attestation verifier and receiving a remote attestation verification from the attestation verifier. The method 500 may further include receiving a trained model from the client device, generating the trained model by retraining the AI ​​model at the client device.

[0059] Fig. 6A and Figure 6B An overview of example components within a computing device in an edge computing system 600 according to an embodiment is provided. The edge computing system 600 can be used, for example, to Figure 3 The method and apparatus 300 shown and described provide secure AI model deployment. In another example, the Fig. 6A and Figure 6B The components depicted in the description may be used to implement any of the computing nodes or devices discussed with reference to the present edge computing systems and environments. A corresponding edge computing node may be embodied as a type of device, equipment, computer, or other "thing" capable of communicating with other edge, networking, or endpoint components. For example, an edge computing device may be embodied as a personal computer, a server, a smartphone, a mobile computing device, a smart device, an in-vehicle computing system (e.g., a navigation system), a self-contained device having an external box, housing, etc., or other device or system capable of performing the described functions.

[0060] exist Fig. 6A In the simplified example depicted in , edge computing node 600 includes a computing engine (also referred to herein as a "computing circuit module") 602, an input / output (I / O) subsystem 608 (also referred to herein as an "I / O circuit module"), a data storage device 610 (also referred to herein as a "data storage circuit module"), a communication circuit module subsystem 612, and optionally one or more peripheral devices 614 (also referred to herein as a "peripheral device circuit module"). In other examples, the corresponding computing device may include other or additional components, such as those typically found in a computer (e.g., a display, peripheral devices, etc.). In addition, in some examples, one or more of the illustrative components may be incorporated into another component, or otherwise form part of another component.

[0061] The computing node 600 may be embodied as any type of engine, device, or collection of devices capable of performing various computing functions. In some examples, the computing node 600 may be embodied as a single device, such as an integrated circuit, an embedded system, a field programmable gate array (FPGA), a system on a chip (SOC), or other integrated systems or devices. In an illustrative example, the computing node 600 includes or is embodied as a processor 604 (also referred to herein as a "processor circuit module") and a memory 606 (also referred to herein as a "memory circuit module"). The processor 604 may be embodied as any type of processor capable of performing the functions described herein (e.g., executing an application). For example, the processor 604 may be embodied as (one or more) multi-core processors, microcontrollers, processing units, specialized or dedicated processing units, or other processors or processing / control circuits.

[0062] In some examples, the processor 604 may be embodied as, include, or be coupled to an FPGA, an application specific integrated circuit (ASIC), reconfigurable hardware or hardware circuit module, or other specialized hardware to facilitate the execution of the functions described herein. In some examples, the processor 604 may be embodied as a specialized x-processing unit (xPU), also referred to as a data processing unit (DPU), an infrastructure processing unit (IPU), or a network processing unit (NPU). Such an xPU may be embodied as an independent circuit or circuit package, integrated within a SOC, or integrated with a networking circuit module (e.g., in a SmartNIC or enhanced SmartNIC), an acceleration circuit module, a storage device, a storage disk, or AI hardware (e.g., a GPU, a programmed FPGA, or an ASIC customized to implement an AI model such as a neural network). Such an xPU may be designed to receive, retrieve, and / or otherwise obtain programming outside of a CPU or general processing hardware to process one or more data streams and perform specific tasks and actions of the data stream (such as hosting microservices, performing service management or orchestration, organizing or managing server or data center hardware, managing a service grid, or collecting and distributing telemetry). However, it will be appreciated that xPUs, SOCs, CPUs, and other variations of processors 604 may work in coordination with one another to perform many types of operations and instructions within and on behalf of compute node 600 .

[0063] The memory 606 may be embodied as any type of volatile (e.g., dynamic random access memory (DRAM), etc.) or non-volatile memory or data storage device capable of performing the functions described herein. Volatile memory may be a storage medium that requires power to maintain the state of data stored by the medium. Non-limiting examples of volatile memory may include various types of random access memory (RAM), such as DRAM or static random access memory (SRAM). One specific type of DRAM that may be used in a memory module is synchronous dynamic random access memory (SDRAM).

[0064] In an example, the memory devices (e.g., memory circuit modules) are any number of block-addressable memory devices, such as those based on NAND or NOR technology (e.g., single-level cell (“SLC”), multi-level cell (“MLC”), quad-level cell (“QLC”), triple-level cell (“TLC”), or other NAND). In some examples, the memory device(s) include byte-addressable in-situ write-in-place three-dimensional cross-point memory devices or other byte-addressable in-situ write-in-place non-volatile memory (NVM) devices, such as single-level or multi-level phase change memory (PCM) or phase change memory with switches (PCMS); NVM devices using chalcogenide phase change materials (e.g., chalcogenide glass); resistive memories including metal oxide-based, oxygen vacancy-based, and conductive bridge random access memories (CB-RAM); nanowire memories; ferroelectric transistor random access memories (FeTRAM); magnetoresistive random access memories (MRAM) combined with memristor technology; spin transfer torque (STT)-MRAM; devices based on spintronic magnetic junction memory; devices based on magnetic tunnel junctions (MTJs); devices based on DW (domain walls) and SOT (spin-orbit transfer); thyristor-based memory devices; any combination of the above; or other suitable memories. The memory device may also include a three-dimensional cross-point memory device (e.g., 3D XPoint TM memory) or other byte-addressable in-place write-nonvolatile memory devices. The memory device can refer to the die itself and / or to the packaged memory product. In some examples, 3D crosspoint memory (e.g. 3DXPoint TM Memory 606 may include a transistor-free stackable cross-point architecture in which memory cells are located at the intersection of word lines and bit lines and are individually addressable and in which bit storage devices are based on changes in bulk resistance. In some examples, all or a portion of memory 606 may be integrated into processor 604. Memory 606 may store various software and data used during operation, such as one or more applications, data operated by (one or more) applications, libraries, and drivers.

[0065] In some examples, resistor-based and / or transistor-free memory architectures include nanoscale phase change memory (PCM) devices in which a volume of phase change material resides between at least two electrodes. Portions of the example phase change material exhibit varying degrees of crystalline and amorphous phases, wherein varying degrees of resistance between the at least two electrodes can be measured. In some examples, the phase change material is a chalcogenide-based glass material. Such resistive memory devices are sometimes referred to as memristive devices, which remember the history of currents that previously flowed through them. Stored data is retrieved from the example PCM device by measuring resistance, wherein the crystalline phase exhibits (one or more) relatively lower resistance values ​​(e.g., logical "0") when compared to an amorphous phase having (one or more) relatively higher resistance values ​​(e.g., logical "1").

[0066] The example PCM device stores data for a long period of time (e.g., about 10 years at room temperature). Writing operations (e.g., set to logic "0", set to logic "1", set to an intermediate resistance value) to the example PCM device are accomplished by applying one or more current pulses to at least two electrodes, where the pulses have a specific current amplitude and duration. For example, a long low current pulse (SET) applied to the at least two electrodes may cause the example PCM device to reside in a low resistance crystalline state, while a relatively short high current pulse (RESET) applied to the at least two electrodes causes the example PCM device to reside in a high resistance amorphous state.

[0067] In some examples, the implementation of the PCM device facilitates a non-von Neumann computing architecture that enables computing power in memory. In general, a conventional computing architecture includes a central processing unit (CPU) connected to one or more memory devices via a bus communication. As such, transmitting data between the CPU and the memory consumes a limited amount of energy and time, which is a known bottleneck of the von Neumann computing architecture. However, the PCM device minimizes and in some cases eliminates data transmission between the CPU and the memory by performing some computing operations in the memory. In other words, the PCM device both stores information and performs computing tasks. Such a non-von Neumann computing architecture can implement vectors with relatively high dimensions to facilitate hyper-dimensional computing, such as vectors with 10,000 bits. Relatively large bit width vectors enable computing paradigms that mimic the human brain, which also processes information similar to wide bit vectors.

[0068] Computer circuit module 602 is communicatively coupled to other components of computing node 600 via I / O subsystem 608, which may be embodied as circuit modules and / or components that facilitate input / output operations with computing circuit module 602 (e.g., having processor 604 and / or main memory 606) and other components of computing circuit module 602. For example, I / O subsystem 608 may be embodied as or otherwise include a memory controller hub, an input / output control hub, an integrated sensor hub, a firmware device, a communication link (e.g., a point-to-point link, a bus link, wires, cables, optical guides, printed circuit board traces, etc.), and / or other components and subsystems that facilitate input / output operations. In some examples, I / O subsystem 608 may form part of a system on chip (SoC) and be incorporated into computing circuit module 602 along with one or more of processor 604, memory 606, and other components of computing circuit module 602.

[0069] One or more illustrative data storage devices / disks 610 may be embodied as one or more of any type(s) of physical device(s) configured for short-term or long-term storage of data, such as, for example, a memory device, a memory, a circuit module, a memory card, a flash memory, a hard drive, a solid-state drive (SSD), and / or other data storage devices / disks. Each data storage device / disk 610 may include a system partition that stores firmware code and data for the data storage device / disk 610. Each data storage device / disk 610 may also include one or more operating system partitions that store executable files and data files for an operating system, depending on, for example, the type of computing node 600.

[0070] The communication circuit module 612 may be embodied as any communication circuit, device, or collection thereof capable of enabling communication between the computer circuit module 602 and another computing device (e.g., an edge gateway implementing an edge computing system) over a network. The communication circuit module 612 may be configured to use any one or more communication technologies (e.g., wired or wireless communications) and associated protocols (e.g., cellular networking protocols such as 3GPP 4G or 5G standards, such as IEEE 802.11 / Wireless LAN protocol, wireless WAN protocol, Ethernet, Bluetooth Low Energy, such as IEEE 802.15.4 or IoT protocols, low power wide area networks (LPWAN) or low power wide area (LPWA) protocols, etc.) to achieve such communication.

[0071] The illustrative communication circuit module 612 includes a network interface controller (NIC) 620, which may also be referred to as a host structure interface (HFI). The NIC 620 may be embodied as one or more add-in boards, daughter cards, network interface cards, controller chips, chipsets, or other devices that may be used by the computing node 600 to connect to another computing device (e.g., an edge gateway node). In some examples, the NIC 620 may be embodied as part of a system on a chip (SoC) including one or more processors, or may be included in a multi-chip package that also includes one or more processors. In some examples, the NIC 620 may include a local processor (not shown) and / or a local memory (not shown) that are both local to the NIC 620. In such an example, the local processor of the NIC 620 may be able to perform one or more of the functions of the computing circuit module 602 described herein. Additionally or alternatively, in such an example, the local memory of the NIC 620 may be integrated into one or more components of the client computing node at the board level, the slot level, the chip level, and / or other levels.

[0072] Additionally, in some examples, the respective computing node 600 may include one or more peripheral devices 614. Such peripheral devices 614 may include any type of peripheral devices found in a computing device or server, such as audio input devices, displays, other input / output devices, interface devices, and / or other peripheral devices, depending on the particular type of computing node 600. In further examples, the computing node 600 may be embodied by a respective edge computing node (whether a client, gateway, or aggregation node) in an edge computing system, or a similar form of device, computer, subsystem, circuit module, or other component.

[0073] In a more detailed example, Figure 6BA block diagram of an example of components that may be present in an edge computing node 650 for implementing the techniques described herein (e.g., operations, processes, methods, and a set of methods) is illustrated. This edge computing node 650 provides a closer view of the corresponding components of the node 600 when implemented as a computing device (e.g., a mobile device, a base station, a server, a gateway, etc.) or as a part of a computing device (e.g., a mobile device, a base station, a server, a gateway, etc.). The edge computing node 650 may include any combination of hardware or logic components referenced herein, and it may include any device that can be used with an edge communication network or a combination of such networks or may be coupled with any device that can be used with an edge communication network or a combination of such networks. The components may be implemented as an integrated circuit (IC), a portion thereof, a discrete electronic device, or other module, an instruction set, a programmable logic or algorithm, hardware, a hardware accelerator, software, firmware, or a combination thereof, mounted in the edge computing node 650, or may be implemented as a component otherwise incorporated into a chassis of a larger system.

[0074] The edge computing device 650 may include a processing circuit module in the form of a processor 652, which may be a microprocessor, a multi-core processor, a multi-threaded processor, an ultra-low voltage processor, an embedded processor, an xPU / DPU / IPU / NPU, a dedicated processing unit, a specialized processing unit, or other known processing elements. The processor 652 may be an integrated circuit or a single package in which the processor 652 and other components form a single integrated circuit or a single package (such as the Edison from Intel Corporation of Santa Clara, California). TM or Galileo TM As an example, the processor 652 may include a system on a chip (SoC) based on ArchitectureCore TM CPU processors (such as Quark TM 、Atom TM , i3, i5, i7, i9 or MCU class processors) or can be obtained from However, any number of other processors may be used, such as those available from Advanced Micro Devices, Inc. of Sunnyvale, California. Based on the MIPS Technologies of Sunnyvale, California designs based on, licensed from ARM Holdings Limited or its customers or their licensees or adopters The processor may include, for example, The company's A5-A13 processors, from Snapdragon Technologies TM Processor or OMAP from Texas Instruments TM A processor unit. It can be included in a limited hardware configuration or include more than Figure 6B Processor 652 and accompanying circuit modules may be provided in a single-socket form factor, a multi-socket form factor, or various other formats in all component-less configurations shown.

[0075] The processor 652 can communicate with the system memory 654 through the interconnect 656 (e.g., a bus). Any number of memory devices can be used to provide a quantitative system memory. As an example, the memory 654 can be a random access memory (RAM) designed according to the Joint Electron Device Engineering Council (JEDEC), such as a DDR or mobile DDR standard (e.g., LPDDR, LPDDR2, LPDDR3, or LPDDR4). In a specific example, the memory component can comply with the DRAM standard issued by JEDEC, such as JESD79F for DDR SDRAM, JESD79-2F for DDR2 SDRAM, JESD79-3F for DDR3 SDRAM, JESD79-4A for DDR4 SDRAM, JESD209 for low power DDR (LPDDR), JESD209-2 for LPDDR2, JESD209-3 for LPDDR3, and JESD209-4 for LPDDR4. Such standards (and similar standards) can be referred to as DDR-based standards, and the communication interface of the storage device that implements such standards can be referred to as a DDR-based interface. In various implementations, the individual memory devices may have any number of different package types, such as a single die package (SDP), a dual die package (DDP), or a quad die package (Q17P). These devices may be soldered directly to the motherboard in some examples to provide a lower profile solution, while in other examples the devices are configured as one or more memory modules that are in turn coupled to the motherboard via a given connector. Any number of other memory implementations may be used, such as other types of memory modules, for example different kinds of dual in-line memory modules (DIMMs) including but not limited to micro DIMMs or mini DIMMs.

[0076] To provide persistent storage of information such as data, applications, operating systems, etc., storage device 658 may also be coupled to processor 652 via interconnect 656. In an example, storage device 658 may be implemented with the aid of a solid state hard drive (SSDD). Other means that may be used for storage device 658 include flash memory cards such as secure digital (SD) cards, micro SD cards, extreme digital (XD) picture cards, etc., and universal serial bus (USB) flash drives. In an example, the memory device may be or may include a memory device using chalcogenide glass; a multi-threshold level NAND flash memory; a NOR flash memory; a single-level or multi-level phase change memory (PCM); a resistive memory; a nanowire memory; a ferroelectric transistor random access memory (FeTRAM); an antiferroelectric memory; a magnetoresistive random access memory (MRAM) memory combined with memristor technology; a resistive memory including metal oxide-based, oxygen vacancy-based, and conductive bridge random access memory (CB-RAM); or a spin transfer torque (STT)-MRAM; a device based on spintronic magnetic junction memory; a device based on a magnetic tunnel junction (MTJ); a device based on a DW (domain wall) and a SOT (spin-orbit transfer); a thyristor-based memory device; or a combination of any of the above; or other memory.

[0077] In a low power implementation, the storage device 658 may be on-die memory or registers associated with the processor 652. However, in some examples, a micro hard disk drive (HDD) may be used to implement the storage device 658. Furthermore, any number of new technologies may be used for the storage device 658, such as resistive change memory, phase change memory, holographic memory, or chemical memory, in addition to or in place of the described technologies.

[0078] Components can communicate via interconnect 656. Interconnect 656 can include any number of technologies, including Industry Standard Architecture (ISA), Extended ISA (EISA), Peripheral Component Interconnect (PCI), Extended Peripheral Component Interconnect (PCIx), PCI Express (PCIe), or any number of other technologies. Interconnect 656 can be a proprietary bus used, for example, in SoC-based systems. Other bus systems can be included, such as an Inter-Integrated Circuit (I2C) interface, a Serial Peripheral Interface (SPI) interface, a point-to-point interface, and a power bus, etc.

[0079] The interconnect 656 can couple the processor 652 to the transceiver 666 for communicating with the connected edge device 662. The transceiver 666 can use any number of frequencies and protocols, such as 2.4 gigahertz (GHz) transmission under the IEEE 802.15.4 standard, using a 100 MHz RF transmission such as that provided by the IEEE 802.15.4 standard. Special Interest Groups Defined Low Energy (BLE) standard, or standards, etc. Any number of radio devices configured for a particular wireless communication protocol configuration may be used for connection to the connected edge device 662. For example, a wireless local area network (WLAN) unit may be used to implement a wireless communication protocol in accordance with the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard. Additionally, wireless wide area communications, such as according to cellular or other wireless wide area protocols, may occur via a wireless wide area network (WWAN) unit.

[0080] The wireless network transceiver 666 (or multiple transceivers) can communicate using a variety of standards or radio devices for communication at different distances. For example, the edge computing node 650 can use a local transceiver based on Bluetooth Low Energy (BLE) or another low-power radio device to communicate with nearby devices (e.g., within about 10 meters) to save power. or other medium power radios to reach more distant connected edge devices 662 (e.g., within about 50 meters). The two communication technologies can be performed on a single radio at different power levels or on separate transceivers (e.g., a local transceiver using BLE and a local transceiver using Bluetooth LE). The network is performed on separate mesh transceivers.

[0081] A wireless network transceiver 666 (e.g., a radio transceiver) may be included to communicate with devices or services in the cloud (e.g., edge cloud 695) by means of a local area network protocol or a wide area network protocol. The wireless network transceiver 666 may be a low power wide area (LPWA) transceiver that complies with IEEE802.15.4 or IEEE 802.15.4g standards, etc. The edge computing node 650 may use the LoRaWAN developed by Semtech and the LoRa Alliance. TM (Long Range Wide Area Network) to communicate over a wide area. The techniques described herein are not limited to these techniques, but may be used with any number of other cloud transceivers that implement long range, low bandwidth communications (such as Sigfox) and other techniques. In addition, other communication techniques such as slotted channel hopping described in the IEEE 802.15.4e specification may be used.

[0082] As described herein, in addition to the systems mentioned for the wireless network transceiver 666, any number of other radio communications and protocols may be used. For example, the transceiver 666 may include a cellular transceiver that uses spread spectrum (SPA / SAS) communications for high-speed communications. In addition, any number of other protocols may be used, such as the provision of 4G LTE for medium-speed communications and network communications. Network. As discussed in further detail at the end of this disclosure, the transceiver 666 may include a radio device compatible with any number of 3GPP (3rd Generation Partnership Project) specifications, such as Long Term Evolution (LTE) and fifth generation (5G) communication systems. A network interface controller (NIC) 668 may be included to provide wired communications to nodes of the edge cloud 695 or to other devices such as a connected edge device 662 (e.g., operating in a grid). Wired communications may provide an Ethernet connection or may be based on other types of networks, such as a controller area network (CAN), a local interconnect network (LIN), DeviceNet, ControlNet, Data Highway +, PROFIBUS, or PROFINET, etc. An additional NIC 668 may be included to enable connection to a second network, for example, a first NIC 668 provides communications to the cloud via Ethernet, and a second NIC 668 provides communications to other devices via another type of network.

[0083] Knowing the diversity of types of applicable communications from a device to another component or network, the applicable communication circuit modules used by the device may include or be embodied by the following components: any one or more of components 664, 666, 668, or 670. Thus, in various examples, applicable means for communication (e.g., receiving, transmitting, etc.) may be embodied by such communication circuit modules.

[0084] The edge computing node 650 may include or be coupled to an acceleration circuit module 664, which may be embodied by one or more artificial intelligence (AI) accelerators, neural computing sticks, neuromorphic hardware, FPGAs, arrangements of GPUs, arrangements of xPU / DPU / IPU / NPUs, one or more SoCs, one or more CPUs, one or more digital signal processors, dedicated ASICs, or other forms of specialized processors or circuit modules designed to perform one or more specialized tasks. These tasks may include AI processing (including machine learning, training, reasoning, and classification operations), visual data processing, network data processing, object detection, rule analysis, and the like. These tasks may also include specific edge computing tasks for service management and service operations discussed elsewhere in this document.

[0085] The interconnect 656 can couple the processor 652 to a sensor hub or external interface 670 for connecting additional devices or subsystems. The devices can include sensors 672 such as accelerometers, level sensors, flow sensors, optical light sensors, camera sensors, temperature sensors, global navigation system (e.g., GPS) sensors, pressure sensors, atmospheric pressure sensors, etc. The hub or interface 670 can further be used to connect the edge computing node 650 to actuators 674 such as power switches, valve actuators, audible sound generators, visual warning devices, etc.

[0086] In some optional examples, various input / output (I / O) devices may be present within or connected to the edge computing node 650. For example, a display or other output device 684 may be included to show information, such as sensor readings or actuator positions. An input device 686 such as a touch screen or keypad may be included to accept input. The output device 684 may include any number of forms of audio or visual display, including simple visual outputs such as binary status indicators (e.g., light emitting diodes (LEDs)) and multi-character visual outputs, or more complex outputs such as display screens (e.g., liquid crystal display (LCD) screens), where outputs of characters, graphics, multimedia objects, etc. are generated or produced by the operation of the edge computing node 650. The display or console hardware may be used in the context of the present system to provide outputs of the edge computing system and receive inputs of the edge computing system; manage components or services of the edge computing system; identify the status of edge computing components or services; or implement any other number of management or regulatory functions or service use cases.

[0087] Battery 676 can power edge computing node 650, although in examples where edge computing node 650 is installed in a fixed location, it can have a power source coupled to the grid, or the battery can be used as a backup or for temporary capabilities. Battery 676 can be a lithium-ion battery or a metal-air battery, such as a zinc-air battery, an aluminum-air battery, a lithium-air battery, and the like.

[0088] A battery monitor / charger 678 may be included in the edge computing node 650 to track the state of charge (SoCh) of the battery 676 (if included). The battery monitor / charger 678 may be used to monitor other parameters of the battery 676 to provide fault prediction, such as the state of health (SoH) and state of function (SoF) of the battery 676. The battery monitor / charger 678 may include a battery monitoring integrated circuit, such as the LTC4020 or LTC2990 from Linear Technologies, the ADT7488A from ON Semiconductor of Phoenix Arizona, or the UCD90xxx series of ICs from Texas Instruments of Dallas, TX. The battery monitor / charger 678 may pass information about the battery 676 to the processor 652 via the interconnect 656. The battery monitor / charger 678 may also include an analog-to-digital (ADC) converter that enables the processor 652 to directly monitor the voltage of the battery 676 or the current from the battery 676. The battery parameters may be used to determine actions that the edge computing node 650 may perform, such as transmission frequency, mesh network operation, sensing frequency, and the like.

[0089] A power block 680 or other power source coupled to the grid can be coupled to the battery monitor / charger 678 to charge the battery 676. In some examples, the power block 680 can be replaced with a wireless power receiver to obtain power wirelessly, for example, through a loop antenna in the edge computing node 650. A wireless battery charging circuit such as the LTC4020 chip from Linear Technologies of Milpitas, California, etc. can be included in the battery monitor / charger 678. A specific charging circuit can be selected based on the size of the battery 676 and the current required accordingly. Charging can be performed using the air fuel standard published by the Airfuel Alliance, the Qi wireless charging standard published by the Wireless Power Consortium, or the Rezence charging standard published by the Alliance for Wireless Power, etc.

[0090] The storage device 658 may include instructions 682 in the form of software, firmware, or hardware commands to implement the techniques described herein. Although such instructions 682 are illustrated as code blocks included in the memory 654 and the storage device 658, it is understood that any of the code blocks may be replaced with hardwired circuits, such as those built into an application specific integrated circuit (ASIC).

[0091] In an example, the instructions 682 provided via the memory 654, the storage device 658, or the processor 652 may be embodied as a non-transitory machine-readable medium 660 including code that directs the processor 652 to perform electronic operations in the edge computing node 650. The processor 652 may access the non-transitory machine-readable medium 660 via the interconnect 656. For example, the non-transitory machine-readable medium 660 may be embodied by the means described for the storage device 658, or the non-transitory machine-readable medium 660 may include a specific storage unit, such as a storage device and / or storage disk including an optical disk (e.g., a digital versatile disk (DVD), a compact disk (CD), a CD-ROM, a Blu-ray disk), a flash drive, a floppy disk, a hard disk drive (e.g., an SSD), or any number of other hardware devices, where information is stored for any duration (e.g., for an extended period of time, permanently, for a transient instance, for temporary caching, and / or caching). The non-transitory machine-readable medium 660 may include instructions to direct the processor 652 to perform a particular sequence of actions or flows, for example, as described in connection with the operational and functional flow chart(s) and block diagram(s) depicted above. As used herein, the terms "machine-readable medium" and "computer-readable medium" are interchangeable. As used herein, the term "non-transitory computer-readable medium" is expressly defined to include any type of computer-readable storage device and / or storage disk and is expressly defined to exclude propagating signals and is expressly defined to exclude transmission media.

[0092] Also in certain examples, instructions 682 on processor 652 (alone or in combination with instructions 682 of machine-readable medium 660) can configure the execution or operation of a trusted execution environment (TEE) 690. In an example, TEE 690 operates as a protected area accessible to processor 652 for secure execution of instructions and secure access to data. This can be accomplished, for example, by using Software Guard Extensions (SGX) or Hardware security extensions, Management Engine (ME) or A Converged Security Manageability Engine (CSME) is provided to provide various implementations of TEE 690 and accompanying secure areas in memory 654 or processor 652. Security hardening, hardware root of trust, and other aspects of trusted or protected operations may be implemented in device 650 through TEE 690 and processor 652.

[0093] Figure 7is a block diagram showing an overview of a configuration for edge computing, including a layer of processing referred to as an "edge cloud" in many of the following examples. As shown, the edge cloud 710 is co-located at an edge location (such as an access point or base station 740, a local processing hub 750, or a central office 720), and therefore may include multiple entities, devices, and device instances. Compared to the cloud data center 730, the edge cloud 710 is located closer to the endpoint (consumer and producer) data sources 760 (e.g., autonomous vehicles 761, user devices 762, commercial and industrial devices 763, video capture devices 764, drones 765, smart city and building devices 766, sensors and IoT devices 767, etc.). The computing, memory, and storage resources provided at the edge in the edge cloud 710 are critical to providing ultra-low latency response times for services and functions used by the endpoint data sources 760 and reducing network backhaul traffic from the edge cloud 710 toward the cloud data center 730, thereby improving energy consumption and overall network usage along with other benefits.

[0094] Compute, memory, and storage are scarce resources and are often reduced depending on the edge location (e.g., fewer processing resources are available at a consumer endpoint device than at a base station or at a central office). However, the closer the edge location is to an endpoint (e.g., a user equipment (UE)), the more space and power are often constrained. Therefore, edge computing attempts to reduce the amount or number of resources required for network services by distributing more resources that are located closer both geographically and in network access time. In this way, edge computing attempts to bring computing resources to workload data or workload data to computing resources where appropriate.

[0095] The following describes aspects of edge cloud architectures that cover a variety of potential deployments and address limitations that some network operators or service providers may have in their own infrastructure. These include variations in configuration based on edge location (because the edge at the base station level, for example, may have more constrained performance and capabilities in a multi-tenant scenario); configuration based on the type of compute, memory, storage, fabric, acceleration, or similar resources available to the edge location, location layer, or location group; services, security, and management and orchestration capabilities; and related goals for availability and performance of end services. These deployments can be handled in the network layer, which can be considered a "near edge", "near edge", "local edge", "middle edge", or "far edge" layer, depending on latency, distance, and timing characteristics.

[0096] Edge computing is a developing paradigm in which computing is performed at or closer to the "edge" of a network, typically using a computing platform (e.g., x86 or ARM computing hardware architecture) implemented at a base station, gateway, network router, or other device closer to the endpoint devices that generate and consume data. For example, an edge gateway server may be equipped with a pool of storage resources and memory to perform computing in real time for low-latency use cases (e.g., autonomous driving or video surveillance) for connected client devices. Or as an example, a base station may be augmented with computing and acceleration resources to directly handle service workloads for connected user devices without further passing data via a backhaul network. Or as another example, central office network management hardware may be replaced with standardized computing hardware that performs virtualized network functions and provides computing resources for the execution of service and consumer functions for connected devices. Within an edge computing network, there may be scenarios where computing resources will be "moved" into the service of data and scenarios where data will be "moved" to computing resources. Or as an example, base station compute, acceleration, and network resources can provide services to facilitate scaling to workload demands on an on-demand basis by activating dormant capacity (subscription, on-demand capacity) to manage extreme situations, emergencies, or to provide longevity for deployed resources over a significantly longer implementation lifecycle.

[0097] Figure 8 The operational layers between the endpoints, edge cloud, and cloud computing environment 800 according to an embodiment are illustrated. Specifically, Figure 8 An example of a computing use case 805 is depicted, utilizing an edge cloud 710 between multiple illustrative layers of network computing. The layers begin with an endpoint (device and thing) layer 800, which accesses the edge cloud 710 to perform data creation, analysis, and data consumption activities. The edge cloud 710 can span multiple network layers, such as an edge device layer 810 with a gateway, on-premise server, or network device (node ​​815) located in a physically adjacent edge system; a network access layer 820 including a base station, radio processing unit, network hub, regional data center (DC), or local network device (device 825); and any devices, devices, or nodes located therebetween (in layer 812, not detailed). Network communications within the edge cloud 710 and between the various layers can occur via any number of wired or wireless media, including via connectivity architectures and technologies not depicted.

[0098] Examples of latency caused by network communication distance and processing time constraints can range from less than milliseconds (ms) when within the endpoint layer 800, less than 5ms at the edge device layer 810, to even between 10 and 40ms when communicating with nodes at the network access layer 820. Outside the edge cloud 710 are the core network 830 and cloud data center 840 layers, each with increasing latency (e.g., between 50-60ms at the core network layer 830, to 100ms or more at the cloud data center layer). As a result, operations at the core network data center 835 or cloud data center 845 with latency of at least 50 to 100ms or more will not be able to complete many time-critical functions of the use case 805. Each of these latency values ​​is provided for the purpose of illustration and comparison; it will be understood that latency can be further reduced using other access network media and technologies. In some examples, respective portions of the network may be classified as "near edge", "local edge", "near edge", "middle edge", or "far edge" layers relative to network sources and destinations. For example, from the perspective of the core network data center 835 or cloud data center 845, a central office or content data network may be considered to be located in a "near edge" layer ("close" to the cloud, having high latency values ​​when communicating with the devices and endpoints of use case 805), while an access point, base station, locally deployed server, or network gateway may be considered to be located in a "far edge" layer ("far away" from the cloud, having low latency values ​​when communicating with the devices and endpoints of use case 805). It will be appreciated that other classifications of specific network layers as constituting a "near," "local," "close," "intermediate," or "far" edge may be based on latency, distance, number of network hops, or other measurable characteristics as measured from a source in any of the network layers 800-840.

[0099] Since multiple services of the edge cloud are utilized, various use cases 805 can access resources under usage pressure from incoming flows. To achieve results with low latency, services executed within the edge cloud 710 balance varying requirements in terms of: (a) priority (throughput or latency) and quality of service (QoS) (e.g., the business of an autonomous vehicle may have a higher priority than a temperature sensor in terms of response time requirements; or, performance sensitivity / bottlenecks may exist at compute / accelerator, memory, storage, or network resources, depending on the application); (b) reliability and resilience (e.g., some input flows need to be acted upon and the business is routed with mission-critical reliability, while some other input flows can tolerate occasional failures, depending on the application); and (c) physical constraints (e.g., power, cooling, and form factor).

[0100] The end-to-end service view of these use cases involves the concept of service flow and is associated with transactions. Transactions detail the overall service requirements for entities consuming services, as well as associated services for resources, workloads, workflows, and business functions and business-level requirements. Services executed with the described "terms" can be managed at each layer in a manner that ensures real-time and runtime contract compliance of the transaction during the life cycle of the service. When a component in a transaction is missing its negotiated service level agreement (SLA), the system as a whole (the component in the transaction) can provide the following capabilities: (1) understand the impact of violating the SLA, and (2) add other components in the system to restore the overall transaction SLA, and (3) implement steps to remedy the situation.

[0101] Therefore, considering these changes and service characteristics, edge computing within edge cloud 710 can provide the ability to serve and respond to multiple applications of use cases 805 (e.g., object tracking, video surveillance, connected cars, etc.) in real time or near real time and meet ultra-low latency requirements for these multiple applications. These advantages enable entirely new classes of applications (e.g., virtual network functions (VNFs), function as a service (FaaS), edge as a service (EaaS), standard processes, etc.) that cannot utilize traditional cloud computing due to latency or other limitations.

[0102] However, with the advantages of edge computing comes the following warning. Devices located at the edge are often resource-constrained and therefore there is pressure on the use of edge resources. Typically, this is solved by pooling storage resources and memory for use by multiple users (tenants) and devices. The edge can be power and cooling constrained, and therefore power usage needs to be explained by the application that is consuming the most power. There may be an inherent power-performance trade-off in these pooled memory resources, because many of them may use emerging memory technologies, where more power requires greater memory bandwidth. Similarly, improved security and root-of-trust trusted functions of hardware are also needed, because edge locations may be unattended and may even require licensed access (for example, when housed in a third-party location). Such problems are magnified in edge clouds 710 in multi-tenant, multi-owner, or multi-access settings, where services and applications are requested by many users, especially when network usage fluctuates dynamically and the combination of multiple stakeholders, use cases, and services changes.

[0103] At a more general level, an edge computing system may be described as comprising any number of deployments at the previously discussed layers (network layers 800-840) operating in an edge cloud 710 that provide coordination from clients and distributed computing devices. One or more edge gateway nodes, one or more edge aggregation nodes, and one or more core data centers may be distributed across the layers of the network to provide implementations of edge computing systems by or on behalf of a telecommunications service provider ("telco" or "TSP"), an IoT service provider, a cloud service provider (CSP), an enterprise entity, or any other number of entities. Various implementations and configurations of edge computing systems may be provided dynamically, such as when orchestrated to meet service objectives.

[0104] Consistent with the examples provided herein, a client computing node may be embodied as any type of endpoint component, device, equipment, or other thing capable of communicating as a producer or consumer of data. Furthermore, the label "node" or "device" as used in an edge computing system does not necessarily refer to such a node or device operating in a client or agent / subordinate / follower role; rather, any of the nodes or devices in an edge computing system refers to an independent entity, node, or subsystem including a discrete or connected hardware or software configuration to facilitate or use edge cloud 710.

[0105] As such, the edge cloud 710 is comprised of network components and functional features operated by and within edge gateway nodes, edge aggregation nodes, or other edge computing nodes within the network layers 810-830. The edge cloud 710 may thus be embodied as any type of network that provides edge computing and / or storage resources located near endpoint devices (e.g., mobile computing devices, IoT devices, smart devices, etc.) having radio access network (RAN) capabilities as discussed herein. In other words, the edge cloud 710 may be envisioned as an “edge” that connects endpoint devices and traditional network access points that serve as entry points into a service provider core network, including mobile operator networks (e.g., Global System for Mobile Communications (GSM) networks, Long Term Evolution (LTE) networks, 5G / 6G networks, etc.), while also providing storage and / or computing capabilities. Other types and forms of network access (eg, Wi-Fi, long-range wireless, wired networks (including optical networks)) may also be utilized in place of or in combination with such 3GPP operator networks.

[0106] The network components of the edge cloud 710 may be servers, multi-tenant servers, device computing devices, and / or any other type of computing devices. For example, the edge cloud 710 may include a device computing device, which is a self-contained electronic device including a housing, a chassis, a case, or a housing. In some cases, the size of the housing may be determined for portability so that it can be carried and / or transported by a person. The example housing may include a material forming one or more outer surfaces that partially or completely protect the contents of the device, wherein the protection may include weather protection, hazardous environment protection (e.g., electromagnetic interference (EMI), vibration, extreme temperature, etc.), and / or enable diving. The example housing may include a power circuit module for providing power for a fixed and / or portable implementation, such as an alternating current (AC) power input, a direct current (DC) power input, (one or more) AC / DC converters, (one or more) DC / AC converters, (one or more) DC / DC converters, power regulators, transformers, charging circuit modules, batteries, wired inputs, and / or wireless power inputs. The example housing and / or its surface may include or be connected to mounting hardware to enable attachment to structures such as buildings, telecommunication structures (e.g., poles, antenna structures, etc.) and / or racks (e.g., server racks, blade mounts, etc.). The example housing and / or its surface may support one or more sensors (e.g., temperature sensors, vibration sensors, light sensors, acoustic sensors, capacitive sensors, proximity sensors, infrared or other visual thermal sensors, etc.). One or more such sensors may be contained in the surface, carried by the surface, or otherwise embedded in the surface and / or mounted to the surface of the device. The example housing and / or its surface may support mechanical connectivity, such as propulsion hardware (e.g., wheels, rotors such as propellers, etc.) and / or articulated hardware (e.g., robotic arms, pivotable attachments, etc.). In some cases, the sensor may include any type of input device, such as user interface hardware (e.g., buttons, switches, dials, sliders, microphones, etc.). In some cases, the example housing includes an output device contained therein, carried by it, embedded therein, and / or attached thereto. Output devices may include displays, touch screens, lights, light emitting diodes (LEDs), speakers, input / output (I / O) ports (e.g., Universal Serial Bus (USB)), and the like. In some cases, an edge device is a device that exists in a network for a specific purpose (e.g., a traffic light), but may have processing and / or other capabilities that may be used for other purposes. Such an edge device may be independent of other networked devices and may be equipped with a housing that has a form factor suitable for its primary purpose; yet may be used for other computing tasks that do not interfere with its primary mission. Edge devices include IoT devices.The device computing device may include hardware and software components to manage local issues such as device temperature, vibration, resource utilization, updates, power issues, physical and network security, etc. Integration. Figure 6B Example hardware for implementing device computing means is described. Edge cloud 710 may also include one or more servers and / or one or more multi-tenant servers. Such a server may include an operating system and implement a virtual computing environment. The virtual computing environment may include a hypervisor for managing (e.g., mass production, deployment, commissioning, destruction, retirement, etc.) one or more virtual machines, one or more containers, etc. Such a virtual computing environment provides an execution environment in which one or more applications and / or other software, code, or script can be executed while being isolated from one or more other applications, software, code, or scripts.

[0107] Fig. 9 An example method for networking and services in an edge computing system according to an embodiment is described. Fig. 9 In the embodiment of the present invention, various client endpoints 910 (in the form of mobile devices, computers, autonomous vehicles, commercial computing equipment, industrial processing equipment) exchange requests and responses specific to the type of endpoint network aggregation. For example, the client endpoint 910 can obtain network access via a wired broadband network by exchanging requests and responses 922 through a locally deployed network system 932. Some client endpoints 910 (such as mobile computing devices) can obtain network access via a wireless broadband network by exchanging requests and responses 924 through an access point (e.g., a cellular network tower) 934. Some client endpoints 910 (such as autonomous vehicles) can obtain network access for requests and responses 926 via a wireless vehicle network through a network system 936 located on the street. However, regardless of the type of network access, the TSP can deploy aggregation points 942, 944 within the edge cloud 710 to aggregate services and requests. Therefore, within the edge cloud 710, the TSP can deploy various computing and storage resources such as at an edge aggregation node 940 to provide requested content. The edge aggregation nodes 940 and other systems of the edge cloud 710 are connected to a cloud or data center 960, which uses a backhaul network 950 to satisfy higher latency requests from the cloud / data center for websites, applications, database servers, etc. Additional or consolidated instances of the edge aggregation nodes 940 and aggregation points 942, 944, including those deployed on a single server framework, may also exist in other areas of the edge cloud 710 or TSP infrastructure.

[0108] Fig.10 An example software distribution platform 1005 for distributing software according to an embodiment is illustrated. The software distribution platform 1005 may include one or more devices (such as (one or more) example processor platforms 1015 and / or Figure 8Example of a connected edge device 810) Figure 6B The example computer readable instructions 682 of the present invention can be stored and transmitted to other computing devices (e.g., third parties, Figure 8 The example software distribution platform 1005 may be implemented by any computer server, data facility, cloud service, etc. that delivers software to the example connected edge device 810 (e.g., an example connected edge device). The example connected edge device may be a customer, client, management device (e.g., a server), a third party (e.g., a customer of an entity that owns and / or operates the software distribution platform 1005). The example connected edge device may operate in a commercial and / or home automation environment. In some examples, the third party is such as Figure 6B The third party may be a consumer, user, retailer, OEM, etc., who purchases and / or licenses the software for use and / or resale and / or sublicense. In some examples, the distributed software causes the display of one or more user interfaces (UIs) and / or graphical user interfaces (GUIs) to identify one or more devices (e.g., connected edge devices) that are geographically and / or logically separated from each other (e.g., physically separated IoT devices that are licensed with responsibility for water distribution control (e.g., pumps), power distribution control (e.g., relays), etc.).

[0109] exist Fig.10 In the example of the description of FIG. 1 , the software distribution platform 1005 includes one or more servers and one or more storage devices. As described above, the storage device stores computer-readable instructions 682, which may correspond to Figure 6B The example computer readable instructions 682 of the example software distribution platform 1005 are provided. One or more servers of the example software distribution platform 1005 communicate with a network 1010, which may correspond to any one or more of the Internet and / or any of the example networks described above. In some examples, as part of a business transaction, one or more servers respond to a request to transfer software to a requesting party. Payment for the delivery, sale, and / or license of the software may be handled by one or more servers of the software distribution platform and / or via a third-party payment entity. The server enables a purchaser and / or licensor to download the computer readable instructions 682 from the software distribution platform 1005. For example, the server 1010 may correspond to Figure 6BThe software of the example computer readable instructions 682 can be downloaded to (one or more) example processor platforms 1015 (e.g., example connected edge devices), which are to execute the computer readable instructions 682 to implement non-dominant resource management for edge multi-tenant applications. In some examples, one or more servers of the software distribution platform 1005 are communicatively connected to one or more security domains and / or security devices, and requests and transmissions of the example computer readable instructions 682 must pass through the one or more security domains and / or security devices. In some examples, one or more servers of the software distribution platform 1005 periodically provide, transmit, and / or force updates to the software (e.g., Figure 6B Example computer readable instructions 682) to ensure that improvements, patches, updates, etc. are distributed and applied to the software at the end-user device.

[0110] exist Fig.10 In the illustrated example, the computer readable instructions 682 are stored in a specific format on the storage device of the software distribution platform 1005. The format of the computer readable instructions includes, but is not limited to, a specific code language (e.g., Java, JavaScript, Python, C, C#, SQL, HTML, etc.) and / or a specific code state (e.g., uncompiled code (e.g., ASCII), interpreted code, linked code, executable code (e.g., binary), etc.). In some examples, the computer readable instructions 682 stored in the software distribution platform 1005 are in a first format when transmitted to the (one or more) example processor platform 1015. In some examples, the first format is an executable binary that can be executed by the (one or more) processor platform 1015 of a specific type. However, in some examples, the first format is an uncompiled code that requires one or more preparation tasks to convert the first format to a second format to enable execution on the (one or more) example processor platform 1015. For example, the (one or more) receiving processor platform 1015 may need to compile the computer readable instructions 682 in the first format to generate executable code in a second format that can be executed on the (one or more) processor platform 1015. In still other examples, the first format is an interpreted code that is interpreted by an interpreter upon arrival at the processor platform(s) 1015 to facilitate execution of the instructions.

[0111] Fig.11An example of an infrastructure processing unit (IPU) is depicted. Different examples of the IPU disclosed herein enable improved performance, management, security, and coordination functions between entities (e.g., cloud service providers) and enable infrastructure offloading or communication coordination functions. As disclosed in further detail below, the IPU can be integrated with an intelligent NIC and storage device or memory (e.g., on the same die, system on chip (SoC), or connected die) located at a local deployment system, base station, gateway, neighborhood central office, etc. Different examples of one or more IPUs disclosed herein can execute applications including any number of microservices, each of which runs in its own process and communicates using a protocol (e.g., HTTP resource API, message service, or gRPC). Microservices can be deployed independently using centralized management of these services. The management system can be written in different programming languages, and the management system can use different data storage technologies.

[0112] In addition, one or more IPUs can perform platform management, networking stack processing operations, security (encryption) operations, storage software, identity and key management, telemetry, logging, monitoring, and service mesh (e.g., controlling how different microservices communicate with each other). The IPU can access the xPU to offload the performance of various tasks. For example, the IPU exposes the XPU, storage devices, memory, and CPU resources and capabilities as services that can be accessed by other microservices for functional combinations. This can improve performance and reduce data movement and latency. The IPU can perform functions such as routers, load balancers, firewalls, TCP / reliable transport, service meshes (e.g., agents or API gateways), security, data conversion, authentication, quality of service (QoS), security, telemetry measurement, event logging, initiation and management of data flows, data placement, or job scheduling of resources on xPU, storage devices, memory, or CPU.

[0113] exist Fig.11 In the illustrated example, the IPU 1100 includes or otherwise accesses a secure resource management circuit module 1102, a network interface controller (NIC) circuit module 1104, a security and root of trust circuit module 1106, a resource combination circuit module 1108, a timestamp management circuit module 1110, a memory and storage device 1112, a processing circuit module 1114, an accelerator circuit module 1116, or a translator circuit module 1118. Any number or combination of other structures (one or more) may be used, such as but not limited to the following. Fig.111100, a compression and encryption circuit module 1120, a memory management and translation unit circuit module 1122, a computing structure data exchange circuit module 1124, a security policy enforcement circuit module 1126, a device virtualization circuit module 1128, a telemetry, tracking, logging and monitoring circuit module 1130, a quality of service circuit module 1132, a search circuit module 1134, a network function circuit module (e.g., routing, firewall, load balancing, network address translation (NAT), etc.) 1136, a reliable transmission, sorting, retransmission, congestion control circuit module 1138, and a high availability, fault handling and migration circuit module 1140. Different examples can use one or more structures (components) of the example IPU 1100 together or separately. For example, the compression and encryption circuit module 1120 can be used as a separate service or can be linked as part of a data flow with a vSwitch and packet encryption.

[0114] In some examples, the IPU 1100 includes a field programmable gate array (FPGA) 1170 configured to receive commands from a CPU, XPU, or application via an API and execute commands / tasks on behalf of the CPU (including workload management and offload or accelerator operations). Fig.11 Examples of the invention may include any number of FPGAs configured or otherwise constructed to perform any operations of any IPU described herein.

[0115] The example computing fabric circuit module 1150 provides connectivity to a local host or device, such as a server or device (such as an xPU, memory, or storage device). In some examples, peripheral component interconnect express (PCIe), ARMAXI, Quick Path Interconnect (QPI), Ultra Path Interconnect (UPI), Connectivity with a local host or device or a smartNIC or another IPU is provided by one or more of On-Chip System Fabric (IOSF), Omnipath, Ethernet, Compute Express Link (CXL), HyperTransport, NVLink, Advanced Microcontroller Bus Architecture (AMBA) Interconnect, OpenCAPI, Gen-Z, CCIX, Wireless Fabric (IF), etc. Different examples of host connectivity provide symmetric memory and cache to enable equal peer operation between the CPU, XPU, and IPU (e.g., via CXL.cache and CXL.mem).

[0116] The example media interface connection circuit module 1160 provides connectivity to a remote smartNIC or another IPU or service via a network medium or fabric. This can be provided over any type of network medium (e.g., wired or wireless) and using any protocol (e.g., Ethernet, InfiniBand, Fibre Channel, ATM, to name a few).

[0117] In some examples, the server / CPU is not the primary component that manages the IPU 1100, which is the root of the system (e.g., a data center or rack of servers) and manages computing resources (e.g., CPUs, xPUs, storage devices, memory, other IPUs, etc.) in and outside the IPU 1100. Different operations of the IPU are described below.

[0118] In some examples, the IPU 1100 performs orchestration to decide which hardware or software is to execute the workload based on available resources (e.g., services and devices) and considers service level agreements and latency to determine whether to allocate resources (e.g., CPU, xPU, storage, memory, etc.) from a local host or from a remote host or pooled resources. In an example, when the IPU 1100 is selected to execute the workload, the secure resource management circuit module 1102 offloads the work to the CPU, xPU or other device and the IPU 1100 accelerates connectivity, reduces latency, CPU, and increases reliability of the distributed runtime.

[0119] In some examples, the secure resource management circuit module 1102 runs a service grid to decide what resources to execute the workload, and provides L7 (application layer) and remote procedure call (RPC) services to completely bypass the kernel so that user space applications can communicate directly with the example IPU 1100 (e.g., the IPU 1100 and the application can share memory space). In some examples, the service grid is a configurable low-latency infrastructure layer designed to handle communications between application microservices using an application programming interface (API) (e.g., through a remote procedure call (RPC)). The example service grid provides fast, reliable, and secure communications between containerized or virtualized application infrastructure services. The service grid can provide key capabilities, including but not limited to service discovery, load balancing, encryption, observability, traceability, authentication and authorization, and support for circuit breaker mode.

[0120] In some examples, the infrastructure services include composite nodes created by the IPU upon or after receiving a workload from an application. In some cases, the composite nodes include access to hardware devices, software using an API, RPC, gRPC, or a communication protocol with instructions, such as but not limited to iSCSI, NVMe-oF, or CXL.

[0121] In some cases, the example IPU 1100 dynamically selects itself to run a given workload (e.g., a microservice) within a composable infrastructure including IPUs, xPUs, CPUs, storage devices, memory, and other devices in a node.

[0122] In some examples, communications are passed back to local services on the same host through the NIC / smartNIC (for cross-node communications) or loopback, through the media interface connection circuit module 1160 of the example IPU 1100. Communications to another IPU through the example media interface connection circuit module 1160 of the example IPU 1100 can then use the shared memory between xPUs exchanged through the local IPU to support transmission. Using IPU to IPU communication can reduce latency and jitter through ingress scheduling and work processing of messages based on service level objectives (SLOs).

[0123] For example, for requests to database applications that require responses, the example IPU 1100 prioritizes its processing to minimize delays to the requesting application. In some examples, the IPU 1100 schedules prioritized message requests that issue events to perform SQL queries against a database, and the example IPU builds a microservice that issues SQL queries and sends the queries to the appropriate device or service.

[0124] Example 1 is a system for supplying a trained artificial intelligence (AI) model, the system comprising: a processor circuit module; and a memory comprising instructions, which, when executed by the processor circuit module, cause the processor circuit module to: receive a deployment request for the AI ​​model from a client device; send a remote attestation request to the client device in response to the deployment request; receive a remote attestation response from the client device; verify the remote attestation response; in response to verifying the remote attestation response, generate an encrypted model based on the AI ​​model; and send the encrypted model to a first trusted environment within the client device.

[0125] In Example 2, the subject matter of Example 1 includes the instructions further causing the processor circuit module to send the encryption model to a second trusted environment associated with an accelerator within the client device; wherein: the first trusted environment includes a first security architecture dedicated to the client device; and the second trusted environment includes a second architecture dedicated to the accelerator and different from the first security architecture.

[0126] In Example 3, the subject matter of Examples 1-2 includes the instructions further causing the processor circuit module to: in response to verifying the remote attestation response, generate a symmetric key to encrypt the AI ​​model, wherein the encrypted model is further generated based on the symmetric key; generate a wrapping key based on the symmetric key; and send the wrapping key to the client device.

[0127] In Example 4, the subject matter of Example 3 includes the instructions further causing the processor circuit module to provision the client computing device with an attestation key prior to receiving the deployment request.

[0128] In Example 5, the subject matter of Example 4 includes, wherein provisioning the client computing device with the attestation key comprises storing the attestation key within a confidentiality protected client memory on the client device.

[0129] In Example 6, the subject matter of Example 5 includes, wherein the confidentiality protected client memory comprises at least one of a confidentiality protected dynamic random access memory and a confidentiality protected flash memory device.

[0130] In Example 7, the subject matter of Examples 4-6 includes, wherein receiving the remote attestation response comprises receiving a device offer signed by an attestation key.

[0131] In Example 8, the subject matter of Examples 4-7 includes the instructions further causing the processor circuit module to: receive a unique device key pair and an attestation certificate chain of an attestation key generated at the client device; and extract the public key from the attestation certificate chain after verifying the remote attestation response.

[0132] In Example 9, the subject matter of Example 8 includes, wherein generating the wrapping key is further based on the public key.

[0133] In Example 10, the subject matter of Examples 1-9 includes, wherein verifying the remote attestation response comprises: sending the remote attestation response to an attestation verifier; and receiving the remote attestation verification from the attestation verifier.

[0134] In Example 11, the subject matter of Examples 1-10 includes, the instructions further causing the processor circuit module to receive the trained model from the client device, generating the trained model by retraining the AI ​​model at the client device.

[0135] Example 12 is a method for provisioning a trained artificial intelligence (AI) model, the method comprising: receiving a deployment request for the AI ​​model from a client device; sending a remote attestation request to the client device in response to the deployment request; receiving a remote attestation response from the client device; verifying the remote attestation response; generating an encrypted model based on the AI ​​model in response to verifying the remote attestation response; and sending the encrypted model to a first trusted environment within the client device.

[0136] In Example 13, the subject matter of Example 12 includes sending an encryption model to a second trusted environment associated with an accelerator within a client device; wherein: the first trusted environment includes a first security architecture dedicated to the client device; and the second trusted environment includes a second architecture dedicated to the accelerator and different from the first security architecture.

[0137] In Example 14, the subject matter of Examples 12-13 includes generating a symmetric key to encrypt the AI ​​model in response to verifying the remote attestation response, wherein the encrypted model is further generated based on the symmetric key; generating a wrapping key based on the symmetric key; and sending the wrapping key to the client device.

[0138] In Example 15, the subject matter of Example 14 includes provisioning the client computing device with an attestation key prior to receiving the deployment request.

[0139] In Example 16, the subject matter of Example 15 includes, wherein provisioning the client computing device with the attestation key comprises storing the attestation key within a confidentiality protected client memory on the client device.

[0140] In Example 17, the subject matter of Example 16 includes, wherein the confidentiality protected client memory comprises at least one of a confidentiality protected dynamic random access memory and a confidentiality protected flash storage device.

[0141] In Example 18, the subject matter of Examples 15-17 includes, wherein receiving the remote attestation response comprises receiving a device offer signed by an attestation key.

[0142] In Example 19, the subject matter of Examples 15-18 includes receiving a unique device key pair and an attestation certificate chain for an attestation key generated at a client device; and extracting the public key from the attestation certificate chain after validating the remote attestation response.

[0143] In Example 20, the subject matter of Example 19 includes, wherein generating the wrapping key is further based on the public key.

[0144] In Example 21, the subject matter of Examples 12-20 includes, wherein verifying the remote attestation response comprises: sending the remote attestation response to an attestation verifier; and receiving the remote attestation verification from the attestation verifier.

[0145] In Example 22, the subject matter of Examples 12-21 includes receiving a trained model from a client device, and generating the trained model by retraining the AI ​​model at the client device.

[0146] Example 23 is at least one non-temporary machine-readable storage medium, comprising instructions that are executed in response to a processor circuit module of a device controlled by a computer to cause the processor circuit module to: receive a deployment request for an AI model from a client device; in response to the deployment request, send a remote attestation request to the client device; receive a remote attestation response from the client device; verify the remote attestation response; in response to verifying the remote attestation response, generate an encrypted model based on the AI ​​model; and send the encrypted model to a first trusted environment within the client device.

[0147] In Example 24, the subject matter of Example 23 includes the instructions further causing the processor circuit module to send the encryption model to a second trusted environment associated with an accelerator within the client device; wherein: the first trusted environment includes a first security architecture dedicated to the client device; and the second trusted environment includes a second architecture dedicated to the accelerator and different from the first security architecture.

[0148] In Example 25, the subject matter of Examples 23-24 includes the instructions further causing the processor circuit module to: in response to verifying the remote attestation response, generate a symmetric key to encrypt the AI ​​model, wherein the encrypted model is further generated based on the symmetric key; generate a wrapping key based on the symmetric key; and send the wrapping key to the client device.

[0149] In Example 26, the subject matter of Example 25 includes the instructions further causing the processor circuit module to provision the client computing device with an attestation key prior to receiving the deployment request.

[0150] In Example 27, the subject matter of Example 26 includes, the instructions further causing the processor circuit module to store the attestation key in a confidentiality protected client memory on the client device.

[0151] In Example 28, the subject matter of Example 27 includes, wherein the confidentiality protected client memory comprises at least one of a confidentiality protected dynamic random access memory and a confidentiality protected flash memory device.

[0152] In Example 29, the subject matter of Examples 26-28 includes, the instructions further causing the processor circuit module to receive a device offer signed by the attestation key.

[0153] In Example 30, the subject matter of Examples 26-29 includes the instructions further causing the processor circuit module to: receive a unique device key pair of an attestation key and an attestation certificate chain generated at the client device; and extract the public key from the attestation certificate chain after verifying the remote attestation response.

[0154] In Example 31, the subject matter of Example 30 includes, wherein generating the wrapping key is further based on the public key.

[0155] In Example 32, the subject matter of Examples 23-31 includes the instructions further causing the processor circuit module to: send a remote attestation response to the attestation verifier; and receive a remote attestation verification from the attestation verifier.

[0156] In Example 33, the subject matter of Examples 23-32 includes, the instructions further causing the processor circuit module to receive the trained model from the client device, generating the trained model by retraining the AI ​​model at the client device.

[0157] Example 34 is a device for supplying a trained artificial intelligence (AI) model, the device comprising: a component for receiving a deployment request for the AI ​​model from a client device; a component for sending a remote attestation request to the client device in response to the deployment request; a component for receiving a remote attestation response from the client device; a component for verifying the remote attestation response; a component for generating an encrypted model based on the AI ​​model in response to verifying the remote attestation response; and a component for sending the encrypted model to the client device.

[0158] Example 35 is at least one machine-readable medium comprising instructions that, when executed by a processing circuit module, cause the processing circuit module to perform operations to implement any of Examples 1-34.

[0159] Example 36 is an apparatus comprising components for implementing any of Examples 1-34.

[0160] Example 37 is a system for implementing any of Examples 1-34.

[0161] Example 38 is a method for implementing any of Examples 1-34.

[0162] As described herein, examples may include logic or multiple components, modules or mechanisms, or may operate on logic or multiple components, modules or mechanisms. A module may be hardware, software or firmware that is communicatively coupled to one or more processors to facilitate the execution of the operations described herein. A module may be a hardware module, and a module like this may be considered as a tangible entity that can perform a specified operation, and a module like this may be configured or arranged in some way. In an example, a circuit may be arranged as a module in a specified manner (e.g., internally or relative to an external entity such as other circuits). In an example, all or part of one or more computer systems (e.g., independent, client or server computer systems) or one or more hardware processors may be configured as a module that operates to perform a specified operation by firmware or software (e.g., instructions, application parts or applications). In an example, software may reside on a machine-readable medium. In an example, software causes hardware to perform a specified operation when executed by the underlying hardware of the module. Therefore, the term hardware module is understood to include a tangible entity that is physically constructed, specially configured (e.g., hardwired) or temporarily (e.g., temporarily) configured (e.g., programmed) to operate in a specified manner or perform part or all of any operations described herein. Consider an example in which a module is temporarily configured, and each module in the module does not need to be instantiated at any one moment in time. For example, where the module includes a general-purpose hardware processor configured using software; the general-purpose hardware processor can be configured as corresponding different modules at different times. The software can configure the hardware processor accordingly, for example, to constitute a specific module at one instance of time and to constitute a different module at a different instance of time. A module can also be a software or firmware module that operates to perform a set of methods described herein.

[0163] As used in this document, a circuit module or circuit may include, for example, individually or in any combination, a hardwired circuit module, a programmable circuit module such as a computer processor including one or more separate instruction processing cores, a state machine circuit module, and / or firmware storing instructions executed by the programmable circuit module. The circuit, circuit module, or module may be collectively or individually embodied as a circuit module that forms part of a larger system (e.g., an integrated circuit (IC), a system on a chip (SoC), a desktop computer, a laptop computer, a tablet computer, a server, a smart phone, etc.).

[0164] As used in any embodiment herein, the term "logic" may refer to firmware and / or circuit modules configured to perform any of the aforementioned operations. The firmware may be embodied as code, instructions or instruction sets and / or data that are hard-coded (e.g., non-volatile) in a memory device and / or circuit module.

[0165] As used in any embodiment herein, a "circuit module" may include, for example, a hardwired circuit module, a programmable circuit module, a state machine circuit module, logic and / or firmware storing instructions executed by a programmable circuit module, for example, individually or in any combination. The circuit module may be embodied as an integrated circuit such as an integrated circuit chip. In some embodiments, the circuit module may be formed at least in part by a processor circuit module that executes a code and / or instruction set (e.g., software, firmware, etc.) corresponding to the functionality described herein, thereby converting a general-purpose processor into a dedicated processing environment to perform one or more of the operations described herein. In some embodiments, the processor circuit module may be embodied as an independent integrated circuit, or may be combined into one of several components on an integrated circuit. In some embodiments, various components and circuit modules of a node or other system may be combined into a system-on-chip (SoC) architecture.

[0166] The above detailed description includes reference to the accompanying drawings, which form a part of the detailed description. By way of illustration, the accompanying drawings show specific embodiments that can be implemented. These embodiments are also referred to as "examples" in this article. Such examples may include elements that have in addition to those shown or described. However, examples including the elements shown or described are also contemplated. In addition, or with respect to a specific example (or one or more aspects thereof), or with respect to other examples (or one or more aspects thereof) shown or described in this article, examples of any combination or permutation of those elements (or one or more aspects thereof) shown or described are also contemplated.

[0167] In this document, as is common in patent documents, the terms "a" or "an" are used to include one or more than one, independent of any other instance or usage of "at least one" or "one or more." In this document, unless otherwise indicated, the term "or" is used to refer to a non-exclusive or, such that "A or B" includes "A but not B," "B but not A," and "A and B." In the appended claims, the terms "including" and "in which" are used as the plain English equivalents of the respective terms "comprising" and "wherein." Moreover, in the following claims, the terms "including" and "comprising" are open-ended, that is, systems, devices, articles, or processes that include elements in addition to those listed after such terms in a claim are still considered to fall within the scope of that claim. Furthermore, in the following claims, the terms "first," "second," and "third," etc. are used merely as labels and are not used to suggest a numerical order of their objects.

[0168] The above description is intended to be illustrative and not restrictive. For example, the examples described above (or one or more aspects thereof) may be used in combination with others. Other embodiments may be used, such as by a person of ordinary skill in the art, when reviewing the above description. The abstract is intended to allow the reader to quickly determine the nature of the technical disclosure. It is understood that it will not be used to interpret or limit the scope or meaning of the claims. Moreover, in the above specific embodiments, various features may be grouped together to simplify the present disclosure. However, the claims may not set forth every feature disclosed herein, as the embodiments may feature a subset of the features. In addition, the embodiments may include fewer features than those disclosed in a particular example. Therefore, the following claims are hereby incorporated into the specific embodiments, wherein the claims themselves serve as independent embodiments. The scope of the embodiments disclosed herein shall be determined by reference to the appended claims together with the full scope of equivalents to which such claims are entitled.

Claims

1. A system for serving a trained artificial intelligence (AI) model, the system comprising: Processor circuit module; as well as a memory including instructions that, when executed by the processor circuit module, cause the processor circuit module to: Receiving a deployment request for the AI ​​model from a client device; In response to the deployment request, sending a remote attestation request to the client device; receiving a remote attestation response from the client device; verifying the remote attestation response; In response to verifying the remote attestation response, generating an encryption model based on the AI ​​model; as well as The encrypted model is sent to a first trusted environment within the client device.

2. The system of claim 1, wherein the instructions further cause the processor circuit module to send the encryption model to a second trusted environment associated with an accelerator within the client device; wherein: The first trusted environment includes a first security architecture dedicated to the client device; as well as The second trusted environment includes a second architecture that is specific to the accelerator and is different from the first security architecture.

3. The system of claim 1 , wherein the instructions further cause the processor circuit module to: In response to verifying the remote attestation response, generating a symmetric key to encrypt the AI ​​model, wherein the encrypted model is further generated based on the symmetric key; generating a wrapping key based on the symmetric key; and The wrapping key is sent to the client device. 4 . The system of claim 3 , the instructions further causing the processor circuit module to provision an attestation key for the client computing device prior to receiving the deployment request.

5. The system of claim 4, wherein: Receiving the remote attestation response includes receiving a device offer signed by the attestation key.

6. The system of claim 4, wherein the instructions further cause the processor circuit module to: receiving a unique device key pair and an attestation certificate chain for the attestation key generated at the client device; and After validating the remote attestation response, a public key is extracted from the attestation certificate chain.

7. The system of claim 1, wherein: Verifying the remote attestation response comprises: sending the remote attestation response to an attestation verifier; and A remote attestation verification is received from the attestation verifier.

8. The system of claim 1, wherein the instructions further cause the processor circuit module to receive a trained model from the client device, and generate the trained model by retraining the AI ​​model at the client device.

9. A method for serving a trained artificial intelligence (AI) model, the method comprising: Receiving a deployment request for the AI ​​model from a client device; In response to the deployment request, sending a remote attestation request to the client device; receiving a remote attestation response from the client device; verifying the remote attestation response; In response to verifying the remote attestation response, generating an encryption model based on the AI ​​model; as well as The encrypted model is sent to a first trusted environment within the client device.

10. The method of claim 9, further comprising sending the encryption model to a second trusted environment associated with an accelerator within the client device; wherein: The first trusted environment includes a first security architecture dedicated to the client device; as well as The second trusted environment includes a second architecture that is specific to the accelerator and is different from the first security architecture.

11. The method of claim 9, further comprising: In response to verifying the remote attestation response, generating a symmetric key to encrypt the AI ​​model, wherein the encrypted model is further generated based on the symmetric key; generating a wrapping key based on the symmetric key; as well as The wrapping key is sent to the client device.

12. The method of claim 11, further comprising provisioning a client computing device with an attestation key prior to receiving the deployment request.

13. The method of claim 12, wherein: Receiving the remote attestation response includes receiving a device offer signed by the attestation key.

14. The method of claim 12, further comprising: receiving a unique device key pair and an attestation certificate chain for the attestation key generated at the client device; as well as After validating the remote attestation response, a public key is extracted from the attestation certificate chain.

15. The method of claim 9, wherein: Verifying the remote attestation response comprises: sending the remote attestation response to an attestation verifier; and A remote attestation verification is received from the attestation verifier.

16. The method of claim 9, further comprising receiving a trained model from the client device, the trained model being generated by retraining the AI ​​model at the client device.

17. At least one non-transitory machine-readable storage medium comprising instructions that, in response to being executed by a processor circuit module of a device controlled by a computer, cause the processor circuit module to: Receiving a deployment request for the AI ​​model from a client device; In response to the deployment request, sending a remote attestation request to the client device; receiving a remote attestation response from the client device; verifying the remote attestation response; In response to verifying the remote attestation response, generating an encryption model based on the AI ​​model; as well as The encrypted model is sent to a first trusted environment within the client device.

18. The at least one non-transitory machine-readable storage medium of claim 17, the instructions further causing the processor circuit module to send the encryption model to a second trusted environment associated with an accelerator within the client device; wherein: The first trusted environment includes a first security architecture dedicated to the client device; as well as The second trusted environment includes a second architecture that is specific to the accelerator and is different from the first security architecture.

19. The at least one non-transitory machine-readable storage medium of claim 17, the instructions further causing the processor circuit module to: In response to verifying the remote attestation response, generating a symmetric key to encrypt the AI ​​model, wherein the encrypted model is further generated based on the symmetric key; generating a wrapping key based on the symmetric key; and The wrapping key is sent to the client device.

20. The at least one non-transitory machine-readable storage medium of claim 19, the instructions further causing the processor circuit module to provision an attestation key for a client computing device prior to receiving the deployment request.

21. The at least one non-transitory machine-readable storage medium of claim 20, the instructions further causing the processor circuit module to receive a device offer signed by the attestation key.

22. The at least one non-transitory machine-readable storage medium of claim 20, the instructions further causing the processor circuit module to: receiving a unique device key pair and an attestation certificate chain for the attestation key generated at the client device; and After validating the remote attestation response, a public key is extracted from the attestation certificate chain.

23. The at least one non-transitory machine-readable storage medium of claim 17, the instructions further causing the processor circuit module to: sending the remote attestation response to an attestation verifier; and A remote attestation verification is received from the attestation verifier.

24. At least one non-transitory machine-readable storage medium as described in claim 17, the instructions further causing the processor circuit module to receive a trained model from the client device and generate the trained model by retraining the AI ​​model at the client device.

25. An apparatus for serving a trained artificial intelligence (AI) model, the apparatus comprising: means for receiving a deployment request for an AI model from a client device; means for sending a remote attestation request to the client device in response to the deployment request; means for receiving a remote attestation response from said client device; means for verifying said remote attestation response; means for generating an encrypted model based on the AI ​​model in response to validating the remote attestation response; as well as Means for sending the encrypted model to the client device.

Images