Heterogeneous computing-oriented cloud edge-end collaborative AI workflow execution method, system and device

Abstract

The application discloses a cloud edge end cooperative AI workflow execution method, system and device facing heterogeneous computing power. The method applied to a terminal device comprises the following steps: receiving a pre-compiled execution component and a task package of a pre-compiled workflow engine; running the pre-compiled execution component in a local sandbox to obtain privacy data; identifying sensitive entities by using a built-in desensitization pipeline and replacing them with desensitization alternative identifiers, and establishing a mapping relationship vault locally; sending a desensitization request carrying the desensitization alternative identifier and platform access credentials to an edge layer, so as to route the upstream model calling credentials hosted by a security credential storage module to a cloud heterogeneous computing power pool or an upstream large model service through a credential proxy; receiving desensitization reasoning results returned by the cloud; and restoring the desensitization alternative identifiers to sensitive entities by using the local vault and outputting. The application reduces the compliance risk caused by the original privacy data leaving the terminal device and improves the flexibility of multi-tenant model service access, computing power scheduling and resource isolation.

Description

Technical Field

[0001] This invention belongs to the field of distributed computing and artificial intelligence system architecture, and in particular relates to a cloud-edge-device collaborative AI workflow execution method, system and device for heterogeneous computing power. Background Technology

[0002] Currently, artificial intelligence (AI) workflow orchestration platforms generally adopt a centralized cloud deployment architecture. Their core computing paradigm is "data-to-code," meaning that data must be uploaded to the cloud before the cloud server executes model inference and business logic processing. While this architecture offers advantages such as centralized management and shared computing power, the inventors have discovered significant technical flaws and application bottlenecks in enterprise-level, high-security, and domestic IT compliance scenarios, as detailed below:

[0003] 1. Significant Data Privacy and Security Risks and Lack of Edge-Side Capabilities: In highly sensitive scenarios such as handling confidential corporate documents, medical records, and financial transaction data, existing platforms force users to upload all raw data containing sensitive entities (such as personal identification information, trade secrets, and privacy data) to the cloud before they can participate in large-scale model inference and workflow execution. This process directly causes data to leave the user's local controllable boundaries, facing multiple privacy and compliance risks such as data leakage, unauthorized access, misuse, and cross-border transmission. Meanwhile, current edge devices (such as terminals, edge nodes, and local servers) only serve as data collection and result display layers, lacking the ability to locally process complex workflow business logic, perform data anonymization, and conduct privacy-preserving computations. The inability of edge devices to complete the preprocessing, encryption, anonymization, and workflow execution of sensitive data locally necessitates exposing sensitive data to the cloud environment, making it difficult to meet compliance requirements such as the Cybersecurity Law 2.0, the Data Security Law, and the Personal Information Protection Law.

[0004] 2. Severe Computing Power Lock-in and Insufficient Domestic Adaptation Hinder Compliance in the Domestic IT Innovation Field: Existing AI workflow orchestration platforms are deeply tied to the proprietary GPU computing power of specific cloud providers, with their underlying architecture, drivers, operators, and runtime environments all relying on foreign hardware ecosystems. This model results in a lack of underlying adaptation capabilities for domestic chips, hindering end-to-end domestic compatibility from hardware to software. In domestic IT innovation substitution and compliance scenarios in key industries (government affairs, finance, energy, and healthcare), the platform's siloed computing power, ecosystem lock-in, and insufficient domestic adaptation prevent it from meeting the core requirements of "independent controllability, security and reliability, and domestic substitution," severely restricting the large-scale deployment of AI workflow technology in the domestic IT innovation field.

[0005] 3. Insufficient computing power isolation in high-concurrency scenarios, and limitations in credential management and resource scheduling: In multi-user, high-concurrency, and multi-tenant scenarios, centralized cloud architectures struggle to achieve physical-level computing power isolation, data isolation, and key isolation based on the separation of platform access credentials and upstream model call credentials. Traditional architectures lack credential hosting and proxy routing mechanisms, resulting in the mixed execution of tasks, data, and keys from different users within a shared computing power pool. This easily leads to resource contention, task interference, data cross-leakage, and key leakage. Furthermore, centralized architectures lack flexible computing power routing, dynamic scheduling, and tenant-level resource isolation capabilities. They cannot achieve physical isolation of computing power, dedicated key management, and independent security domains based on user needs, data sensitivity, and security levels. This results in billing confusion, resource waste, and blurred security boundaries, making it difficult to meet the application requirements of enterprise-level multi-tenancy, high security, and strong isolation. Summary of the Invention

[0006] The purpose of this invention is to provide a cloud-edge-device collaborative AI workflow execution method and system for heterogeneous computing power, so as to solve the technical problems in the prior art that original privacy data must be uploaded to the cloud, heterogeneous computing power is difficult to schedule uniformly, and concurrent resources are difficult to isolate.

[0007] The core technical solution of this invention lies in achieving "code to data reversal" and "physical layer decoupling of workflow engines." The cloud compiles the workflow into a multi-platform binary execution engine and distributes it to the terminal device; the terminal device executes a de-identified pipeline locally, builds a local mapping vault, and only uploads de-identified inference requests with de-identified substitution identifiers; the edge gateway identifies the user based on platform access credentials, reads the managed upstream model call credentials for proxy routing and result verification retry; the cloud layer smooths out the differences between underlying heterogeneous chips through a unified tensor abstraction layer.

[0008] To achieve the above objectives, the technical solution of the present invention is summarized as follows:

[0009] On one hand, this invention provides a cloud-edge-device collaborative AI workflow execution method, applied to terminal devices. This method includes the following steps:

[0010] Receive pre-compiled execution components and task packages from the pre-compiled AI workflow execution engine distributed from the edge layer or cloud layer;

[0011] The pre-compiled execution component is run in the local memory sandbox of the terminal device to obtain local privacy data to be processed.

[0012] The privacy data is identified by using the desensitization pipeline built into the pre-compiled execution component. The identified sensitive entities are replaced with typed desensitization replacement identifiers. A mapping vault between the typed desensitization replacement identifiers and real sensitive entities is established and maintained in the local memory sandbox.

[0013] The desensitized prompt word containing the typed desensitized replacement identifier and the platform access credential are encapsulated into an inference request and sent to the edge layer gateway, so that the edge layer gateway can identify the tenant identity based on the platform access credential and read the hosted upstream model call credential to proxy route the inference request to the cloud heterogeneous computing power pool or the target model service.

[0014] Receive the inference result returned from the edge layer gateway, which is generated by the cloud or target model service and contains the typed de-identification substitution identifier;

[0015] Based on the mapping relationship vault in the local memory sandbox, the typed de-identified replacement identifiers in the inference result are reverse-engineered into real sensitive entities, and the business results are output locally on the terminal device. The mapping relationship vault can reside in the terminal device's memory during the task's lifecycle, or it can be persistently stored locally on the terminal device in encrypted form with user authorization. When the task is completed, the session times out, or the user triggers cleanup, the terminal device deletes or overwrites the mapping relationship vault.

[0016] The step of using the de-identification pipeline built into the pre-compiled execution component to perform entity recognition on the privacy data includes:

[0017] Identify and replace sensitive data in standard format using regular expressions;

[0018] The lightweight runtime environment embedded in the pre-compiled execution component is invoked to run the quantized named entity recognition model locally and identify context-sensitive entities in non-standard formats; the privacy data does not leave the terminal device during the entire recognition process.

[0019] The operation of the pre-compiled execution component includes:

[0020] It provides two operating modes: a command-line interactive interface and a background resident service;

[0021] The command-line interaction interface outputs instructions with a dual semantic format containing structured data and markup language text to adapt to calls from third-party external artificial intelligence agent tools.

[0022] When the network is disconnected, the background persistent service caches the received task packets in a local encrypted queue; after the network is restored, it automatically resumes execution from the breakpoint and synchronizes the backlog of inference requests to the edge layer gateway.

[0023] The method further includes:

[0024] Analyze the node computation location requirements in the task package and execute scheduling decisions based on the frontier computation-ready node set;

[0025] If the data sensitivity of the current task node is determined to be higher than a preset threshold, then the terminal device is forced to call the neural network processor or graphics processor to load the lightweight model on the edge side to complete local inference, without sending inference requests to the edge layer.

[0026] On the other hand, the present invention also provides an AI workflow inference request routing method applied to an edge gateway device, the method comprising:

[0027] Receive an inference request sent by a terminal device, the inference request carrying de-identified prompts and platform access credentials;

[0028] The platform access credentials are verified, and the tenant identity, permission scope, or billing entity of the target customer is determined based on the verification result. Based on the tenant identity, the upstream model invocation credentials and model access address pre-configured by the target customer are read from or referenced from the security credential storage module. The upstream model invocation credentials include the upstream model vendor API Key and Base... The URL, model identifier, access token, or a combination thereof; based on the upstream model call credential, model access address, and preset compliance, cost, or latency policies, determine the target model service or cloud heterogeneous computing power pool; encapsulate the de-identified prompt words into an upstream inference request matching the target model service or cloud heterogeneous computing power pool, and have the edge gateway device proxy the inference of the target model service or cloud heterogeneous computing power pool; receive the returned preliminary inference result; verify the preliminary inference result using the preset structured pattern specification in the task package; if the verification fails, intercept the preliminary inference result, and generate a natural language retry prompt word containing the expected format requirements and error fields based on the abstract syntax tree error parsing mechanism; append the retry prompt word to the historical context, re-initiate the inference request, and continue until the verification passes, then send the result to the terminal device.

[0029] Furthermore, this invention also provides an AI model inference scheduling method for heterogeneous computing power, applied to a cloud server. This method includes:

[0030] Receive inference requests that are routed and forwarded by the edge gateway device;

[0031] The input data in the inference request is converted into a unified tensor data structure that is independent of the underlying hardware through a unified tensor abstraction layer.

[0032] Query the operator mapping registry and dynamically map the standard operator call instructions to the underlying native operator library instructions corresponding to the target heterogeneous chip based on the target heterogeneous chip type to which the inference request is routed;

[0033] The dynamic memory pooling management module calls the underlying interface of the target heterogeneous chip to allocate memory and copies the unified tensor data structure to the device to perform inference calculations.

[0034] The inference results are returned to the edge gateway device.

[0035] Furthermore, this invention also provides a global interaction method for cloud-edge-device collaborative AI workflows, the method comprising:

[0036] The cloud server breaks down the workflow specifications into task packages that follow a unified asynchronous coordination protocol, and distributes them to the terminal devices along with the pre-compiled execution components of the execution engine;

[0037] The terminal device runs the pre-compiled execution component locally to perform de-identification and generates an inference request carrying a de-identification replacement identifier, which is then sent to the edge gateway device.

[0038] The edge gateway device identifies the tenant's identity based on the platform access credentials in the inference request, reads the hosted upstream model call credentials from the security credential storage module, and proxies and routes the request to a specific heterogeneous computing power pool or upstream large model service.

[0039] After the cloud server or upstream large model service completes the underlying operator mapping and heterogeneous inference, it returns the results to the edge gateway device for structured verification and intelligent retry.

[0040] After the verification is successful, the result is sent to the terminal device, which then uses its local vault to restore the anonymized data to obtain the final business result.

[0041] In another aspect, the present invention also provides a terminal device, including: a processor and a memory;

[0042] The memory is used to store computer programs, and the processor is used to execute the computer programs to implement the methods described above.

[0043] In another aspect, the present invention also provides an edge gateway device, comprising: a processor and a memory;

[0044] The memory is used to store computer programs, and the processor is used to execute the computer programs to implement the methods described above.

[0045] In another aspect, the present invention also provides a cloud server, comprising: a processor and a memory;

[0046] The memory is used to store computer programs, and the processor is used to execute the computer programs to implement the methods described above.

[0047] This invention employs an architecture reversal of "code distribution, edge execution," preventing raw privacy data from being uploaded to the edge layer or cloud with inference requests within a pre-defined workflow execution path. This reduces compliance risks associated with raw privacy data leaving the terminal device. Furthermore, it enhances the flexibility of multi-tenant model service access, computing power scheduling, and resource isolation through a proxy routing mechanism that decouples platform access credentials from upstream model call credentials. This invention ensures data remains on the local physical device, with only de-identified replacement identifier requests uploaded, thus mitigating compliance risks from the ground up. This invention also achieves seamless compatibility with heterogeneous computing power. A unified tensor abstraction layer and operator mapping table are built in the cloud, allowing for improved heterogeneous computing power switching efficiency across different chip architectures (such as NVIDIA and domestic AI chips) without requiring modifications to the business layer code. Finally, this invention achieves high availability and automatic error correction. The edge layer introduces a schema-based intelligent retry mechanism based on AST error parsing, effectively overcoming the "illusion" problem of large language models and ensuring the accuracy of structured data extraction. The technical solution of this invention realizes computing power isolation and pay-as-you-go pricing. Through the decoupled hosting mechanism of platform access credentials and upstream model call credentials, the edge gateway can read the corresponding upstream model call credentials or computing power resource configuration for each tenant based on the unified identification of tenant identity, permission scope and billing entity, thereby realizing isolated routing, independent auditing and fine metering of multi-tenant requests. Attached Figure Description

[0048] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of the invention. Wherein:

[0049] Figure 1 This is an overview diagram of the cloud-edge-device three-layer AI workflow execution system architecture provided in this embodiment of the invention;

[0050] Figure 2 A comparison diagram of data flow and system architecture between the traditional "Data to Code" mode and the "Code to Data" mode of the present invention provided in the embodiments of the present invention;

[0051] Figure 3 This is a logical diagram of edge layer credential decoupling proxy routing and cloud heterogeneous computing power scheduling provided in an embodiment of the present invention;

[0052] Figure 4 The flowchart of the edge layer task packet verification, AST error parsing and closed-loop retry state machine provided in the embodiments of the present invention;

[0053] Figure 5 This is a diagram illustrating the secure distribution and local desensitization and restoration data flow of the endpoint execution engine provided in this embodiment of the invention. Detailed Implementation

[0054] Unless otherwise expressly stated, throughout the specification and claims, the term "comprising" or its variations such as "including" or "comprises" shall be understood to include the stated elements or components without excluding other elements or other components.

[0055] The technical solution of the present invention is illustrated below through specific embodiments. It should be understood that the one or more steps mentioned in the present invention do not preclude the existence of other methods and steps before or after the combined steps, or that other methods and steps may be inserted between these explicitly mentioned steps. It should also be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Unless otherwise stated, the numbering of each method step is only for the purpose of identifying each method step, and not for limiting the order of each method or limiting the scope of the present invention. Changes or adjustments to their relative relationships, without substantial changes to the technical content, can also be considered as within the scope of the present invention.

[0056] The raw materials and instruments used in the examples are not subject to any specific restrictions on their source; they can be purchased from the market or prepared according to conventional methods known to those skilled in the art.

[0057] To address the shortcomings of traditional technologies, this invention provides a cloud-edge-device collaborative AI workflow execution method, applied to terminal devices. This method includes the following steps:

[0058] Receive pre-compiled execution components and task packages from the pre-compiled AI workflow execution engine distributed from the edge layer or cloud layer;

[0059] The pre-compiled execution component is run in the local memory sandbox of the terminal device to obtain local privacy data to be processed.

[0060] The privacy data is identified by using the desensitization pipeline built into the pre-compiled execution component. The identified sensitive entities are replaced with typed desensitization replacement identifiers. A mapping vault between the typed desensitization replacement identifiers and real sensitive entities is established and maintained in the local memory sandbox.

[0061] The desensitized prompt word containing the typed desensitization replacement identifier and the platform access credential are encapsulated into an inference request and sent to the edge layer gateway. This allows the edge layer gateway to identify the tenant's identity based on the platform access credential and read the hosted upstream model invocation credential to proxy and route the inference request to the cloud heterogeneous computing power pool or the target model service. In some embodiments, the upstream model invocation credential can be a user-provided key.

[0062] Receive the inference result returned from the edge layer gateway, which is generated by the cloud or target model service and contains the typed de-identification substitution identifier;

[0063] Based on the mapping vault in the local memory sandbox, the typed desensitized replacement identifiers in the inference results are reversed and restored to real sensitive entities, and the business results are output locally on the terminal device.

[0064] The step of using the de-identification pipeline built into the pre-compiled execution component to perform entity recognition on the privacy data includes:

[0065] Identify and replace sensitive data in standard format using regular expressions;

[0066] The lightweight runtime environment embedded in the pre-compiled execution component is invoked to run the quantized named entity recognition model locally and identify context-sensitive entities in non-standard formats; the privacy data does not leave the terminal device during the entire recognition process.

[0067] The operation of the pre-compiled execution component includes:

[0068] It provides two operating modes: a command-line interactive interface and a background resident service;

[0069] The command-line interaction interface outputs instructions with a dual semantic format containing structured data and markup language text to adapt to calls from third-party external artificial intelligence agent tools.

[0070] When the network is disconnected, the background persistent service caches the received task packets in a local encrypted queue; after the network is restored, it automatically resumes execution from the breakpoint and synchronizes the backlog of inference requests to the edge layer gateway.

[0071] The method further includes:

[0072] Analyze the node computation location requirements in the task package and execute scheduling decisions based on the frontier computation-ready node set;

[0073] If the data sensitivity of the current task node is determined to be higher than a preset threshold, then the terminal device is forced to call the neural network processor or graphics processor to load the lightweight model on the edge side to complete local inference, without sending inference requests to the edge layer.

[0074] To address the shortcomings of traditional technologies, this invention also provides an AI workflow routing and verification method for heterogeneous computing power, applied to edge gateway devices. This method includes:

[0075] Receive an inference request sent by a terminal device, the inference request carrying de-identified prompts and platform access credentials;

[0076] The platform access credentials are verified to determine the tenant's identity. Based on the tenant's identity, the corresponding upstream model call credentials and model access address are read or referenced from the security credential storage module. Based on the preset compliance, cost, or latency policies, the de-identified inference request is proxied and routed to the corresponding cloud heterogeneous computing power pool or upstream large model service.

[0077] Receive the preliminary inference results returned by the heterogeneous computing power pool in the cloud;

[0078] The preliminary reasoning results are verified using the pre-defined structured pattern specifications in the task package;

[0079] If the verification fails, the preliminary inference result is intercepted, and a natural language retry prompt containing the expected format requirements and the error field is generated based on the abstract syntax tree error parsing mechanism.

[0080] The retry prompt is added to the historical context, and the inference request is re-initiated to the heterogeneous computing power pool in the cloud until the verification is successful, and then the result is sent to the terminal device.

[0081] To address the shortcomings of traditional technologies, this invention also provides an AI model inference scheduling method for heterogeneous computing power, applied to a cloud server. This method includes:

[0082] Receive inference requests that are routed and forwarded by the edge gateway device;

[0083] The input data in the inference request is converted into a unified tensor data structure that is independent of the underlying hardware through a unified tensor abstraction layer.

[0084] Query the operator mapping registry and dynamically map the standard operator call instructions to the underlying native operator library instructions corresponding to the target heterogeneous chip based on the target heterogeneous chip type to which the inference request is routed;

[0085] The dynamic memory pooling management module calls the underlying interface of the target heterogeneous chip to allocate memory and copies the unified tensor data structure to the device to perform inference calculations.

[0086] The inference results are returned to the edge gateway device.

[0087] To address the shortcomings of traditional technologies, this invention also provides a global interaction method for cloud-edge-device collaborative AI workflows, which includes:

[0088] The cloud server breaks down the workflow specifications into task packages that follow a unified asynchronous coordination protocol, and distributes them to the terminal devices along with the pre-compiled execution components of the execution engine;

[0089] The terminal device runs the pre-compiled execution component locally to perform de-identification and generates an inference request carrying a de-identification replacement identifier, which is then sent to the edge gateway device.

[0090] The edge gateway device identifies the tenant's identity based on the platform access credentials in the inference request, reads the hosted upstream model call credentials from the security credential storage module, and proxies and routes the request to a specific heterogeneous computing power pool or upstream large model service.

[0091] The cloud server completes heterogeneous inference based on unified tensor abstraction and operator mapping, or the upstream large model service completes model inference based on proxy call requests and returns the results to the edge gateway device for structured verification and intelligent retry; after the verification is passed, the results are sent to the terminal device, which uses the local vault to restore the de-identified data to obtain the final business result.

[0092] To address the shortcomings of conventional technologies, the present invention also provides a terminal device, comprising: a processor and a memory;

[0093] The memory is used to store computer programs, and the processor is used to execute the computer programs to implement the methods described above.

[0094] To address the shortcomings of conventional technologies, the present invention also provides an edge gateway device, comprising: a processor and a memory;

[0095] The memory is used to store computer programs, and the processor is used to execute the computer programs to implement the methods described above.

[0096] To address the shortcomings of conventional technologies, the present invention also provides a cloud server, comprising: a processor and a memory;

[0097] The memory is used to store computer programs, and the processor is used to execute the computer programs to implement the methods described above.

[0098] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0099] The system architecture of this invention is summarized below:

[0100] refer to Figure 1 This invention proposes a three-layer AI workflow execution system for heterogeneous computing power, encompassing cloud, edge, and device layers. The system is physically decoupled based on responsibilities and data sensitivity, specifically including:

[0101] Cloud Layer 100: Serves as the computing power hub and model pool. It includes a heterogeneous model inference pool 110, a workflow registry center 120, and a billing engine 130. The heterogeneous model inference pool 110 encapsulates a unified inference interface for multi-vendor GPUs / NPUs, and is backward compatible with heterogeneous hardware such as NVIDIA A100 / H100, Huawei Ascend Atlas 900, Cambricon MLU370, and Hygon DCU. The billing engine 130 supports independent metering for each user based on token usage and inference attempts.

[0102] Edge Layer 200: Serves as the scheduling hub. It includes a traffic scheduling gateway 210, a task packet distributor 220, and a result aggregator 230. Its main responsibilities include proximity access, credential proxy routing, rate limiting authentication, and schema verification of asynchronous results.

[0103] Edge Layer 300: Serves as a data sovereignty fortress. It includes a local execution engine 310, a privacy sandbox 320, and an edge-side inference module 330. It is responsible for the local execution of business logic and the anonymization of privacy data, ensuring that raw data does not leave the edge.

[0104] The following is an embodiment of the Frontier node-level scheduling decision-making technology of the present invention:

[0105] After the workflow is put into operation, the system first needs to make node-level execution location decisions. This invention introduces an evaluation mechanism for the Frontier computationally ready node set. When the workflow reaches a certain node, the following decision tree logic is executed:

[0106] First, assess the node's data sensitivity. If the node involves processing local high-security files, then a forced decision is made to "execute on the edge," calling the edge inference module 330 to load a lightweight model and perform inference using the local NPU / GPU, with data not leaving the edge.

[0107] Secondly, if a node requires a large model with extremely high parameters to perform complex inference, the decision is to "infer in the cloud" and forward it to the cloud layer 100 through the edge layer 200.

[0108] Finally, if the user has configured a specific local agent, the decision is to "distribute tasks in packages," directing the tasks to that external agent. This mechanism enables flexible computing power scheduling at the node level.

[0109] End-side execution embodiment of the technical solution of this invention: Binary secure distribution and Vault desensitization and restoration (corresponding to) Figure 2 , Figure 5 )

[0110] To achieve the "Code to Data" architecture reversal, the Workflow Registry 120 compiles core business logic into multi-platform pre-compiled execution components for distribution. These pre-compiled execution components are executable programs that are not in plaintext source code form. Their specific forms include, but are not limited to: native binary files specific to the underlying architecture of a particular operating system (such as .exe or MSIX packages for Windows, Universal Binary for macOS, ELF for Linux, .so combined with JNI for Android, and the Framework for iOS); cross-platform bytecode files; and WebAssembly (Wasm) modules based on the Web technology stack. Before distribution, the cloud layer obfuscates the logic of these pre-compiled execution components and attaches digital code signatures to increase the difficulty of reverse engineering the source code, reduce the risk of the execution components being directly tampered with or leaked, and increase the difficulty of reverse engineering.

[0111] refer to Figure 2 , Figure 2 This intuitively demonstrates the essential differences between the traditional centralized architecture and the architecture of this invention in terms of data flow and computing paradigm.

[0112] Figure 2 The upper part (Part A) illustrates the traditional "Data to Code" model. In this model, a terminal device holding raw privacy data (such as the name "Zhang San," mobile phone number "138****1234," ID number, address, and business data) needs to directly encapsulate the raw privacy data within a prompt (e.g., "Please analyze the contract risks of 'Zhang San, mobile phone number 13800001234'") and upload it in full to a cloud server via a network link, where it is centrally processed by cloud code or a cloud model. The inventors discovered that this model has serious technical flaws: the raw privacy data is forced to leave the security boundary of the terminal device, completely exposed to the cloud transmission link and cloud processing environment, which can easily lead to data leakage and violate compliance requirements for data export and privacy protection.

[0113] Figure 2 The lower half (Part B) illustrates the "Code to Data" model proposed in this invention. To overcome the aforementioned shortcomings, this invention introduces a collaborative mechanism between cloud servers, edge gateways, and terminal devices in its system architecture. The specific data flow and execution logic are as follows:

[0114] Step ① (Code / Task Deployment): The cloud server (including modules such as a workflow registry, pre-compiled execution components, and heterogeneous computing power inference pool) transforms the business logic into pre-compiled execution components and workflow task packages, and actively deploys them to the terminal devices where the data resides. This achieves a paradigm reversal in computation, "letting the code execute where the data is located."

[0115] Step ② (Local Anonymization and Request Forwarding): After receiving the code, the terminal device performs privacy verification and local anonymization in its local execution engine. The terminal device establishes and maintains a "local mapping vault" locally (e.g., establishing...).<PERSON_1> ↔ Zhang San,<PHONE_1> The mapping relationship for 13800001234). This vault is physically stored only locally on the terminal. Subsequently, the terminal device will only display a prompt carrying a de-identified replacement identifier (e.g., "Please analyze").<PERSON_1> ,Phone number<PHONE_1> The "contractual risk" is encapsulated as an inference request and sent to the edge gateway. After performing tenant identification, credential hosting reading, and proxy route verification, the edge gateway only forwards the de-identified request to the heterogeneous computing power inference pool in the cloud or the upstream large model service. Throughout the process, under the preset workflow execution path, the original privacy data is not uploaded to the edge layer or the cloud along with the inference request.

[0116] It should also be noted that proxy routing strategies may include tenant level, data region, model vendor whitelist, model capability label, request latency threshold, model call cost, upstream service health status, chip type, concurrency limit, or a combination thereof.

[0117] Step 3 (Result Return and Local Restoration): After the upstream service called by the cloud server or proxy completes the large model inference based on the de-identified prompts, it returns the inference result containing the de-identified replacement identifier to the terminal device via the edge gateway. Finally, the terminal device uses the local mapping vault to reverse-engineer the de-identified replacement identifier back to the original privacy information and outputs the business result.

[0118] Through the above Figure 2 As can be seen from the architecture comparison shown, the core technical effect of this invention is that it ensures that the original privacy data does not leave the terminal from the physical architecture and data flow to the bottom layer, and that the cloud and transmission links only come into contact with and process the desensitized content, thus completely solving the data privacy security and information technology compliance issues in the execution of cloud AI workflows.

[0119] refer to Figure 2 and Figure 5 The local execution engine 310 starts within the privacy sandbox 320 on the client side. Taking "Medical Record Summary" as an example, the client side obtains the raw data: "Patient Zhang Jianguo fainted today at ByteDance, costing 119.58 yuan." The de-identification pipeline 311 starts:

[0120] 1. The regular expression matcher 311a identifies the standard amount "119.58 yuan".

[0121] 2. The local NER model 311b (an ONNX quantization mini-model embedded in the pre-compiled execution component, running offline) identified “Zhang Jianguo” (PERSON) and “ByteDance” (ORG).

[0122] Subsequently, a local mapped vault map 312 is established in the endpoint memory, recording the mapping: {"<PERSON_1> Zhang Jianguo<ORG_1> ": "ByteDance"}.

[0123] In this invention, the engine replaces the identified sensitive entities with "de-identified replacement identifiers". These de-identified replacement identifiers refer to a substitute that does not have actual privacy meaning in the large cloud model, but can be restored through mapping relationships on the client side. Specific forms include, but are not limited to: (1) Type-specific placeholders, such as using...<PERSON_1> Replace personal name, use<ORG_1> (1) Replace company names to preserve the syntactic context required for large model inference; (2) Cryptographic hash, such as using SHA-256 truncated values ​​to replace sensitive order numbers; (3) Virtual pronouns, such as using “Company A” or “Company A” to replace real names to enhance the coherence of natural language; (4) Random obfuscated strings or irreversible cryptographic tokens.

[0124] In this embodiment, the engine uses typed placeholders as an example, replacing the original text with desensitized prompts: "patient".<PERSON_1> Today in<ORG_1> The message "Fainting..." is encapsulated as an inference request and sent to edge layer 200. Upon receiving the result from the cloud containing the de-identified replacement identifier, the engine queries the local mapping vault 312, reverses the de-identified replacement identifier to the real data, and outputs it. During runtime, the engine immediately performs memory zeroing on sensitive data. Furthermore, the local execution engine 310 provides two modes:

[0125] It should be noted that the typed desensitization replacement identifier retains the entity type, sequence number, role attribute or semantic category of the sensitive entity, enabling the cloud model to maintain the reasoning context without knowing the real sensitive entity.

[0126] CLI Interaction Mode: Supports built-in Agent (BYOA) mode. The instruction set adopts a "dual semantic format", which includes both JSON for AI IDEs (such as Cursor) to parse and Markdown for human review.

[0127] Background persistent service: When the network is disconnected, the task is cached in a local SQLite encrypted queue for offline processing; after the network is restored, the de-identified request is automatically synchronized from the breakpoint to the edge layer 200.

[0128] An edge-cloud collaboration embodiment of the technical solution of this invention: Platform access credential verification, credential hosting, and proxy routing. In this embodiment, the platform access credential and the upstream model call credential are different types of credentials. The platform access credential is distributed to the target customer by the cloud-edge-device collaborative AI workflow platform and is used to identify the customer's identity, tenant affiliation, permission scope, billing entity, or workspace at the edge gateway device. The upstream model call credential is pre-configured by the target customer into the platform's secure credential storage module and is used to access the target customer's own or designated upstream large model service. The secure credential storage module can be implemented using key management services, hardware security modules, trusted execution environments, database field-level encryption, or a combination thereof. The upstream model call credential is stored in encrypted form and bound to a tenant identifier, workspace identifier, application identifier, or credential reference identifier. After the platform access credential verification is successful, the edge gateway device reads the temporary decryption result or temporary access token based on the credential reference identifier.

[0129] refer to Figure 3The inference request sent from the endpoint carries platform access credentials (such as a platform API Key or signature token), but not the user's upstream model vendor API Key. Upon reaching the edge layer 200, the platform access credential verification module 211 in the traffic scheduling gateway 210 reads and verifies the credential in memory to determine the tenant identity of the target customer. Subsequently, the security credential reading module 212, based on the tenant identity, reads the upstream model call credentials (such as the upstream model vendor API Key, Base URL, and model name) pre-configured for the target customer from the security credential storage module. The proxy routing module 213 routes the request to the Huawei Ascend cluster in the cloud according to a policy (such as compliance with domestic IT innovation), or converts the anonymized inference request into a request format compatible with the aforementioned upstream model service interface, and calls the upstream model service in a proxy manner. The gateway supports multi-key health checks and automatic failover upon failure. After routing is completed, the key in memory is immediately zeroed out and overwritten, and is not persisted. The security credential storage module stores the upstream model call credentials in encrypted form. The edge gateway device does not return the upstream model invocation credentials to the terminal device, nor does it write the upstream model invocation credentials to the inference results or scheduling logs. By decoupling the platform access credentials from the upstream model invocation credentials, the edge gateway device can uniformly perform tenant identification, permission verification, and audit logging while selecting the pre-configured upstream model service for each tenant. This approach allows different tenants to reuse their existing upstream model service accounts or model access resources, thereby improving the flexibility of model service access and reducing the pressure on the platform to uniformly maintain a fixed model provider resource pool.

[0130] Request access to heterogeneous model inference pool 110 at cloud layer 100. For compatibility with the underlying chip:

[0131] The Unified Tensor Abstraction Layer 111 converts the input Token IDs into a unified tensor structure.

[0132] The dynamic memory pooling module 113 calls the target chip interface (such as Ascend acl.rt.malloc) to allocate device memory.

[0133] Operator mapping registry 112 intercepts standard operation calls and dynamically maps them to native operators of the target architecture (e.g., mapping matrix multiplication to cublas on CUDA architecture chip 114a, or aclnnMatmul on CANN architecture chip 114b). This enables heterogeneous scheduling that is transparent to upper-layer services.

[0134] An embodiment of the edge layer task flow of the present invention: task packet distribution and AST intelligent retry reference. Figure 4The WorkflowSpec is decomposed into task packages 221 by the task package dispatcher 220 in the edge layer. The task package adopts a JSONSchema structure and includes: task_id, node_spec, input_context, expected_output_schema, deadline, and callback_url.

[0135] After the client-side Agent completes its task, it sends a callback to the callback_url. Upon receiving the callback, the aggregator 230 first performs idempotent deduplication based on the task_id; if no callback is received within the timeout period, it triggers redistribution.

[0136] After obtaining the initial inference results, the schema validator 231 in the result aggregator 230 validates the results. If the large model exhibits "illusion" (such as the expected number being output as "$119.58"), the validator will intercept it.

[0137] At this point, the AST parsing and retry module 232 starts: based on the abstract syntax tree, it extracts the expected type and the actual error field, and automatically generates retry instructions in natural language (e.g., "The expected cost is a number, but "$119.58" was output. Please correct it and output only pure numbers"). The edge layer 200 appends the instructions to the historical context and automatically re-initiates inference to the cloud layer 100, forming a closed-loop retry until the verification is successful and then it is sent to the end layer 300.

[0138] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0139] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the method described above; the electronic device includes the processor, which is configured to perform the method described above.

[0140] The example electronic devices of embodiments of the present invention are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their patterns are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0141] The electronic device includes a computing unit that can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). The RAM may also store various programs and data required for the operation of the electronic device. The computing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0142] Multiple components in an electronic device are connected to an I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

Claims

1. A cloud-edge-device collaborative AI workflow execution method, applied to terminal devices, characterized in that, Includes the following steps: Receive pre-compiled execution components and task packages from the pre-compiled AI workflow execution engine distributed from the edge layer or cloud layer; The pre-compiled execution component is run in the local memory sandbox of the terminal device to obtain local privacy data to be processed. The privacy data is identified by using the desensitization pipeline built into the pre-compiled execution component. The identified sensitive entities are replaced with typed desensitization replacement identifiers. A mapping vault between the typed desensitization replacement identifiers and real sensitive entities is established and maintained in the local memory sandbox. The desensitized prompt word containing the typed desensitized replacement identifier and the platform access credential are encapsulated into an inference request and sent to the edge layer gateway, so that the edge layer gateway can identify the tenant identity based on the platform access credential and read the hosted upstream model call credential to proxy route the inference request to the cloud heterogeneous computing power pool or the target model service. Receive the inference result returned from the edge layer gateway, which is generated by the cloud or target model service and contains the typed de-identification substitution identifier; Based on the mapping vault in the local memory sandbox, the typed desensitized replacement identifiers in the inference results are reversed and restored to real sensitive entities, and the business results are output locally on the terminal device.

2. The method according to claim 1, characterized in that, The step of using the de-identification pipeline built into the pre-compiled execution component to perform entity recognition on the privacy data includes: Identify and replace sensitive data in standard format using regular expressions; The lightweight runtime environment embedded in the pre-compiled execution component is invoked to run the quantized named entity recognition model locally and identify context-sensitive entities in non-standard formats; the privacy data does not leave the terminal device during the entire recognition process.

3. The method according to claim 1, characterized in that, Running the pre-compiled execution component includes: It provides two operating modes: a command-line interactive interface and a background resident service; The command-line interaction interface outputs instructions with a dual semantic format containing structured data and markup language text to adapt to calls from third-party external artificial intelligence agent tools. When the network is disconnected, the background persistent service caches the received task packets in a local encrypted queue; after the network is restored, it automatically resumes execution from the breakpoint and synchronizes the backlog of inference requests to the edge layer gateway.

4. The method according to claim 1, characterized in that, The method further includes: Analyze the node computation location requirements in the task package and execute scheduling decisions based on the frontier computation-ready node set; If the data sensitivity of the current task node is determined to be higher than a preset threshold, then the terminal device is forced to call the neural network processor or graphics processor to load the lightweight model on the edge side to complete local inference, without sending inference requests to the edge layer.

5. A method for routing AI workflow inference requests applied to edge gateway devices, characterized in that, include: Receive an inference request sent by a terminal device, the inference request carrying de-identified prompts and platform access credentials; Verify the platform access credentials and determine the tenant identity, permission scope, or billing entity of the target customer based on the verification results; Based on the tenant identity, the upstream model call credentials and model access address pre-configured by the target customer are read or referenced from the security credential storage module. The upstream model call credentials include the upstream model vendor API Key, BaseURL, model identifier, access token or a combination thereof. Based on the upstream model call credentials, model access address, and preset compliance, cost, or latency policies, determine the target model service or cloud heterogeneous computing power pool; The de-identified prompts are encapsulated into an upstream inference request that matches the target model service or the cloud heterogeneous computing power pool, and the edge gateway device proxies and calls the target model service or the cloud heterogeneous computing power pool. Receive the returned preliminary reasoning results; The preliminary reasoning results are verified using the pre-defined structured pattern specifications in the task package; If the verification fails, the preliminary inference result is intercepted, and a natural language retry prompt containing the expected format requirements and the error field is generated based on the abstract syntax tree error parsing mechanism. The retry prompt is appended to the historical context, and the inference request is re-initiated until the verification passes, at which point the result is sent to the terminal device.

6. A method for scheduling AI model inference based on heterogeneous computing power, applied to cloud servers, characterized in that, include: Receive inference requests that are routed and forwarded by the edge gateway device; The input data in the inference request is converted into a unified tensor data structure that is independent of the underlying hardware through a unified tensor abstraction layer. Query the operator mapping registry and dynamically map the standard operator call instructions to the underlying native operator library instructions corresponding to the target heterogeneous chip based on the target heterogeneous chip type to which the inference request is routed; The dynamic memory pooling management module calls the underlying interface of the target heterogeneous chip to allocate memory and copies the unified tensor data structure to the device to perform inference calculations. The inference results are returned to the edge gateway device.

7. A global interaction method for cloud-edge-device collaborative AI workflow, characterized in that, include: The cloud server breaks down the workflow specifications into task packages that follow a unified asynchronous coordination protocol, and distributes them to the terminal devices along with the pre-compiled execution components of the execution engine; The terminal device runs the pre-compiled execution component locally to perform de-identification and generates an inference request carrying a de-identification replacement identifier, which is then sent to the edge gateway device. The edge gateway device identifies the tenant's identity based on the platform access credentials in the inference request, reads the hosted upstream model call credentials from the security credential storage module, and proxies and routes the request to a specific heterogeneous computing power pool or upstream large model service. After the cloud server or upstream large model service completes the underlying operator mapping and heterogeneous inference, it returns the results to the edge gateway device for structured verification and intelligent retry. After the verification is successful, the result is sent to the terminal device, which then uses its local vault to restore the anonymized data to obtain the final business result.

8. A terminal device, characterized in that, include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program to implement the method according to any one of claims 1 to 4.

9. An edge gateway device, characterized in that, include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program to implement the method of claim 5.

10. A cloud server, characterized in that, include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program to implement the method of claim 6.

Images