AI process processing method and related equipment
By generating and automating AI processes through a process component library, the problems of long AI process development cycles and resource waste have been solved, enabling modular development and automated deployment, and improving development efficiency and resource utilization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGZHOU GUANGHUA ZHIDIAN TECH CO LTD
- Filing Date
- 2025-12-31
- Publication Date
- 2026-04-24
Smart Images
Figure CN121920802A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for processing AI processes and related equipment. Background Technology
[0002] In related technologies, applying AI technology to specific industrial scenarios, such as the intelligent inspection process of power equipment, can effectively improve the efficiency and accuracy of power inspection. However, the current development process of AI processes for specific scenarios faces the following problems: First, it heavily relies on expert experience, requiring each scenario to be developed from scratch in a "workshop" style, resulting in long project cycles and high costs; second, the resulting solutions are strongly coupled with specific scenarios, making it difficult to effectively reuse algorithm models and business logic in other scenarios, creating a large number of "AI silos"; and third, the lack of unified technical standards prevents the effective accumulation and sharing of results between different projects.
[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention
[0004] The main objective of this application is to propose an AI process processing method and related equipment, which can realize the modular development, building block combination and automated deployment of industry AI application processes, thereby shortening the project development cycle, reducing development costs and improving resource utilization.
[0005] To achieve the above objectives, one aspect of this application proposes a method for processing AI workflows. This method is executed based on a workflow component library, which includes a domain knowledge-driven task meta-library, an AI operator library, and a neural network model library. The method includes the following steps: Based on the target application scenario, an executable AI process is generated according to the process building library. The executable AI process includes data flow and control flow. The executable AI process is analyzed to obtain the computational characteristics and resource requirements of each node; Obtain the real-time resource status of the deployment environment; Based on the computational characteristic information, the resource requirement information, and the real-time resource status, resource adaptation is performed on the executable AI process to obtain the target deployment scheme of the executable AI process; The executable AI process is deployed according to the target deployment scheme to run the executable AI process in the deployment environment.
[0006] In some embodiments, generating an executable AI process based on the process library according to the target application scenario includes: Obtain the task element corresponding to the target application scenario from the domain knowledge-driven task element library as the target task element; Obtain the neural network model corresponding to the target task element from the neural network model library as the target neural network model; After decomposing the target neural network model, the target AI operator is extracted from the AI operator library; Based on the process template and combination logic, the executable AI process is generated according to the target task element, the target neural network model, and the target AI operator.
[0007] In some embodiments, obtaining the neural network model corresponding to the target task element from the neural network model library as the target neural network model includes: Extract the feature vector of the target task element; The target neural network model is obtained by searching and matching the feature vectors in the neural network model library.
[0008] In some embodiments, the step of extracting target AI operators from the AI operator library after decomposing the target neural network model includes: Obtain the structure definition file of the target neural network model; Traverse the structural computation graph corresponding to the target neural network model and extract all operator nodes; The operators corresponding to the operator nodes are identified to obtain custom optimized operators; Based on the type name of the custom optimization operator and the target hardware platform, the corresponding target AI operator is extracted from the AI operator library.
[0009] In some embodiments, the step of adapting the executable AI process to resources based on the computing characteristic information, the resource requirement information, and the real-time resource status to obtain a target deployment scheme for the executable AI process includes: Based on the computational characteristic information, the resource requirement information, and the real-time resource status, a reinforcement learning agent performs resource adaptation for each task node in the executable AI process, calculates a reward value based on the environmental feedback information corresponding to each resource adaptation scheme, and determines the target deployment scheme of the executable AI process based on the reward value.
[0010] In some embodiments, the deployment operation of the executable AI process according to the target deployment scheme includes: The executable AI process, the corresponding neural network model, AI operators, and dependent environment are containerized and packaged to obtain a containerized image; The containerized image is distributed and scheduled in the edge-cloud cluster according to the target deployment plan.
[0011] In some embodiments, when running the executable AI process in the deployment environment, the method further includes the following steps: Obtain performance monitoring information during the execution process; Adjust the target deployment plan based on the performance monitoring information.
[0012] To achieve the above objectives, another aspect of this application proposes an AI process processing apparatus. The apparatus executes based on a process component library, which includes a domain knowledge-driven task meta-library, an AI operator library, and a neural network model library. The apparatus includes: The first module is used to generate an executable AI process based on the target application scenario and the process building library, wherein the executable AI process includes a data flow and a control flow; The second module is used to analyze the executable AI process to obtain the computational characteristics and resource requirements of each node. The third module is used to obtain the real-time resource status of the deployment environment; The fourth module is used to perform resource adaptation on the executable AI process based on the computing characteristic information, the resource requirement information and the real-time resource status, so as to obtain the target deployment scheme of the executable AI process; The fifth module is used to deploy the executable AI process according to the target deployment plan, so as to run the executable AI process in the deployment environment.
[0013] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.
[0014] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.
[0015] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method.
[0016] The embodiments of this application include at least the following beneficial effects: This application provides a method and related equipment for processing AI workflows. This solution utilizes a pre-built workflow construction library comprising a domain knowledge-driven task meta-library, an AI operator library, and a neural network model library. Each task meta-library in the task meta-library contains input description information, output description information, and functional description information. Each AI operator in the AI operator library carries performance indicators and hardware dependency labels. Each neural network model in the neural network model library is associated with a preset task meta-library in the task meta-library and carries annotation information corresponding to the preset AI operators in the AI operator library. This allows the AI workflow processing to be carried out according to… The process building library generates executable AI processes corresponding to the target application scenario. Then, it analyzes the executable AI processes to obtain the computational characteristics and resource requirements of each node. Simultaneously, it obtains the real-time resource status of the deployment environment and performs resource adaptation on the executable AI processes to obtain the target deployment plan. Then, it deploys the executable AI processes according to the target deployment plan to run the executable AI processes in the deployment environment. This enables modular development, modular combination, and automated deployment of AI application scenarios, thereby effectively shortening the project development cycle, reducing development costs, and improving resource utilization for the target application scenario. Attached Figure Description
[0017] Figure 1 This is a flowchart of the AI process processing method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the architecture corresponding to the AI process processing method provided in the embodiments of this application; Figure 3 This is a schematic diagram illustrating the content and relationships of each sub-library within the process construction library provided in the embodiments of this application; Figure 4 This is a flowchart of the process deployment and resource adaptation provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the AI process processing device provided in the embodiments of this application; Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.
[0019] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”
[0020] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.
[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0022] In related technologies, applying AI technology to specific industrial scenarios, such as the intelligent inspection process of power equipment, can effectively improve the efficiency and accuracy of power inspection. However, the development process of AI processes for specific scenarios currently faces the following problems: First, the development process is highly dependent on expert experience and extremely fragmented. For example, taking the "transmission line hazard identification" application, developers need to collect data, train models, and develop inference programs separately for different targets such as bird nests, missing insulators, and large vehicles. Each sub-task is an independent, customized AI project. This "workshop" style development model results in long project cycles, high labor costs, and difficulty in reusing results between different projects, creating numerous "AI silos" that hinder resource reuse and model iteration.
[0023] Second, AI models are disconnected from industry business processes. Most current AI platforms provide model training and deployment tools, but a complete industry application is a complex process that may include data preprocessing, serial or parallel inference of multiple models, post-processing logic, and result distribution. Existing technologies lack a high-level abstraction that can "glue" individual AI models into a complete, business-logic-compliant intelligent process. This forces developers to invest significant effort in low-level coding to implement the business logic, rather than focusing on the value of AI itself.
[0024] Third, the performance optimization of AI models and operators is tightly coupled with specific hardware platforms. To achieve real-time requirements on resource-constrained edge devices, it is often necessary to develop high-performance custom operators or optimize model structures for specific hardware. This work has an extremely high technical threshold, and the optimization results are often tied to specific projects and hardware, making it impossible to smoothly migrate to other processes, resulting in redundant development and wasted resources.
[0025] In view of this, this application provides an AI process processing method and related equipment, which can realize the modular development, building block combination and automated deployment of industry AI application processes, thereby shortening the project development cycle, reducing development costs and improving resource utilization.
[0026] The AI process processing method provided in this application relates to the field of artificial intelligence technology. This AI process processing method can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the AI process processing method, but is not limited to the above forms.
[0027] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
[0028] The embodiments of this application will be described in detail below with reference to the accompanying drawings: Figure 1 This is an optional flowchart of the AI process processing method provided in this application embodiment. The method of this embodiment can be executed based on a process component library, which includes a domain knowledge-driven task meta-library, an AI operator library, and a neural network model library. Specifically, the process component library includes a domain knowledge-driven task meta-library, an AI operator library, and a neural network model library. Each task meta-library in the task meta-library has input description information, output description information, and functional description information. Each AI operator in the AI operator library carries performance indicators and hardware dependency labels. Each neural network model in the neural network model library is associated with a preset task meta-library in the task meta-library and carries annotation information corresponding to a preset AI operator in the AI operator library.
[0029] Specifically, Figure 1 The method shown may include, but is not limited to, steps S110 to S150: Step S110: Build the process library, where; Step S110: Based on the target application scenario, generate an executable AI process according to the process building library. The executable AI process includes data flow and control flow. Step S120: Analyze the executable AI process to obtain the computational characteristics and resource requirements of each node; Step S130: Obtain the real-time resource status of the deployment environment; Step S140: Based on the computing characteristics information, resource requirements information and real-time resource status, perform resource adaptation on the executable AI process to obtain the target deployment plan for the executable AI process; Step S150: Deploy the executable AI process according to the target deployment plan to run the executable AI process in the deployment environment.
[0030] It is understandable that, such as Figure 2 As shown, in this embodiment, the process construction library is set in the process construction layer. This process construction layer serves as the core asset library for subsequent execution processes, providing task elements, high-performance AI operators, and neural network models for multiple scenarios. Figure 3 As shown, the domain knowledge-driven task meta-library, AI operator library, and neural network model library in the process building library are pre-set with corresponding mapping or association relationships, so that the subsequent application process can quickly and accurately complete the calling process of task meta-library, AI operator, and neural network model through the mapping and association relationships.
[0031] Specifically, the domain knowledge-driven task element library in this embodiment stores standardized decompositions of industry problems. For example, in the field of power line inspection, "transmission line inspection" can be decomposed into multiple standard task elements such as "bird nest recognition," "insulator missing recognition," and "large vehicle recognition." Each task element defines input description information, output description information, and functional description information. The AI operator library is a high-performance AI operator library used to store high-performance custom operators optimized for specific hardware (such as Huawei Ascend NPU). For example, there is the "adhesion segmentation operator" developed for vehicle occlusion, and the "SmokeDetectionOperator" developed for image preprocessing. Each AI operator in the AI operator library has clear performance metrics and hardware dependency labels. The neural network model library is used to store pre-trained and optimized neural network models for specific tasks. For example, there is the "glass insulator missing recognition model" based on YOLOv5, and the "respirator silicone color change recognition model" based on ResNet-50. Each model is associated with its most suitable task element, and the accuracy, speed, and required AI operator support for each neural network model are labeled.
[0032] It is understandable that, such as Figure 2 As shown, in this embodiment, the generation process of the executable AI process can be automated in the process assembly and generation layer. Specifically, the process of generating the executable AI process in this embodiment may include, but is not limited to, the following steps: Obtain the task elements corresponding to the target application scenario from the domain knowledge-driven task element library as the target task elements; Obtain the neural network model corresponding to the target task element from the neural network model library as the target neural network model; After decomposing the target neural network model, the target AI operator is extracted from the AI operator library; Based on process templates and combinational logic, an executable AI process is generated according to the target task element, the target neural network model, and the target AI operator.
[0033] like Figure 2 As shown, the process assembly and generation layer of this embodiment pre-configures a graphical process designer and an automatic mapping and filling engine. The graphical process designer in this embodiment provides a graphical interface, allowing users to select required task elements from a domain knowledge-driven task element library via drag-and-drop, and connect them into a flowchart according to business logic. The automatic mapping and filling engine automatically maps each selected task element to the most suitable target neural network model in the neural network model library after the user completes the flowchart design. It then parses and extracts the high-performance AI operators relied upon by the target neural network model from the AI operator library as the target AI operators, automatically generating a complete executable AI process containing data flow and control flow. For example, using... Figure 3 Taking the process of building the library as an example, the user drags in the "Insulator Missing Identification" task element, which automatically matches it with the YOLOv5 insulator model and loads the required AI operators such as Conv2D and ResizeNearestNeighbor.
[0034] It is understood that the core objective of the automatic mapping and filling engine in this embodiment is to transform an abstract target task element that describes "what to do" into a concrete, executable task node that knows "how to do it." It acts like an intelligent automation assistant, translating the user's business intent into detailed instructions for machine execution. Specifically, the processing of the automatic mapping and filling engine includes, but is not limited to, the following steps: Step 1: Automatic Mapping to Obtain the Target Neural Network Model: Finding the most suitable target neural network model for the target task elements. The goal of this embodiment is to automatically select the most suitable pre-trained model from the neural network model library as the target neural network model based on the definition of the target task elements. The specific process is as follows: Step 1.1: Extract the feature vectors of the target task elements: When a user drags a target task element (such as "insulator missing identification") into the process designer, the automatic mapping and filling engine reads the standardized metadata of that target task element and transforms it into a feature vector that the system can understand. This feature vector may include, but is not limited to, the following: task_type: Task type (e.g., object detection, image classification, semantic segmentation).
[0035] domain: The field or area of expertise (e.g., power line inspection, security, industrial quality inspection).
[0036] target_object: The target object (e.g., insulator, bird's nest, silicone respirator).
[0037] input_type: Input data type (e.g., RGB image, infrared image, video stream).
[0038] performance_requirement: Performance requirements (e.g., high-precision mode, high-speed mode).
[0039] Step 1.2: Obtain the target neural network model by searching and matching the feature vectors in the neural network model library.
[0040] Specifically, this embodiment uses the aforementioned feature vectors to perform a search in a neural network model library. Each neural network model in the library is associated with similar, standardized labels. Then, a matching algorithm is used to calculate the matching degree between each target task element and the corresponding label of each neural network model in the library. The matching algorithm can be based on rules, weighted scoring, or machine learning ranking models to calculate the matching degree between the task element features and each model label. In this embodiment, the matching algorithm's rules include primary and secondary rules. The primary rule is that task_type and target_object must match completely. For example, the task element "insulator missing identification" must match a model with task_type "object detection" and target_object containing "insulator". The secondary rule selects the best candidate model from those satisfying the primary rule based on the following dimensions: Dimension 1, Accuracy First: Select the model with the highest accuracy or mAP index.
[0041] Dimension 2, Speed Priority: If the process has real-time requirements, then select the model with the fastest inference_speed.
[0042] Dimension 3, Hardware Affinity: Prioritize model versions that have been optimized for the target hardware (such as Ascend NPU).
[0043] Step 1.3: Obtain the following from the neural network model library obtained through output mapping: Specifically, the matching algorithm outputs one or more recommended models (which can be sorted by confidence). In this embodiment, the target neural network model ranked first is selected as the "best-fit model" for the target task element. Therefore, the "insulator missing identification" task element is specifically mapped to the model file "YOLOv5-insulator-defect-v3.pt".
[0044] Step 2: Decompose the target neural network model and extract the target AI operators that the target neural network model depends on from the AI operator library. The goal of this embodiment is to analyze the target neural network model selected in the previous step and identify the custom operators necessary for the operation of the target neural network model, especially those that require high-performance optimization. Specifically, the execution process of this embodiment includes, but is not limited to, the following steps: Step 2.1, Target neural network model structure definition file: In this embodiment, after locating the selected target neural network model, the corresponding structure definition file of the target neural network model is read. The structure definition file can be an ONNX file, a TensorFlow SavedModel directory or GraphDef file, a PyTorch torchscript file, or other intermediate representation (IR) files.
[0045] Step 2.2: Traverse the structural computation graph corresponding to the target neural network model and extract all operator nodes: The target neural network model in this embodiment is essentially a computation graph, composed of nodes (operators) and edges (tensors). Therefore, the parsing engine in this embodiment traverses this computation graph and extracts all operator nodes. For example, a classic CNN model may contain operators such as Conv2D, ReLU, MaxPool, and Resize.
[0046] Step 2.3: Identify the operators corresponding to the operator nodes to obtain custom optimized operators, and extract the corresponding target AI operators from the AI operator library based on the type name of the custom optimized operators and the target hardware platform: The parsing engine in this embodiment does not simply list all basic operators, but possesses intelligent recognition capabilities. Specifically, this embodiment identifies which operators are common standard operators natively supported by inference frameworks (such as TensorRT, CANN), and which are critical custom optimized operators that need to be loaded from a high-performance AI operator library. For each identified custom optimized operator (such as SmokeDetectionOperator), this embodiment queries the high-performance AI operator library using its operator type name and target hardware platform (such as Ascend 310) as keys. Since the operator library stores multiple implementation versions of the custom optimized operator (such as CPU version, GPU CUDA version, NPU version), this embodiment also precisely loads the optimized version (such as .so dynamic library file) corresponding to the target hardware platform determined during the process deployment phase.
[0047] It is understood that when this embodiment decomposes the target neural network model, it can also perform dependency checks to analyze the dependency information of the target neural network model or the target AI operator, such as specific software library versions, driver versions, etc., and record this dependency information to improve the accuracy of subsequent AI process execution.
[0048] It is understandable that, such as Figure 4 As shown, in this embodiment, after obtaining the executable AI process, in Figure 2 The resource adaptation engine in the process deployment and optimization layer, as shown, performs resource requirement analysis on the executable AI process to obtain resource requirements including task node computing characteristics, NPU / GPU affinity, memory and bandwidth, and real-time requirements. Combined with real-time resource status obtained from real-time environmental monitoring of the target application scenario, the reinforcement learning agent in the reinforcement learning optimizer performs resource adaptation for each task node in the executable AI process. Based on the environmental feedback information corresponding to each resource adaptation scheme, a reward value is calculated, and the target deployment scheme for the executable AI process is determined according to the reward value. In this embodiment, the real-time resource status includes, but is not limited to, monitoring terminal device load, monitoring terminal edge node resources, monitoring cloud resource status, and monitoring network bandwidth and latency. Figure 4 As shown, in this embodiment, the target deployment scheme obtained by the reinforcement learning agent can deploy task A in the executable AI process to the edge NPU, task B to the edge NPU, task C to the cloud CPU, and task D to the end device.
[0049] Specifically, after obtaining the target deployment scheme, this embodiment can use containerization technology to package the executable AI process, the corresponding neural network model, AI operators, and dependent environment into a containerized image. Then, it can distribute and schedule the image in the edge-cloud cluster through the target deployment scheme, so that the corresponding processes of the executable AI process can be executed on the end device, edge node, and cloud platform respectively, thereby completing the execution process of the entire executable AI process.
[0050] Specifically, such as Figure 4 As shown, in this embodiment, when running the executable AI process in the deployment environment, performance monitoring information of the execution process is also obtained, and the target deployment plan is adjusted according to the performance monitoring information to improve the matching degree between the target deployment plan and the target application scenario, thereby improving the execution efficiency and accuracy of the executable AI process.
[0051] For example, taking a substation intelligent inspection process comprising three task nodes as an example, node A in this process is used for fire smoke recognition (computationally intensive, large model), node B is used for respirator silicone color change recognition (computationally intensive, medium model), and node C is used for result aggregation and alarm (logic control intensive, lightweight). The available computing resources determined based on real-time resource status in this substation intelligent inspection scenario include: Edge computing: Atlas 200 DK (Ascend 310 NPU, with strong computing power but limited memory) at the substation site Cloud: GPU servers in remote data centers (powerful computing power, elastic resources, but with network latency).
[0052] The optimization goal of this embodiment is to minimize the overall end-to-end execution time while meeting real-time requirements (e.g., overall process latency < 2 seconds), and to take into account system energy consumption and bandwidth usage.
[0053] This embodiment models the resource allocation problem in the above scenario as a reinforcement learning problem according to this application embodiment, and first defines the following key elements based on the above scenario: Intelligent Agent: Resource Adaptation Engine.
[0054] Environment: A system consisting of heterogeneous hardware clusters at the edge and cloud, running AI processes, network status, etc.
[0055] State: A snapshot of the environment at time t. S(t) = [Computational cost of task A, computational cost of task B, computational cost of task C, remaining NPU computing power, remaining GPU computing power, network bandwidth, current deployment scheme].
[0056] Action: The behavior that an agent can take, namely, reallocating computing devices to one or more task nodes. A = {allocate to task A: NPU or GPU; allocate to task B: NPU or GPU; allocate to task C: Edge-CPU or Cloud}; a specific action example: [A->NPU, B->GPU, C->Edge-CPU].
[0057] Reward: The score given by the environment after the agent performs an action. Our optimization goals are all reflected here.
[0058] Positive Rewards: +R1: Reduced total process latency. +R2: Successfully meeting deadlines (e.g., <2 seconds). +R3: Saving network bandwidth (e.g., task C is processed at the edge).
[0059] Negative rewards: -R4: Increased total process latency. -R5: Deadline violation. -R6: Individual device resource overload (e.g., NPU memory overflow). -R7: High bandwidth consumption.
[0060] Based on the definitions of the above key elements, this application provides an example of the processing procedure for its embodiments. In this example, it is assumed that the initial deployment scheme is [A->NPU, B->NPU, C->Cloud], and the specific execution process includes the following steps: Step 1: State Awareness: The engine perceives the current state S(t): The NPU load is very high, tasks A and B are queuing on the NPU, and network bandwidth is tight, resulting in a total latency of 2.5 seconds, which exceeds the requirements.
[0061] Step 2: Decision: Based on the current policy (a function that maps states to actions), the agent decides to try a new deployment scheme. It selects an action: migrate task B from the NPU to the GPU. That is, the new action is [A->NPU, B->GPU, C->Cloud].
[0062] Step 3: Execution and Evaluation: Execute this new plan: Task A has exclusive access to the NPU, resulting in faster inference speed; Task B runs on the cloud GPU, which has sufficient computing power despite slight network transmission overhead; Task C remains in the cloud and its results are aggregated with those of Task B; the system measures a new total latency of 1.8 seconds, and the NPU load returns to normal.
[0063] Step 4: Reward Calculation and Learning The environment calculates rewards based on the results: Big bonus: Because the total latency has been reduced from 2.5 seconds to 1.8 seconds (latency reduction + R1).
[0064] +Medium Reward: Because the time limit requirement of <2 seconds was met (the deadline was met +R2).
[0065] - Minor penalty: Because the data for task B needs to be uploaded to the cloud, some bandwidth was consumed (bandwidth consumption -R7).
[0066] Total reward = +R1 + R2 - R7, which is a positive feedback.
[0067] Step 5: Strategy Update Based on this positive reward, the agent updates its internal policy. Therefore, the agent remembers: "When the NPU is overloaded and the network is still adequate, offloading some computationally intensive tasks (such as B) to the cloud GPU is a good option." The probability of the agent taking the same or similar action the next time it encounters a similar situation increases.
[0068] In another scenario-based decision-making example, when the network suddenly interrupts or bandwidth drops sharply, the information in the reinforcement learning agent is as follows: State S(t): [Network bandwidth ≈ 0].
[0069] Agent decision-making: All tasks must be forcibly scheduled to be executed on edge devices. The action becomes [A->NPU,B->NPU,C->Edge-CPU].
[0070] Result: Although latency may increase (because B needs to queue on the NPU), the process is guaranteed to at least be able to run.
[0071] Reward: Although a -R4 penalty is incurred due to increased latency, the overall decision remains the lesser of two evils because the process was prevented from completely failing (which can be considered a huge negative reward). The agent thus learns to prioritize local self-consistent operation during network failures.
[0072] As can be seen from the above example, the reinforcement learning agent in the resource adaptation engine of this embodiment can explore various resource allocation schemes through the cyclical execution of perception-decision-execution-learning, evaluate the performance of these schemes in the real environment (latency, resource utilization, etc.), and accumulate experience. Finally, it learns an optimal strategy that can automatically and in real time generate an optimal resource deployment scheme when facing complex and dynamically changing heterogeneous computing environments, thereby achieving the best balance between performance, cost and reliability.
[0073] As can be seen from the above, the method of this application embodiment does not require engineers to write complex hard-coded rules, but rather achieves modular development, building block combination and automated deployment of industry AI application processes through a data-driven approach.
[0074] Please see Figure 5 This application also provides an AI process processing apparatus, the apparatus comprising: The first module is used to build a process construction library, which includes a domain knowledge-driven task meta-library, an AI operator library, and a neural network model library. Each task meta-library in the task meta-library has input description information, output description information, and functional description information. Each AI operator in the AI operator library carries performance indicators and hardware dependency labels. Each neural network model in the neural network model library is associated with a preset task meta-library in the task meta-library and carries annotation information corresponding to the preset AI operators in the AI operator library. The first module is used to generate executable AI processes based on the target application scenario and the process building library. The executable AI processes include data flow and control flow. The second module is used to analyze the executable AI process and obtain the computational characteristics and resource requirements of each node. The third module is used to obtain the real-time resource status of the deployment environment; The fourth module is used to adapt resources to the executable AI process based on computing characteristics, resource requirements and real-time resource status, so as to obtain the target deployment plan for the executable AI process. The fifth module is used to deploy executable AI processes according to the target deployment plan, so as to run the executable AI processes in the deployment environment.
[0075] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0076] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.
[0077] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0078] Please see Figure 6 , Figure 6 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 610 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 620 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 620 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 620 and is called and executed by the processor 610 using the methods described in the embodiments of this application. The input / output interface 630 is used to realize information input and output; The communication interface 640 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 650 transmits information between various components of the device (e.g., processor 610, memory 620, input / output interface 630, and communication interface 640); The processor 610, memory 620, input / output interface 630 and communication interface 640 are connected to each other within the device via bus 650.
[0079] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.
[0080] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.
[0081] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.
[0082] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0083] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0084] The AI workflow processing method and related equipment provided in this application have the following beneficial effects: (1) This embodiment transforms complex AI projects into a “building block” process assembly by establishing an asset library of domain task elements, AI models and operators, which greatly improves development efficiency and quality consistency.
[0085] (2) This embodiment enables domain experts who do not have in-depth AI programming knowledge to participate in process construction through a graphical process designer and an automatic mapping engine.
[0086] (3) In this embodiment, by calling an AI operator library optimized for specific hardware, the generated process can fully utilize the hardware computing power and improve execution efficiency.
[0087] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.
[0088] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.
[0089] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0090] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or appropriate combinations thereof.
[0091] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0092] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.
[0093] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0094] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0095] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0096] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0097] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.
Claims
1. A method for processing AI workflows, characterized in that, The method is executed based on a process component library, which includes a domain knowledge-driven task meta-library, an AI operator library, and a neural network model library. The method includes the following steps: Based on the target application scenario, an executable AI process is generated according to the process building library. The executable AI process includes data flow and control flow. The executable AI process is analyzed to obtain the computational characteristics and resource requirements of each node; Obtain the real-time resource status of the deployment environment; Based on the computational characteristic information, the resource requirement information, and the real-time resource status, resource adaptation is performed on the executable AI process to obtain the target deployment scheme of the executable AI process; The executable AI process is deployed according to the target deployment scheme to run the executable AI process in the deployment environment.
2. The method according to claim 1, characterized in that, The step of generating an executable AI process based on the target application scenario and the process library includes: Obtain the task element corresponding to the target application scenario from the domain knowledge-driven task element library as the target task element; Obtain the neural network model corresponding to the target task element from the neural network model library as the target neural network model; After decomposing the target neural network model, the target AI operator is extracted from the AI operator library; Based on the process template and combination logic, the executable AI process is generated according to the target task element, the target neural network model, and the target AI operator.
3. The method according to claim 2, characterized in that, The step of obtaining the neural network model corresponding to the target task element from the neural network model library as the target neural network model includes: Extract the feature vector of the target task element; The target neural network model is obtained by searching and matching the feature vectors in the neural network model library.
4. The method according to claim 2, characterized in that, After decomposing the target neural network model, the process of extracting the target AI operator from the AI operator library includes: Obtain the structure definition file of the target neural network model; Traverse the structural computation graph corresponding to the target neural network model and extract all operator nodes; The operators corresponding to the operator nodes are identified to obtain custom optimized operators; Based on the type name of the custom optimization operator and the target hardware platform, the corresponding target AI operator is extracted from the AI operator library.
5. The method according to claim 1, characterized in that, The step of adapting the executable AI process to resources based on the computational characteristic information, the resource requirement information, and the real-time resource status to obtain the target deployment scheme for the executable AI process includes: Based on the computational characteristic information, the resource requirement information, and the real-time resource status, a reinforcement learning agent performs resource adaptation for each task node in the executable AI process, calculates a reward value based on the environmental feedback information corresponding to each resource adaptation scheme, and determines the target deployment scheme of the executable AI process based on the reward value.
6. The method according to claim 1, characterized in that, The deployment operation of the executable AI process according to the target deployment plan includes: The executable AI process, the corresponding neural network model, AI operators, and dependent environment are containerized and packaged to obtain a containerized image; The containerized image is distributed and scheduled in the edge-cloud cluster according to the target deployment plan.
7. The method according to claim 1, characterized in that, In running the executable AI process in the deployment environment, the method further includes the following steps: Obtain performance monitoring information during the execution process; Adjust the target deployment plan based on the performance monitoring information.
8. A processing device for an AI process, characterized in that, The device executes based on a process component library, which includes a domain knowledge-driven task meta-library, an AI operator library, and a neural network model library. The device includes: The first module is used to generate an executable AI process based on the target application scenario and the process building library, wherein the executable AI process includes a data flow and a control flow; The second module is used to analyze the executable AI process to obtain the computational characteristics and resource requirements of each node. The third module is used to obtain the real-time resource status of the deployment environment; The fourth module is used to perform resource adaptation on the executable AI process based on the computing characteristic information, the resource requirement information and the real-time resource status, so as to obtain the target deployment scheme of the executable AI process; The fifth module is used to deploy the executable AI process according to the target deployment plan, so as to run the executable AI process in the deployment environment.
9. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of claims 1 to 7.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.