Cloud edge collaborative algorithm arrangement and deployment method, system and device and storage medium

Abstract

The invention provides a cloud edge collaboration algorithm arrangement and deployment method, system and device and a storage medium, and relates to the technical field of cloud edge collaboration.The method comprises the steps that algorithm task requirements input by a user are analyzed, and after corresponding task information is obtained, an intermediate representation file is generated; collecting computing power state information of the plurality of edge devices to determine a target edge device used for executing an algorithm link, and generating an operation parameter configuration file according to a hardware architecture of the target edge device and algorithm characteristics of the intermediate representation file; and issuing the intermediate representation file and the operation parameter configuration file to the target edge device, and performing automatic compiling and deployment according to the chip type of the target edge device. By issuing the intermediate representation file and the operation parameter configuration file to the target edge device and automatically executing compiling and deployment, the effects of cross-platform automatic adaptation, rapid deployment and efficient operation are achieved, and the problem that in the prior art, task organization and deployment flexibility is insufficient is solved.

Description

Technical Field

[0001] This application relates to the technical field of cloud-edge collaboration, and in particular to a cloud-edge collaboration algorithm orchestration and deployment method, system, device and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology and edge computing systems, an increasing number of distributed business scenarios are relying on multi-node collaborative processing to complete intelligent identification and data analysis tasks. In applications such as video surveillance, industrial inspection, IoT sensing, and smart cities, there is often a simultaneous need for centralized computing power in the cloud and real-time processing requirements from edge devices. To improve the overall system response speed and reduce the bandwidth pressure caused by large-scale data backhaul, various cloud-edge collaborative processing models are gradually becoming the industry mainstream. However, different applications differ significantly in algorithm combination methods, resource scheduling modes, and deployment processes, leading to a need for more flexible task organization methods and more efficient algorithm execution processes between the cloud and the edge.

[0003] Existing solutions typically train, manage, and uniformly configure algorithm models through a cloud platform. The cloud distributes model files, inference programs, or execution scripts to edge devices according to a pre-defined task flow. Upon receiving the relevant files, the edge devices load the corresponding models and programs according to a fixed algorithm execution order and perform inference processing based on locally available hardware resources. In some solutions, the cloud also periodically obtains the operational status of the edge devices and updates versions or migrates tasks accordingly to ensure algorithm availability and performance. This approach enables unified management of algorithm content, reduces manual configuration workload to some extent, and achieves basic real-time inference capabilities at the edge.

[0004] While unified cloud management and edge execution of fixed task processes can achieve algorithm distribution, model deployment, and inference execution, there are problems such as fixed processes, difficulty in flexible orchestration, and lack of automatic adaptation capabilities for different devices when facing scenarios involving multiple algorithm combinations, cross-hardware platform deployment, or dynamic adjustment of task links. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies in terms of task organization and deployment flexibility, this application provides a cloud-edge collaborative algorithm orchestration and deployment method, system, device, and storage medium.

[0006] On one hand, this invention provides a cloud-edge collaborative algorithm orchestration and deployment method, comprising: parsing user-inputted algorithm task requirements to obtain corresponding task information; providing multiple optional algorithm modules for users to drag and drop and combine, and setting execution order and logical relationships according to the task information, to generate corresponding algorithm links; converting the algorithm links into intermediate representation files; collecting computing power status information of multiple edge devices according to the intermediate representation files; determining the target edge device for executing the algorithm links according to the computing power status information; and generating a running parameter configuration file according to the hardware architecture of the target edge device and the algorithm characteristics of the intermediate representation files; distributing the intermediate representation files and the running parameter configuration files to the target edge device; and automatically compiling and deploying according to the chip type of the target edge device.

[0007] Optionally, the step of parsing the algorithm task requirements input by the user to obtain corresponding task information, and providing multiple optional algorithm modules for the user to drag and drop to combine and set the execution order and logical relationship to generate the corresponding algorithm link includes: receiving the algorithm task requirements input by the user on the cloud management platform, parsing the algorithm task requirements into task information through natural language processing or structured forms; retrieving the corresponding algorithm modules from a preset algorithm library according to the task information, presenting the algorithm modules as graphical nodes in a visual arrangement interface, and arranging the algorithm modules according to a preset canvas area in response to the user's drag and drop operation; establishing the data flow relationship and execution order between the algorithm modules according to the connection lines drawn by the user between the algorithm modules; validating the connection relationship between the algorithm modules, and converting the valid algorithm modules and their connection relationships into an algorithm link.

[0008] Optionally, the step of converting the algorithm link into an intermediate representation file and collecting computing power status information of multiple edge devices based on the intermediate representation file includes: traversing all algorithm module nodes in the algorithm link, extracting the algorithm type, model identifier, input / output data format, and dependencies between nodes for each node; structurally encoding the algorithm type, model identifier, input / output data format, and dependencies between nodes for each node according to a preset intermediate representation specification to generate an intermediate representation file; parsing the intermediate representation file to obtain the computational complexity, model size, and inference accuracy requirements of the algorithm link to construct algorithm characteristics; and sending computing power query commands to multiple edge devices registered on the cloud management platform based on the algorithm characteristics, and receiving computing power status information fed back by the multiple edge devices.

[0009] Optionally, the step of determining the target edge device for executing the algorithm link based on the computing power status information, and generating a running parameter configuration file based on the hardware architecture of the target edge device and the algorithm characteristics of the intermediate representation file, includes: calculating the comprehensive load index of each edge device based on the computing power status information; parsing the intermediate representation file to obtain the computing requirements of the algorithm link; performing a matching degree evaluation between the computing requirements and the computing power peaks of the multiple edge devices to select edge devices whose comprehensive load index is lower than a preset threshold and whose matching degree evaluation meets the execution conditions as a candidate device set; filtering the candidate device set based on network latency, device geographical location, and task priority to obtain the target edge device; identifying the chip type of the target edge device; selecting the corresponding inference engine based on the chip type; determining the inference accuracy type based on the hardware architecture of the target edge device and the model accuracy requirements in the intermediate representation file; calculating the batch size based on the available memory of the target edge device and the throughput requirements of the algorithm link; and generating a running parameter configuration file based on the inference engine, the inference accuracy type, and the batch size.

[0010] Optionally, the step of evaluating the matching degree between the computational demand and the peak computing power of the multiple edge devices to select edge devices whose comprehensive load index is lower than a preset threshold and whose matching degree evaluation meets the execution conditions as a candidate device set includes: parsing the number of floating-point operations of each algorithm module in the algorithm link from the intermediate representation file, summing the number of floating-point operations of all algorithm modules in the algorithm link to obtain the total computational demand of the algorithm link; obtaining the theoretical peak computing power of each edge device from the static configuration information of the multiple edge devices, calculating the actual available computing power of each edge device based on the theoretical peak computing power, and dividing the total computational demand by the actual available computing power of each edge device to obtain the estimated value. Execution time; calculate the matching score for each edge device. When the estimated execution time is less than or equal to the preset target execution time, the matching score is 1. When the estimated execution time is greater than the preset target execution time, the matching score is the quotient of the target execution time divided by the estimated execution time. Select edge devices whose comprehensive load index is lower than a preset load threshold as a first selection set. Select edge devices whose matching score is greater than or equal to the preset matching threshold from the first selection set as a second selection set. Verify whether the available memory of each edge device in the second selection set is greater than or equal to the memory required by the algorithm link marked in the intermediate representation file. Select the edge devices that pass the verification as a candidate device set.

[0011] Optionally, the step of selecting a candidate device set from the first screening set includes: selecting edge devices whose matching score is greater than or equal to a preset matching threshold from the first screening set as a second screening set; verifying whether the available memory of each edge device in the second screening set is greater than or equal to the memory required by the algorithm link marked in the intermediate representation file, and selecting the edge devices that pass the verification as a candidate device set.

[0012] Optionally, the step of distributing the intermediate representation file and the runtime parameter configuration file to the target edge device, and automatically compiling and deploying them according to the chip type of the target edge device, includes: encapsulating the intermediate representation file and the runtime parameter configuration file into a deployment data packet through a microservice interface, and transmitting the deployment data packet to the target edge device through a preset secure communication connection; after the target edge device receives the deployment data packet, verifying the digital signature of the deployment data packet and parsing out the intermediate representation file and the runtime parameter configuration file; calling the corresponding heterogeneous hardware adaptation layer module on the target edge device according to the inference engine in the runtime parameter configuration file, and reading the model information in the intermediate representation file through the heterogeneous hardware adaptation layer module; compiling the model information using the chip type of the target edge device to obtain a compiled file, optimizing the compiled file according to the batch size and inference precision type in the runtime parameter configuration file, and loading the compiled and optimized model file.

[0013] On the other hand, a cloud-edge collaborative algorithm orchestration and deployment system is also provided, comprising: a parsing module, used to parse the algorithm task requirements input by the user, obtain the corresponding task information, and provide multiple optional algorithm modules for the user to drag and drop to combine and set the execution order and logical relationship to generate the corresponding algorithm link; a collection module, used to convert the algorithm link into an intermediate representation file, and collect the computing power status information of multiple edge devices based on the intermediate representation file; a generation module, used to determine the target edge device for executing the algorithm link based on the computing power status information, and generate a running parameter configuration file based on the hardware architecture of the target edge device and the algorithm characteristics of the intermediate representation file; and a deployment module, used to distribute the intermediate representation file and the running parameter configuration file to the target edge device, and automatically compile and deploy according to the chip type of the target edge device.

[0014] On the other hand, an electronic device is also provided, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the cloud-edge collaborative algorithm orchestration and deployment method as described in any of the above.

[0015] On the other hand, a computer-readable storage medium is also provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the cloud-edge collaborative algorithm orchestration and deployment method as described in any of the above.

[0016] This application provides a cloud-edge collaborative algorithm orchestration and deployment method, system, device, and storage medium with the following technical effects: By parsing the user-inputted algorithm task requirements, corresponding task information is obtained. Based on the task information, multiple selectable algorithm modules are provided in a visual interface, allowing users to combine and set the execution order and logical relationships through drag-and-drop, thereby generating the corresponding algorithm chain. Subsequently, the algorithm chain is converted into an intermediate representation file, and computing power status information is collected from multiple edge devices based on the intermediate representation file. After obtaining the computing power status information of multiple edge devices, a target edge device for executing the algorithm chain is selected based on the computing power status information, and a running parameter configuration file is generated by combining the hardware architecture of the target edge device and the algorithm characteristics of the intermediate representation file. Finally, the intermediate representation file and the running parameter configuration file are distributed to the target edge device, and automatic compilation and deployment are performed according to the chip type of the target edge device, enabling the algorithm chain to complete inference execution on the target edge device, overcoming the problem of insufficient flexibility in task organization and deployment in existing technologies. Attached Figure Description

[0017] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the cloud-edge collaborative algorithm orchestration and deployment method provided in this embodiment of the invention; Figure 2 This is a schematic block diagram of the cloud-edge collaborative algorithm orchestration and deployment system provided in this embodiment of the invention; Figure 3 This is a schematic block diagram of the structure of the electronic device provided in the embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0020] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or sub-modules is not necessarily limited to those steps or sub-modules explicitly listed, but may include other steps or sub-modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0021] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0022] This application provides a cloud-edge collaborative algorithm orchestration and deployment method, which can parse, orchestrate, match resources, and automatically deploy user-input algorithm task requirements, enabling different algorithm chains to automatically complete compilation and inference execution on adapted edge devices. The method mainly includes the following steps: Step S1: Parse the algorithm task requirements input by the user to obtain the corresponding task information. Based on the task information, provide multiple optional algorithm modules for the user to drag and drop to combine and set the execution order and logical relationship to generate the corresponding algorithm chain.

[0023] The system receives algorithm task requirements from users on a cloud management platform and parses these requirements into task information, including task type, processing object, output requirements, and constraints. Based on this information, it selects algorithm modules corresponding to the task type from a pre-defined algorithm library and loads them into a visual orchestration interface as graphical nodes. Each module is assigned a node identifier representing its functional attributes. Users drag and drop multiple algorithm modules into the canvas area in the visual orchestration interface and connect the input and output ports of the nodes, thus establishing the data flow relationship and execution order between the algorithm modules. Simultaneously, it performs interface type matching verification on the user-drawn connections to ensure that the output data format of the preceding algorithm module can be correctly parsed by the following algorithm module. After successful verification, it constructs the algorithm link using node identifiers, node types, input / output port definitions, and connection edge information as components.

[0024] When the user inputs the algorithm task requirement as "license plate detection and recognition," the task information will include the task type "composite detection task," the processing object "vehicle license plate region," and the output requirement "license plate text and location." Based on the task information, the algorithm library selects a target detection module (e.g., a detection algorithm based on YOLOv5), an optical character recognition module (e.g., a character recognition algorithm based on CRNN), and a post-processing module, displaying these modules as "detection nodes," "recognition nodes," and "text parsing nodes," respectively. The user drags the "detection node" onto the canvas, places the "recognition node" after it, and specifies the output box coordinates of the "detection node" as the input image region of the "recognition node" through a connection operation. After interface matching and verification, this link is converted into an algorithm link.

[0025] Step S2: Convert the algorithm link into an intermediate representation file, and collect computing power status information of multiple edge devices based on the intermediate representation file.

[0026] The algorithm iterates through each algorithm module node in the algorithm chain, extracting the algorithm type, model identifier, input / output data format, and node dependencies. This information is then structured and encoded according to a predefined intermediate representation specification. The encoded results are used to construct an intermediate representation file. The intermediate representation file is parsed, and the computational complexity, model size, and inference accuracy requirements for each algorithm module are calculated sequentially. Algorithm characteristics are then constructed based on these parsing results. Subsequently, a computing power query command is sent to multiple edge devices registered on the cloud management platform. This command includes algorithm characteristic parameters and task information. Upon receiving the query command, the edge devices return computing power status information, including CPU utilization, GPU utilization, neural network processor utilization, available memory size, video memory capacity, chip type identifier, supported inference framework types, and the number of currently running tasks. This computing power status information is then associated and stored with the edge device's static configuration information.

[0027] For algorithm links containing "detection nodes" and "recognition nodes," the intermediate representation file records the model paths, input size formats (e.g., RGB640×640), output formats (e.g., target bounding box coordinate arrays or character confidence matrices), and dependency edges between the detection model "det_model_v1" and the recognition model "ocr_model_v2." After algorithm characteristic analysis, the computational complexity of the detection node is a multi-scale convolutional structure, and the computational complexity of the recognition node is a sequence decoding structure, with the inference precision requirement set to 16-bit floating-point precision. Subsequently, a computing power query command is sent to multiple edge devices (e.g., A10 devices based on NVIDIA graphics processors, Atlas devices based on Huawei Ascend, etc.), and the CPU utilization rate (e.g., 23%), GPU utilization rate (e.g., 35%), available memory, and other information returned by each device are received, forming a computing power status information set.

[0028] Step S3: Determine the target edge device for executing the algorithm link based on the computing power status information, and generate a running parameter configuration file based on the hardware architecture of the target edge device and the algorithm characteristics of the intermediate representation file.

[0029] The comprehensive load index of each edge device is calculated based on the computing power status information of multiple edge devices. The intermediate representation file is parsed to obtain the computational requirements of the algorithm link. Then, the matching degree between the computational requirements and the peak computing power of multiple edge devices is evaluated. Edge devices with a comprehensive load index below a preset threshold and whose matching degree evaluation meets the execution conditions are selected as a candidate device set. The candidate device set is further filtered based on network latency, device geographic location, and task priority to finally determine the target edge device. The chip type of the target edge device is read, and the corresponding inference engine is selected based on the chip type. The inference accuracy type is determined by combining the model accuracy requirements of the intermediate representation file. The batch size is then calculated based on the available memory of the target edge device and the throughput requirements of the algorithm link, and finally, a runtime parameter configuration file is generated.

[0030] For the algorithm chain that includes both detection and recognition models, the total computational requirement is approximately 7.5 × 10⁻⁶. 9 The computational cost (calculated using the actual model structure) is matched against the peak computing power of each edge device (e.g., 100 × 10¹² operations per second for NVIDIA A10) to obtain the estimated execution time and calculate a matching score. When an NVIDIA A10 device has a comprehensive load index of 0.18, an estimated execution time of 0.003 seconds, a matching score of 1, and sufficient available memory, it is added to the candidate device set. Then, based on network latency (e.g., 8ms), it is identified as the target edge device, and TensorRT is selected as the inference engine. The inference precision type is set to 16-bit floating-point precision, and the batch size is determined to be 8 based on throughput requirements, thereby generating a runtime parameter configuration file.

[0031] Step S4: Distribute the intermediate representation file and runtime parameter configuration file to the target edge device, and automatically compile and deploy them according to the chip type of the target edge device.

[0032] The intermediate representation file and runtime parameter configuration file are encapsulated into a deployment data package via a microservice interface. This deployment data package includes a version identifier and digital signature, and a secure communication connection is established between the cloud management platform and the target edge device. The deployment data package is then transmitted to the target edge device. Upon receiving the deployment data package, the target edge device verifies the digital signature to ensure transmission integrity and source credibility. After successful verification, the intermediate representation file and runtime parameter configuration file are parsed. Based on the inference engine in the runtime parameter configuration file, the corresponding heterogeneous hardware adaptation layer module is invoked. This module reads the model information from the intermediate representation file and compiles the model information using the target edge device's chip type, resulting in a compiled file. The compiled file is then optimized and configured according to the batch size and inference precision type. The algorithm execution engine is initialized, and the optimized model file is loaded, enabling the deployed algorithm pipeline to perform inference on the target edge device.

[0033] When the target edge device is an NVIDIA A10 device, the inference engine field in the deployment data package indicates that TensorRT should be called. After parsing the intermediate representation file, the edge device compiles the detection model and the recognition model into TensorRT engine files respectively. It further optimizes the model according to the batch size of 8 and the inference precision of 16-bit floating-point precision in the configuration file. The running engine is initialized in the GPU memory and the optimized model is loaded, thereby completing the compilation and deployment and starting inference.

[0034] In this example, by parsing the algorithm task requirements, the user-input natural language description or structured configuration can be converted into explicit task information, enabling accurate identification of the required algorithm type, processing objective, and output result. By providing algorithm modules to the user in the form of graphical nodes and allowing the user to freely combine algorithm modules by dragging and dropping, this embodiment implements a visual construction method for algorithm chains, enabling users to organize complex algorithm flows intuitively. After generating the algorithm chains, by converting the algorithm chains into intermediate representation files, this embodiment structures and standardizes the algorithm execution flow, allowing subsequent computing power analysis, device matching, and parameter configuration to be performed based on a unified description format, thereby improving overall scalability and cross-device adaptability.

[0035] Furthermore, by collecting computing power status information from multiple edge devices based on intermediate representation files, this embodiment can comprehensively grasp the real-time operating status of different edge devices, including CPU utilization, GPU utilization, neural network processor utilization, available memory size, video memory capacity, and supported inference framework types. This enables precise device selection under dynamically changing operating conditions. Based on the computing power status information, target edge devices are determined, and a configuration file for operating parameters is generated by combining hardware architecture and algorithm characteristics. This allows the algorithm pipeline to automatically adapt parameters such as inference accuracy, batch size, and inference engine when deployed to different devices, solving the problems of cumbersome manual configuration and cross-device incompatibility in traditional cloud-edge collaborative deployment.

[0036] Finally, by distributing intermediate representation files and runtime parameter configuration files to the target edge device and automatically completing compilation and deployment, this embodiment realizes an automated algorithm deployment process across platforms and architectures. This enables the algorithm chain to be executed on the target device with optimal configuration, improves overall inference efficiency, reduces deployment complexity, and ensures high real-time performance and high stability in a multi-device environment. It overcomes the problems of insufficient flexibility in task organization and deployment in existing technologies.

[0037] In another example, step S1 can be achieved through the following steps: It receives algorithm task requirements input by users on the cloud management platform and parses these requirements into task information, including task type, processing object, and output requirements, through natural language processing or structured forms.

[0038] The system listens to the task creation interface of the cloud management platform and receives algorithm task requirements input by the user. These requirements can originate from text input boxes, structured form fields, or preset task templates. The system then invokes the natural language processing module to perform intent recognition and key field extraction on the text content. Through lexical segmentation, entity recognition, and dependency parsing, it accurately extracts the task type, processing object, and output requirements. If the user inputs via a form, the system directly generates a task information structure based on the field content. The final task information is structured data, containing parameters such as task type, processing object, output requirements, input data format, and constraints.

[0039] When a user submits "identify whether the fruit is rotten and output the category and confidence score" in the input box, the natural language processing module identifies the task type as "image classification task" based on a deep bidirectional encoding model (such as the BERT-BiLSTM-CRF structure commonly used in existing technologies), the processing object as "fruit image region", the output requirement as "rotten state category and confidence score value", and generates task information containing the above fields.

[0040] The corresponding algorithm module is retrieved from the preset algorithm library based on the task information. The algorithm module includes at least one of the following: target detection module, optical character recognition module, image segmentation module, image classification module, key point detection module, and post-processing module. The algorithm module is presented as a graphical node in the visual arrangement interface, and the algorithm module is arranged and placed according to the preset canvas area in response to the user's drag operation.

[0041] Based on the task type field in the task information, algorithm modules matching the task type are retrieved from the algorithm library index table. The algorithm library structure includes information such as module identifier, model path, input format, output format, operator type, and version number. The retrieved algorithm modules are loaded as graphical nodes in the visualization interface. Each graphical node includes a node icon, node name, and port representation. The visualization interface uses a two-dimensional coordinate canvas. Upon receiving a user's drag event, the coordinate parameters of the nodes are updated based on the mouse position information, and the node layout is displayed in real time on the canvas area, ensuring that users can freely organize the algorithm module structure.

[0042] For the task type "image classification task" indicated in the task information, an image classification module (such as the ResNet50-based image classification algorithm, which belongs to the classic convolutional neural network classification architecture in existing technology) is retrieved from the algorithm library. At the same time, an image preprocessing module is matched for the content of the object to be processed and loaded as "classification node" and "preprocessing node". The user drags the "preprocessing node" and "classification node" into the canvas in the visualization interface and places the nodes on the left and right sides of the canvas according to the user's mouse coordinates.

[0043] Based on the connection lines drawn by the user between algorithm modules, the data flow relationship and execution order between algorithm modules are established, with the output of the preceding algorithm module serving as the input of the following algorithm module.

[0044] The system monitors connection drawing events triggered by users in the visual orchestration interface, including the selection of starting and target ports. When a user drags a connection from the output port of one algorithm module to the input port of another, a connection edge is generated according to the port data format definition, and the connection edge is recorded as a data flow relationship. Port binding information, execution order, and dependency chains are maintained for all connection edges to construct a complete algorithm link topology. The system also detects in real time whether the connection direction is from the output of the preceding algorithm module to the input of the following algorithm module, ensuring the correct causal logic of the execution order.

[0045] When a user connects the output port (output format: RGB image tensor) of the "preprocessing node" to the input port (input format: RGB image tensor) of the "classification node", the input and output formats of the two are confirmed to be completely consistent according to the port definition table. Then, the data flow relationship is recorded as "preprocessing node → classification node", and the execution order is set to perform preprocessing first and then classification.

[0046] The connection relationships between algorithm modules are validated to ensure that the data interface types of adjacent algorithm modules match and there is no circular dependency. The validated algorithm modules and their connection relationships are converted into algorithm links containing node identifiers, node types, input and output port definitions, and connection edge information.

[0047] Each user-drawn connection edge is validated individually, including port data type checks, port shape consistency checks, and input / output direction checks. During validation, the data interface types are verified to be consistent according to the algorithm module's input / output format definitions; for example, data dimensions, number of channels, and format types must be completely identical. Directed cycle detection is also performed on the connection topology, using topology sorting to determine if circular dependencies exist. If any connection fails to meet the requirements, the user is prompted to correct it interactively. After all validations pass, the node identifier, node type, input / output port definitions, and connection edge information are packaged into an algorithm link data structure.

[0048] When a user mistakenly connects the output port of the "Classification Node" to the input port of the "Preprocessing Node," topology detection reveals a circular dependency, immediately prompting the message "The algorithm link has a circular dependency; please adjust the connection method." After the user readjusts the connection, the validated link is converted into an algorithm link containing the "Preprocessing Node (Type: Preprocessing Module)," the "Classification Node (Type: Image Classification Module)," and edge information.

[0049] In another example, step S2 can be achieved through the following steps: Traverse all algorithm module nodes in the algorithm chain and extract the algorithm type, model identifier, input and output data format, and dependencies between nodes for each node.

[0050] The algorithm chain structure is traversed topologically in all algorithm module nodes. During the traversal, the model identifier, algorithm type, input port definition, output port definition, and adjacent dependent nodes of each node are read to form a node-level parameter set. The parameter set is stored in a structure list for subsequent generation of intermediate representation files. Furthermore, it is ensured that all nodes in the algorithm chain are arranged in a directed acyclic structure to avoid recursive dependencies that could cause model parsing failures.

[0051] The detection node is classified as an "object detection module," with a model identifier of "det_model_v1," and input / output data formats of "RGB 640×640 tensor" and "boundary coordinate array." The recognition node is classified as an "optical character recognition module," with a model identifier of "ocr_model_v2," and input / output data formats of "image region tensor" and "character sequence probability matrix." The dependency between the two will be recorded as "detection node -- recognition node."

[0052] The algorithm type, model identifier, input and output data format, and dependencies between nodes of each node are structured and encoded according to a preset intermediate representation specification to generate an intermediate representation file containing the algorithm link topology, model parameter paths, and data flow definitions.

[0053] According to the predefined intermediate representation standard, node fields are converted into a unified key-value structure, and the topological relationships of the algorithm links are written into the graph structure description segment. Model identifiers are used to find the corresponding model parameter paths in the model library, and these paths are encoded into the model section of the intermediate representation file. The data flow definition section includes descriptions of the input, output, data format, and data transmission between edges for each node. The final generated intermediate representation file can be stored in JSON, XML, or a dedicated binary format, ensuring parsability and cross-device compatibility.

[0054] For example, the detection node encoding in the intermediate representation file is: The encoded content, "node_id:det_01;type:detection;model_path: / models / det_model_v1.trt;input_format:RGB_640_640;output_format:bbox_array;next:[ocr_01]", is used to assemble a complete intermediate representation file.

[0055] The intermediate representation file is parsed to obtain the computational complexity, model size, and inference accuracy requirements of the algorithm chain, in order to construct the algorithm characteristics.

[0056] The node information in the intermediate representation file is analyzed, and the computational complexity of the model structure corresponding to each model identifier is estimated, including the number of convolution operations, feature map size changes, and parameter statistics. Simultaneously, the model size field is read to determine the model file size. Preset inference precision requirements are read based on the model type; for example, object detection models require 16-bit floating-point precision, while optical character recognition models require 32-bit floating-point precision. The computational complexity, model size, and inference precision requirements are integrated into algorithm characteristics for subsequent computational power matching.

[0057] The computational cost of the convolutional layers in the statistical detection model is 2.8 × 10⁻⁶. 9 The computational cost of the recognition model is 4.7 × 10⁻⁶. 9 The total computational complexity is 7.5 × 10⁻⁶. 9 The model sizes are 85MB and 112MB respectively, and the inference accuracy requirements are "detection node FP16, recognition node FP32", thus constructing the algorithm characteristics.

[0058] Based on algorithm characteristics, the system sends computing power query commands to multiple edge devices registered on the cloud management platform. These commands carry algorithm characteristic parameters and task information. The system also receives computing power status information from multiple edge devices, including CPU utilization, GPU utilization, neural network processor utilization, available memory size, video memory capacity, chip type identifier, supported inference framework types, and the number of currently running tasks. The system associates and stores the computing power status information with the static configuration information of the edge devices, including device model, hardware architecture type, peak computing power, and network bandwidth parameters.

[0059] A computing power query command is constructed for each registered edge device. During the construction process, the computational complexity, model size, and inference accuracy requirements from the algorithm characteristics, along with the task type, processing object, and output requirements from the task information, are loaded into the command data body. Upon receiving the computing power query command, the edge device invokes its local monitoring module to read real-time CPU utilization, GPU utilization, neural network processor utilization, available memory, and video memory capacity, and returns computing power status information. This computing power status information is then combined with static configuration information for subsequent device evaluation.

[0060] For NVIDIA A10 devices, the computing power status information includes a CPU utilization of 25%, a GPU utilization of 40%, available memory of 9GB, video memory of 24GB, a chip type identifier of "NVIDIA-A10", a supported inference framework type of "TensorRT", and 3 currently running tasks. The static configuration information includes the device model "A10-24GB", the hardware architecture type "CUDA", a peak computing power of 100 × 10¹² times per second, and a network bandwidth parameter of 10 Gbps. This information is then stored in a structured, associated manner.

[0061] In another example, step S3 can be achieved through the following steps: The comprehensive load index of each edge device is calculated based on the computing power status information of multiple edge devices. The comprehensive load index is calculated by weighting the CPU utilization, GPU utilization, and the ratio of available memory to required memory. The intermediate representation file is parsed to obtain the computational requirements of the algorithm link.

[0062] The computing power assessment module in the cloud management platform standardizes the CPU utilization, GPU utilization, and the ratio of available memory to required memory for each edge device. These three standardized values ​​are then weighted and summed according to preset weight parameters. The weighted sum is used as the comprehensive load index for the corresponding edge device. The comprehensive load index is represented by a value between 0 and 1, with values ​​close to 1 indicating high resource consumption. During the calculation of the comprehensive load index, CPU utilization, GPU utilization, and the ratio of available memory are obtained by reading fields from the computing power status information. When parsing the intermediate representation file to obtain the computational requirements of the algorithm chain, the node entries in the intermediate representation file are traversed. For each node's corresponding algorithm model entry, the computational quantity field is read. The computational quantity field is expressed in floating-point operations. After traversal, the computational quantity fields of all nodes are summed to obtain the computational requirements of the algorithm chain.

[0063] If the CPU utilization of a device is read as 0.55, the GPU utilization as 0.40, the available memory is 4GB, and the algorithm link requires 2GB of memory, then the available memory ratio is 4 divided by 2, which equals 2.0. The CPU utilization, GPU utilization, and available memory ratio are standardized and denoted as 0.55, 0.40, and 1.0 respectively. These are then weighted using parameters of 0.4, 0.4, and 0.2 to obtain a comprehensive load index: 0.55 multiplied by 0.4, 0.40 multiplied by 0.4, and 1.0 multiplied by 0.2, resulting in a total load index of 0.58. When parsing the intermediate representation file, if it contains three nodes—an object detection module, an image segmentation module, and a post-processing module—with computational requirements of 2GFLOPs, 3GFLOPs, and 1GFLOPs respectively, then the total computational requirement of the algorithm link is 6GFLOPs.

[0064] The computational load requirement is matched with the peak computing power of multiple edge devices to select edge devices whose comprehensive load index is lower than a preset threshold and whose matching degree meets the execution conditions as a set of candidate devices.

[0065] The theoretical execution time of each edge device under the current computing demand is calculated by dividing the peak computing power of multiple edge devices by the computing demand. The theoretical execution time is then adjusted according to the comprehensive load index by dividing the theoretical execution time by one and subtracting the comprehensive load index to obtain the adjusted estimated execution time. The adjusted estimated execution time is then compared with the preset maximum acceptable execution time. When the comprehensive load index is less than the preset threshold and the adjusted estimated execution time is less than or equal to the maximum acceptable execution time, the edge device is added to the candidate device set.

[0066] If the computational requirement of the algorithm link is 6 GFLOPs, and the peak computing power of an edge device is 12 GFLOPs / s, then the theoretical execution time is 6 divided by 12, which equals 0.5 seconds. The overall load index of the device is 0.58, so the corrected estimated execution time is 0.5 divided by (1 minus 0.58), which equals 1.19 seconds. If the maximum acceptable execution time is set to 1.2 seconds and the overall load index threshold is set to 0.7, then since the overall load index of the edge device is 0.58, which is less than 0.7, and the corrected estimated execution time is less than or equal to 1.2 seconds, the device is added to the candidate device set.

[0067] The candidate device set is filtered based on network latency, device geographic location, and task priority to obtain the target edge device. The chip type of the target edge device is identified, and the corresponding inference engine is selected based on the chip type.

[0068] The communication monitoring module of the cloud management platform collects network round-trip latency data from each device in the candidate device set to the cloud, and records the spatial coordinates of the device location and the area to which the task belongs. By calculating the Euclidean distance between the device location and the task location, a communication score is formed in combination with the network round-trip latency. Based on the task priority field, the candidate device set is reordered from high to low priority. Based on the sorting results, the device with the highest communication score and the smallest geographical distance is selected as the target edge device. Then, the chip type identifier field of the target edge device is read, and the corresponding inference engine mapping table is looked up according to the chip type field to select an inference engine that matches the chip type.

[0069] When the candidate device set includes devices A, B, and C, device A has a network round-trip latency of 20 milliseconds and a distance of 3 kilometers, device B has 30 milliseconds and a distance of 1 kilometer, and device C has 15 milliseconds and a distance of 10 kilometers. Given a high task priority, the communication score is weighted by network latency and distance, with weights of 0.7 and 0.3 respectively. Device A's communication score is 20 multiplied by 0.7 plus 3 multiplied by 0.3, equaling 15.9; device B's score is 30 multiplied by 0.7 plus 1 multiplied by 0.3, equaling 21.3; and device C's score is 15 multiplied by 0.7 plus 10 multiplied by 0.3, equaling 15.5. Therefore, device C is selected as the target edge device. If device C's chip type is identified as "HUAWEI_ASCEND", the Mind Spore inference engine is selected based on the mapping table.

[0070] The inference precision type is automatically determined based on the hardware architecture of the target edge device and the model precision requirements in the intermediate representation file. The inference precision types include 32-bit floating-point precision, 16-bit floating-point precision, and 8-bit integer precision. The batch size is calculated based on the available memory of the target edge device and the throughput requirements of the algorithm link.

[0071] By parsing the hardware architecture field of the target edge device, we obtain its supported integer precision, half-precision floating-point, and full-precision floating-point operation capabilities. Combining this with the model precision requirement field in the intermediate representation file, we compare the model precision requirement with the hardware's supported precision range. If the model precision requirement allows for lower precision, we select the lowest available precision type within the device's supported range to improve inference speed. Subsequently, we read the remaining usable memory capacity based on the target edge device's available memory field, and calculate the maximum batch size using integer division based on the single inference memory usage and throughput requirements of the algorithm chain. We then set a lower limit of 1 for the batch size.

[0072] If the model precision requirement field is marked as "Supports FP32 / FP16 / INT8", and the device's hardware architecture indicates that it supports INT8 operations, then the inference precision type is determined to be 8-bit integer precision. If the device's available memory is 6GB and the single inference memory usage of the algorithm link is 500MB, then the batch size is 6GB divided by 500MB, which equals 12. Combining this with the minimum batch size set in the throughput requirement field to be 8, the final batch size is 12.

[0073] Generate a runtime parameter configuration file based on the inference engine, inference precision type, and batch size.

[0074] The parameter generation module of the cloud management platform writes the inference engine field, inference precision field, batch size field, and model path field in the intermediate representation file into the configuration file according to the preset parameter template format. The configuration file structure adopts the key-value pair form, including engine type key, precision type key, batch size key, device identifier key, and model file path key. After generation, the field integrity is checked to ensure that all necessary configuration keys have been written correctly.

[0075] If the inference engine is TensorRT, the inference precision is INT8, the batch size is 12, and the model file path is " / model / ir / model.bin", then write the fields "engine=TensorRT", "precision=INT8", "batch=12", and "model_path= / model / ir / model.bin" into the generated runtime parameter configuration file; the field integrity check will verify the existence of the four fields engine, precision, batch, and model_path in turn.

[0076] Furthermore, the step of evaluating the matching degree between the computational demand and the peak computing power of multiple edge devices to select edge devices with a comprehensive load index lower than a preset threshold and whose matching degree evaluation meets the execution conditions as a candidate device set can be preferably: The number of floating-point operations for each algorithm module in the algorithm chain is parsed from the intermediate representation file. The number of floating-point operations for all algorithm modules in the algorithm chain is summed to obtain the total computational requirement of the algorithm chain.

[0077] By traversing the node list in the intermediate representation file, the computational cost field of each node is read. This computational cost field records the number of floating-point operations required for the algorithm model of that node to perform one inference. The total computational cost requirement of the algorithm chain is obtained by adding the computational cost fields of all nodes. The node order is kept consistent during the traversal process to ensure the accuracy of the computational cost statistics.

[0078] If the computational cost of the target detection module in the intermediate representation file is 4 GFLOPs, the computational cost of the image classification module is 2 GFLOPs, and the computational cost of the post-processing module is 0.5 GFLOPs, then the total computational cost requirement of the algorithm link is 6.5 GFLOPs.

[0079] The theoretical peak computing power of each edge device is obtained from the static configuration information of multiple edge devices. The theoretical peak computing power is expressed in terms of floating-point operations per second. The actual available computing power of each edge device is calculated based on the theoretical peak computing power. The actual available computing power is equal to the difference between the theoretical peak computing power multiplied by one and the comprehensive load index. The total computing demand is divided by the actual available computing power of each edge device to obtain the estimated execution time.

[0080] By consulting the peak computing power field in the static configuration information of edge devices, the theoretical peak computing power of each device, expressed in GFLOPs / s, is read. Then, combined with the comprehensive load index field, the actual available computing power is calculated by multiplying the theoretical peak computing power by one and subtracting the comprehensive load index. Subsequently, the estimated execution time is obtained by dividing the total computing demand by the actual available computing power, and the estimated execution time is used as the basis for subsequent matching degree calculation.

[0081] If the theoretical peak computing power of device A is 20 GFLOPs / s and the comprehensive load index is 0.4, then the actual available computing power is 20 multiplied by (1 minus 0.4) to get 12 GFLOPs / s; the total computing power requirement of the algorithm link is 6.5 GFLOPs, then the estimated execution time is 6.5 divided by 12 to get 0.54 seconds.

[0082] A matching score is calculated for each edge device. The matching score is based on the ratio of the estimated execution time to the preset target execution time. When the estimated execution time is less than or equal to the target execution time, the matching score is 1. When the estimated execution time is greater than the target execution time, the matching score is the quotient of the target execution time divided by the estimated execution time.

[0083] The matching score is calculated by comparing the estimated execution time with the target execution time. If the estimated execution time is less than or equal to the target execution time, the matching score is set to 1. If the estimated execution time is greater than the target execution time, the matching score is calculated by dividing the target execution time by the estimated execution time. The matching score is represented between 0 and 1.

[0084] If the estimated execution time is 0.54 seconds and the target execution time is 0.5 seconds, the matching score is 0.93, which is 0.5 divided by 0.54, since the estimated execution time is greater than the target execution time. If the estimated execution time is 0.48 seconds, the matching score is 1.

[0085] Edge devices with a comprehensive load index lower than a preset load threshold are selected as the first selection set. Edge devices with a matching score greater than or equal to a preset matching threshold are selected from the first selection set as the second selection set.

[0086] By comparing the comprehensive load index with a preset load threshold, devices with a comprehensive load index lower than the preset load threshold are selected as the first selection set. Then, in the first selection set, the matching score of each device is compared with a preset matching score threshold, and devices with a matching score greater than or equal to the matching score threshold are selected to form the second selection set.

[0087] If the load threshold is set to 0.7 and the matching score threshold is set to 0.85, and if device A has a comprehensive load index of 0.4 and a matching score of 0.93, then device A meets both requirements and is included in the second screening set; if device B has a comprehensive load index of 0.6 but a matching score of 0.80, then device B only meets the load requirement and is included in the first set but not in the second set.

[0088] Verify whether the available memory of each edge device in the second screening set is greater than or equal to the memory required by the algorithm link as marked in the intermediate representation file, and select the edge devices that pass the verification as the candidate device set.

[0089] By reading the available memory field of each device in the second filter set and the required memory field in the intermediate representation file, the available memory field is compared with the required memory field; when the available memory field is greater than or equal to the required memory field, the device is added to the candidate device set; a one-time verification is performed on all devices to ensure no omissions.

[0090] If the algorithm link requires 3GB of memory and device A has 4.5GB of available memory, then device A passes the verification; if device B has only 2.8GB of available memory, then device B fails the memory verification.

[0091] Furthermore, the step of identifying the chip type of the target edge device and selecting the corresponding inference engine based on the chip type can preferably be: Identify the chip type of the target edge device, wherein the chip type includes any one of NVIDIA graphics processor, Intel central processing unit, Huawei Ascend neural network processor or Lingxi neuromorphic chip.

[0092] When the chip type is an NVIDIA graphics processor, select the TensorRT inference engine.

[0093] When the chip type is an Intel CPU, select the OpenVINO inference engine.

[0094] When the chip type is Huawei Ascend Neural Processor, select the Mind Spore inference engine.

[0095] When the chip type is the Lynxi neuromorphic chip, select the Lynxi inference engine.

[0096] By reading the chip type identifier field of the target edge device, the identifier field is matched one by one with the preset chip-engine mapping table; when the chip type is "GPU_NVIDIA", it is mapped to TensorRT; when the chip type is "CPU_INTEL", it is mapped to OpenVINO; when the chip type is "ASCEND_NPU", it is mapped to MindSpore; and when the chip type is "LYNXI_NEURO", it is mapped to Lynxi; after the selection is completed, the engine field is written to the parameter structure.

[0097] When the device's chip type field is read as "GPU_NVIDIA_XAVIER", the inference engine is TensorRT; when the chip type field is "CPU_INTEL_XEON", the inference engine is OpenVINO; when the field is "ASCEND_310B", the inference engine is MindSpore; and when the field is "LYNXI_9000", the inference engine is Lynxi.

[0098] In another example, step S4 can be achieved through the following steps: The intermediate representation file and runtime parameter configuration file are encapsulated into a deployment data package through the microservice interface. The deployment data package contains a version identifier and a digital signature. A secure communication connection is established between the cloud management platform and the target edge device. The deployment data package is transmitted to the target edge device through the preset secure communication connection.

[0099] The deployment encapsulation module in the cloud management platform writes the intermediate representation file and the runtime parameter configuration file as two independent file entries into the same deployment data packet structure. During the encapsulation process, a unique version identifier field is generated for the deployment data packet, and the deployment data packet is digitally signed using the private key module built into the cloud management platform. The digital signature is generated based on the SHA-256 hash algorithm and the RSA private key encryption process. The generated deployment data packet is written to the data stream buffer. Then, the secure communication management module is called to establish a TLS 1.3 encrypted channel with the target edge device through a preset secure communication connection. This encrypted channel requires certificate chain verification and key negotiation during establishment. After the channel is established, the deployment data packet is sent to the target edge device in streaming mode through the encrypted channel. During the transmission, the fragment number is recorded and integrity verification is performed to prevent data corruption.

[0100] If the intermediate representation file is named "ir_graph.json" and the runtime parameter configuration file is named "config.ini", the deployment and packaging module writes the two file entries into a data packet named "package_v102.tar" and generates the version identifier "v1.0.2". The deployment and packaging module uses the built-in RSA-2048 private key to generate a digital signature field "signature.bin" for the entire data packet content. The secure communication module establishes a TLS1.3 encrypted channel by verifying the device certificate sent by the target edge device. The data packet is divided into 128 data fragments, each containing 4KB of data content, and sent one by one. The edge device compares the checksum of each fragment at the receiving end to ensure transmission integrity.

[0101] After the target edge device receives the deployment data packet, it verifies the digital signature of the deployment data packet and parses out the intermediate representation file and the runtime parameter configuration file.

[0102] The system reads the digital signature field from the deployment data packet using the built-in verification module of the target edge device and extracts the main content of the data packet. It then verifies the digital signature using the public key module corresponding to the cloud management platform. The verification method involves generating a SHA-256 hash value from the main content of the data packet and matching it byte-by-byte with the signature decryption result. Once the digital signature verification is successful, the system calls the data packet parsing module to unpack the data packet structure, parsing the intermediate representation file and the runtime parameter configuration file from the data packet and writing them to the " / opt / ir / " and " / opt / config / " directories, respectively. Simultaneously, a parsing log is generated to record the parsing time and result of each file entry.

[0103] If the edge device reads the digital signature "signature.bin" in the deployment data packet, it decrypts the signature using the locally stored RSA-2048 public key to obtain the hash value "3f5c…a91b"; then it performs SHA-256 operation on the data packet body to generate the hash value "3f5c…a91b". Since the two are completely consistent, the verification passes; the parsing module writes "ir_graph.json" to " / opt / ir / ir_graph.json" and "config.ini" to " / opt / config / config.ini", and records the parsing completion time "11ms" in the parsing log.

[0104] The inference engine in the runtime parameter configuration file calls the corresponding heterogeneous hardware adaptation layer module on the target edge device, and reads the model information in the intermediate representation file through the heterogeneous hardware adaptation layer module.

[0105] By reading the inference engine fields in the runtime parameter configuration file, the heterogeneous hardware adaptation layer modules that need to be invoked on the target edge device are determined. These adaptation layer modules include NVIDIA, Intel, Ascend, and Lynxi adaptation layers. After the device starts the corresponding adaptation layer module, it reads the model structure information, operator list information, and model weight path information from the intermediate representation file through the file reading interface provided by the adaptation layer. During the reading process, data type validation is performed on each field, such as verifying whether the model structure field contains a node sequence and whether the input dimension field is a positive integer. After successful validation, a model loading structure is constructed to prepare the necessary structured information for subsequent model compilation.

[0106] When the inference engine field in the runtime parameter configuration file is "TensorRT", the edge device automatically calls the NVIDIA adaptation layer module. This adaptation layer obtains the operator sequence, including convolution operators, pooling operators, and fully connected operators, by reading the "nodes" field in " / opt / ir / ir_graph.json". It obtains the model weight path " / opt / ir / weights.bin" by reading the "weights_path" field. After verifying that the input dimension is "[1,3,224,224]", the adaptation layer writes this information into the model loading structure "model_info_struct".

[0107] The model information is compiled using the chip type of the target edge device to obtain the compiled file. The compiled file is then optimized by configuring the batch size and inference accuracy type in the runtime parameter configuration file. The algorithm runtime engine is then initialized and the compiled and optimized model file is loaded.

[0108] By querying the chip type field of the target edge device, the corresponding model compiler is invoked to execute the compilation process. When the chip type is a graphics processor, the TensorRT compiler is invoked; when it is a central processing unit, the OpenVINO compiler is invoked; when it is an Ascend neural network processor, the Mind Spore Lite compiler is invoked; and when it is a neuromorphic chip, the Lynxi compiler is invoked. The compiler generates a device-executable model file based on the intermediate representation file and the model weight file. Then, the batch size field and the inference precision field in the runtime parameter configuration file are read. The batch size field controls the input batch size of the compiled model, and the inference precision field controls the quantization configuration of the model. After the optimization configuration is completed, the algorithm runtime engine initialization function is invoked to load the generated compiled model file and allocate contiguous memory space for the input buffer and output buffer to ensure that the model can be executed directly on the hardware.

[0109] When the chip type field of the target edge device is "GPU_NVIDIA_XAVIER", the device calls the TensorRT compiler. After parsing the model structure, the compiler generates a "model.plan" file. The batch size field in the runtime parameter configuration file is "8" and the inference precision field is "INT8". Based on this information, the compiler converts all convolution operators and matrix multiplication operators into 8-bit integer operation forms. After optimization, the algorithm runtime engine loads "model.plan" through the interface "init_engine()" and allocates an input buffer "input_buffer" and an output buffer "output_buffer" in the device's video memory. Each buffer is 16MB in size, and the model loading process is finally completed.

[0110] like Figure 2 As shown, this application also provides a cloud-edge collaborative algorithm orchestration and deployment system 10, which mainly includes the following modules: The parsing module 11 is mainly used to parse the algorithm task requirements input by the user, obtain the corresponding task information, and provide multiple optional algorithm modules for the user to drag and drop to combine and set the execution order and logical relationship to generate the corresponding algorithm chain.

[0111] The acquisition module 12 is mainly used to convert the algorithm link into an intermediate representation file and collect computing power status information of multiple edge devices based on the intermediate representation file.

[0112] The generation module 13 is mainly used to determine the target edge device for executing the algorithm link based on the computing power status information, and to generate the running parameter configuration file based on the hardware architecture of the target edge device and the algorithm characteristics of the intermediate representation file.

[0113] The deployment module 14 is mainly used to distribute intermediate representation files and runtime parameter configuration files to the target edge devices, and to automatically compile and deploy them according to the chip type of the target edge devices.

[0114] In this embodiment, by using the parsing module 11, the acquisition module 12, the generation module 13, and the deployment module 14 as the core functional units of the system, the system can completely construct, analyze, and deploy the algorithm task process in the cloud environment, improve the system's ability to manage complex algorithm tasks, make the cloud-edge collaborative algorithm orchestration and deployment operations clearer, more efficient, and more reliable, help reduce the difficulty of manual configuration, improve the resource utilization of edge devices, and ensure the consistency and scalability of the algorithm link in a multi-device environment.

[0115] This application also provides an electronic device 20, which includes a processor 22 and a memory 24. The memory 24 stores at least one instruction or at least one program. The at least one instruction or at least one program is loaded and executed by the processor 22 to implement the cloud-edge collaborative algorithm orchestration and deployment method as described above.

[0116] In one alternative embodiment, an electronic device 20 is provided, such as Figure 3 As shown, Figure 3 The illustrated electronic device includes a processor 22 and a memory 24. The processor 22 and the memory 24 are connected, for example, via a bus 21. Optionally, the electronic device 20 may further include a transceiver 23, which can be used for data interaction between the electronic device 20 and other electronic devices 20, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 23 is not limited to one unit, and the structure of the electronic device 20 does not constitute a limitation on the embodiments of this application.

[0117] In this embodiment, the processor 22 and memory 24 in the electronic device 20 work together at the hardware level to achieve the complete execution of the above-mentioned cloud-edge collaborative algorithm orchestration and deployment method. The processor 22, as the core computing unit of the device, executes the instructions or programs stored in the memory 24 to perform processes such as parsing algorithm task requirements, constructing algorithm links, generating intermediate representation files, collecting edge device computing power status information, generating running parameter configuration files, and deploying models. The instructions or programs pre-stored in the memory 24 exist in the form of structured code, containing calling logic corresponding to multiple functional modules. Furthermore, the instructions or programs integrate a visual orchestration interface calling module, an intermediate representation file generation module 13, a computing power query module, a device filtering module, and an automatic compilation module, ensuring that the processor 22 can complete multi-step processing of the logical links during invocation. During the operation of electronic device 20, processor 22 first loads a program for task parsing at the application layer and parses the user-inputted algorithm task requirements into task information. Then, it calls the algorithm module retrieval routine in memory 24, retrieves the corresponding algorithm module from the algorithm library based on the task information, and constructs the algorithm link. Next, processor 22 executes an intermediate representation generation program, uniformly encoding the structure, model identifier, and data flow of the algorithm link into an intermediate representation file, and calls a computing power acquisition program to broadcast computing power query instructions to multiple edge devices. After receiving computing power status information from multiple edge devices, processor 22 executes a device filtering program, automatically determining the target edge device based on operating parameters and device status, and generating an operating parameter configuration file. Finally, it calls a deployment program to send deployment data packets to the target edge device via secure communication and performs automatic compilation and deployment. Through the collaboration of the above hardware and software, electronic device 20 in this embodiment can completely implement all the technical processes described in the cloud-edge collaborative algorithm orchestration and deployment method, ensuring that the software logic runs stably and reliably on the hardware platform.

[0118] This application also provides a computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the cloud-edge collaborative algorithm orchestration and deployment method as described above.

[0119] In this embodiment, by providing complete and executable software logic in the storage medium, the cloud-edge collaborative algorithm orchestration and deployment method can be reused across platforms and hardware, significantly improving the portability and scalability of the algorithm deployment system. Simultaneously, since the program is organized sequentially in the storage medium, the processor can directly call the corresponding functional modules during execution, automating the algorithm task from parsing to deployment. This avoids errors caused by complex manual configuration and improves the stability and consistency of algorithm deployment.

[0120] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0121] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device, equipment, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0122] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing the relevant hardware to implement them. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0123] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A cloud-edge collaborative algorithm orchestration and deployment method, characterized in that, include: The algorithm task requirements input by the user are parsed to obtain the corresponding task information. Based on the task information, multiple optional algorithm modules are provided for the user to drag and drop to combine and set the execution order and logical relationship to generate the corresponding algorithm chain. The algorithm link is converted into an intermediate representation file, and the computing power status information of multiple edge devices is collected based on the intermediate representation file. Based on the computing power status information, the target edge device for executing the algorithm link is determined, and a running parameter configuration file is generated based on the hardware architecture of the target edge device and the algorithm characteristics of the intermediate representation file; The intermediate representation file and the runtime parameter configuration file are sent to the target edge device, and automatically compiled and deployed according to the chip type of the target edge device.

2. The cloud-edge collaborative algorithm orchestration and deployment method according to claim 1, characterized in that, The steps of parsing the user-inputted algorithm task requirements to obtain corresponding task information, and providing multiple optional algorithm modules for the user to drag and drop, combine, and set the execution order and logical relationship to generate the corresponding algorithm chain include: Receive algorithm task requirements input by users on the cloud management platform, and parse the algorithm task requirements into task information through natural language processing or structured forms; Based on the task information, the corresponding algorithm module is retrieved from the preset algorithm library, and the algorithm module is presented as a graphical node in the visual arrangement interface. The algorithm module is also arranged and placed according to the preset canvas area in response to the user's drag operation. Based on the user's operation of drawing connection lines between the algorithm modules, establish the data flow relationship and execution order between the algorithm modules; The connection relationships between the algorithm modules are validated, and the validated algorithm modules and their connection relationships are converted into algorithm links.

3. The cloud-edge collaborative algorithm orchestration and deployment method according to claim 1, characterized in that, The step of converting the algorithm link into an intermediate representation file and collecting computing power status information of multiple edge devices based on the intermediate representation file includes: Traverse all algorithm module nodes in the algorithm chain and extract the algorithm type, model identifier, input and output data format, and dependencies between nodes for each node; The algorithm type, model identifier, input and output data format, and dependencies between nodes of each node are structured and encoded according to a preset intermediate representation specification to generate an intermediate representation file. The intermediate representation file is parsed to obtain the computational complexity, model size, and inference accuracy requirements of the algorithm chain, so as to construct the algorithm characteristics; Based on the characteristics of the algorithm, the system sends computing power query commands to multiple edge devices registered on the cloud management platform and receives computing power status information from the multiple edge devices.

4. The cloud-edge collaborative algorithm orchestration and deployment method according to claim 1, characterized in that, The step of determining the target edge device for executing the algorithm link based on the computing power status information, and generating a running parameter configuration file based on the hardware architecture of the target edge device and the algorithm characteristics of the intermediate representation file, includes: Calculate the comprehensive load index of each edge device based on the computing power status information, and parse the intermediate representation file to obtain the computing requirements of the algorithm link; The computational requirements are matched with the peak computing power of the multiple edge devices to select edge devices whose comprehensive load index is lower than a preset threshold and whose matching degree evaluation meets the execution conditions as a set of candidate devices. The candidate device set is filtered based on network latency, device geographic location, and task priority to obtain target edge devices. The chip type of the target edge devices is identified, and the corresponding inference engine is selected based on the chip type. The inference accuracy type is determined based on the hardware architecture of the target edge device and the model accuracy requirements in the intermediate representation file. The batch size is calculated based on the available memory of the target edge device and the throughput requirements of the algorithm link; A runtime parameter configuration file is generated based on the inference engine, the inference precision type, and the batch size.

5. The cloud-edge collaborative algorithm orchestration and deployment method according to claim 4, characterized in that, The step of evaluating the matching degree between the computational demand and the peak computing power of the multiple edge devices to select edge devices whose comprehensive load index is lower than a preset threshold and whose matching degree evaluation meets the execution conditions as a candidate device set includes: The number of floating-point operations for each algorithm module in the algorithm chain is parsed from the intermediate representation file. The number of floating-point operations for all algorithm modules in the algorithm chain is summed to obtain the total computational requirement of the algorithm chain. The theoretical peak computing power of each edge device is obtained from the static configuration information of the multiple edge devices, and the actual available computing power of each edge device is calculated based on the theoretical peak computing power. The total computing power requirement is divided by the actual available computing power of each edge device to obtain the estimated execution time. Calculate the matching score for each edge device. When the estimated execution time is less than or equal to the preset target execution time, the matching score is 1. When the estimated execution time is greater than the preset target execution time, the matching score is the quotient of the target execution time divided by the estimated execution time; Edge devices whose comprehensive load index is lower than a preset load threshold are selected as the first set of filters, and a set of candidate devices is selected from the first set of filters.

6. The cloud-edge collaborative algorithm orchestration and deployment method according to claim 5, characterized in that, The step of filtering the candidate device set from the first filtering set includes: The edge devices whose matching score is greater than or equal to a preset matching threshold are selected from the first filtering set as the second filtering set; Verify whether the available memory of each edge device in the second filtering set is greater than or equal to the memory required by the algorithm link marked in the intermediate representation file, and select the edge devices that pass the verification as the candidate device set.

7. The cloud-edge collaborative algorithm orchestration and deployment method according to claim 1, characterized in that, The steps of distributing the intermediate representation file and the runtime parameter configuration file to the target edge device, and automatically compiling and deploying them according to the chip type of the target edge device, include: The intermediate representation file and the runtime parameter configuration file are encapsulated into a deployment data package through a microservice interface, and the deployment data package is transmitted to the target edge device through a preset secure communication connection; After the target edge device receives the deployment data packet, it verifies the digital signature of the deployment data packet and parses out the intermediate representation file and the running parameter configuration file; The inference engine in the running parameter configuration file calls the corresponding heterogeneous hardware adaptation layer module on the target edge device, and reads the model information in the intermediate representation file through the heterogeneous hardware adaptation layer module; The model information is compiled using the chip type of the target edge device to obtain a compiled file. The compiled file is then optimized using the batch size and inference accuracy type in the runtime parameter configuration file, and the optimized model file is loaded.

8. A cloud-edge collaborative algorithm orchestration and deployment system, characterized in that, include: The parsing module is used to parse the algorithm task requirements input by the user, obtain the corresponding task information, and provide multiple optional algorithm modules for the user to drag and drop to combine and set the execution order and logical relationship to generate the corresponding algorithm chain. The acquisition module is used to convert the algorithm link into an intermediate representation file and acquire computing power status information of multiple edge devices based on the intermediate representation file. The generation module is used to determine the target edge device for executing the algorithm link based on the computing power status information, and to generate a running parameter configuration file based on the hardware architecture of the target edge device and the algorithm characteristics of the intermediate representation file. The deployment module is used to distribute the intermediate representation file and the running parameter configuration file to the target edge device, and to automatically compile and deploy them according to the chip type of the target edge device.

9. An electronic device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the cloud-edge collaborative algorithm orchestration and deployment method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the cloud-edge collaborative algorithm orchestration and deployment method as described in any one of claims 1 to 8.

Images