Algorithm pipeline arrangement method and device, electronic equipment and storage medium

Abstract

The application provides an algorithm pipeline arrangement method and device, electronic equipment and a storage medium. The technical solution is as follows: a task description file is obtained, the task description file includes a plurality of module description information; based on the plurality of module description information, a plurality of general modules in a module library are called for processing to obtain a plurality of application modules; and based on the plurality of application modules, an algorithm pipeline corresponding to the task description file is obtained. According to the embodiment of the application, the workload of algorithm development can be reduced, and the speed of algorithm deployment can be accelerated.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more particularly to an algorithm pipeline orchestration method, apparatus, electronic device, and storage medium. Background Technology

[0002] AI (Artificial Intelligence) technology based on DP (Deep Learning) algorithms has developed rapidly in recent years. Utilizing AI to empower various IoT (Internet of Things) scenarios has also become a popular application. From smart home products related to individuals to applications related to security and production, such as vehicle traffic sensing, construction site safety helmet detection, and life jacket detection, AI is becoming an important means of liberating manpower and improving productivity.

[0003] In IoT scenarios, after the algorithm model is trained, it needs to be deployed and run on various hardware platforms with different computing power according to application requirements. Algorithm deployment requires organizing and connecting different data processing modules into an algorithm pipeline. In related technologies, algorithm pipelines for different types of algorithm tasks or algorithm tasks running on different hardware platforms are generally obtained through manual hard coding, which presents a huge workload problem. Summary of the Invention

[0004] This application provides an algorithm pipeline orchestration method, apparatus, electronic device, and storage medium to solve the problems existing in related technologies. The technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide an algorithm pipeline orchestration method, including:

[0006] Obtain the task description file; the task description file includes description information for multiple modules;

[0007] Based on the description information of multiple modules, multiple general modules are called in the module library for processing to obtain multiple application modules;

[0008] Based on multiple application modules, an algorithm pipeline corresponding to the task description file is obtained.

[0009] Secondly, embodiments of this application provide an algorithm pipeline orchestration apparatus, wherein the apparatus includes:

[0010] The application layer is used to obtain the task description file; the task description file includes description information for multiple modules.

[0011] The framework layer is used to call multiple general modules in the module library for processing based on multiple module description information, to obtain multiple application modules, and based on multiple application modules, to obtain the algorithm pipeline corresponding to the task description file.

[0012] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements the method provided in any embodiment of this application when executing the computer program.

[0013] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method provided in any embodiment of this application.

[0014] The technical solution of this application embodiment can pre-configure multiple general modules in a module library. By writing a task description file containing description information of multiple modules, the general modules can be called to process and obtain application modules suitable for specific algorithm tasks, thus obtaining an algorithm pipeline. In this way, for tasks of different types or on different hardware platforms, as long as different task description files are written, the above method can be reused to obtain an algorithm pipeline adapted to the task. That is, by using a modular approach, low-code development can be achieved, reducing development workload and accelerating the speed of algorithm deployment.

[0015] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of this application will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0016] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0017] Figure 1 This is a schematic diagram illustrating an exemplary application scenario of an embodiment of this application;

[0018] Figure 2 A flowchart of an algorithm pipeline orchestration method provided in an embodiment of this application;

[0019] Figure 3 This is a schematic diagram of the algorithm pipeline orchestration framework in one embodiment of this application;

[0020] Figure 4 This is a visual rendering of the algorithm pipeline in one embodiment of this application;

[0021] Figure 5 A structural block diagram of an algorithm pipeline orchestration apparatus provided in an embodiment of this application;

[0022] Figure 6 This is a structural block diagram of an electronic device used to implement the algorithmic pipeline orchestration method of the embodiments of this application. Detailed Implementation

[0023] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0024] To more clearly demonstrate the algorithm pipeline orchestration method provided in the embodiments of this application, we first introduce the application scenarios that can be used to implement this method. Figure 1 This diagram illustrates an exemplary application scenario of an embodiment of this application. This scenario is an IoT scenario. In IoT scenarios, algorithm pipelines need to be generated for algorithmic tasks, and the hardware platforms for applying these pipelines are diverse. For example... Figure 1 As shown, the algorithm pipeline in this application embodiment can be used on hardware platforms such as servers, all-in-one computers, mobile terminals, and cameras. The computing power of these platforms can vary greatly, ranging from edge servers equipped with high-performance GPUs (Graphics Processing Units) to common mobile terminals, and even to security cameras with extremely limited resources. Hardware architectures also often vary, from common GPUs and CPUs (Central Processing Units) to heterogeneous systems-on-a-chip (SoCs) that simultaneously incorporate CPUs, DPS (Digital Processing Systems), and NPUs (Neural-network Processing Units).

[0025] For example, the algorithm in the embodiments of this application may refer to an AI algorithm. Accordingly, the algorithm task in the embodiments of this application may include a computing task implemented based on AI technology, such as a computer vision task, including a vehicle detection task, an area intrusion detection task, etc.

[0026] In this embodiment, the algorithm pipeline orchestration device is used to orchestrate the algorithm pipelines (i.e., pipelines) corresponding to each algorithm task, thereby implementing the algorithm pipeline orchestration method provided in this embodiment. For example, as... Figure 1As shown, developers can write task description files based on task attributes and input them into the algorithm pipeline orchestration device. The device then orchestrates the algorithm pipeline based on these task description files, and this pipeline can be applied to the aforementioned hardware platforms. Task attributes include task type, hardware platform type, etc. For example, developers can write task description files for license plate recognition tasks that need to run on cameras, vehicle traffic perception tasks that can run on edge servers, or facial recognition tasks that can run on any platform, to orchestrate the corresponding algorithm pipelines.

[0027] For example, the algorithm pipeline orchestration device can provide an algorithm pipeline orchestration method in the form of a computing framework. That is, a computing framework can be pre-built, integrating multiple general and reusable algorithm modules. In practical applications, the algorithm modules in the computing framework are then instantiated and orchestrated according to the task attributes of a specific algorithm task, resulting in an algorithm pipeline that can be ported to a specific hardware platform. This modular approach enables low-code development, reduces development workload, and accelerates algorithm deployment.

[0028] Optionally, the computing framework can also integrate low-level operation operators for various hardware platforms. In practical applications, the corresponding low-level operation operators are called for the hardware platform corresponding to the algorithm task, so that the hardware platform can execute the algorithm task according to the algorithm pipeline. Optionally, the computing framework can also include a runtime, which integrates different inference engines for performing data operations on each algorithm module.

[0029] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0030] Figure 2 A flowchart of an algorithmic pipeline orchestration method provided in an embodiment of this application is shown. Figure 2 As shown, the method may include:

[0031] S210. Obtain the task description file; wherein, the task description file includes description information for multiple modules;

[0032] S220. Based on multiple module description information, multiple general modules are called from the module library for processing to obtain multiple application modules;

[0033] S230. Based on multiple application modules, an algorithm pipeline corresponding to the task description file is obtained.

[0034] In this embodiment, the task description file can be used to describe the algorithm task. For example, the task description file may include identification information of the algorithm task, such as the name and category of the algorithm task. The task description file may also include multiple module descriptions corresponding to multiple data processing modules (or data processing stages) of the algorithm task, each module description describing the corresponding data processing module. For example, each module description may include a module name, module category, module type, module parameters, input / output information, etc.

[0035] In practical applications, the algorithm tasks described by different task description files can include different data processing modules. For example, the data processing modules included in an algorithm task can be set based on its task attributes, such as the task type and the type of hardware platform executing the task. For instance, a facial recognition task deployed on an edge server may include data processing modules such as video decoding, video frame extraction, color space conversion, face detection, face tracking, face quality scoring, face optimization filtering, facial feature extraction, and face recognition. A facial recognition task deployed on a panel device may include data processing modules such as color space conversion, face detection, face quality scoring, and facial feature extraction.

[0036] Since the data processing modules in an algorithm task are determined based on task attributes, the task description file can be associated with these attributes. Specifically, developers can determine the task description information and module description information in the task description file based on task attributes, including module category, module type, module parameters, input / output information, etc. For example, multiple data processing modules can be determined based on task attributes, and the task description file can be written based on these modules and preset rules. These preset rules can include predefined writing requirements such as data format and required fields. Based on these preset rules, the relevant information for each data processing module can be parsed from the task description file.

[0037] In practical applications, different algorithm tasks can be summarized and decomposed into a series of general data processing modules, which can then be stored in a module library. For example, computer vision tasks generally include sub-tasks such as image processing, video processing, model inference, and applied data processing. Image processing can include various data processing modules such as color space conversion and image compression; video processing can include various data processing modules such as video decoding and video frame extraction; and model inference can include various data processing modules such as object detection, classification, regression, multi-object tracking, feature extraction, and recognition.

[0038] In different algorithmic tasks, the data processing module of the same type follows the same data processing flow, such as containing the same functions / operators, but there may be differences in configuration, such as different parameters or different input and output information. For example, in face detection and vehicle detection tasks, the image processing module's processing flow is the same, but the output detection boxes of different sizes can be set according to actual needs. In the embodiments of this application, the general module may include an uninstantiated data processing module determined based on the module category and type, such as a data processing module containing a defined function / operator but without configured related parameters.

[0039] Optionally, the module description information in the task description information may include information such as module category, module type, and parameters. This allows for the invocation of general modules from the module library based on module category and type information, and the instantiation of these general modules based on parameter configuration information, i.e., configuring parameters and other attributes for these general modules to obtain application modules. In other words, the application modules in this embodiment may include instantiated modules obtained by configuring general modules, such as data processing modules that contain defined functions / operators and have configured parameters. Since the application modules are instantiated modules, organizing and chaining them can yield an algorithm pipeline.

[0040] As can be seen, the method provided in this application pre-configures multiple general modules in a module library, writes a task description file containing description information of multiple modules, and calls the general modules to process and obtain application modules suitable for specific algorithm tasks, thus obtaining an algorithm pipeline. In this way, for tasks of different types or on different hardware platforms, as long as different task description files are written, the above method can be reused to obtain an algorithm pipeline adapted to the task. It features low coding and modularity.

[0041] Optionally, in an exemplary embodiment, step S220 involves calling multiple general modules from a module library for processing based on multiple module description information to obtain multiple application modules, including:

[0042] Based on the module types contained in each module description information in multiple module description information, multiple general modules are identified in the module library;

[0043] Based on the configuration information contained in the description information of each module, multiple general modules are instantiated to obtain multiple application modules.

[0044] In other words, the description information for each module includes the module type, configuration information, etc. The module type can be used to identify common modules in the module library; the configuration information can be used to instantiate common modules.

[0045] In this embodiment, the module type refers to the type of data processing module described in the module description information, such as video decoding, video frame extraction, color space conversion, face detection, and face tracking. Optionally, the module description information may also include the category of the described data processing module, such as image processing, video processing, model inference, and application data processing. Accordingly, a general module can be called from the module library based on the module type and category in the module description information. For example, based on the video processing category and video decoding type, the corresponding general video decoding module can be accurately called from the module library.

[0046] Configuration information may include algorithm parameters, which are one or more parameters of the data processing module described in the module description information. For example, for an object detection module, these algorithm parameters may include the detection box length, detection box width, etc. For a batch processing module, these algorithm parameters may include the batch size.

[0047] Optionally, in an exemplary implementation, step S230, obtaining an algorithm pipeline corresponding to the task description file based on multiple application modules, includes:

[0048] Based on the order information of multiple module descriptions, the arrangement order of multiple application modules is obtained;

[0049] Based on the input and output information contained in the module description information of multiple modules, the data types between multiple application modules are determined.

[0050] Based on the arrangement order of multiple application modules and the data types among them, an algorithm pipeline is obtained.

[0051] In other words, the task description file also contains the order information of multiple module descriptions, used to determine the arrangement order of the corresponding application modules. Optionally, this order information may include indications of whether the data processing modules are parallel, their serial order, etc. For example, the order of each module can be explicitly recorded as data in the task description file, or the module description information can be written sequentially according to the order of the modules, thus implicitly including the order information between the module descriptions in the task description file.

[0052] Based on the above approach, the description information for each module also includes input and output information to determine the data types between modules, such as images, image sets, and numerical values. Thus, based on this information, multiple application modules can be accurately orchestrated to obtain an algorithm pipeline.

[0053] For example, in this embodiment of the application, the task description file may include a JSON (JavaScript Object Notation) file. This JSON file includes task description information input according to a preset field arrangement structure, as well as multiple module description information. JSON is a lightweight data exchange format, easily parsed and generated by machines. The field arrangement structure in the JSON file can be predefined, such as the writing positions of the task description information and module description information, the types of fields included, etc. Then, the task description information and module description information are input based on this field arrangement structure, so that the algorithm pipeline orchestration device can accurately parse the task description information and module description information from the JSON file based on this field arrangement structure.

[0054] Optionally, in one exemplary embodiment, the above method further includes:

[0055] Based on the user-inputted general module registration information, add the corresponding general module to the module library.

[0056] In other words, the embodiments of this application support users to register custom modules in the module library. By registering custom modules in the module library, the algorithm pipeline that the device can orchestrate can be expanded, that is, the application scope based on the above method can be expanded, without modifying the core code used to implement the above method.

[0057] Optionally, in one exemplary embodiment, the above method further includes:

[0058] In response to the algorithm pipeline of the first hardware platform, based on the hardware type of the first hardware platform, the underlying operation operator corresponding to the hardware type is called in multiple underlying operation operators;

[0059] The system provides underlying operation operators for the first hardware platform through a preset interface; the preset interface is used for hardware platforms corresponding to multiple hardware types to call.

[0060] The first hardware platform is the hardware platform that executes the algorithm pipeline. This first hardware platform can be a hardware platform associated with the task description file; for example, the task description file is written for an algorithm task that runs on the first hardware platform. In some scenarios, the first hardware platform can also be any hardware platform. For example, the algorithm task corresponding to the task description file can be implemented on any hardware platform, in which case the underlying operation operators are adapted according to the actual hardware platform executing the algorithm task.

[0061] In the above exemplary embodiments, hardware type can refer to the hardware architecture of the hardware platform, including processor type, model, and other information. According to the above method, the algorithm pipeline orchestration device defines a preset interface as a common processing interface for different hardware platforms. The device can call the corresponding low-level operation operator based on the hardware type, without requiring each hardware platform to implement task execution through a dedicated interface. That is, a unified hardware programming interface shields the implementation details differences in data computation across different hardware, thereby providing efficient, flexible, and multi-platform hardware-accelerated algorithm pipeline orchestration capabilities.

[0062] To more clearly illustrate the technical concept of this application, a specific application example is provided below. In this application example, the above method is implemented in the form of a computing framework (hereinafter referred to as the pipeline framework). Figure 3 A schematic diagram of the algorithm pipeline orchestration framework is shown. For example... Figure 3 As shown, the framework includes: pipeline layer, layer (data processing stage), framework layer, hardware abstraction layer, and hardware platform layer.

[0063] The pipeline layer is the application layer of the framework. For different application scenarios, it's often necessary to create a suitable pipeline based on the processing stages, such as a face detection pipeline, a vehicle detection pipeline, or a region intrusion detection pipeline. This framework can organize algorithm pipelines using JSON files. For algorithm implementation, developers only need to write the JSON file according to the rules, and the pipeline framework will use the JSON file description for runtime orchestration, executing each step of the algorithm to obtain the final output. Describing algorithm pipelines using JSON enables low-code development of algorithm engineering, accelerating algorithm implementation and iteration.

[0064] The Layer is a data processing module library within the pipeline framework. A Layer is a fundamental building block of an algorithm pipeline, a node (step) in data processing. Connecting different Layers creates algorithm pipelines with different functionalities. Currently, the Layer component mainly consists of image processing-related Layers, video processing-related Layers, model inference-related Layers, application data processing Layers, and custom Layers. Among these, the model inference Layer primarily implements general inference task modules for computer vision scenarios, such as object detection, classification, regression, multi-object tracking, feature extraction, and recognition. The implementation of these modules is a general template and is not bound to specific algorithm scenarios. The difference between different tasks lies in the different input and output data (with different meanings) of the Layers, while the data processing methods of the Layers are universal. Therefore, in practical applications, a single Layer can often be flexibly applied to different tasks, such as face detection / recognition and vehicle / license plate detection and recognition. Furthermore, in the implementation, the principle of decoupling the Layer implementation from hardware details is followed. The Layer code does not directly expose the hardware interface details of different platforms. Instead, it shields hardware differences through a unified hardware abstraction layer. The advantage of this is that the Layer module of the Layer layer is more universal, allowing algorithm porters of different hardware platforms to share the Layer module of the Layer layer.

[0065] The framework layer contains the core components of the pipeline framework, including a registry, graph resolver, runtime, and data manager. The registry registers custom layers. If existing layer implementations cannot cover specific data processing operations in the algorithm flow, the registry mechanism allows users to flexibly implement and register custom layers without modifying the framework's core code. The graph resolver parses JSON files, identifies layer configurations within them, and instantiates layers to construct the algorithm pipeline's execution graph. The runtime executes data operations at different nodes according to the order of the execution graph, currently supporting integration with different inference engines. The runtime allows for flexible multi-threaded concurrency at both the layer and pipeline levels, making fuller use of the parallel capabilities of different hardware. The data manager manages the input and output data structures of layers. Inputs and outputs between different layers are deconstructed through these data structures. Data is consumed only when the input queue is available, and after computation, results are added to the output queue for downstream layers to consume. This eliminates the need for layers to explicitly handle complex data dependencies, reducing complexity. The data manager is also responsible for tracking the lifecycle of intermediate data at different stages of the pipeline and releasing the data.

[0066] The Hardware Abstraction Layer (HAL) is the framework's unified hardware programming interface. It defines common processing interfaces for different hardware, shielding them from the implementation details of data computation. During pipeline execution, the underlying operators corresponding to the hardware platform are selected for execution when running layer operations, such as selecting the appropriate image operator interface, video operator interface, or inference engine interface. When developers need to interface with a new hardware platform, they must re-add the relevant operator implementations of HAL for that new platform according to the HAL interface specification.

[0067] The hardware platform layer represents the hardware acceleration platforms currently supported by the framework, mainly including platforms based on hardware architectures such as CPU, GPU, and DSP, covering edge servers, all-in-one computing machines, consumer-grade mobile devices, security cameras, etc.

[0068] To more clearly describe the modular and low-code characteristics of the method in this application's embodiments, an example JSON file for a facial object detection task is provided below to demonstrate the pipeline's processing flow. An example of the information in the JSON file is as follows:

[0069]

[0070] The JSON file mentioned above includes task description information such as the name and version of the algorithm task, as well as module description information. The example above uses JPG (Joint Photographic Group) image decoding to illustrate the module description information. In practical applications, the JSON file can also include descriptions of multiple other modules. Each module description can include the module's name, category, type, input and output information, etc. For example, the module description for the JPG image decoding module indicates that its name is JPG image decoding, its category is video processing, its type is JPG image decoding, its input information is batch data of JPG images, and its output information is batch data of BGR (blue-green-red). Therefore, in the pipeline orchestration JSON file, users can select different layer modules as needed and set corresponding parameters. Different layers ultimately combine to form the entire algorithm flow. Figure 4 This shows a visual rendering of the pipeline orchestrated from the JSON file in the example. At runtime, the pipeline framework's graph parser parses the JSON file, instantiates the pipeline, executes it, and obtains the computation results.

[0071] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provide an algorithm pipeline orchestration apparatus 500. (See reference...) Figure 5 The device 500 may include:

[0072] Application layer 510 is used to obtain the task description file; the task description file includes description information for multiple modules;

[0073] Framework layer 520 is used to call multiple general modules in the module library for processing based on multiple module description information, to obtain multiple application modules, and based on multiple application modules, to obtain the algorithm pipeline corresponding to the task description file.

[0074] The device 500 may also include a module library.

[0075] Exemplarily, the device 500 may further include:

[0076] The hardware abstraction layer is used to respond to the algorithm pipeline of the first hardware platform. Based on the hardware type of the first hardware platform, it calls the underlying operation operator corresponding to the hardware type from multiple underlying operation operators, and provides the underlying operation operator to the first hardware platform through a preset interface; wherein, the preset interface is used for hardware platforms corresponding to multiple hardware types to call.

[0077] For example, the framework layer may include a graph resolver, which is used to determine multiple general modules in the module library based on the module types contained in each module description information; and to instantiate the multiple general modules based on the configuration information contained in each module description information to obtain multiple application modules.

[0078] For example, the graph resolver in the framework layer can also be used for:

[0079] Based on the order information of multiple module descriptions, the arrangement order of multiple application modules is obtained;

[0080] The framework layer may also include a data manager, which determines the data types between multiple application modules based on the input and output information contained in the module description information of each module, so that the framework layer can obtain an algorithm pipeline based on the arrangement order of multiple application modules and the data types between multiple application modules.

[0081] For example, the framework layer may also include a registrar for adding the corresponding general module to the module library based on the general module registration information input by the user.

[0082] For example, the framework layer may also include an execution runtime for executing the above-described algorithm pipeline.

[0083] For example, the data manager in the framework layer is also used to track the lifecycle of intermediate data in each application module of the algorithm pipeline and to release the data.

[0084] For example, the task description file includes a JSON file, which includes task description information input according to a preset field arrangement structure and description information of multiple modules.

[0085] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0086] This application also provides an electronic device for implementing the above method. Figure 6 A structural block diagram of an electronic device according to an embodiment of this application is shown. Figure 6 As shown, the electronic device includes a memory 610 and a processor 620. The memory 610 stores a computer program that can run on the processor 620. When the processor 620 executes the computer program, it implements the algorithm pipeline orchestration method in the above embodiments. The number of memories 610 and processors 620 can be one or more.

[0087] The electronic device also includes:

[0088] The communication interface 630 is used to communicate with external devices and perform data exchange and transmission.

[0089] If the memory 610, processor 620, and communication interface 630 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0090] Optionally, in a specific implementation, if the memory 610, processor 620, and communication interface 630 are integrated on a single chip, then the memory 610, processor 620, and communication interface 630 can communicate with each other through an internal interface.

[0091] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods provided in any embodiment of this application.

[0092] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the methods provided in any embodiment of this application.

[0093] This application also provides a chip, which includes a processor for calling and executing instructions stored in a memory, causing a communication device on which the chip is installed to perform the method provided in this application.

[0094] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0095] It should be understood that the aforementioned processor can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0096] Further, optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0097] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0098] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0099] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0100] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0101] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0102] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0104] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An algorithm pipeline orchestration method, comprising: Obtain a task description file; wherein the task description file includes description information for multiple modules; Based on the description information of the multiple modules, multiple general modules are called from the module library for processing to obtain multiple application modules; Based on the multiple application modules, an algorithm pipeline corresponding to the task description file is obtained; in response to the first hardware platform executing the algorithm pipeline, based on the hardware type of the first hardware platform, the underlying operation operator corresponding to the hardware type is called from multiple underlying operation operators; the underlying operation operator is provided to the first hardware platform through a preset interface; wherein, the preset interface is used for hardware platforms corresponding to multiple hardware types to call.

2. The method according to claim 1, wherein, Based on the description information of the multiple modules, multiple general modules are called from the module library for processing to obtain multiple application modules, including: Based on the module types contained in each module description information in the plurality of module description information, the plurality of general modules are determined in the module library; Based on the configuration information contained in the description information of each module, the multiple general modules are instantiated to obtain the multiple application modules.

3. The method according to claim 1 or 2, wherein, The process of obtaining the algorithm pipeline corresponding to the task description file based on the multiple application modules includes: Based on the order information of the description information of the multiple modules, the arrangement order of the multiple application modules is obtained; Based on the input and output information contained in each module description information in the multiple module description information, the data types among the multiple application modules are determined; The algorithm pipeline is obtained based on the arrangement order of the multiple application modules and the data types among the multiple application modules.

4. The method according to claim 1 or 2, wherein, The task description file includes a JSON file, which includes task description information input according to a preset field arrangement structure and description information of the multiple modules.

5. The method according to claim 1 or 2, wherein, The method further includes: Based on the general module registration information entered by the user, the corresponding general module is added to the module library.

6. An algorithm pipeline orchestration apparatus, wherein, The device includes: The application layer is used to obtain the task description file; wherein the task description file includes description information of multiple modules; The framework layer is used to call multiple general modules in the module library for processing based on the multiple module description information to obtain multiple application modules, and to obtain the algorithm pipeline corresponding to the task description file based on the multiple application modules. A hardware abstraction layer is used to respond to the execution of the algorithm pipeline by the first hardware platform. Based on the hardware type of the first hardware platform, it calls the underlying operation operator corresponding to the hardware type from multiple underlying operation operators, and provides the underlying operation operator to the first hardware platform through a preset interface; wherein, the preset interface is used for hardware platforms corresponding to multiple hardware types to call.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 1-5.

8. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1-5.

Images