Multi-algorithm multi-service scenario AI application system and working method thereof

Abstract

The application provides a multi-algorithm multi-service scenario AI application system and a working method thereof, and aims at problems caused by common application of multi-algorithms and multi-services in the scenario AI application, realizes data flow processing between multiple algorithms and multiple services through layered structure design and modular design, more conveniently expands algorithms and services, realizes popularization of the AI scenario application, adopts an efficient module scheduling algorithm, so that terminal equipment can also achieve real-time analysis of the scenario AI application, and through use of an algorithm request callback (ARI) technology, loop structure of a module scheduling diagram is avoided, decoupling of the module is realized, and performance of the AI application is maximized.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a scenario-based AI application system with multiple algorithms and multiple services, and its working method. Background Technology

[0002] In recent years, with the continuous deepening of urban digitalization and industrial digitalization, the demand for AI in different industries has become increasingly diverse and fragmented, and the requirements for the applicability and implementation capabilities of solutions have become increasingly stringent.

[0003] Currently, AI based solely on algorithms and frameworks is insufficient to meet the real-world needs of various industries. Only by bringing AI from the laboratory to all aspects of production and daily life, and implementing "scenario-based AI" based on actual needs, can we bring about truly feasible new ideas and solutions, and promote the comprehensive integration and upgrading of traditional industries. However, the application of scenario-based AI requires a wide variety of algorithms. Different scenarios rely on different algorithms, and even the same AI monitoring device may need to be configured with different business functions and use different algorithmic functions.

[0004] Therefore, in order to better realize the application of scenario-based AI, it is necessary to use new methods that support the expansion of business and algorithms, and achieve high-performance computing and analysis to empower various AI scenario applications with high real-time requirements. Summary of the Invention

[0005] To address the challenges of multiple algorithms and services involved in scenario-based AI applications, this invention provides a scenario-based AI application system and its working method that integrates multiple algorithms and services. Through layered structure and modular design, it enables data flow processing between various algorithms and services. Furthermore, by using Algorithm Request Callback (ARI) technology, it avoids the loop structure of the module scheduling graph, achieving module decoupling and maximizing the performance of AI applications.

[0006] In a first aspect, the present invention provides a scenario-based AI application system with multiple algorithms and multiple services, the system comprising a session layer, a processor layer and a module layer;

[0007] The session layer is designed as a container for processors and modules, used for external communication and parallel processing of frame reception and frame computation. It is also used to store all static configurations of this session channel, including the service configurations of the channel and their associated algorithm configurations. It is also used to parse the configuration parameters of the current channel, analyze all the configured services and algorithms, and divide them into multiple separate processors.

[0008] The processor layer consists of one or more virtual processors for session creation and management. Each processor encapsulates an algorithm module and a business module, with each processor containing one business module and at least one algorithm module.

[0009] The module layer includes an algorithm module and a business module. The algorithm module encapsulates the AI ​​algorithm model and uses the AI ​​model to perform inference calculations. The business module encapsulates the scenario business logic and performs logical judgments based on the results of the algorithm model.

[0010] Secondly, this invention provides a working method for a scenario-based AI application system with multiple algorithms and multiple services, including:

[0011] The session parses the configuration parameters of the current channel, analyzes all the configured services and algorithms, divides them into multiple processors, and configures N algorithm modules plus one service module for each processor;

[0012] After image data flows into the processor, it first flows into the algorithm module for inference calculation. The inference result then flows into the business module for business analysis. After the analysis is completed, the analysis result is output. The input and output of the algorithm module and the business module are transmitted to the outside world using data packets. Each processor has a unique data packet.

[0013] Optionally, the processor partitioning adopts the following rules:

[0014] Given a set of algorithm modules A{a1,a2,…ai,…} and a set of business modules B{b}, where all elements in set A are distinct, these two sets are divided into multiple subsets S1{a1,…,ai,…,b}, S2{a1,…,ai,…,b},…,Si{a1,…,ai,…,b} according to different business functions. For any ai, if it appears in multiple subsets Sum(Si), then these subsets Sum(Si) will be combined into a processor, and all identical ai in each subset Si will be merged to avoid duplication. For any subset Sk, if its element ai belongs only to this Sk, then Sk constitutes a processor on its own.

[0015] Optionally, when image data flows from the processor into the algorithm module, it needs to be executed according to specified rules, mainly including the following rules:

[0016] Single algorithm module

[0017] Image data flows directly into the algorithm module, and after inference is completed, it is output to the business module.

[0018] Multiple algorithm modules

[0019] Processors with multiple algorithm modules are divided into two categories:

[0020] One type is where algorithm modules execute in parallel. Image data flows into the algorithm modules simultaneously, and when it flows out of the algorithm modules, it needs to wait for all algorithm modules to complete their inference before inputting the inference results into the business module.

[0021] One type involves a sequential relationship between some algorithm modules, where subsequent algorithm modules rely on the results output by previous algorithm modules to complete inference calculations and input the inference results into the business module.

[0022] Optionally, when multiple algorithm modules are performing inference calculations, the processor uses a DAG to construct a module scheduling graph for algorithm module scheduling, and employs algorithm request callback technology to avoid the formation of loops in the module scheduling graph.

[0023] This invention employs a layered architecture design for scenario-based AI applications, creatively introducing processor design to achieve virtualization of underlying modules. This enables different processors to perform parallel processing, maximizing AI processing performance. Heuristic rules are used to automatically partition processors without user intervention. For module scheduling, a module scheduling graph and algorithm request callback technology are introduced, which avoids scheduling graph loops and improves module processing performance, maximizing parallel processing and enhancing performance for better deployment and application on terminal devices.

[0024] It should be understood that the description in the foregoing summary section is not intended to limit the key or essential features of the embodiments of the present invention, nor is it intended to restrict the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A flowchart of a single algorithm module system provided in this disclosure embodiment;

[0027] Figure 2 A system flowchart comprising multiple algorithm modules is provided as an embodiment of this disclosure;

[0028] Figure 3 for Figure 2 The module scheduling diagram of the system flowchart.

[0029] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0030] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While some embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the invention. It should be understood that the accompanying drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the invention.

[0031] As business scope expands, AI application scenarios will no longer be limited to a single field. For example, in intelligent transportation, it's no longer solely reliant on algorithms for detecting pedestrians, vehicles, and non-motorized vehicles. Smart road infrastructure maintenance scenarios require algorithms for detecting and recognizing traffic lights, lane lines, and traffic signs. Smart communities and smart campuses, on the other hand, demand accurate face detection and recognition algorithms. At this point, conventional algorithm-based business frameworks are insufficient for these applications. They need to expand from supporting only a single algorithm to supporting multiple algorithms, and then perform business analysis based on the algorithm analysis results. Therefore, it's necessary to design a business analysis model based on multi-algorithm scheduling to build scenario-based AI applications and create greater value.

[0032] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0033] Example 1

[0034] This embodiment provides a scenario-based AI application system with multiple algorithms and multiple services. It adopts a layered structure, which consists of three layers from top to bottom: session, processor, and module. The upper layer can only call the interface of the lower layer. In this way, when each layer is expanded, it only affects the layer that depends on it.

[0035] 1. Session Layer

[0036] The session layer is designed as a container for processors and modules, used for external communication and parallel processing of frame reception and frame computation. It is also used to store all static configurations of this session channel, including the service configurations for channel activation and their associated algorithm configurations. It is also used to parse the configuration parameters of the current channel, analyze all configured services and algorithms, and divide them into multiple separate processors. Each processor is configured with N algorithm modules and one service module. Different processors can process in parallel without the use of any locks for control, which improves processing performance.

[0037] 2. Processor layer

[0038] The processor layer consists of one or more virtual processors that create and manage sessions. The processor is the smallest unit that implements complete image data algorithms and business analysis. The processor encapsulates algorithm modules and business modules. Each processor contains one business module and at least one algorithm module. If a processor has multiple algorithm modules, they flow into different algorithm modules according to predetermined rules. They may be executed in parallel or sequentially. There is no relationship between different processors, and they can be executed in parallel.

[0039] The processor layer is responsible for one or more business analyses, and the number of processors is not directly specified by the user; it is created and managed by the session. Users typically submit business requirements for an analysis channel without specifying corresponding algorithm modules. Therefore, the session needs to analyze the business data in the configuration file according to predetermined rules to find the required one or more algorithm modules, along with a business module, to achieve the desired business analysis function. However, considering algorithm module reuse and performance issues, the session virtualizes one or more processors by analyzing all the algorithm and business modules required for this channel.

[0040] For example, if a user submits a task for this analysis channel that includes three business functions: congestion event analysis, red light violation analysis, and guardrail detection, then through pre-defined rule analysis, the current channel requires a pedestrian / vehicle non-motorized vehicle (PVN) detection algorithm module, a traffic light detection algorithm module, a guardrail detection algorithm module, and a business module. For congestion events, only the PVAN detection algorithm module and the business module are needed; for red light violations, the PVAN detection algorithm module, traffic light detection algorithm module, and business module are needed; and for guardrail detection, only the guardrail detection algorithm module and the business module are needed. Analysis shows that this session can be divided into two processors: one processor contains the PVAN detection algorithm module, the traffic light detection algorithm module, and the business module; the other processor contains the guardrail detection algorithm module and the business module. In this way, the detection of PVANs and guardrails can be executed in parallel, improving analysis performance.

[0041] 3. Module layer

[0042] It includes an algorithm module and a business module. The algorithm module encapsulates the AI ​​algorithm model and uses the AI ​​model to perform inference calculations. The business module encapsulates the scenario business logic and performs logical judgments based on the results of the algorithm model.

[0043] 1) Algorithm Module

[0044] Algorithm module classes encapsulate algorithms. Since each algorithm has different input and output parameters, and modules provide a unified interface, data is typically transmitted via data packets. Therefore, during processing, it's necessary to convert the unified interface parameters to the specific algorithm's input and output parameters. Each algorithm module encapsulates one algorithm; subsequent expansion only requires adding new algorithm modules and their corresponding input and output data.

[0045] 2) Business Module

[0046] The business module encapsulates the business logic of the scenario. Similar to the algorithm module, the input and output data are also based on data packets.

[0047] Example 2

[0048] This embodiment discloses a working method for the scenario-based AI application system with multiple algorithms and multiple services as described in Embodiment 1, including:

[0049] Step 1: Parse the configuration parameters of the current channel, analyze all the configured services and algorithms, divide them into multiple processors, and configure N algorithm modules plus one service module for each processor;

[0050] Specifically, the processor partitioning follows these rules: Given an algorithm module set A{a1,a2,…ai,…} and a business module set B{b}, where all elements in set A are distinct, these two sets are divided into multiple subsets S1{a1,…,ai,…,b}, S2{a1,…,ai,…,b},…,Si{a1,…,ai,…,b} according to different business functions. For any ai, if it appears in multiple subsets Sum(Si), then these subsets Sum(Si) will be combined into a processor. Furthermore, all identical ai in each subset Si are merged to avoid duplication. For any subset Sk, if its element ai belongs only to this Sk, then Sk constitutes a separate processor.

[0051] Step 2: After the image data flows into the processor, it first flows into the algorithm module for inference calculation. The inference result then flows into the business module for business analysis. After the analysis is completed, the analysis result is output. The input and output of the algorithm module and the business module to the outside world are transmitted through data packets. Each processor has a unique data packet.

[0052] Since different modules have different structured data, but the external interfaces of the modules are unified, in order to make the data to be processed adaptable to each module, it is necessary to combine and encapsulate the data of each module into a single data packet and transfer it between different modules.

[0053] Data packets (DPs) are the carriers of data, primarily used to transfer data between modules, i.e., runtime data. This data mainly includes input and output data. Since different algorithm SDKs and business SDKs have different input and output data formats, a dedicated design is needed to store the input and output data of different SDKs in the DP. Each processor has a unique DP, allowing the DP's data buffer to be reused during the processing of each frame, avoiding memory allocation and deallocation each time. Because the DP is a member object of the processor, its lifecycle is consistent with the processor, and its validity is limited to the processing of the current frame of data on the current processor.

[0054] Data packets, acting as a medium for transmission between modules, contain the input and output data items of all modules in the current processor, as well as the frame data to be processed. Each module's input and output data items are encapsulated into a packet and accessed via a module tag. A packet encapsulates the input and output data of a module, and this data is structured, using a specific structure `any` to provide storage space for various types of structured data. In addition to the input and output data, each packet has a timestamp to store the timestamp information of the current frame to be processed. Furthermore, different modules may use specific methods to implement specific functions, such as serialization, to sequence structured data into JSON format.

[0055] When image data flows from the processor into the algorithm module, it needs to be executed according to specified rules, mainly the following:

[0056] Single algorithm module

[0057] Image data flows directly into the algorithm module, inference is completed, and the output is sent to the business module, such as... Figure 1 As shown in processor 1;

[0058] Multiple algorithm modules

[0059] Processors with multiple algorithm modules are divided into two categories:

[0060] One type involves parallel execution between algorithm modules. Image data flows into the algorithm modules simultaneously, and when it flows out, it needs to wait for all algorithm modules to complete their inference before inputting the inference results into the business module. Figure 2 As shown in processor 2;

[0061] One type involves algorithms with a specific order in which subsequent modules rely on the results of earlier modules to perform inference calculations and then input the inference results into the business module. For example... Figure 2 As shown in processor 2.

[0062] For the same processor, if two modules have no dependencies, they can be executed in parallel. However, if dependencies exist, the current module must wait for the preceding module to finish processing before it can execute. If a module depends on the results of multiple modules, it must wait for all dependent modules to finish processing before it can execute. Thus, if a processor contains N algorithm modules, a Directed Acyclic Graph (DAG) is used to construct the module scheduling graph to achieve maximum parallel processing between modules. Each module's processing is encapsulated into a separate task, and each independent task can only run on a single core and single thread, which can be considered an execution unit. Tasks can then run in parallel. Furthermore, dependencies can be added between tasks. For example, `taskA.precede(taskB)` indicates that `taskB` depends on `taskA`; or `taskA.succeed(taskB)` indicates that `taskA` depends on `taskB`. Figure 2 The module scheduling diagram of the system flowchart is as follows: Figure 3 As shown.

[0063] During the processor's scheduling of modules, sometimes module B depends on module A, but module A also depends on module B. Constructing a module scheduling graph in this case would result in a loop. Solving loops is extremely difficult. To overcome this challenge, Algorithm Request Callback (ARI) technology is introduced to break down loop dependencies. Module A then requests the processor to run module B's computation via ARI, obtains the result, and returns it to A, instead of directly using the module scheduling graph. This avoids the scheduling loop problem while simultaneously achieving maximum parallel processing between independent modules.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A scenario-based AI application system with multiple algorithms and multiple services, characterized in that, The system includes a session layer, a processor layer, and a module layer; The session layer is designed as a container for processors and modules, used for external communication and parallel processing of frame reception and frame computation. It is also used to store all static configurations of this session channel, including the service configurations of the channel and their associated algorithm configurations. It is also used to parse the configuration parameters of the current channel, analyze all the configured services and algorithms, and divide them into multiple separate processors. The processor layer consists of one or more virtual processors for session creation and management. Each processor encapsulates an algorithm module and a business module, with each processor containing one business module and at least one algorithm module. The module layer includes an algorithm module and a business module. The algorithm module encapsulates the AI ​​algorithm model and uses the AI ​​algorithm model to perform inference calculations. The business module encapsulates the scenario business logic and performs logical judgments based on the algorithm model results. The processor partitioning follows these rules: Given a set of algorithm modules A{a1, a2, … ai, …} and a set of business modules B{b}, where all algorithm modules in set A are distinct, these two sets are divided into multiple subsets S1{a1, …,ai, …, b}, S2{a1, …,ai, …, b}, …, Si{a1, …,ai, …, b} according to different business functions. For any algorithm module ai, if it appears in multiple subsets Sum(Si), then these subsets Sum(Si) will be combined into a processor. Furthermore, all identical algorithm modules ai in each subset Si are merged to avoid duplication. For any subset Sk, if its algorithm module ai belongs only to this Sk, then Sk constitutes a processor on its own.

2. A method for operating a scenario-based AI application system with multiple algorithms and multiple services as described in claim 1, characterized in that, include: The session parses the configuration parameters of the current channel, analyzes all the configured services and algorithms, divides them into multiple processors, and configures N algorithm modules plus one service module for each processor; After the image data flows into the processor, it first flows into the algorithm module for inference and calculation. The inference result then flows into the business module for business analysis. After the analysis is completed, the analysis result is output. The input and output of the algorithm module and the business module are transmitted to the outside world using data packets as the carrier. Each processor has a unique data packet. The processor partitioning follows these rules: Given a set of algorithm modules A{a1, a2, … ai, …} and a set of business modules B{b}, where all algorithm modules in set A are distinct, and each algorithm module ai encapsulates an AI algorithm model, these two sets are divided into multiple subsets S1{a1, …,ai, …, b}, S2{a1, …,ai, …, b}, …, Si{a1, …,ai, …, b} according to different business requirements. For any algorithm module ai, if it appears in multiple subsets Sum(Si), these subsets Sum(Si) will be combined into a processor. Furthermore, all identical algorithm modules ai in each subset Si will be merged to avoid duplication. For any subset Sk, if its algorithm module ai belongs only to this Sk, then Sk constitutes a processor on its own.

3. The working method of the scenario-based AI application system with multiple algorithms and multiple services according to claim 2, characterized in that, When image data flows from the processor into the algorithm module, it needs to be executed according to specified rules, mainly the following: Single algorithm module Image data flows directly into the algorithm module, and after inference is completed, it is output to the business module. Multiple algorithm modules Processors with multiple algorithm modules are divided into two categories: One type is where algorithm modules execute in parallel. Image data flows into the algorithm modules simultaneously, and when it flows out of the algorithm modules, it needs to wait for all algorithm modules to complete their inference before inputting the inference results into the business module. One type involves a sequential relationship between some algorithm modules, where subsequent algorithm modules rely on the results output by previous algorithm modules to complete inference calculations and input the inference results into the business module.

4. The working method of the scenario-based AI application system with multiple algorithms and multiple services according to claim 3, characterized in that, When multiple algorithm modules perform inference calculations, the processor uses a Directed Acyclic Graph (DAG) to construct a module scheduling graph for algorithm module scheduling, and employs an algorithm request callback technique to prevent the module scheduling graph from forming loops.

Images