AI workflow execution device and method with parallel processing capability

By using a parallel processing architecture and scheduling algorithm, the problems of slow processing speed and low resource utilization of existing AI workflow execution devices are solved, realizing efficient and real-time parallel processing of AI workflows and improving processing speed and resource utilization.

CN119045971APending Publication Date: 2024-11-29SHANGHAI SHUHENG INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411183105.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Existing AI workflow execution devices use a serial processing method, resulting in slow processing speed, low resource utilization, and poor scalability, which cannot meet the real-time and high-efficiency requirements of highly complex datasets.

Method used

It adopts a parallel processing architecture, including a data input module, a task scheduling module, parallel processing units, a result merging module, and an output interface. Through load balancing and dynamic scheduling algorithms, it realizes the parallel operation of multiple processing units and the merging of results.

Benefits of technology

It significantly improves processing speed and resource utilization, has good scalability and reliability, can meet real-time requirements, and increases processing speed by more than 3 times and resource utilization by more than 50%.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119045971A_ABST
    Figure CN119045971A_ABST
Patent Text Reader

Abstract

The invention relates to an AI workflow execution device and method with parallel processing capability, and the device comprises a data input module which is used for receiving data input by a user, including but not limited to images, texts and audios, and carrying out the preprocessing of the data; the task scheduling module is used for distributing the preprocessed data tasks to a plurality of parallel processing units, and task load balancing of each parallel processing unit is ensured by adopting scheduling algorithms such as load balancing and dynamic scheduling; the parallel processing unit is used for enabling each processing unit to independently execute the reasoning task of the AI model through a plurality of processors or processing nodes so as to realize parallel operation of the plurality of processing units; the result merging module is used for summarizing results processed by the parallel processing units to generate a final output result; the output interface is used for outputting the result to a user or a downstream system; the problems of low processing speed, low resource utilization rate, poor expansibility and the like in the prior art can be solved.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an AI workflow execution device and method with parallel processing capability. [Background Technology]

[0002] At present, in the field of artificial intelligence (AI), with the emergence of big data and complex models, the execution efficiency of AI workflows has become a key issue. Traditional AI workflow execution devices usually use serial processing, which is slow and inefficient when processing complex and large data sets, and often cannot meet the requirements of real-time and high efficiency. For example, in high-complexity fields such as image processing and natural language processing, serial processing will cause the system response time to be too long, thus affecting the user experience and actual application effect.

[0003] The main structure of the existing technology usually includes: a data input unit, which is responsible for receiving and preprocessing input data, such as images, text, etc. A model execution unit: uses a pre-trained AI model to analyze and process the preprocessed data. A result output unit: outputs the processed results for users to view or further process. These units are connected in a single-threaded sequence to complete the data processing process in turn. However, although this structure can work normally in simple applications, it exposes the following problems and shortcomings in the face of big data and scenarios that require high concurrent processing:

[0004] (1) Slow processing speed: The serial processing method limits the processing speed of the overall system and cannot fully utilize the parallel processing capabilities of multi-core CPUs or GPUs.

[0005] (2) Low resource utilization: During the data processing stage when some steps are waiting, the processor may be idle most of the time, and resource utilization is not high.

[0006] (3) Poor scalability: It is difficult to increase the system's processing power by adding hardware devices, making it difficult to meet growing computing needs. [Summary of the invention]

[0007] The purpose of the present invention is to solve the above-mentioned deficiencies and provide an AI workflow execution device and method with parallel processing capabilities, which can solve the problems of slow processing speed, low resource utilization, poor scalability and so on in the prior art, realize efficient and real-time AI workflow parallel processing, significantly improve the processing speed and resource utilization, and have good scalability and reliability.

[0008] In one aspect, the present invention provides an AI workflow execution device with parallel processing capability, comprising:

[0009] A data input module is used to receive data input by users, including but not limited to images, texts, and audios, and to pre-process the data;

[0010] A task scheduling module is used to distribute the pre-processed data tasks to multiple parallel processing units, and adopt a scheduling algorithm including but not limited to load balancing and dynamic scheduling to ensure the task load balance of each parallel processing unit;

[0011] Parallel processing units are used to enable each processing unit to independently perform the reasoning task of the AI ​​model through multiple processors or processing nodes, thereby realizing the parallel operation of multiple processing units;

[0012] The result merging module is used to aggregate the results processed by each parallel processing unit to generate the final output result;

[0013] Output interface: used to output results to users or downstream systems.

[0014] In a possible implementation, in the data input module, the preprocessing steps include data cleaning, format conversion and feature extraction, and data cleaning is used to remove noise and useless information, format conversion is used to convert data into a standard format, and feature extraction is used to extract key features as input for subsequent processing.

[0015] In another possible implementation, in the task scheduling module, the workload of each parallel processing unit is made the same through a load balancing algorithm, and the task allocation is dynamically adjusted based on the current system status through a dynamic scheduling algorithm.

[0016] In another possible implementation, in the parallel processing unit, each unit is configured with different types of AI models to adapt to different application scenarios. The AI ​​models include but are not limited to convolutional neural networks and recursive neural networks. Through reasonable task allocation and scheduling, parallel operation of multiple processing units is achieved.

[0017] In other possible implementation schemes, the result merging module includes a data aggregation unit and a decision fusion unit. The data aggregation unit aggregates the intermediate results of each parallel processing unit to generate a comprehensive result. For classification or prediction tasks, the decision fusion unit adopts a majority voting and confidence weighted strategy to fuse the decision results of each unit.

[0018] In another possible implementation, the output interface supports multiple output formats and interface protocols, and the output types include but are not limited to text, image, and report formats to meet different user needs.

[0019] In another aspect, the present invention provides an AI workflow execution method with parallel processing capability, comprising the following steps:

[0020] 1) When the user inputs data, the data input module will first initialize the data and then pre-process the data to ensure that the data meets the processing requirements;

[0021] 2) After receiving the pre-processed data tasks, the task scheduling module calculates the best allocation scheme through the scheduling algorithm and allocates the tasks to each parallel processing unit;

[0022] 3) After receiving the task, each processing unit independently performs model reasoning and returns the result to the task scheduling module;

[0023] 4) After receiving the results of each processing unit, the result merging module generates the final result through aggregation and fusion algorithms and prepares for output;

[0024] 5) After receiving the final result generated by the result merging module, the output interface outputs the result to the specified target location or system according to the preset format and protocol.

[0025] In a possible implementation, in step 1), the preprocessing step includes data cleaning, format conversion and feature extraction, and data cleaning is used to remove noise and useless information, format conversion is used to convert data into a standard format, and feature extraction is used to extract key features as input for subsequent processing.

[0026] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a program, and when the program is executed by a processor, the above method is implemented.

[0027] In a fourth aspect of the present invention, an electronic device is proposed, comprising a memory and a processor, wherein the memory stores a program executable on the processor, and when the program is executed by the processor, the electronic device implements the above method.

[0028] Compared with the prior art, the present invention has the following advantages:

[0029] (1) Improving processing speed: The present invention can simultaneously schedule multiple processing units to run in parallel, effectively improving the speed of data processing and meeting real-time requirements.

[0030] (2) Optimizing resource utilization: The present invention reasonably allocates tasks through a task scheduling module, so that the system can fully utilize the computing power of multi-core CPUs and GPUs, thereby improving resource utilization.

[0031] (3) Improve scalability: The modular design of the system of the present invention can easily improve the overall computing power by adding processing units to meet the growing computing needs.

[0032] (4) High reliability: The system of the present invention is designed with an error detection and recovery mechanism, so that the failure of a processing unit will not affect the operation of the entire system.

[0033] (5) Energy saving: The present invention improves processing efficiency due to parallel processing, and the time required for the system to complete the same amount of work is shortened, thereby reducing energy consumption overall.

[0034] In summary, the present invention solves the problems of slow processing speed, low resource utilization, and poor scalability in the prior art; and the above technical effects are confirmed by theoretical analysis and experimental data. For example, in large-scale image processing tasks, the parallel processing device proposed in the present invention has a processing speed increased by more than 3 times and a resource utilization increased by more than 50% compared with the traditional serial processing system. [Drawings]

[0035] Figure 1 It is a schematic diagram of the structure of the device of the present invention;

[0036] Figure 2 It is a schematic flow diagram of the method of the present invention;

[0037] Figure 3 It is a schematic diagram of the principle of the present invention;

[0038] Figure 4 It is a structural schematic diagram of the electronic device of the present invention. [Specific implementation method]

[0039] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present invention. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0040] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments:

[0041] As attached Figure 1 and attached Figure 3As shown, in some embodiments, the present invention provides an AI workflow execution device with parallel processing capabilities, aiming to solve the problems of slow processing speed, low resource utilization, poor scalability, etc. in the prior art. The main structure of the device includes a data input module, a task scheduling module, a parallel processing unit, a result merging module, and an output interface. Specifically, the device includes:

[0042] A data input module is used to receive data input by users, including but not limited to images, texts, and audios, and to pre-process the data;

[0043] A task scheduling module is used to distribute the pre-processed data tasks to multiple parallel processing units, and adopt a scheduling algorithm including but not limited to load balancing and dynamic scheduling to ensure the task load balance of each parallel processing unit;

[0044] Parallel processing units are used to enable each processing unit to independently perform the reasoning task of the AI ​​model through multiple processors or processing nodes, thereby realizing the parallel operation of multiple processing units;

[0045] The result merging module is used to aggregate the results processed by each parallel processing unit to generate the final output result;

[0046] Output interface: used to output results to users or downstream systems.

[0047] In some other embodiments, the functions and working principles of each module are as follows:

[0048] 1. Data input module

[0049] Function: used to receive various types of data input by users, including but not limited to images, text, audio, etc., and pre-process the data.

[0050] The preprocessing steps include: data cleaning (removing noise and useless information), format conversion (converting data into a standard format), and feature extraction (extracting key features as input for subsequent processing).

[0051] The specific process is: when the user inputs data, the data input module will first initialize the data, and then pre-process the data to ensure that the data meets the processing requirements.

[0052] 2. Task Scheduling Module

[0053] The function is to distribute the pre-processed data tasks to multiple parallel processing units, and use advanced scheduling algorithms (such as load balancing, dynamic scheduling, etc.) to ensure that the task load of each processing unit is balanced.

[0054] Load balancing algorithm: Ensures that the workload of each parallel processing unit is roughly the same to prevent some units from being overloaded and causing performance degradation.

[0055] Dynamic scheduling algorithm: dynamically adjust task allocation based on the current system status to optimize resource utilization and processing efficiency.

[0056] The workflow is as follows: after receiving the pre-processed data task, the task scheduling module calculates the best allocation plan through the scheduling algorithm and allocates the task to each parallel processing unit.

[0057] 3. Parallel Processing Unit

[0058] Function: multiple processors or processing nodes, each unit can independently perform the reasoning task of the AI ​​model.

[0059] Processing model: Each unit can be configured with different types of AI models (such as convolutional neural networks, recursive neural networks, etc.) to adapt to different application scenarios.

[0060] Parallel execution: Through reasonable task allocation and scheduling, multiple processing units can run in parallel to improve processing speed and efficiency.

[0061] Process description: After receiving the task, each processing unit independently performs model reasoning and returns the result to the task scheduling module.

[0062] 4. Result Merging Module

[0063] The function is to summarize the results processed by each parallel processing unit and generate the final output result.

[0064] Data aggregation: Aggregate the intermediate results of each parallel processing unit to generate comprehensive results.

[0065] Decision fusion: For classification or prediction tasks, strategies such as majority voting and confidence weighting are used to fuse the decision results of each unit to improve the accuracy and reliability of the final result.

[0066] Specific process: After receiving the results of each processing unit, the result merging module generates the final result through aggregation and fusion algorithms and prepares for output.

[0067] 5. Output interface

[0068] Function: Output the results to users or downstream systems, and support multiple output formats and interface protocols.

[0069] The output types are: text, image, report and other forms to meet the needs of different users.

[0070] Process description: After the output interface receives the final result generated by the result merging module, it outputs the result to the specified target location or system according to the preset format and protocol.

[0071] Through the above modules and steps, the device of the present invention realizes efficient and real-time parallel processing of AI workflows, significantly improves the processing speed and resource utilization, and has good scalability and reliability.

[0072] In some other embodiments, the present invention provides an AI workflow execution method with parallel processing capability, comprising the following steps:

[0073] 1) After the user inputs data, the data input module will first initialize the data and then preprocess the data to ensure that the data meets the processing requirements; the preprocessing steps include data cleaning, format conversion and feature extraction. Data cleaning removes noise and useless information, format conversion converts the data into a standard format, and feature extraction extracts key features as input for subsequent processing;

[0074] 2) After receiving the pre-processed data tasks, the task scheduling module calculates the best allocation scheme through the scheduling algorithm and allocates the tasks to each parallel processing unit;

[0075] 3) After receiving the task, each processing unit independently performs model reasoning and returns the result to the task scheduling module;

[0076] 4) After receiving the results of each processing unit, the result merging module generates the final result through aggregation and fusion algorithms and prepares for output;

[0077] 5) After receiving the final result generated by the result merging module, the output interface outputs the result to the specified target location or system according to the preset format and protocol.

[0078] The present invention is further described below by means of specific embodiments:

[0079] Embodiment 1: Embodiment for text processing tasks

[0080] 1. Data input module

[0081] Receiving data: The system receives long-form text data, which could be social media comments, news articles, customer feedback, etc.

[0082] Text splitting: Split long text into paragraphs and sentences for parallel processing.

[0083] Noise filtering: remove meaningless characters, advertising information and other noise data from the text.

[0084] Feature extraction: Extract features such as word vectors and sentence embeddings to generate data for model input.

[0085] 2. Task Scheduling Module

[0086] Task allocation: Assign preprocessed text data to multiple parallel processing units. Use dynamic scheduling algorithms to reasonably allocate tasks based on the current load of the system.

[0087] Scheduling algorithm: Load balancing and dynamic scheduling are combined to ensure that the task processing efficiency of each processing unit is maximized.

[0088] 3. Parallel Processing Unit

[0089] Model inference: Each processing unit runs pre-trained natural language processing (NLP) models, such as BERT, GPT, etc., to perform semantic analysis and sentiment recognition on text.

[0090] Independent processing: Each processing unit performs reasoning tasks independently and can process different text data simultaneously.

[0091] 4. Result merging module

[0092] Summarize results: Summarize the analysis results of each processing unit.

[0093] Majority voting: For the sentiment classification results of each sentence, the majority voting method is used to determine the final sentiment classification.

[0094] Confidence weighting: Based on the confidence of each processing unit in sentiment classification, the results are weighted and summarized to improve the overall accuracy.

[0095] 5. Output interface

[0096] Result output: The final semantic analysis and sentiment recognition results are output. The results can be presented in the form of text reports or charts for easy viewing and analysis by users.

[0097] Effects of the above specific embodiments: This embodiment can greatly improve the speed and accuracy of text processing, whether it is a long text or a short text.

[0098] In sentiment analysis tasks, the parallel execution system achieved nearly 20% higher accuracy and 2 to 3 times faster processing speed than the traditional serial approach.

[0099] The present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a computer, the method described in the above method embodiment is implemented. Specific beneficial effects can be found in the above method embodiment.

[0100] The present invention also provides a computer program product, which implements the method described in the above method embodiment when executed by a computer. Specific beneficial effects can be found in the above method embodiment.

[0101] In some other embodiments of the present invention, an electronic device 400 is disclosed, as shown in the attached figure. Figure 4 As shown, the electronic device includes a processor and a memory, the processor and the memory are connected via a bus, and the memory stores a program that can be run on the processor. When the program is executed by the processor, the electronic device implements the above method. Specifically, the hardware composition of the electronic device includes: one or more processors 401; memory 402; display 403; one or more application programs (not shown); and one or more computer programs 404, and the above devices can be connected via one or more communication buses 405. The one or more computer programs 404 are stored in the above memory 402 and are configured to be executed by the one or more processors 401, and the one or more computer programs 404 include instructions.

[0102] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0103] Each functional unit in each embodiment of the present invention can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as flash memory, mobile hard disk, read-only memory, random access memory, disk or optical disk.

[0104] The above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit it. Under the idea of ​​the present invention, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present invention as described above. Although the present invention is described in detail with reference to the above embodiments, it should be understood by those skilled in the art that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0105] The present invention is not limited to the above-mentioned implementation modes, and any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement modes and shall be included in the protection scope of the present invention.

Claims

1. An AI workflow execution device with parallel processing capability, characterized in that: include: A data input module is used to receive data input by users, including but not limited to images, texts, and audios, and to pre-process the data; A task scheduling module is used to distribute the pre-processed data tasks to multiple parallel processing units, and adopt a scheduling algorithm including but not limited to load balancing and dynamic scheduling to ensure the task load balance of each parallel processing unit; Parallel processing units are used to enable each processing unit to independently perform the reasoning task of the AI ​​model through multiple processors or processing nodes, thereby realizing the parallel operation of multiple processing units; The result merging module is used to aggregate the results processed by each parallel processing unit to generate the final output result; Output interface: used to output results to users or downstream systems.

2. The device according to claim 1, characterized in that: In the data input module, the preprocessing steps include data cleaning, format conversion and feature extraction. Noise and useless information are removed through data cleaning, data is converted into a standard format through format conversion, and key features are extracted through feature extraction as input for subsequent processing.

3. The device according to claim 1, characterized in that: In the task scheduling module, the workload of each parallel processing unit is made the same through a load balancing algorithm, and the task allocation is dynamically adjusted based on the current system state through a dynamic scheduling algorithm.

4. The device according to claim 1, characterized in that: In the parallel processing unit, each unit is configured with different types of AI models to adapt to different application scenarios. The AI ​​models include but are not limited to convolutional neural networks and recursive neural networks. Through reasonable task allocation and scheduling, the parallel operation of multiple processing units is achieved.

5. The device according to claim 1, characterized in that: The result merging module includes a data aggregation unit and a decision fusion unit. The data aggregation unit aggregates the intermediate results of each parallel processing unit to generate a comprehensive result. For classification or prediction tasks, the decision fusion unit adopts majority voting and confidence weighting strategies to fuse the decision results of each unit.

6. The device according to claim 1, characterized in that: The output interface supports a variety of output formats and interface protocols, and its output types include but are not limited to text, image, and report formats to meet different user needs.

7. A method for executing an AI workflow with parallel processing capability, characterized in that: The following steps are involved: 1) When the user inputs data, the data input module will first initialize the data and then pre-process the data to ensure that the data meets the processing requirements; 2) After receiving the pre-processed data tasks, the task scheduling module calculates the best allocation scheme through the scheduling algorithm and allocates the tasks to each parallel processing unit; 3) After receiving the task, each processing unit independently performs model reasoning and returns the result to the task scheduling module; 4) After receiving the results of each processing unit, the result merging module generates the final result through aggregation and fusion algorithms and prepares for output; 5) After receiving the final result generated by the result merging module, the output interface outputs the result to the specified target location or system according to the preset format and protocol.

8. The method according to claim 7, characterized in that: In step 1), the preprocessing steps include data cleaning, format conversion and feature extraction. Data cleaning is used to remove noise and useless information, format conversion is used to convert data into a standard format, and feature extraction is used to extract key features as input for subsequent processing.

9. A computer-readable storage medium, wherein a program is stored in the computer-readable storage medium, characterized in that: When the program is executed by a processor, the method according to claim 7 or 8 is implemented.

10. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores a program executable on the processor, and when the program is executed by the processor, the electronic device implements the method as claimed in claim 7 or 8.