Mass machine learning model reasoning distributed operation optimization system

Abstract

The invention discloses a massive machine learning model reasoning distributed operation optimization system, which relates to the technical field of computers, and is technically characterized by comprising a task scheduling module, a model deployment module and a performance optimization module, and the task scheduling module dynamically creates a Booster instance according to task characteristics by virtue of an improved Funboost framework, so that efficient receiving, storage, distribution and frequency regulation of tasks are realized, and a multi-concurrency mode and automatic resource allocation are supported. The model deployment module accurately positions the model through a function, calls a pre-parameter processing function, stores parameters in a database and issues a task by means of Booster, dynamically manages the model parameters through the database, and reasonably controls the operation interval through a dormancy mechanism. The performance optimization module reads the data from the InfluxDB, converts the data into a NumPy array, and stores the data after batch processing, so that the calculation efficiency is remarkably improved. The system is applied to the field of industrial big data, in particular to monitoring and early warning of a thermal power system, resource occupation can be greatly reduced, and operation efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a massive machine learning model inference distributed operation optimization system. Background Art

[0002] With the widespread adoption of digital technologies in the manufacturing industry, industrial production and operations are generating vast and diverse data sets. This data provides decision makers, researchers, and engineers with analytical insights into production processes, equipment status, and product quality. However, industrial big data, characterized by large volumes, diverse data types, rapid generation, and low value density, requires more advanced and specialized analytical platforms and methods to enable fault diagnosis, fault prediction, health assessment, maintenance management, and performance analysis. For example, thermal power systems are characterized by numerous modules, high coupling, and complex changing trends. To achieve precise control and automated modeling, the laboratory has developed an artificial intelligence prediction system based on time series data to intelligently predict current system status, including processing large amounts of alarm data. Traditional alarm monitoring methods suffer from high false alarm rates, alarm information overload, and difficulty in determining fault causes. Therefore, optimization of the alarm logic calculation and scheduling system is needed to improve the system's intelligence and operational efficiency.

[0003] The system currently uses a distributed task scheduling framework based on Celery to run predictive models. Celery is a powerful distributed task queue framework that supports asynchronous and scheduled task scheduling and can handle highly concurrent tasks. It distributes and consumes tasks through message middleware (such as Redis), and features high availability, fast response, and flexible scalability. The system also uses message queues to implement asynchronous task processing, decoupling applications through a producer-consumer model to improve the system's scalability and fault tolerance. Furthermore, the system uses the InfluxDB time series database to store and manage large-scale time series data, and attempts to use the NumPy scientific computing library to accelerate the filling of intermediate variables.

[0004] The existing Celery framework consumes excessive system resources (particularly CPU and memory) when processing large numbers of tasks, leading to server performance bottlenecks. Celery's design doesn't align with the specific needs of model execution functions in the system, preventing the framework from fully leveraging its advantages in scheduled tasks. Dependencies between intermediate variables frequently lead to population failures, and the computational efficiency of the population process is low, especially when processing large amounts of data, where using Python's built-in functions to perform individual calculations takes a long time. Summary of the Invention

[0005] The purpose of the present invention is to solve the above problems and provide a massive machine learning model reasoning distributed operation optimization system to solve the problems of high resource usage, low task scheduling efficiency and failure to fill intermediate variables in the existing technology.

[0006] In order to achieve the above-mentioned object, the technical solution of the present invention is as follows: a massive machine learning model inference distributed operation optimization system, including a task scheduling module, a model deployment module and a performance optimization module;

[0007] The task scheduling module creates a corresponding Booster instance for the task function through the improved Funboost framework. The Booster instance is responsible for receiving, storing, distributing tasks and controlling the frequency of task execution.

[0008] The model deployment module searches for the model through a function, determines whether the model exists, calls the pre-parameter processing function, writes the processed model parameters to the database, and publishes the task through the Booster instance;

[0009] The performance optimization module reads data from InfluxDB and converts it into a NumPy object array, processes the data in batches using NumPy's matrix operation capability, and writes the calculation results into the object and stores them in InfluxDB.

[0010] Furthermore, the task scheduling module supports multiple concurrent modes such as multi-threading, multi-process and coroutine, and can automatically adjust the size of the thread pool or process pool according to the number of tasks;

[0011] The task scheduling module achieves efficient scheduling and execution of tasks by configuring queue name, task execution times per second, concurrency mode, concurrency quantity, whether to use distributed frequency control, and middleware type parameters.

[0012] Furthermore, the model deployment module dynamically stores and updates model parameters through a database, supporting real-time adjustment of the model operation frequency;

[0013] The model deployment module controls the model running time interval through the sleep mechanism to avoid excessive resource occupation.

[0014] Furthermore, the system is applied in the field of industrial big data processing, especially in real-time monitoring and fault warning of thermal power systems;

[0015] The system significantly reduces system resource usage and improves operational efficiency by optimizing task scheduling, model deployment, and intermediate variable filling.

[0016] Furthermore, the Booster instance of the Funboost framework supports dynamic scheduling and frequency control of tasks, and can dynamically adjust the execution strategy of tasks according to task priority and resource usage; the Booster instance implements persistent storage and grouped consumption of tasks through data structures.

[0017] Compared with existing technologies, this solution offers the following advantages: It implements dynamic task scheduling and frequency control through an improved Booster instance of the Funboos framework, dynamically adjusting task execution strategies based on task priority and resource usage. The framework supports multiple concurrency modes and implements persistent storage and grouped consumption of tasks through data structures, significantly reducing system resource usage and improving resource utilization efficiency.

[0018] The task scheduling module of the present invention supports multiple concurrency modes, including multithreading, multiprocessing, and coroutines, and can automatically adjust the size of the thread pool or process pool according to the number of tasks. By configuring parameters such as queue name, number of task executions per second, concurrency mode, number of concurrent tasks, whether to use distributed frequency control, and middleware type, efficient task scheduling and execution are achieved, meeting the requirements for task scheduling flexibility in industrial big data processing. The performance optimization module of the present invention uses NumPy's matrix computing capabilities to batch process data, significantly improving computational efficiency, reducing fill time, and ensuring efficient system operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is an architectural diagram of a massive machine learning model inference distributed operation optimization system according to an embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram of the vmstat output results in an embodiment of the present invention;

[0021] Figure 3 This is a diagram showing the framework operation effect test results in an embodiment of the present invention;

[0022] Figure 4 This is a log chart of the running frequency of the Funboost framework model in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the present invention, the technical solution of the present invention will be further described in detail below in conjunction with the embodiments of the present invention and the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0024] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the embodiments.

[0025] Example 1: Massive Machine Learning Model Inference Distributed Operation Optimization System

[0026] The system of the present invention is based on a distributed architecture design and uses the Funboost framework to replace the traditional Celery framework to achieve efficient scheduling and execution of tasks. Figure 1 As shown, the system mainly consists of the following modules:

[0027] Task scheduling module: responsible for the release, scheduling and execution of tasks. This module is the core of the system, ensuring that tasks can be efficiently distributed to various computing nodes and executed at a predetermined frequency. The present invention creates a corresponding Booster for the task function through the improved Funboos. Booster is similar to the combination of Broker and Worker in Celery, responsible for the reception, storage and distribution of tasks. Compared with Celery, Funboost is more lightweight, supports multiple concurrency modes (such as multi-threading, multi-process, coroutine, etc.), and can flexibly configure the execution frequency and concurrency of tasks.

[0028] Model deployment module: responsible for model publishing, parameter preprocessing and operation. This module ensures that the model can be correctly loaded, initialized, and run in the specified Booster. The present invention searches for the model through the export_model_funboost function, determines whether the model exists, and calls the pre-parameter processing function. The processed model parameters are written to the Redis database and the task is published through the Booster. Storing model parameters in the Redis database ensures that the required data can be quickly obtained when the model is running, while supporting dynamic adjustment of the model operation frequency.

[0029] Performance Optimization Module: Uses NumPy to accelerate computations and reduce resource usage. This module optimizes computational efficiency, reducing computation time and resource consumption. This module reads data from InfluxDB and converts it into NumPy object arrays. It then leverages NumPy's matrix computing capabilities to process data in batches, eliminating the need for individual calculations. The computation results are written to a JSON object and stored in InfluxDB. NumPy's high-performance computing capabilities significantly improve data processing efficiency, reduce fill times, and reduce system resource usage.

[0030] The above task scheduling module is implemented using the Funboost framework. The specific steps are as follows:

[0031] 1. Use the @boost decorator: Create a corresponding Booster for the task function. The Booster is responsible for receiving, storing, and distributing the task. Use the @boost decorator to add metadata to the task function, specifying parameters such as the task queue name, execution frequency, and concurrency mode.

[0032] 2. Publish the task through the publish method: pass the task parameters to the Booster and trigger the execution of the task. Use the publish method to send the task parameters to the specified queue. After the Booster receives the task, it assigns it to the consumer process.

[0033] 3. Use the multi_process_consume(n) method: Start multiple consumer processes to improve task processing efficiency. The multi_process_consume(n) method starts multiple consumer processes, and each process independently processes tasks in the task queue.

[0034] 4. Configure key parameters:

[0035] queue_name: Each task function uses a different queue name to avoid task conflicts;

[0036] qps: controls the number of task executions per second, supports decimals (such as 0.01 means once every 100 seconds), and is used for current limiting;

[0037] concurrent_mode: concurrent mode, the default is multi-threading (THREADING), supporting multi-process, coroutine and other modes;

[0038] concurrent_num: the number of concurrent tasks. The framework will automatically adjust the thread pool size according to the number of tasks.

[0039] is_using_distributed_frequency_control: whether to use distributed frequency control to ensure that multiple consumers execute tasks at the set frequency globally;

[0040] broker_kind: middleware type, using REDIS_STREAM, based on the Stream data structure of Redis 5.0 or later, supporting consumption confirmation, persistence, and grouped consumption.

[0041] The present invention divides the model into multiple arrays and publishes tasks to different Booster respectively, reducing the task load of each Booster. After many experiments, the Booster size is set to 42 and the number of consumers is set to 3. The resource occupation is relatively ideal and can meet the operation effect. In addition, the current preset total number of boosters is 1, that is, when the model is actually deployed, an attempt will be made to fill one booster. After that, if there is a new model to be run, a new booster will be created and the current total number of boosters will be modified.

[0042] The algorithms for framework-related functions are shown in Table 1-2 below:

[0043] Table 1 Create booster algorithm try_create_booster_list pseudo code

[0044]

[0045] Table 2 Push object selection algorithm choose_booster_to_export_model pseudo code

[0046]

[0047]

[0048] The above model deployment module includes the following steps:

[0049] 1. Model Release: This method uses the export_model_funboost function to search for a model, determine if the model exists, call the pre-parameter processing function, and update the model's operating status in the database. The model is searched based on information such as the model number, version, and unit to which it belongs. Once the model is confirmed to exist, the pre-parameter processing function is called, the model parameters are written to the Redis database, and the model's operating status is updated. The pseudocode for the export_model_funboost model release algorithm is shown in Table 3 below.

[0050] Table 3 Pseudocode of export_model_funboost model algorithm

[0051]

[0052] 2. Parameter Preprocessing: This method processes model parameters using the predict_task_provider_funboost function, writes the model information to a Redis database, and selects an appropriate Booster to publish the task. This process generates the parameter objects required for model execution and stores them in the Redis database. The task is then published through the Booster to ensure correct execution. The parameter preprocessing algorithm is shown in Table 4 below.

[0053] Table 4 Parameter preprocessing algorithm predict_task_provider_funboost pseudo code

[0054]

[0055] 3. Model Run: This invention uses the predict_funboost_task function to retrieve model information from Redis, initialize data, run the model, and write the results to Redis. If the run time is less than the interval, the system goes into hibernation to avoid excessive resource usage. The system reads model parameters from the Redis database, initializes the model runtime environment, runs the model, and writes the results to the Redis database. If the model run time is less than the set interval, the system goes into hibernation to avoid excessive resource usage.

[0056] Table 5 Pseudocode of model deployment algorithm predict_funboost_task

[0057]

[0058] The performance optimization module is implemented through the following steps:

[0059] 1. Data Reading and Conversion: This paper reads data from InfluxDB and converts it into NumPy object arrays. This paper reads time series data from the InfluxDB database and converts it into NumPy object arrays for efficient matrix operations.

[0060] 2. Batch calculation: This paper uses NumPy's matrix operations to process data in batches, avoiding the need for one-by-one traversal calculations. Leveraging NumPy's matrix operation capabilities, batch calculations on data significantly improve computational efficiency and reduce fill time.

[0061] 3. Result storage: The present invention writes the calculation results into a JSON object and stores it in InfluxDB. The calculation results are encapsulated as a JSON object and stored in the InfluxDB database for subsequent use.

[0062] 4. Exception handling: This invention detects exceptions when using NumPy for filling. If filling fails, it reverts to the original method and refills. It detects exceptions during the filling process and, if any, switches back to the original, item-by-item calculation method, ensuring the reliability of the filling process.

[0063] Example 2: Application of the system described in Example 1 in a thermal power system

[0064] Thermal power systems require massive amounts of alarm logic and model prediction tasks to achieve real-time monitoring of equipment status and fault warnings. Traditional systems use the Celery framework for task scheduling, but this suffers from high resource usage and low task scheduling efficiency. Therefore, this embodiment aims to reduce system resource usage and improve operational efficiency by optimizing task scheduling and model deployment.

[0065] The test environment was based on the actual operating environment of a thermal power system, using a 222 server, which was primarily responsible for running the models for units 3 and 4. Prior to testing, the system had already deployed 1,019 models, which needed to operate efficiently to meet real-time monitoring requirements. The implementation steps are as follows:

[0066] Task scheduling optimization: Using the Funboost framework, we divided the 500 models into 12 boosters, each responsible for 42 models. We configured qps = 0.5 to ensure each model ran once every 2 seconds. We used three consumer processes to handle each booster's tasks, significantly reducing CPU and memory usage.

[0067] Model deployment optimization: Publish the model using the export_model_funboost function to ensure that the model parameters are correctly written to the Redis database. Run the model using the predict_funboost_task function to ensure that the model execution frequency meets the requirements.

[0068] Performance optimization: During operation, the system CPU usage dropped to around 10%, and the memory usage dropped to around 30%. The model runs at a stable frequency of once every 2 seconds, meeting system requirements.

[0069] The test indicators include the following:

[0070] CPU usage: recorded by vmstat, top, and MonitorPro plug-in;

[0071] Memory usage: recorded by vmstat and MonitorPro plug-in;

[0072] Model running frequency: record the number of model runs and the time interval through logs;

[0073] 1. Task Scheduling Optimization Test Results

[0074] Test method: In the pre-optimization environment, the system runs 1019 models and uses the Funboost framework to run these models. After the model is started and the system resource usage rate stabilizes, use vmstat to record the CPU usage and free memory amount every 3 seconds for 20 times. The vmstat output results are shown in the attached figure. Figure 2 At the same time, use top and MonitorPro plug-ins to calculate the memory usage and get the average value as the test result. The test result is shown in the attached figure. Figure 3 The data was recorded again after 30 minutes and 1 hour of running, and compared with the data when using the Celery framework.

[0075] 2. Model running frequency test results

[0076] Test method: This paper uses the Funboost framework to run the model and records the model's running frequency to ensure that it meets the predetermined running interval (such as running once every 2 seconds). By logging the number of model runs and the time interval, the stability of the model operation and the frequency control effect are verified. The Funboost framework model running frequency log is as follows Figure 4 shown.

[0077] Test results: Over the course of more than an hour, the model's run frequency remained essentially within the required two-second interval. Log records indicate that the model's run frequency and intervals met expectations, and overall stability met requirements.

[0078] The Funboost framework of the present invention can effectively control the model running frequency, ensure that the model runs at predetermined intervals, and meet the demand of thermal power system for real-time monitoring.

[0079] This embodiment significantly reduces system resource usage and improves operational efficiency by optimizing task scheduling and model deployment. Specific performance is as follows:

[0080] Resource usage: The Funboost framework is significantly better than Celery in terms of CPU and memory usage, and can effectively reduce the system burden.

[0081] Model running frequency: Funboost can accurately control the model running frequency to meet the real-time monitoring needs of thermal power systems.

[0082] The above specific embodiments are merely explanations of the present invention and are not limitations of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the embodiments as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by patent law.

Claims

1. A massive machine learning model inference distributed operation optimization system, characterized by: Includes task scheduling module, model deployment module and performance optimization module; The task scheduling module creates a corresponding Booster instance for the task function through the improved Funboost framework. The Booster instance is responsible for receiving, storing, distributing tasks and controlling the frequency of task execution. The model deployment module searches for the model through a function, determines whether the model exists, calls the pre-parameter processing function, writes the processed model parameters to the database, and publishes the task through the Booster instance; The performance optimization module reads data from InfluxDB and converts it into a NumPy object array, processes the data in batches using NumPy's matrix operation capability, and writes the calculation results into the object and stores them in InfluxDB.

2. The system according to claim 1, wherein: The task scheduling module supports multiple concurrent modes such as multi-threading, multi-process and coroutine, and can automatically adjust the size of the thread pool or process pool according to the number of tasks; The task scheduling module achieves efficient scheduling and execution of tasks by configuring queue name, task execution times per second, concurrency mode, concurrency quantity, whether to use distributed frequency control, and middleware type parameters.

3. The system according to claim 1, wherein: The model deployment module dynamically stores and updates model parameters through a database, supporting real-time adjustment of the model operation frequency; The model deployment module controls the model running time interval through the sleep mechanism to avoid excessive resource occupation.

4. The system according to claim 1, wherein: The system is applied in the field of industrial big data processing, especially in the real-time monitoring and fault warning of thermal power systems; The system significantly reduces system resource usage and improves operational efficiency by optimizing task scheduling, model deployment, and intermediate variable filling.

5. The system according to claim 1, wherein: The Booster instance of the Funboost framework supports dynamic scheduling and frequency control of tasks, and can dynamically adjust the execution strategy of tasks according to task priority and resource usage; the Booster instance implements persistent storage and grouped consumption of tasks through data structures.

Images