Elastic concurrent AI model optimization productivity acceleration middle table

Abstract

The invention relates to an elastic concurrent AI (artificial intelligence) model optimization productivity acceleration middle station, which comprises a resource management and scheduling module, a data processing and storage module, an elastic concurrent control module, a productivity optimization module, a system monitoring and feedback module and a user interaction and management module, the data processing and storage module is used for preprocessing and storing data, the elastic concurrency control module is used for elastically adjusting system resources and task processing capacity, the productivity optimization module is used for accelerating task distribution and result output, and the system monitoring and feedback module is used for monitoring the system state and providing feedback. And the user interaction and management module is used for providing high-efficiency middle station management. The system has the beneficial effect that the system shows extremely high adaptation capability in various scenes such as multi-task parallel, burst load processing and high-reliability storage.

Description

Technical Field

[0001] The invention relates to the field of intelligent services and data processing, and specifically to a flexible and concurrent AI model optimization productivity acceleration middle platform. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, more and more companies and organizations have begun to apply AI models to various business fields for intelligent decision-making, automated processing and data analysis. However, the development, deployment and operation and maintenance of AI models still face many challenges, especially in multi-task, high-load and resource-intensive application scenarios. How to efficiently manage and schedule computing resources, accelerate task processing, optimize system performance, and ensure system stability and availability are still key issues that need to be urgently addressed in AI middleware systems.

[0003] Most existing AI middleware systems focus on optimization in a single field, such as training acceleration, resource management or task scheduling of a single model, and lack global, multi-dimensional optimization solutions. Most traditional AI systems tend to adopt fixed resource allocation and task scheduling strategies, which cannot cope with real-time changing loads and complex business needs. In this case, the system is susceptible to resource bottlenecks, resulting in task delays or resource waste, which in turn affects the overall productivity improvement. In addition, due to the increasing computational complexity of AI model training and reasoning tasks, the processing capacity of a single hardware resource can no longer meet the needs of efficient concurrency. How to achieve flexible concurrent allocation of hardware resources has become the key to promoting the performance improvement of AI middleware.

[0004] Under the existing AI middle-end technology framework, the intelligence level of resource management, task scheduling and optimization is still relatively low. Many systems are prone to uneven resource allocation, waste of computing resources and task conflicts when facing multi-tasking, resulting in low overall processing efficiency. Existing technologies rely on fixed resource configuration and preset scheduling rules when dealing with large-scale concurrent tasks. They lack real-time perception and adaptability of system operating status and task load. The lack of an effective elastic scheduling mechanism makes it impossible for the system to dynamically adjust resource configuration according to load changes, further leading to inefficient use of computing resources. At the same time, traditional feedback mechanisms mostly focus on feedback from a single task, lack of linkage optimization between multiple tasks and multiple modules, resulting in the inability to effectively perform cross-module collaborative optimization when bottlenecks occur during task processing.

[0005] In addition, with the continuous in-depth application of AI technology, especially in large-scale data processing and real-time reasoning, the task execution process is becoming increasingly complex, and the demand for cross-task and cross-module collaboration is also increasing. In this context, how to efficiently handle a large number of heterogeneous tasks, cross-module collaboration, dynamic task allocation and resource scheduling has become a difficult problem that needs to be urgently solved by the AI ​​middle platform. Existing systems often lack efficient task allocation and scheduling strategies, which makes it difficult to efficiently parse and process the dependencies between different tasks, thereby affecting the execution efficiency of the task and the overall productivity of the system.

[0006] In addition, the management and interaction methods of the AI ​​middle platform also face great challenges. Traditional AI middle platform systems mostly use fixed operating interfaces and interaction methods, lack intelligent identification and response to user needs, and the user experience is often relatively simple and rigid. For the operation and maintenance personnel of the AI ​​middle platform, how to monitor and schedule a large number of tasks in real time, analyze system performance, and respond to system bottlenecks and failures in a timely manner is also an issue that cannot be ignored. Most existing AI middle platforms rely on manual intervention for system monitoring and troubleshooting, lack intelligent feedback mechanisms and adaptive optimization capabilities, resulting in low problem troubleshooting efficiency and serious waste of resources. Summary of the invention

[0007] In response to the above problems, the present invention aims to provide a flexible and concurrent AI model optimization productivity acceleration middle platform to solve the problems raised in the above background technology.

[0008] The purpose of the present invention is achieved through the following technical solutions:

[0009] The present invention provides a flexible and concurrent AI model optimization productivity acceleration middle platform, including a resource management and scheduling module, a data processing and storage module, a flexible and concurrent control module, a productivity optimization module, a system monitoring and feedback module, and a user interaction and management module. The resource management and scheduling module is used to efficiently allocate and schedule system resources. By real-time monitoring of the CPU, GPU, memory, storage and network bandwidth indicators of computing resources, a dynamic state map is constructed using multi-dimensional resource data to reflect the current resource load and distribution of the system. The data processing and storage module is used to preprocess and store data. The flexible and concurrent control module is used to flexibly adjust system resources and task processing capabilities, and adopts a cold and hot data separation strategy to improve data storage efficiency and access speed. The productivity optimization module combines hardware and software optimization strategies to provide performance acceleration for different types of tasks, and is used to accelerate task distribution and result output. The system monitoring and feedback module is used to monitor the system status and provide feedback. The user interaction and management module is used to provide efficient middle platform management.

[0010] Furthermore, the resource management and scheduling module monitors the CPU, GPU, memory, storage and network bandwidth indicators of computing resources in real time, and uses multi-dimensional resource data to build a dynamic status map to reflect the current resource load and distribution of the system.

[0011] Furthermore, the resource management and scheduling module ensures fairness and traceability of multi-tenant resource use by using a resource allocation record system based on blockchain technology. In the face of sudden resource demands, a rapid elastic expansion mechanism is adopted, as follows:

[0012] S1. Predict system resource requirements:

[0013] The system load at time t is Load(t), and the resource demand is Request(t). The resource demand depends on the historical load data and task complexity. A prediction model based on time series analysis and machine learning is introduced. The set of historical loads is: {Load(t-1), Load(t-2), ...}, which is converted into future resource demand prediction. It is expressed as:

[0014]

[0015] Among them, ARIMA difference integrated moving average autoregressive function, Load(t) is the load at time t, Load(t-1) is the load at time t-1, Load(t-2) is the load at time t-2, Load(tk) is the load at time tk, k is the length of the historical window, ε is the prediction error, and a specific weight factor w is introduced into the prediction model. i , considering the impact of task loads at different time points on current demand, the resource demand forecast is optimized as follows:

[0016]

[0017] in, The resource demand forecast for optimization, i.e., the expected value of allocation g(C) is the task complexity correction function, w i is the weight of the complexity correction function, which is dynamically adjusted based on the computing requirements of the real-time monitoring task;

[0018] S2: Rapid resource allocation:

[0019] In order to achieve rapid expansion, the resource allocation module allocates the predicted resource requirements to each computing node through a distributed resource allocation algorithm. In the system, there are N computing nodes S1, S2, ..., S N In the case of , the allocation goal is to minimize the delay of resource allocation and the resource waste of computing nodes. The optimization goal of resource allocation can be modeled as:

[0020]

[0021] Request i is assigned to node S i The actual amount of resources, is the expected value of allocation. By introducing the Lagrange multiplier method, the total resource constraint is added to the optimization objective

[0022]

[0023] Where La( ) is the Lagrangian function, λ is the Lagrangian multiplier, and the partial derivative of the Lagrangian function is obtained to obtain the optimization condition:

[0024]

[0025] Among them, Request total is the sum of resources allocated to all nodes;

[0026] S3. Load-aware resource release:

[0027] In the process of resource release, the real-time load of nodes is monitored to identify nodes with low resource utilization and dynamically release excess resources. i The real-time load is Load i (t), its resource utilization rate U i (t) is:

[0028]

[0029] The system sets the resource release threshold θ. When U i When (t)<θ, the resource release logic is triggered. The optimization goal of resource release is to minimize the impact on task execution time. The objective function is:

[0030]

[0031] Where, ΔT i It is the additional delay of the task caused by resource release, and dynamic programming is used to solve the resource release plan;

[0032] S4. Mechanism expansion and dynamic adjustment:

[0033] In order to adapt to the real-time changing system status, the fast elastic expansion mechanism is dynamically adjusted. By monitoring resource utilization efficiency and task execution performance, the expansion parameters are dynamically adjusted, including:

[0034]

[0035] Among them, α and β are weight parameters, It is task Tj Delay time, M is the total number of tasks, and according to the feedback value F(t), the rate of resource expansion and release is adjusted to ensure that the system is always in an efficient state.

[0036] Furthermore, the data processing and storage module performs efficient preprocessing and persistent storage on the input data, so that the data flow in the system has low latency, reliability and consistency. At the storage level, a hot and cold data separation strategy is adopted to improve the storage efficiency and access speed of the data. The dynamic tiering method based on access frequency prediction stores the hot data with high frequency access on the NVMe SSD high-performance storage medium, while the cold data with low frequency access is transferred to the low-cost mechanical hard disk storage medium with larger capacity. This tiered storage mechanism combines time series analysis and machine learning technology, and can dynamically adjust the distribution of data between different storage media, thereby greatly reducing storage costs and improving data access performance. At the same time, the module supports high-throughput writing and query optimization of real-time data streams, and uses in-memory computing technology and pre-compiled query accelerators to complete large-scale data reading and writing tasks in a high-concurrency environment within milliseconds.

[0037] Furthermore, in order to solve the data consistency and disaster recovery problems in the distributed storage environment, the data processing and storage module integrates a distributed data consensus protocol. The protocol records data operations through a distributed ledger in the blockchain network, so that even if some nodes fail or the network is partitioned, the system can still ensure data consistency and reliability.

[0038] Furthermore, the elastic concurrency control module implements collaborative scheduling between the task layer, node layer and system layer based on the multi-level elastic strategy of tasks and resources. The task layer elastic strategy optimizes the task granularity, dynamically adjusts the size of the task execution unit, divides large tasks into multiple microtasks, and executes them in parallel when resources are idle, thereby making full use of computing power; for short-term high-priority tasks, resource preemption and priority scheduling mechanisms are adopted to ensure the rapid processing of critical tasks. At the node layer, the module implements elastic scaling of computing resources through containerization technology, and dynamically increases or decreases the number of container instances according to the task load through the automatic expansion function of the Kubernetes container orchestration system, thereby meeting task requirements while minimizing resource waste. The system layer elastic strategy further integrates the resource distribution between nodes, and realizes system-level resource elasticity through cross-node task scheduling and load balancing.

[0039] Furthermore, in a multi-task parallel processing system, the system has M concurrent tasks T1, T2, ..., T M , it is necessary to have N computing nodes S1,S2,…,S N On scheduling and execution, each task T i The resource requirement is R i(t), the processing time is τ i , the total resource capacity of the computing node is C j , the goal is to minimize the following total cost function:

[0040]

[0041] Among them, J is the total cost function, a, b, r are weight parameters, mean() is the mean function, and there are the following constraints:

[0042]

[0043] Among them, formula ① indicates that the total resource demand of each node cannot exceed its capacity, and formula ② indicates that each task must be assigned to at least one node. The goal of resource allocation is to minimize load imbalance while satisfying the above constraints. The load balancing function is introduced

[0044]

[0045] There exists a gradient function:

[0046]

[0047] Among them, u, v are weights, λ1, λ2 are two independent Lagrange operators, and the priority of the task is determined by the dynamic weight P i (t), and the weight is determined by the task urgency Em i (t), estimated completion time and resource requirements R i (t) is:

[0048]

[0049] Using the priority queue algorithm, tasks are assigned to appropriate nodes, and the scheduling order meets the following optimization conditions:

[0050]

[0051] Among them, σ(T i ,S j ) is task T i Is it assigned to node S? j The optimization goal is achieved through dynamic programming:

[0052]

[0053] Furthermore, the productivity optimization module combines hardware and software optimization strategies to provide performance acceleration for different types of tasks. For compute-intensive tasks, the module uses GPU acceleration and model quantization technology to automatically select the optimal computing device and optimization parameters to significantly reduce task execution time. For input-output intensive tasks, the module uses data preloading and multi-level caching technology to complete data preparation before task start, reducing task waiting time. The productivity optimization module has cross-module collaboration capabilities. By sharing real-time data and task status information, each module can synchronously update the operation strategy. When the productivity optimization module detects that the task processing speed is lower than expected, it can send a resource expansion request to the elastic concurrency control module, or notify the data processing module to optimize the data loading strategy, thereby forming a dynamic collaborative and efficient operation system.

[0054] Furthermore, the system monitoring and feedback module adopts a full-stack monitoring technology based on distributed microservices, which realizes full coverage of real-time monitoring from hardware resources, operating system, network communication to application layer logic. By deploying lightweight monitoring agents on each node, it can collect CPU, GPU, memory, disk I / O, network bandwidth hardware indicators, as well as thread pool status, queue length, task execution time application layer data at millisecond granularity, and transmit the data to the central monitoring service for aggregation and analysis; in the feedback process, the system monitoring and feedback module analyzes the monitoring data in real time. When it is detected that the CPU usage of a node is too high, the module can trigger the resource management and scheduling module to reallocate the task load and notify the elastic concurrency control module to dynamically expand resources.

[0055] Furthermore, the user interaction and management module includes a context-aware dynamic interface generation engine that can dynamically adjust the interface layout and interactive elements based on the user's operation history, current task status and system resource status. When the user is processing resource-intensive tasks, the module will automatically simplify the interface to reduce the cognitive burden, and display the operation items in advance to ensure that the user can quickly complete the necessary operations. The module has a built-in intelligent assistant function that supports users to perform task scheduling, resource query, and system operations through natural language instructions, and automatically recommends resource allocation plans based on the user's task priority.

[0056] Beneficial effects of the present invention: The present invention uses a resource allocation record system based on blockchain technology in the resource management and scheduling module, and the resource management and scheduling module ensures the fairness and traceability of multi-tenant resource use. When facing sudden resource demands, a rapid elastic expansion mechanism is adopted, including predicting the resource demand of the system, rapid resource allocation, load-aware resource release, mechanism expansion and dynamic adjustment. In combination with ARIMA time series analysis and nonlinear regression, a multi-factor correction prediction model is adopted to improve the accuracy of resource demand prediction. Through Lagrangian optimization and distributed computing, the optimal resource allocation plan is quickly determined, and the delay of resource allocation is significantly reduced. Based on the real-time load of the node and the resource utilization rate, a load-aware resource release mechanism is designed to dynamically adjust resource allocation to avoid resource waste. By monitoring task delays and resource utilization efficiency, a feedback adjustment mechanism is introduced to ensure that the expansion strategy adapts to the dynamic changes of the system. By introducing elastic concurrency control mechanism and intelligent task scheduling algorithm, the system resource configuration can be adjusted dynamically, and elastic expansion or contraction can be performed according to real-time load changes, effectively avoiding the resource waste and task delay problems in the traditional fixed resource allocation method. This elastic scheduling and adaptive optimization greatly improves resource utilization and system stability, especially in high-load, multi-task concurrency scenarios, and can maintain the system's efficiency and responsiveness. The data processing and storage module adopts a cold and hot data separation strategy, combined with an access frequency prediction model, to store high-frequency access data on high-performance media, and low-frequency access data is transferred to low-cost media. This dynamic tiering mechanism effectively reduces storage costs while ensuring the rapidity of data access in a high-concurrency environment. In addition, data consistency and reliability are guaranteed through a distributed consensus protocol, which solves the problems of data loss and network partitioning in a distributed storage environment, laying the foundation for the stable operation of the system. The productivity optimization module combines hardware acceleration (such as GPU optimization) and software technology (such as model quantization and cache optimization) to dynamically select the optimal strategy according to the task type. Computation-intensive tasks are accelerated through parallel computing, and I / O-intensive tasks reduce waiting time through data preloading and multi-level caching. This targeted optimization effectively reduces the task execution time, making the system perform well when handling multiple tasks, and significantly improving adaptability and flexibility. The system monitoring and feedback module uses full-stack monitoring technology to achieve millisecond-level data collection and aggregate analysis from hardware to application layer. Combined with dynamic adjustment strategies, the system can respond to abnormal loads in a timely manner, automatically trigger resource allocation adjustments or task rescheduling, and ensure that the system is in an efficient and stable operating state for a long time. The feedback mechanism achieves a balance between response speed and adjustment accuracy, avoiding instability caused by excessive adjustment, and improving the self-healing ability of the system. The user interaction and management module uses a context-aware dynamic interface generation engine to adjust the interface layout in real time according to user operations and system status, simplifying complex operations and improving the intuitiveness and convenience of task execution.The integrated intelligent assistant supports natural language operation and automatically recommends resource allocation solutions, which lowers the technical threshold for users and significantly improves the user interaction experience. At the same time, through automatic analysis of task priorities, users can quickly complete key tasks and greatly improve operational efficiency.

[0057] The present invention demonstrates strong adaptability in various scenarios such as multi-task parallelism, burst load processing, and high-reliability storage. Through modular design and flexible configuration parameters, it can meet AI computing needs of different scales and complexities. It is suitable for data centers, cloud computing platforms, and edge computing scenarios, and provides solid technical support for diversified industry applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The invention is further illustrated by the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0059] Figure 1 It is a structural schematic diagram of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] See also Figure 1 The present invention will be further described in conjunction with the following examples.

[0062] See also Figure 1The present invention aims to provide a flexible and concurrent AI model optimization productivity acceleration middle platform, including a resource management and scheduling module, a data processing and storage module, a flexible concurrency control module, a productivity optimization module, a system monitoring and feedback module, and a user interaction and management module. The resource management and scheduling module is used to efficiently allocate and schedule system resources. By real-time monitoring of the CPU, GPU, memory, storage and network bandwidth indicators of computing resources, a dynamic state map is constructed using multi-dimensional resource data to reflect the current resource load and distribution of the system. The data processing and storage module is used to preprocess and store data. The flexible concurrency control module is used to flexibly adjust system resources and task processing capabilities, and adopts a cold and hot data separation strategy to improve data storage efficiency and access speed. The productivity optimization module combines hardware and software optimization strategies to provide performance acceleration for different types of tasks, which is used to accelerate task distribution and result output. The system monitoring and feedback module is used to monitor the system status and provide feedback. The user interaction and management module is used to provide efficient middle platform management.

[0063] Specifically, the resource management and scheduling module monitors the CPU, GPU, memory, storage and network bandwidth indicators of computing resources in real time, and uses multi-dimensional resource data to build a dynamic state map to reflect the current resource load and distribution of the system.

[0064] Specifically, the resource management and scheduling module ensures fairness and traceability of multi-tenant resource use by using a resource allocation record system based on blockchain technology. When faced with sudden resource demands, a rapid elastic expansion mechanism is adopted, as follows:

[0065] S1. Predict system resource requirements:

[0066] The system load at time t is Load(t), and the resource demand is Request(t). The resource demand depends on the historical load data and task complexity. A prediction model based on time series analysis and machine learning is introduced. The set of historical loads is: {Load(t-1), Load(t-2), ...}, which is converted into future resource demand prediction. It is expressed as:

[0067]

[0068] Among them, ARIMA difference integrated moving average autoregressive function, Load(t) is the load at time t, Load(t-1) is the load at time t-1, Load(t-2) is the load at time t-2, Load(tk) is the load at time tk, k is the length of the historical window, ε is the prediction error, and a specific weight factor w is introduced into the prediction model. i, considering the impact of task loads at different time points on current demand, the resource demand forecast is optimized as follows:

[0069]

[0070] in, is the optimal resource demand forecast, i.e., the expected value of allocation, g(C) is the task complexity correction function, w i is the weight of the complexity correction function, which is dynamically adjusted based on the computing requirements of the real-time monitoring task;

[0071] S2: Rapid resource allocation:

[0072] In order to achieve rapid expansion, the resource allocation module allocates the predicted resource requirements to each computing node through a distributed resource allocation algorithm. In the system, there are N computing nodes S1, S2, ..., S N In the case of , the allocation goal is to minimize the delay of resource allocation and the resource waste of computing nodes. The optimization goal of resource allocation can be modeled as:

[0073]

[0074] Request i is assigned to node S i The actual amount of resources, is the expected value of allocation. By introducing the Lagrange multiplier method, the total resource constraint is added to the optimization objective

[0075]

[0076] Where La( ) is the Lagrangian function, λ is the Lagrangian multiplier, and the partial derivative of the Lagrangian function is obtained to obtain the optimization condition:

[0077]

[0078] Among them, Request total is the sum of resources allocated to all nodes;

[0079] S3. Load-aware resource release:

[0080] In the process of resource release, the real-time load of nodes is monitored to identify nodes with low resource utilization and dynamically release excess resources. i The real-time load is Load i (t), its resource utilization rate U i (t) is:

[0081]

[0082] The system sets the resource release threshold θ. When Ui When (t)<θ, the resource release logic is triggered. The optimization goal of resource release is to minimize the impact on task execution time. The objective function is:

[0083]

[0084] Where, ΔT i It is the additional delay of the task caused by resource release, and dynamic programming is used to solve the resource release plan;

[0085] S4. Mechanism expansion and dynamic adjustment:

[0086] In order to adapt to the real-time changing system status, the fast elastic expansion mechanism is dynamically adjusted. By monitoring resource utilization efficiency and task execution performance, the expansion parameters are dynamically adjusted, including:

[0087]

[0088] Among them, α and β are weight parameters, It is task T j Delay time, M is the total number of tasks, and according to the feedback value F(t), the rate of resource expansion and release is adjusted to ensure that the system is always in an efficient state.

[0089] Specifically, the data processing and storage module performs efficient preprocessing and persistent storage on the input data, so that the data flow in the system has low latency, reliability and consistency. At the storage level, a hot and cold data separation strategy is adopted to improve the storage efficiency and access speed of the data. The dynamic tiering method based on access frequency prediction stores the hot data with high frequency access on the NVMe SSD high-performance storage medium, while the cold data with low frequency access is transferred to the larger capacity and low-cost mechanical hard disk storage medium. This tiered storage mechanism combines time series analysis and machine learning technology, and can dynamically adjust the distribution of data between different storage media, thereby greatly reducing storage costs and improving data access performance. At the same time, the module supports high-throughput writing and query optimization of real-time data streams, and uses memory computing technology and pre-compiled query accelerators to complete large-scale data reading and writing tasks in a high-concurrency environment within milliseconds.

[0090] Specifically, in order to solve the problems of data consistency and disaster recovery in a distributed storage environment, the data processing and storage module integrates a distributed data consensus protocol. The protocol records data operations through a distributed ledger in the blockchain network, so that even if some nodes fail or the network is partitioned, the system can still ensure data consistency and reliability.

[0091] Specifically, the elastic concurrency control module implements collaborative scheduling between the task layer, node layer and system layer based on the multi-level elastic strategy of tasks and resources. The task layer elastic strategy optimizes the task granularity, dynamically adjusts the size of the task execution unit, divides large tasks into multiple microtasks, and executes them in parallel when resources are idle, thereby making full use of computing power; for short-term high-priority tasks, resource preemption and priority scheduling mechanisms are adopted to ensure the rapid processing of critical tasks. At the node layer, the module implements elastic scaling of computing resources through containerization technology, and dynamically increases or decreases the number of container instances according to the task load through the automatic expansion function of the Kubernetes container orchestration system, thereby meeting task requirements while minimizing resource waste. The system layer elastic strategy further integrates the resource distribution between nodes, and realizes system-level resource elasticity through cross-node task scheduling and load balancing.

[0092] Specifically, in a multi-task parallel processing system, the system has M concurrent tasks T1, T2, ..., T M , it is necessary to have N computing nodes S1,S2,…,S N On scheduling and execution, each task T i The resource requirement is R i (t), the processing time is τ i , the total resource capacity of the computing node is C j , the goal is to minimize the following total cost function:

[0093]

[0094] Among them, J is the total cost function, a, b, r are weight parameters, mean() is the mean function, and there are the following constraints:

[0095]

[0096] Among them, formula ① indicates that the total resource demand of each node cannot exceed its capacity, and formula ② indicates that each task must be assigned to at least one node. The goal of resource allocation is to minimize load imbalance while satisfying the above constraints. The load balancing function is introduced

[0097]

[0098] There exists a gradient function:

[0099]

[0100] Among them, u, v are weights, λ1, λ2 are two independent Lagrange operators, and the priority of the task is determined by the dynamic weight P i (t), and the weight is determined by the task urgency Em i(t), estimated completion time and resource requirements R i (t) is:

[0101]

[0102] Using the priority queue algorithm, tasks are assigned to appropriate nodes, and the scheduling order meets the following optimization conditions:

[0103]

[0104] Among them, σ(T i ,S j ) is task T i Is it assigned to node S? j The optimization goal is achieved through dynamic programming:

[0105]

[0106] Specifically, the productivity optimization module combines hardware and software optimization strategies to provide performance acceleration for different types of tasks. For compute-intensive tasks, the module uses GPU acceleration and model quantization technology to automatically select the optimal computing device and optimization parameters to significantly reduce task execution time. For input-output intensive tasks, the module uses data preloading and multi-level caching technology to complete data preparation before task start, reducing task waiting time. The productivity optimization module has cross-module collaboration capabilities. By sharing real-time data and task status information, each module can synchronously update the operation strategy. When the productivity optimization module detects that the task processing speed is lower than expected, it can send a resource expansion request to the elastic concurrency control module, or notify the data processing module to optimize the data loading strategy, thereby forming a dynamic collaborative and efficient operation system.

[0107] Specifically, the system monitoring and feedback module adopts a full-stack monitoring technology based on distributed microservices, which realizes full coverage of real-time monitoring from hardware resources, operating system, network communication to application layer logic. By deploying lightweight monitoring agents on each node, it can collect CPU, GPU, memory, disk I / O, network bandwidth hardware indicators, as well as thread pool status, queue length, task execution time application layer data at millisecond granularity, and transmit the data to the central monitoring service for aggregation and analysis; in the feedback process, the system monitoring and feedback module analyzes the monitoring data in real time. When it is detected that the CPU usage of a node is too high, the module can trigger the resource management and scheduling module to reallocate the task load and notify the elastic concurrency control module to dynamically expand resources.

[0108] Specifically, the user interaction and management module includes a context-aware dynamic interface generation engine that can dynamically adjust the interface layout and interactive elements based on the user's operation history, current task status and system resource status. When the user is processing resource-intensive tasks, the module will automatically simplify the interface to reduce the cognitive burden, and display the operation items in advance to ensure that the user can quickly complete the necessary operations. The module has a built-in intelligent assistant function that supports users to perform task scheduling, resource query, and system operations through natural language instructions, and automatically recommends resource allocation plans based on the user's task priority.

[0109] Beneficial effects of the present invention: The present invention uses a resource allocation record system based on blockchain technology in the resource management and scheduling module, and the resource management and scheduling module ensures the fairness and traceability of multi-tenant resource use. When facing sudden resource demands, a rapid elastic expansion mechanism is adopted, including predicting the resource demand of the system, rapid resource allocation, load-aware resource release, mechanism expansion and dynamic adjustment. In combination with ARIMA time series analysis and nonlinear regression, a multi-factor correction prediction model is adopted to improve the accuracy of resource demand prediction. Through Lagrangian optimization and distributed computing, the optimal resource allocation plan is quickly determined, and the delay of resource allocation is significantly reduced. Based on the real-time load of the node and the resource utilization rate, a load-aware resource release mechanism is designed to dynamically adjust resource allocation to avoid resource waste. By monitoring task delays and resource utilization efficiency, a feedback adjustment mechanism is introduced to ensure that the expansion strategy adapts to the dynamic changes of the system. By introducing elastic concurrency control mechanism and intelligent task scheduling algorithm, the system resource configuration can be adjusted dynamically, and elastic expansion or contraction can be performed according to real-time load changes, effectively avoiding the resource waste and task delay problems in the traditional fixed resource allocation method. This elastic scheduling and adaptive optimization greatly improves resource utilization and system stability, especially in high-load, multi-task concurrency scenarios, and can maintain the system's efficiency and responsiveness. The data processing and storage module adopts a cold and hot data separation strategy, combined with an access frequency prediction model, to store high-frequency access data on high-performance media, and low-frequency access data is transferred to low-cost media. This dynamic tiering mechanism effectively reduces storage costs while ensuring the rapidity of data access in a high-concurrency environment. In addition, data consistency and reliability are guaranteed through a distributed consensus protocol, which solves the problems of data loss and network partitioning in a distributed storage environment, laying the foundation for the stable operation of the system. The productivity optimization module combines hardware acceleration (such as GPU optimization) and software technology (such as model quantization and cache optimization) to dynamically select the optimal strategy according to the task type. Computation-intensive tasks are accelerated through parallel computing, and I / O-intensive tasks reduce waiting time through data preloading and multi-level caching. This targeted optimization effectively reduces the task execution time, making the system perform well when handling multiple tasks, and significantly improving adaptability and flexibility. The system monitoring and feedback module uses full-stack monitoring technology to achieve millisecond-level data collection and aggregate analysis from hardware to application layer. Combined with dynamic adjustment strategies, the system can respond to abnormal loads in a timely manner, automatically trigger resource allocation adjustments or task rescheduling, and ensure that the system is in an efficient and stable operating state for a long time. The feedback mechanism achieves a balance between response speed and adjustment accuracy, avoiding instability caused by excessive adjustment, and improving the self-healing ability of the system. The user interaction and management module uses a context-aware dynamic interface generation engine to adjust the interface layout in real time according to user operations and system status, simplifying complex operations and improving the intuitiveness and convenience of task execution.The integrated intelligent assistant supports natural language operation and automatically recommends resource allocation solutions, which lowers the technical threshold for users and significantly improves the user interaction experience. At the same time, through automatic analysis of task priorities, users can quickly complete key tasks and greatly improve operational efficiency.

[0110] The present invention demonstrates strong adaptability in various scenarios such as multi-task parallelism, burst load processing, and high-reliability storage. Through modular design and flexible configuration parameters, it can meet AI computing needs of different scales and complexities. It is suitable for data centers, cloud computing platforms, and edge computing scenarios, and provides solid technical support for diversified industry applications.

[0111] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to replace some of the technical features therein. Any modification, replacement, and improvement made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A flexible and concurrent AI model optimization productivity acceleration middle platform, including a resource management and scheduling module, a data processing and storage module, a flexible concurrency control module, a productivity optimization module, a system monitoring and feedback module, and a user interaction and management module. The resource management and scheduling module is used to efficiently allocate and schedule system resources. By real-time monitoring of the CPU, GPU, memory, storage and network bandwidth indicators of computing resources, a dynamic state map is constructed using multi-dimensional resource data to reflect the current resource load and distribution of the system. The data processing and storage module is used to pre-process and store data. The flexible concurrency control module is used to flexibly adjust system resources and task processing capabilities, and adopts a cold and hot data separation strategy to improve data storage efficiency and access speed. The productivity optimization module combines hardware and software optimization strategies to provide performance acceleration for different types of tasks, and is used to accelerate task distribution and result output. The system monitoring and feedback module is used to monitor the system status and provide feedback. The user interaction and management module is used to provide efficient middle platform management.

2. According to claim 1, a flexible and concurrent AI model optimization productivity acceleration platform is characterized in that: The resource management and scheduling module monitors the CPU, GPU, memory, storage and network bandwidth indicators of computing resources in real time, and uses multi-dimensional resource data to build a dynamic state map to reflect the current resource load and distribution of the system.

3. According to claim 2, a flexible and concurrent AI model optimization productivity acceleration platform is characterized in that: The resource management and scheduling module ensures fairness and traceability of multi-tenant resource use by using a resource allocation record system based on blockchain technology. When faced with sudden resource demands, a rapid elastic expansion mechanism is adopted, as follows: S1. Predict system resource requirements: The system load at time t is Load(t), and the resource demand is Request(t). The resource demand depends on the historical load data and task complexity. A prediction model based on time series analysis and machine learning is introduced. The set of historical loads is: {Load(t-1), Load(t-2), ...}, which is converted into future resource demand prediction. It is expressed as: Among them, ARIMA difference integrated moving average autoregressive function, Load(t) is the load at time t, Load(t-1) is the load at time t-1, Load(t-2) is the load at time t-2, Load(tk) is the load at time tk, k is the length of the historical window, ε is the prediction error, and a specific weight factor w is introduced into the prediction model. i , considering the impact of task loads at different time points on current demand, the resource demand forecast is optimized as follows: in, is the optimal resource demand forecast, i.e., the expected value of allocation, g(C) is the task complexity correction function, w i is the weight of the complexity correction function, which is dynamically adjusted based on the computing requirements of the real-time monitoring task; S2: Rapid resource allocation: In order to achieve rapid expansion, the resource allocation module allocates the predicted resource requirements to each computing node through a distributed resource allocation algorithm. In the system, there are N computing nodes S1, S2, ..., S N In the case of , the allocation goal is to minimize the delay of resource allocation and the resource waste of computing nodes. The optimization goal of resource allocation can be modeled as: Request i is assigned to node S i The actual amount of resources, is the expected value of allocation. By introducing the Lagrange multiplier method, the total resource constraint is added to the optimization objective Where La() is the Lagrangian function, λ is the Lagrangian multiplier, and the partial derivative of the Lagrangian function is obtained to obtain the optimization condition: Among them, Request total is the sum of resources allocated to all nodes; S3. Load-aware resource release: In the process of resource release, the real-time load of nodes is monitored to identify nodes with low resource utilization and dynamically release excess resources. i The real-time load is Load i (t), its resource utilization rate U i (t) is: The system sets a resource release threshold θ. When U i When (t)<θ, the resource release logic is triggered. The optimization goal of resource release is to minimize the impact on task execution time. The objective function is: Where, ΔT i It is the additional delay of the task caused by resource release, and dynamic programming is used to solve the resource release plan; S4. Mechanism expansion and dynamic adjustment: In order to adapt to the real-time changing system status, the fast elastic expansion mechanism is dynamically adjusted. By monitoring resource utilization efficiency and task execution performance, the expansion parameters are dynamically adjusted, including: Among them, α and β are weight parameters, It is task T j Delay time, M is the total number of tasks, and according to the feedback value F(t), the rate of resource expansion and release is adjusted to ensure that the system is always in an efficient state.

4. According to claim 1, a flexible and concurrent AI model optimization productivity acceleration platform is characterized in that: The data processing and storage module performs efficient preprocessing and persistent storage on the input data, so that the data flow in the system has low latency, reliability and consistency. At the storage level, a hot and cold data separation strategy is adopted to improve the storage efficiency and access speed of the data. The dynamic tiering method based on access frequency prediction stores the hot data with high frequency access on the NVMe SSD high-performance storage medium, while the cold data with low frequency access is transferred to the mechanical hard disk storage medium with larger capacity and low cost. This tiered storage mechanism combines time series analysis and machine learning technology, and can dynamically adjust the distribution of data between different storage media, thereby greatly reducing storage costs and improving data access performance. At the same time, the module supports high-throughput writing and query optimization of real-time data streams, and uses memory computing technology and pre-compiled query accelerators to complete large-scale data reading and writing tasks in a high-concurrency environment within milliseconds.

5. According to claim 1, a flexible and concurrent AI model optimization productivity acceleration platform is characterized in that: In order to solve the data consistency and disaster recovery problems in the distributed storage environment, the data processing and storage module integrates a distributed data consensus protocol. The protocol records data operations through the distributed ledger in the blockchain network, so that even if some nodes fail or the network is partitioned, the system can still ensure data consistency and reliability.

6. According to claim 1, a flexible and concurrent AI model optimization productivity acceleration platform is characterized in that: The elastic concurrency control module is based on a multi-level elastic strategy for tasks and resources to achieve collaborative scheduling between the task layer, node layer, and system layer. The task layer elastic strategy optimizes the task granularity, dynamically adjusts the size of the task execution unit, divides large tasks into multiple microtasks, and executes them in parallel when resources are idle, thereby making full use of computing power; for short-term high-priority tasks, resource preemption and priority scheduling mechanisms are used to ensure the rapid processing of critical tasks. At the node layer, the module achieves elastic scaling of computing resources through containerization technology, and dynamically increases or decreases the number of container instances according to the task load through the automatic expansion function of the Kubernetes container orchestration system, thereby meeting task requirements while minimizing resource waste. The system layer elastic strategy further integrates the resource distribution between nodes, and achieves system-level resource elasticity through cross-node task scheduling and load balancing.

7. According to claim 6, a flexible and concurrent AI model optimization productivity acceleration platform, characterized in that: In a multi-task parallel processing system, the system has M concurrent tasks T1, T2, ..., T M , it is necessary to have N computing nodes S1,S2,…,S N On scheduling and execution, each task T i The resource requirement is R i (t), the processing time is τ i , the total resource capacity of the computing node is C j , the goal is to minimize the following total cost function: Among them, J is the total cost function, a, b, r are weight parameters, mean() is the mean function, and there are the following constraints: Among them, formula ① indicates that the total resource demand of each node cannot exceed its capacity, and formula ② indicates that each task must be assigned to at least one node. The goal of resource allocation is to minimize load imbalance while satisfying the above constraints. The load balancing function is introduced There exists a gradient function: Among them, u, v are weights, λ1, λ2 are two independent Lagrange operators, and the priority of the task is determined by the dynamic weight P i (t), and the weight is determined by the task urgency Em i (t), estimated completion time and resource requirements R i (t) is: Using the priority queue algorithm, tasks are assigned to appropriate nodes, and the scheduling order meets the following optimization conditions: Among them, σ(T i ,S j ) is task T i Is it assigned to node S? j The optimization goal is achieved through dynamic programming:

8. According to claim 1, a flexible and concurrent AI model optimization productivity acceleration platform is characterized in that: The productivity optimization module combines hardware and software optimization strategies to provide performance acceleration for different types of tasks. For compute-intensive tasks, the module uses GPU acceleration and model quantization technology to automatically select the optimal computing device and optimization parameters to significantly reduce task execution time. For input-output intensive tasks, the module uses data preloading and multi-level caching technology to complete data preparation before task start, reducing task waiting time. The productivity optimization module has cross-module collaboration capabilities. By sharing real-time data and task status information, each module can synchronously update the operation strategy. When the productivity optimization module detects that the task processing speed is lower than expected, it can send a resource expansion request to the elastic concurrency control module, or notify the data processing module to optimize the data loading strategy, thereby forming a dynamic collaborative and efficient operation system.

9. According to claim 1, a flexible and concurrent AI model optimization productivity acceleration platform, characterized in that: The system monitoring and feedback module adopts full-stack monitoring technology based on distributed microservices, and realizes full coverage of real-time monitoring from hardware resources, operating system, network communication to application layer logic. By deploying lightweight monitoring agents on each node, it can collect CPU, GPU, memory, disk I / O, network bandwidth hardware indicators, as well as thread pool status, queue length, task execution time application layer data at millisecond granularity, and transmit the data to the central monitoring service for aggregation and analysis; in the feedback process, the system monitoring and feedback module analyzes the monitoring data in real time. When it is detected that the CPU usage of a node is too high, the module triggers the resource management and scheduling module to reallocate the task load and notify the elastic concurrency control module to dynamically expand resources.

10. According to claim 1, a flexible and concurrent AI model optimization productivity acceleration platform, characterized in that: The user interaction and management module includes a context-aware dynamic interface generation engine that can dynamically adjust the interface layout and interactive elements based on the user's operation history, current task status and system resource status. When the user is processing resource-intensive tasks, the module will automatically simplify the interface to reduce the cognitive burden, and display the operation items in advance to ensure that the user can quickly complete the necessary operations. The module has a built-in intelligent assistant function that supports users to perform task scheduling, resource query, and system operations through natural language instructions, and automatically recommends resource allocation plans based on the user's task priority.

Images