AIGC high-availability elastic calculation system and method

Abstract

The invention provides an AIGC high-availability elastic computing system and method, which are used for solving the problems of high concurrency, low resource utilization rate, poor expandability and the like faced by AIGC service. The system comprises an API server, message middleware, a plurality of computing nodes, a monitoring system and a management system. And the computing node actively pulls the task from the task queue according to the processing capacity of the computing node, selects proper computing power resources after analyzing the demand, and executes the AIGC task in the containerization environment. The monitoring system tracks the state of each component in real time, and the management system dynamically adjusts resource configuration according to monitoring data. The method has the advantages that the concurrent processing capacity and the resource utilization rate of the system are improved, elastic stretching and intelligent operation and maintenance are achieved, the fault-tolerant capacity of the system is enhanced, meanwhile, the operation cost is reduced, and an efficient, reliable and economical service platform is provided for AIGC application scenes of various scales.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence computing services, and in particular to a high-availability elastic computing system for artificial intelligence generated content (AIGC) tasks and a processing method thereof. Background Art

[0002] Artificial intelligence generated content (AIGC) technology has developed rapidly in recent years and has important applications in content creation, design, entertainment and other fields. AIGC technology can quickly generate multimedia content such as text, images, audio and video, greatly improving the efficiency of creation and bringing revolutionary changes to all walks of life. With the continuous advancement and popularization of AIGC technology, the demand for efficient and reliable AIGC computing services is also growing.

[0003] At present, the traditional cloud computing architecture is mainly used to support AIGC services. This architecture usually includes components such as load balancers, application server clusters, and backend databases. User requests are first distributed to application servers through load balancers, and then the application servers call the corresponding AI models to generate content and finally return the results to users. Another common solution is to use the Serverless architecture to encapsulate AIGC tasks as functions and execute them on demand through the FaaS (Function as a Service) platform.

[0004] However, these solutions have some problems when dealing with AIGC tasks. First, AIGC tasks are usually computationally intensive and have large fluctuations in resource requirements. Traditional architectures are difficult to effectively respond to sudden high-concurrency requests, which can easily cause system overload or resource waste. Second, different types of AIGC tasks have different requirements for computing resources, and fixed resource allocation methods are difficult to meet diverse task requirements. In addition, there is also room for optimization in task scheduling and resource utilization, resulting in low overall system efficiency and high operating costs.

[0005] To solve the above problems, a study proposed a microservice-based AIGC computing architecture, which improves the flexibility and scalability of the system by splitting different types of AIGC tasks into independent microservices. Although this method improves resource utilization to a certain extent, it still faces challenges such as low scheduling efficiency and unstable service quality when dealing with large-scale and diverse AIGC tasks. At the same time, the complexity of the microservice architecture also increases the development and maintenance costs of the system.

[0006] Therefore, there is an urgent need for a computing system that can efficiently handle diverse AIGC tasks, has strong elastic scaling capabilities, and can optimize resource utilization. Developing such a system can not only improve the performance and reliability of AIGC services, but also reduce operating costs and provide solid infrastructure support for the widespread application of AIGC technology. This has become an important research direction and industry demand in the field of AIGC. Summary of the invention

[0007] The purpose of the present invention is to solve the problems of insufficient high-concurrency processing capability, low resource utilization, poor scalability, etc. existing in the existing AIGC computing service.

[0008] In addition, the present invention also aims to provide a highly available elastic computing system that can adapt to diverse AIGC task requirements, has intelligent operation and maintenance capabilities, and reduces operating costs.

[0009] To achieve the above objectives, the present invention provides an AIGC high-availability elastic computing system and method, which is characterized by: including an API server, a message middleware, multiple computing nodes, a monitoring system and a management system, and realizing efficient task scheduling and resource allocation through the active pull mechanism of task queues and computing nodes.

[0010] In some embodiments, the system can also dynamically select different types of computing resources, such as Serverless, virtual machines, or self-built clusters based on task characteristics.

[0011] Specifically, the API server is responsible for receiving AIGC task requests submitted by users, encapsulating the tasks into messages and publishing them to the task queue of the message middleware.

[0012] Furthermore, computing nodes actively pull tasks from the task queue based on their own processing capabilities, achieving more flexible load balancing.

[0013] Preferably, after analyzing the task requirements, the computing node can select the most suitable resource type from a variety of computing resources to meet the special needs of different AIGC tasks.

[0014] Optionally, the system uses containerization technology to execute AIGC tasks, improving resource isolation and utilization efficiency.

[0015] In one embodiment, the monitoring system tracks the status of each component in real time and triggers an alarm when an abnormality is found, and the management system dynamically adjusts the number of computing nodes and resource configuration based on this information.

[0016] In some embodiments, the message middleware is deployed in a multi-copy manner to ensure high availability of the system.

[0017] In addition, the present invention can also support automatic task retry and node failure transfer, further enhancing the fault tolerance of the system.

[0018] In other embodiments, the system supports task priority management to ensure that high-priority tasks are processed first to meet the needs of different business scenarios.

[0019] In a preferred embodiment, when the system receives an AIGC task request, the API server encapsulates the task into a message and publishes it to the task queue. The computing node actively pulls the task according to the current load situation, analyzes the task requirements and selects the appropriate computing resources (such as GPU-accelerated virtual machines). The task is executed in a containerized environment, and the results are uploaded to the cloud storage after completion. The computing node confirms the completion to the message middleware and releases the resources. During the whole process, the monitoring system continuously tracks the status of each component, and the management system dynamically adjusts the resource configuration according to the monitoring data to realize intelligent operation and maintenance.

[0020] By adopting the above scheme, the present invention has the following beneficial effects: (1) The system's concurrent processing capability has been significantly improved, and it can effectively handle high concurrent requests for AIGC services; (2) The resource utilization rate is greatly improved through the active task acquisition mechanism of computing nodes and the selection of diversified computing resources; (3) The system achieves elastic scaling and intelligent operation and maintenance, and can automatically adjust resource configuration according to load changes; (4) Enhanced the system's fault tolerance and improved service reliability and stability; (5) Reduced the operating costs of AIGC services while providing more flexible service options.

[0021] In summary, the present invention provides an efficient, reliable and economical AIGC high-availability elastic computing system and method, which not only solves the key problems in the prior art, but also provides strong infrastructure support for the wide application and innovation of AIGC technology, and has important practical value and promotion significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1This is the AIGC high-availability elastic computing system architecture diagram. It shows the main components of the system, including API server, message middleware, computing nodes, cloud storage service, monitoring system and management system, as well as the interaction between them.

[0024] Figure 2 It is the AIGC task processing flow chart. It describes in detail the entire process from user submission of a task to task completion, including steps such as task verification, queue management, resource allocation, task execution, result storage and notification.

[0025] Figure 3 It is a functional diagram of a computing node. It shows the internal structure of a computing node, including modules such as task acquisition, analysis, resource selection, execution, and result processing, as well as self-monitoring and adaptive control mechanisms.

[0026] Figure 4 It is a diversified computing resource scheduling diagram. It describes how the system manages and allocates different types of computing resources, including global scheduling and local scheduling strategies, as well as various types of computing instances.

[0027] Figure 5 It is a system monitoring and intelligent management diagram. It shows the monitoring and management architecture of the system, including data collection, processing, analysis, decision-making and execution, as well as adaptive control loops.

[0028] Figure 6 It is a high availability and fault tolerance mechanism diagram. It describes the various strategies adopted by the system to ensure high availability and fault tolerance, including data redundancy, service high availability, computing node fault tolerance, task execution guarantee, network partition processing and disaster recovery mechanisms. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as a limitation on the present invention.

[0031] Example 1: Basic system architecture.

[0032] Reference Figure 1 The AIGC high-availability elastic computing system provided by the present invention includes an API server 101, a message middleware 102, multiple computing nodes 103, a monitoring system 104, and a management system 105. These components are interconnected through a network to form an efficient and reliable AIGC task processing platform.

[0033] The API server 101 is the entry point of the system and is responsible for receiving AIGC task requests submitted by users. It adopts RESTful API design and supports multiple authentication methods (such as OAuth 2.0) to ensure security. The API server uses asynchronous I / O technology (such as Node.js or Python's asyncio) to handle a large number of concurrent requests, effectively improving the system's throughput.

[0034] The message middleware 102 includes a task queue 121 and a result queue 122. The task queue 121 is used to store the task messages encapsulated by the API server, and the result queue 122 is used to store the task execution results. In this embodiment, the message middleware is implemented using Apache Kafka, which has high throughput, low latency and reliable message delivery capabilities. Kafka's partitioning mechanism allows the system to scale horizontally to cope with the growing amount of tasks.

[0035] The computing node 103 is the core component of the system and is responsible for actually executing AIGC tasks. Each computing node is an independent computing unit, which can be a physical server, a virtual machine, or a container. The computing node adopts an active task acquisition mechanism to pull tasks from the task queue 121 according to its own processing power. This innovative design significantly improves resource utilization and avoids the node overload problem that may be caused by the traditional push mode.

[0036] The computing node 103 integrates a task analysis module 131, a resource selection module 132, and a task execution module 133. The task analysis module 131 is responsible for parsing the task requirements, including the required computing resources, expected execution time, etc. The resource selection module 132 selects the most suitable type from a variety of computing resources (such as Serverless functions, GPU acceleration instances, dedicated AI chips, etc.) according to the task requirements. The task execution module 133 is responsible for starting the containerized environment on the selected resources and executing the AIGC task. Containerization technology (such as Docker) ensures the consistency and isolation of the task execution environment, greatly improving the reliability and security of the system.

[0037] The monitoring system 104 tracks the operating status of each component in real time, including but not limited to key indicators such as CPU usage, memory usage, network throughput, and task queue length. It uses distributed tracing technology (such as Jaeger) to monitor complex task execution processes and uses a time series database (such as Prometheus) to store monitoring data. When an abnormality is found, the monitoring system will trigger an alarm and notify the operation and maintenance personnel through a preset notification channel (such as email, SMS).

[0038] The management system 105 is another innovative feature of the present invention. It dynamically adjusts the computing resource configuration based on monitoring data and system policies. The management system uses machine learning algorithms (such as time series analysis and anomaly detection) to predict task load trends and automatically adjust the number and specifications of computing nodes accordingly. This intelligent resource management mechanism greatly improves the elasticity and cost-effectiveness of the system.

[0039] In this embodiment, the system also integrates a cloud storage service 106 for storing the execution results of the AIGC task. The cloud storage uses object storage technology (such as Amazon S3 compatible interface) to provide high reliability and scalability.

[0040] The system's workflow is as follows: (1) The user submits an AIGC task request through the API server 101; (2) The API server encapsulates the task into a message and publishes it to the task queue 121 of the message middleware 102; (3) Computing node 103 actively pulls tasks from task queue 121; (4) The task analysis module 131 of the computing node analyzes the task requirements; (5) The resource selection module 132 selects appropriate computing resources; (6) The task execution module 133 starts the containerized environment on the selected resources and executes the AIGC task; (7) The task result is uploaded to the cloud storage service 106, and the result reference is sent to the result queue 122; (8) The computing node confirms to the message middleware that the task is completed and releases resources; (9) The monitoring system 104 tracks the status of each component throughout the process and triggers an alarm when an abnormality is found; (10) The management system 105 dynamically adjusts the number of computing nodes and resource allocation based on monitoring data and load forecasts.

[0041] The innovation of this embodiment lies in: the active task acquisition mechanism of computing nodes improves resource utilization and load balancing effect. The dynamic selection of diversified computing resources adapts to the special needs of different AIGC tasks. Intelligent resource management based on machine learning realizes the automatic elastic scaling of the system. Full-link monitoring and distributed tracing improve the observability and fault diagnosis capabilities of the system.

[0042] It should be noted that this embodiment is only one of the preferred implementations of the present invention, and those skilled in the art can make various modifications and improvements without departing from the essence of the present invention. For example, the message middleware can be replaced by other high-performance message queue systems (such as RabbitMQ), and the monitoring system can integrate more observability tools (such as ELK stack). These changes and improvements should be regarded as falling within the scope of protection of the present invention.

[0043] Example 2: Task processing flow.

[0044] Reference Figure 2 ,This embodiment describes in detail the task processing flow of the AIGC high availability elastic computing system, ,from the user submitting a task request to the completion of the task.

[0045] Users submit AIGC task requests through the API interface. The API server adopts RESTful design and supports multiple request methods. Task requests usually contain information such as task type, input data or parameters, task priority (optional), resource requirements (optional), and callback URL (optional).

[0046] After receiving the request, the API server performs request verification and user authentication. After authentication, the server generates a globally unique task ID and encapsulates the task information into a standardized task message. The task message uses a structured format and contains all necessary information as well as a timestamp and task status mark automatically generated by the system.

[0047] Subsequently, the API server publishes the encapsulated task message to the task queue of the message middleware. The task queue is designed to support multi-partition structure with parallel processing and load balancing. The computing node actively pulls tasks from the task queue and uses the consumer group mechanism to achieve balanced distribution of tasks.

[0048] The task analysis module of the computing node parses the task message, extracts key information such as task type, resource requirements, etc., and estimates the complexity and expected execution time of the task. Based on the analysis results, the resource selection module selects the most suitable resource type from the available computing resource pool. The selection strategy considers factors such as task type and resource requirements, current system load, and resource utilization efficiency and cost.

[0049] The task execution module starts the containerized environment on the selected resources. The container image contains the required AIGC models and dependent libraries in advance to ensure fast startup and environmental consistency. When executing AIGC tasks in the containerized environment, the system records the task status and performance indicators in real time. This information is collected and stored by the monitoring system for subsequent performance analysis and optimization.

[0050] After the task is completed, the result data is uploaded to the cloud storage service. The system generates a temporary URL with access rights and sends the task result information to the result queue. The computing node then confirms the task completion to the message middleware, updates the task status, and releases the used resources.

[0051] If the user provides a callback URL in the task request, the system will send a request containing the task result information to the URL to implement asynchronous notification. During the entire task processing process, the monitoring system continues to track the execution of each step, using distributed tracing technology to associate different processing stages of the same task, facilitating subsequent performance analysis and fault diagnosis.

[0052] The management system dynamically adjusts the system configuration based on monitoring data and historical statistics. For example, when a backlog of tasks is detected, the management system may increase the number of computing nodes or upgrade the configuration of existing nodes. This adaptive adjustment mechanism ensures that the system can maintain optimal performance when the load changes.

[0053] The innovations of this embodiment are mainly reflected in the full life cycle management of tasks, flexible resource selection mechanism, containerized execution environment, decoupling design based on message queues, and real-time monitoring and adaptive adjustment. Through this carefully designed task processing flow, the system can efficiently handle a large number of concurrent AIGC tasks while maintaining low operating costs.

[0054] Example 3: Computing node functions.

[0055] Reference Figure 3 This embodiment describes in detail the functions and working principles of computing nodes in the AIGC high-availability elastic computing system. The computing node is the core component of the system and is responsible for actually executing AIGC tasks. Its innovative design significantly improves the efficiency and flexibility of the system.

[0056] The computing node adopts a modular design, which mainly includes task acquisition module, task analysis module, resource selection module, task execution module and result processing module. This modular design improves the maintainability and scalability of the system, allowing each functional module to be independently upgraded and optimized.

[0057] The task acquisition module implements an innovative active task acquisition mechanism. Different from the traditional task push mode, the computing node actively pulls tasks from the task queue according to its own processing capacity. This mechanism effectively avoids the problems of unbalanced task distribution and node overload. The task acquisition module also implements an adaptive task pull strategy, dynamically adjusting the pull frequency and batch size according to the current node load status to optimize resource utilization efficiency.

[0058] The task analysis module is responsible for parsing task messages, extracting key information and evaluating task complexity. This module uses machine learning algorithms to predict the resource requirements and execution time of new tasks based on historical task execution data. This intelligent task analysis provides an important basis for subsequent resource selection and helps improve the accuracy and efficiency of resource allocation.

[0059] The resource selection module is another innovation of this system. It selects the most suitable type from a variety of computing resources based on the results of task analysis. The resource selection strategy considers multiple factors, including task type, estimated complexity, current system load, resource utilization efficiency, and operating costs. This module also implements dynamic resource pool management, which can dynamically adjust the type and quantity of available resources according to the overall system load.

[0060] The task execution module is responsible for starting the containerized environment on the selected resources and executing AIGC tasks. The application of containerization technology ensures the consistency and isolation of the task execution environment, greatly improving the reliability of the system. The module also implements real-time monitoring and resource limitation of task execution to prevent a single task from over-consuming system resources and affecting the execution of other tasks.

[0061] The result processing module is responsible for a series of operations after the task is completed, including uploading result data, updating task status, releasing resources, etc. This module also implements an intelligent result caching mechanism. For common task inputs, the system will cache the corresponding output results to improve response speed and reduce the consumption of computing resources.

[0062] An important feature of computing nodes is their adaptability. Through built-in performance monitoring and self-diagnosis mechanisms, computing nodes can adjust their behavior based on their own status and the overall situation of the system. For example, when the node detects that its own load is too high, it can actively reduce the frequency of task acquisition; when system resources are tight, the node can give priority to more resource-saving execution methods.

[0063] In addition, the computing nodes also implement fault tolerance and recovery mechanisms. When an abnormality in task execution is detected, the node can automatically retry or transfer the task to other nodes for processing. This mechanism significantly improves the reliability and stability of the system.

[0064] The computing nodes use a decentralized collaboration model. Through the distributed protocol, the nodes can exchange load information and resource status to achieve a more balanced and efficient task allocation. This design avoids single point failures and improves the scalability and fault tolerance of the system.

[0065] The innovation of this embodiment lies in: the active task acquisition mechanism improves resource utilization and load balancing effect. Intelligent task analysis and resource selection achieve more accurate resource allocation. Adaptive capability and decentralized collaboration enhance the flexibility and scalability of the system. The application of containerization technology improves the consistency and isolation of task execution. The intelligent result caching mechanism optimizes system performance and resource utilization.

[0066] Through these innovative designs, computing nodes can efficiently and flexibly process various AIGC tasks while maintaining high availability and scalability of the system. This design not only improves the overall performance of the system, but also significantly reduces operating costs.

[0067] Embodiment 4: Diversified computing resource scheduling.

[0068] Reference Figure 4 This embodiment elaborates on the scheduling mechanism of diversified computing resources in the AIGC high-availability elastic computing system. This mechanism is an important innovation of the present invention, which can select and schedule the most suitable computing resources according to the characteristics of different AIGC tasks, thereby optimizing resource utilization and maximizing system performance.

[0069] This system supports multiple types of computing resources, including general computing instances, GPU accelerated instances, FPGA instances, AI accelerators, Serverless functions, and dedicated AIGC clusters. These diverse resources provide a flexible execution environment for AIGC tasks of different types and complexities.

[0070] The system's resource scheduler adopts a multi-level scheduling strategy, including global scheduling and local scheduling. Global scheduling is responsible for the allocation and balance of overall resources, maintaining the global resource pool, and dynamically adjusting the ratio of various resources based on historical data and prediction models. Local scheduling occurs within each computing node and is responsible for selecting the most suitable computing resources for specific AIGC tasks.

[0071] In the resource selection process, the system considers multiple factors, including task type, complexity, performance requirements, resource utilization efficiency, cost factors, and current resource availability. In order to quantify these factors and make the best decision, the system uses a multi-objective optimization algorithm. Specifically, for task i and resource type j, the present invention defines a utility function U(i, j): U(i, j) = w1 * P(i, j) + w2 * E(i, j) - w3 * C(i, j) Among them, P(i, j) represents the performance score, E(i, j) represents the efficiency score, C(i, j) represents the cost score, and w1, w2, and w3 are the corresponding weight coefficients. The goal of the system is to find the resource type j that maximizes U(i, j) for each task i.

[0072] This system innovatively introduces a resource affinity mechanism. Through machine learning algorithms, the system can learn the affinity relationship between different types of AIGC tasks and various computing resources. Specifically, the present invention uses a collaborative filtering algorithm to predict the effect of task-resource matching. Let R be the task-resource matching matrix, and R_ij represents the performance score of task type i on resource type j. For a new task u and resource type v, the present invention can predict its matching score by the following formula: R_uv = μ_u + Σ(sim(u, i) * (R_iv - μ_i)) / Σ|sim(u, i)| Where μ_u is the average score of task u, and sim(u, i) is the similarity between tasks u and i.

[0073] Another innovation is the dynamic resource scaling mechanism. The system uses time series forecasting models (such as ARIMA) to predict future resource requirements and adjust the resource quantity accordingly. The general form of the forecasting model is: X_t = c + φ_1 X_(t-1) + ... + φ_p X_(tp) + ε_t + θ_1 ε_(t-1) + ... + θ_q ε_(tq) Among them, X_t is the resource demand at time t, c is a constant term, φ and θ are model parameters, and ε is white noise.

[0074] In order to deal with resource conflicts and priority issues, the system implements a multi-level queue scheduling algorithm. Tasks of different priorities are assigned to different queues, and tasks in high-priority queues can get resources first. At the same time, the system uses dynamic time slice allocation and resource preemption mechanisms to ensure that low-priority tasks can also be processed in a timely manner to avoid starvation.

[0075] The system also introduces a task migration mechanism. When a resource type is detected to be overloaded, the system can dynamically migrate some tasks to other types of resources for execution. Migration decisions are based on a load balancing algorithm, such as using the following load imbalance metrics: Imbalance = (L_max - L_avg) / L_avg Among them, L_max is the maximum load, and L_avg is the average load. When Imbalance exceeds the preset threshold, task migration is triggered.

[0076] In order to optimize long-term operating efficiency, the system uses a reinforcement learning algorithm to continuously optimize the scheduling strategy. Specifically, the present invention uses a Q-learning algorithm, the core of which is the formula for updating the Q value: Q(s, a) ← Q(s, a) + α[r + γ*max_a'Q(s', a') - Q(s, a)] Where s is the current state, a is the action, r is the immediate reward, α is the learning rate, γ is the discount factor, and s' is the next state.

[0077] Through these innovative designs, the system can efficiently manage and schedule diverse computing resources, providing the most suitable execution environment for various AIGC tasks.

[0078] Embodiment 5: System monitoring and intelligent management.

[0079] Reference Figure 5 This embodiment describes in detail the monitoring and intelligent management mechanism of the AIGC high-availability elastic computing system. This mechanism is one of the key innovations of the present invention, which realizes the adaptive adjustment and intelligent operation and maintenance of the system, greatly improving the reliability, efficiency and cost-effectiveness of the system.

[0080] The monitoring and management architecture of this system adopts a layered design, including data collection layer, data processing layer, analysis and decision layer, and execution layer. This layered design makes the system highly flexible and scalable, and can adapt to AIGC application scenarios of different scales and complexities.

[0081] The data collection layer is responsible for collecting real-time operation data from various components of the system. It uses distributed data collection technology and deploys lightweight agents on each computing node and key components. These agents use adaptive sampling rate algorithms to dynamically adjust the sampling frequency according to the system load and data change frequency, which not only ensures the real-time nature of the data, but also avoids unnecessary resource consumption. The collected data includes but is not limited to key indicators such as CPU usage, memory usage, network throughput, task queue length, response time, etc.

[0082] The data processing layer preprocesses and aggregates the collected raw data. It uses stream processing technology to process incoming data streams in real time, performing data cleaning, standardization, and preliminary aggregation. In order to process large-scale data, the system adopts a distributed stream processing framework. The core algorithm of data processing can be expressed as: ProcessedData = f(RawData) where f = Normalize ∘ Aggregate ∘ Clean Here, the Clean function handles outliers and missing data, the Aggregate function performs time window aggregation, and the Normalize function normalizes the data to a uniform scale.

[0083] The analytical decision layer is the core of the system intelligence. It uses a variety of advanced analytical techniques, including time series analysis, anomaly detection, predictive modeling and optimization algorithms, to conduct in-depth analysis of processed data and generate system insights and optimization suggestions.

[0084] Among them, the anomaly detection module uses a multivariate Gaussian distribution model. For a set of observation data x, its anomaly score is calculated as follows: anomaly_score(x) = (x - μ)^T Σ^(-1) (x - μ) Here, μ is the mean vector and Σ is the covariance matrix. If the anomaly score exceeds the preset threshold, the system will trigger an alarm.

[0085] The prediction module uses LSTM (Long Short-Term Memory) network for time series prediction. The core equation of LSTM unit is as follows: f_t = σ(W_f · [h_(t-1), x_t] + b_f) i_t = σ(W_i · [h_(t-1), x_t] + b_i) o_t = σ(W_o · [h_(t-1), x_t] + b_o) c_t = f_t * c_(t-1) + i_t * tanh(W_c · [h_(t-1), x_t] + b_c) h_t = o_t * tanh(c_t) Among them, f_t, i_t, o_t are the forget gate, input gate and output gate respectively, c_t is the cell state, and h_t is the hidden state.

[0086] The optimization module uses a genetic algorithm to solve the multi-objective optimization problem of resource allocation. The algorithm's fitness function takes into account system performance, resource utilization, and operating costs: = w1 * performance + w2 * utilization - w3 * fitness cost The execution layer is responsible for converting the optimization suggestions of the analysis and decision-making layer into specific system adjustment operations. It includes resource scaling module, load balancing module and configuration management module. The resource scaling module can dynamically adjust the number and specifications of computing nodes. The load balancing module adjusts the task allocation strategy in real time to ensure system load balancing. The configuration management module is responsible for updating the configuration parameters of each component of the system.

[0087] An important innovation of this system is the introduction of an adaptive control loop. This loop continuously monitors system performance indicators and compares them with predetermined targets. When significant deviations are detected, the system automatically triggers adjustment actions. The control loop uses a PID (proportional-integral-derivative) control algorithm, which has the general form: u(t) = K_p * e(t) + K_i * ∫e(τ)dτ + K_d * de(t) / dt Here, u(t) is the control signal, e(t) is the error signal, K_p, K_i, K_d are the proportional, integral, and differential gains, respectively.

[0088] Another innovation is the system's self-learning capability. By continuously recording and analyzing system behavior and its results, the management system is able to continuously optimize its decision-making model. This process uses online learning algorithms, allowing the system to adapt to changing workloads and environmental conditions.

[0089] The innovation of this embodiment is that it realizes all-round and real-time system monitoring, providing a reliable data basis for intelligent decision-making. It introduces advanced data analysis and machine learning technologies to achieve accurate modeling and prediction of system behavior. Through adaptive control loops and online learning, the system has the ability of continuous self-optimization. The multi-objective optimization algorithm ensures the best balance between performance, resource utilization and cost.

[0090] Embodiment 6: High availability and fault tolerance mechanism.

[0091] Reference Figure 6 This embodiment describes in detail the high availability and fault tolerance mechanisms of the AIGC high availability elastic computing system. These mechanisms are the key to ensuring that the system can maintain stable operation in the face of various failures and abnormal situations, and reflect the innovative design of the invention in terms of reliability and robustness.

[0092] The high availability and fault-tolerant design of this system follows the principle of "defense in depth" and implements a series of innovative measures at multiple levels of the system. First, in terms of data redundancy and consistency assurance, the system adopts a multi-copy storage strategy to ensure the redundancy of key data. For the task queue in the message middleware, the present invention implements an improved Raft consensus algorithm to ensure data consistency. The core of the algorithm lies in the leader election and log replication mechanism, which ensures that the system can maintain data consistency and availability even when some nodes fail.

[0093] In terms of high availability of services, the core service components of the system (such as API server, message middleware, management system, etc.) are all deployed in multiple instances. The present invention innovatively introduces a dynamic service discovery and health check mechanism. This mechanism can detect the health status of service instances in real time and automatically replace them when problems are found, thereby ensuring the continuous availability of services.

[0094] The compute nodes use an innovative self-healing design. Each node runs a self-monitoring process that performs health checks regularly. When an anomaly is detected, the node attempts to self-repair. If the self-repair fails, the node triggers an automatic restart process. This design greatly improves the robustness and self-recovery capabilities of the system.

[0095] In terms of task execution guarantee, the system implements an innovative task checkpoint and recovery mechanism. For long-running AIGC tasks, the system periodically saves the task execution status. If a failure occurs during task execution, the system can restore the task from the most recent checkpoint without having to start from scratch. This greatly improves the system's fault tolerance and resource utilization efficiency.

[0096] In order to deal with possible network partition problems, the system implements a distributed fault detection mechanism based on the Gossip protocol. Each node periodically exchanges heartbeat information with other randomly selected nodes. By analyzing these heartbeat messages, the system can quickly detect and isolate the failed network partition. This mechanism significantly improves the stability of the system in complex network environments.

[0097] In terms of disaster recovery, the system has designed a complete solution, including regular full backup and continuous incremental backup. The present invention innovatively introduces an intelligent backup scheduling algorithm to dynamically adjust the backup strategy according to the importance and change frequency of data. This method not only ensures the security of critical data, but also optimizes the efficiency of backup resource utilization.

[0098] The innovation of this embodiment lies in the improved consensus algorithm, dynamic service discovery and health check mechanism, self-healing design of computing nodes, innovative task checkpoint and recovery mechanism, distributed fault detection based on Gossip protocol, and intelligent backup scheduling algorithm. These innovative designs enable the AIGC high-availability elastic computing system to maintain high availability and reliability in various complex and extreme situations, providing solid infrastructure support for AIGC applications.

[0099] At the same time, the automation and intelligent characteristics of these mechanisms have greatly reduced the burden of system operation and maintenance, and improved the overall efficiency and cost-effectiveness of the system. In addition, the high availability design of this system has also significantly improved resource utilization.

[0100] It should be noted that the specific algorithms and implementations described in this embodiment are only preferred solutions, and those skilled in the art can make appropriate adjustments and optimizations according to actual needs. For example, different consensus algorithms can be selected according to specific application scenarios, or the fault detection strategy can be adjusted according to the characteristics of the network environment. These changes do not affect the core ideas and technical solutions of the present invention and should be deemed to fall within the scope of protection of the present invention.

[0101] Example 7: Task priority management and resource scheduling optimization.

[0102] This embodiment elaborates on the task priority management mechanism and corresponding resource scheduling optimization strategy in the AIGC high-availability elastic computing system. This mechanism is one of the important innovations of the present invention, which ensures that the system can effectively meet the needs of different business scenarios while processing a large number of concurrent AIGC tasks and achieve efficient use of resources.

[0103] In AIGC applications, different types of tasks often have different priorities and time sensitivities. For example, real-time interactive AIGC applications may require faster response times, while background batch tasks can tolerate longer waiting times. In order to effectively manage these diverse task requirements, this system implements a multi-level priority queue mechanism.

[0104] The system divides tasks into multiple priority levels, such as critical, high, medium, and low. Each priority level corresponds to a task queue. When a task enters the system, it is assigned to the corresponding priority queue based on predefined rules or dynamically calculated priority scores. The calculation of the priority score takes into account multiple factors, including but not limited to: task type, user level, service level agreement (SLA) requirements, current system load, etc.

[0105] The calculation formula of the priority score can be expressed as: P = w1 * T + w2 * U + w3 * S + w4 * L Among them, P is the priority score, T, U, S, and L represent the task type factor, user level factor, SLA factor, and load factor respectively, and w1 to w4 are the corresponding weight coefficients. These weights can be dynamically adjusted according to the system operation status and business needs.

[0106] In terms of resource scheduling, the system adopts an innovative hybrid scheduling strategy that combines the characteristics of priority scheduling and fair sharing scheduling. For high-priority tasks, the system will prioritize resource allocation to ensure their rapid execution. At the same time, in order to prevent low-priority tasks from being left unprocessed for a long time, the system introduces an aging mechanism. As the waiting time increases, the actual priority of low-priority tasks will gradually increase.

[0107] The core idea of ​​the scheduling algorithm can be expressed by the following mathematical model: S(i, t) = P(i) + α * W(i, t) Among them, S(i, t) is the scheduling priority of task i at time t, P(i) is the initial priority of task i, W(i,t) is the waiting time of task i at time t, and α is the aging system. This model ensures the timely processing of high-priority tasks and avoids the long-term starvation of low-priority tasks.

[0108] In order to further optimize resource utilization, the system also implements task preheating and resource reservation mechanisms. For high-priority tasks that appear periodically, the system will preheat the necessary resources in advance, such as preloading AI models, pre-allocating computing resources, etc. This mechanism significantly reduces the startup time of high-priority tasks and improves the system's response speed to important tasks.

[0109] At the same time, the system also implements a dynamic resource allocation strategy. By real-time monitoring of the task backlog and resource utilization of each priority queue, the system can dynamically adjust the proportion of resources allocated to each priority. This adaptive mechanism ensures that system resources can be flexibly allocated according to actual needs and maximize overall processing efficiency.

[0110] Another innovation of this embodiment is the introduction of a task splitting and parallel processing mechanism. For some AIGC tasks that can be parallelized, the system will automatically split them into multiple subtasks and assign them to different computing nodes for parallel processing. This method not only improves the processing speed of a single large task, but also improves the overall throughput of the system.

[0111] The task splitting strategy is dynamically determined based on the characteristics of the task and the current resource status of the system. The splitting decision takes into account the following factors: the degree of parallelization of the task, the data dependencies between subtasks, the number of available computing nodes, network bandwidth, etc. The system uses a heuristic algorithm to find the optimal splitting solution to balance processing speed and resource overhead.

[0112] In addition, the system also implements an intelligent task merging mechanism. For some small but similar AIGC tasks, the system will try to merge them into batch tasks. This approach reduces the overhead of task scheduling and resource allocation and improves the overall efficiency of the system. The merging strategy is dynamically determined based on factors such as task similarity, scale, and time window.

[0113] The innovation of this embodiment is reflected in the following aspects: The multi-dimensional task priority calculation model realizes more refined and flexible task management. The hybrid scheduling strategy combines priority scheduling and fair sharing to balance efficiency and fairness. The task preheating and resource reservation mechanism significantly improves the response speed of high-priority tasks. The dynamic resource allocation strategy realizes the adaptive optimization of system resources. The intelligent task splitting and merging mechanism improves the overall processing efficiency and resource utilization of the system.

[0114] Through these innovative designs, the system can effectively manage and schedule AIGC tasks of various priorities, significantly improving the overall performance of the system and user satisfaction.

[0115] It should be noted that the specific algorithms and parameters described in this embodiment are only preferred solutions, and those skilled in the art can make appropriate adjustments and optimizations according to actual application scenarios and requirements. These changes do not affect the core ideas and technical solutions of the present invention and should be deemed to fall within the scope of protection of the present invention.

[0116] Example 8: System optimization and performance improvement.

[0117] This example describes in detail the innovative design and implementation strategies of the AIGC high-availability elastic computing system in terms of system optimization and performance improvement. These optimization measures cover multiple levels of the system and aim to comprehensively improve the processing efficiency of AIGC tasks, system scalability, and resource utilization.

[0118] First, in terms of computing optimization, this system introduces an intelligent computing resource allocation mechanism. Traditional AIGC systems often adopt fixed resource allocation strategies, which are difficult to adapt to the diverse needs of different AIGC tasks. This system innovatively designs an adaptive resource allocation algorithm that can dynamically adjust the computing resources allocated to each task according to the complexity of the task, the size of the input data, and the real-time system load.

[0119] This adaptive algorithm is based on a machine learning model that predicts the computing resources required for AIGC tasks of different types and sizes by analyzing historical task execution data. The prediction model considers multiple factors, including task type, input data characteristics, and expected output quality. The core of the model can be expressed as: R = f(T, D, Q, L) Where R represents the predicted resource demand, T is the task type, D is the input data characteristics, Q is the expected output quality, and L is the current system load. Function f is a trained machine learning model such as a random forest or a neural network.

[0120] Through this intelligent resource allocation, the system can more accurately match task requirements and resource supply, avoid resource waste or shortage, and thus significantly improve overall computing efficiency and resource utilization.

[0121] Secondly, in terms of storage optimization, the system implements a multi-level cache mechanism and intelligent data pre-fetching strategy. Considering the large-scale data reading requirements commonly seen in AIGC tasks, the system designs a multi-level cache architecture including memory cache, SSD cache, and distributed cache. The cache strategy uses an improved LRU-K algorithm, which can better adapt to the characteristics of AIGC workloads.

[0122] In addition, the system also implements an intelligent data pre-fetching mechanism based on access pattern analysis. By analyzing historical data access patterns, the system can predict the data that will be accessed and load it to a faster storage layer in advance, thereby reducing I / O waiting time. The core algorithm of the pre-fetching strategy can be expressed as: P(d|C) = Σ(w_i * f_i(d, C)) Where P(d|C) represents the probability of data d being accessed in the current context C, fi is a different feature function, and wi is the corresponding weight. The system optimizes these weights through continuous learning to improve the accuracy of prefetching.

[0123] In terms of network optimization, the system implements intelligent data transmission scheduling and compression mechanisms. Considering the large-scale data transmission that may be involved in AIGC tasks, the system uses an adaptive data compression algorithm to dynamically select the most suitable compression method based on data characteristics and network conditions. At the same time, the system also implements priority-based network resource allocation to ensure that data transmission of critical tasks will not be interfered with.

[0124] Another important optimization is the improvement of task scheduling. The system adopts an innovative multi-objective scheduling algorithm, which takes into account multiple objectives such as task priority, resource utilization efficiency, load balancing and energy consumption. The objective function of the scheduling algorithm can be expressed as: minimize F = w1 f1(x) + w2 f2(x) + ... + wn*fn(x)subject to g_i(x) <=0, i = 1, ..., mh_j(x) = 0, j = 1, ..., p Where f1, f2, ..., fn represent different optimization goals (such as response time, resource utilization, etc.), w1, w2, ..., wn are the corresponding weights, and g_i and h_j are constraints. The system dynamically adjusts these weights to achieve the optimal scheduling strategy in different scenarios.

[0125] In terms of system scalability, this embodiment introduces a resilient design based on a microservice architecture. The core functions are split into independent microservices, each of which can be expanded independently. The system uses service mesh technology to manage communication between microservices, achieving fine-grained flow control and fault isolation. This architecture enables the system to flexibly perform local expansion according to load conditions, greatly improving the system's scalability and resource utilization efficiency.

[0126] Finally, this embodiment also implements a comprehensive performance monitoring and automatic optimization mechanism. The system continuously collects performance indicators of various components and uses time series analysis and anomaly detection algorithms to identify performance bottlenecks in real time. Based on these analyses, the system can automatically adjust configuration parameters such as cache size, thread pool capacity, load balancing strategy, etc. to adapt to the changing workload.

[0127] For those skilled in the art, many improvements and modifications can be made without violating the principles of the present invention, all of which fall within the scope of protection of the present invention. For example, the present invention can be combined with virtualization, edge computing and other technologies to further expand the application scenarios and effects of intelligent scheduling.

Claims

1. An AIGC high-availability elastic computing system, characterized in that: include: API server, used to receive AIGC task requests submitted by users; Message middleware, including task queue, used to store task messages encapsulated by API server; Multiple computing nodes, each of which is configured to: actively pull tasks from the task queue according to its own processing capacity; analyze task requirements and select appropriate computing resources from a variety of computing resources; start a containerized environment on the selected computing resources to execute AIGC tasks; after the task is completed, confirm the completion to the message middleware and release resources; Monitoring system, used to monitor the status of each component in real time and trigger alarms when abnormalities are found; Management system, used to dynamically adjust the number of computing nodes and resource configuration based on monitoring data and alarms.

2. The AIGC high-availability elastic computing system according to claim 1, characterized in that: The multiple computing resources include at least one of Serverless, virtual machines, and self-built clusters.

3. The AIGC high-availability elastic computing system according to claim 1, characterized in that: It also includes a cloud storage service for storing the execution results of AIGC tasks.

4. The AIGC high-availability elastic computing system according to claim 1, characterized in that: The message middleware adopts a multi-copy deployment mode to ensure high availability.

5. The AIGC high-availability elastic computing system according to claim 1, characterized in that: The computing nodes are also configured to support automatic task retry and node failover.

6. The AIGC high-availability elastic computing system according to claim 1, characterized in that: The management system is also configured to automatically scale the system up or down according to the task queue status.

7. The AIGC high-availability elastic computing system according to claim 1, characterized in that: The computing node is also configured to support task priority management to ensure that high priority tasks are processed first.

8. An AIGC task processing method based on the AIGC high-availability elastic computing system according to any one of claims 1 to 7, characterized in that: The following steps are involved: (1) Receive AIGC task requests submitted by users; (2) Encapsulate the task request as a message and publish it to the task queue of the message middleware; (3) The computing node actively pulls tasks from the task queue based on its own processing capabilities; (4) The computing node analyzes the task requirements and selects appropriate computing resources from a variety of computing resources; (5) Start the containerized environment on the selected computing resources and execute the AIGC task; (6) After the task is completed, the results are uploaded to the designated storage and the completion is confirmed to the message middleware; (7) Release computing resources and prepare to pull the next task; (8) The monitoring system tracks the status of each component throughout the process and triggers an alarm when an abnormality is found; (9) The management system dynamically adjusts the number of computing nodes and resource allocation based on monitoring data and alarms.

9. The AIGC task processing method according to claim 8, characterized in that: The method further includes: during the task execution process, calling other cloud services, including but not limited to storage services and database services.

10. The AIGC task processing method according to claim 8, characterized in that: The method also includes: selecting the most appropriate processing method according to task characteristics, including but not limited to GPU acceleration and Serverless processing.

Images