Distributed computing system based on pre-scheduling resource collaborative adaptation

Abstract

The invention provides a distributed computing system based on pre-scheduling resource collaborative adaptation, and relates to the technical field of resource collaborative allocation systems. The distributed computing system based on pre-scheduling resource collaborative adaptation aims at solving the problems that in the prior art, resource scheduling is not flexible, and application configuration and resource scheduling are lack of coordination. By introducing a dynamic scheduling mechanism of the resources, organic combination of resource pre-scheduling and flexible allocation is realized. Meanwhile, through cooperation of application configuration and resource scheduling, optimal distribution and execution of calculation tasks are achieved, the system is composed of a server side and a client side, the client side has pre-scheduling resources and is used for feeding back the resource state in real time and receiving and executing the calculation tasks, and the server side is responsible for monitoring and scheduling of global resources and task distribution. And through a designed heuristic collaborative adaptation algorithm, the server can dynamically schedule resources in the pre-scheduling resources, and high efficiency and stability of task execution are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of resource collaborative allocation systems, and in particular to a distributed computing system based on collaborative adaptation of pre-scheduled resources. Background Art

[0002] With the explosive growth of data volume, increasing computing demands and the complexity of computing tasks, the traditional centralized computing model faces bottlenecks and challenges. Distributed computing, as a solution, can handle large-scale data sets and complex computing tasks by utilizing multiple computing resources, including cloud computing, edge computing and other distributed architectures. In addition, distributed computing supports parallel processing, which allows multiple tasks to be processed simultaneously and improves the utilization of computing resources. This is of great significance in fields such as large-scale data analysis, artificial intelligence and autonomous driving. In particular, edge computing, as an important component of distributed computing, has emerged in recent years.

[0003] Edge computing pushes computing resources and data processing capabilities to the edge of the network, which can achieve faster data processing and real-time response. By deploying edge servers and computing nodes closer to data sources and terminal devices, edge computing effectively reduces data transmission delays and network congestion, and improves the overall performance and efficiency of the system. For example, in the application of vehicle-machine collaboration, the information obtained by smart vehicles through cameras and sensors needs to be processed and calculated quickly, but the servers and large computing nodes at a long distance cannot meet the low-latency requirements, often with large transmission delays and queuing delays, and the vehicle's own computing equipment alone is not enough to support it. At this time, a part of the computing tasks can be assigned to devices with idle resources through a distributed system, such as mobile phones, laptops and other computing devices connected to the Internet of Vehicles, so as to reduce the burden on the vehicle-mounted computing equipment, thereby improving the performance and response speed of the entire system. In addition, through wireless communication between mobile devices and vehicle-mounted systems, real-time data transmission and interaction can be achieved, allowing vehicles to obtain and process information more quickly, improving the stability and safety of the autonomous driving system.

[0004] In order to fully and safely utilize the redundant resources of computing devices, it is necessary to utilize container virtualization technology. Nowadays, most enterprises and users are turning to multi-cloud infrastructure to reduce the risk of over-dependence on suppliers and obtain the best services and resources with optimized performance. Virtualization is one of the key technologies for distributed computing and cloud computing, and most cloud-based systems are based on virtualization [1]. Docker is a container-based virtualization technology used to develop various types of multi-cloud distributed systems. It isolates and manages resources by dividing the resources of a physical server into multiple virtual machine instances, making it possible to run multiple applications on the same physical device at the same time. This flexibility allows the system to dynamically adjust resource configuration according to changes in workloads, improving the scalability and performance of the system. In addition, virtualization technology also provides fault tolerance and high availability for the system. It responds to physical server failures by quickly migrating virtual machines to ensure continuous and stable operation of the system. At the same time, virtualization also improves the utilization of hardware resources, reduces costs, and simplifies the deployment and management process of the system. In summary, virtualization technology provides basic support for distributed computing and provides important guarantees for system reliability and efficiency. However, Docker is mainly used for software development, and less attention is paid to resource control and data processing based on this container.

[0005] In addition to virtualization technology, edge computing also needs to rely on a large computing platform and communication base, because the edge computing environment involves a large number of computing tasks and data processing requirements. A centralized platform for receiving and distributing tasks to multiple computing nodes and managing and optimizing resources is essential. This platform can effectively schedule and allocate tasks to ensure that tasks are completed on time, and uniformly manage and optimize resources, including resource scheduling and load balancing. In addition, this platform can also receive, integrate and analyze data generated in the edge computing environment to provide support for decision-making and applications. Therefore, the computing platform plays a vital role in edge computing and can promote the development and application of edge computing technology.

[0006] While edge computing provides a powerful solution to meet current computing needs, it also faces some challenges. One of the most prominent issues is the complexity of resource management encountered by large computing platforms. In a distributed environment, factors such as resource heterogeneity of each node and network latency make the effective use of resources more complicated. For example, in a distributed computing network, some nodes may be equipped with high-performance processors and large memories, while other nodes may only provide lower-performance hardware. In addition, challenges in task scheduling, load balancing, and data transmission also limit the performance and efficiency of distributed systems.

[0007] In order to deal with these problems, dynamic resource scheduling has become an attractive concept. By real-time monitoring and analyzing the resource status of computing nodes, the system can intelligently adjust the resource allocation strategy so that each node can fully utilize its resources while avoiding overload or idle resources, thus improving the efficiency of system resource utilization. Summary of the invention

[0008] In view of the shortcomings of the prior art, the present invention provides a distributed computing system based on pre-scheduled resource collaborative adaptation, wherein the pre-scheduled resource module and the information storage module are implemented based on the Docker container and the Redis database. This system is a study and design of a resource monitoring and control module, providing services and support for the resource scheduling function of the OpenSAI platform. OpenSAI is a general distributed network artificial intelligence platform that can unify the elastic computing-storage-network scheduling of the end-edge system, provide a service customization network, and realize the collaborative two-way optimization of network and application computing, that is, the collaborative configuration and online adaptation of computing network resources and application execution, and support dynamic unloading of computing tasks to ensure the execution efficiency and reliability of application services. On the one hand, a scheduling mechanism combining resource pre-scheduling and flexible allocation is proposed to improve the adaptability and flexibility of resource scheduling. On the other hand, the collaborative optimization of application configuration and resource scheduling is realized, solving the isolation and inefficiency problems existing in traditional scheduling methods.

[0009] To achieve the above objectives, the present invention is implemented through the following technical solutions: A distributed computing system based on collaborative adaptation of pre-scheduled resources specifically includes the following steps: S1. System design S101. The server needs to obtain some resource information of the client and store this information in the database. This resource information includes the client's CPU, GPU, network and memory usage, which is used for subsequent dynamic resource scheduling and monitoring; S102. The server needs to build multiple clients and pre-schedule the required resources for rapid deployment and use; S103. The server is responsible for sending different tasks and parameter settings to different clients. Through communication with the client, the server can send the task file to the corresponding client according to the needs of the system for calculation and processing; S104. The server also needs to monitor the system's operating status and resource utilization, and perform dynamic resource scheduling according to the dynamic scheduling algorithm to achieve system efficiency improvement and performance optimization; S105. The server needs to receive the calculation results and integrate and store them. Once the client completes the calculation task and sends the calculation results, the result type message will be received by the server, and the server will store the corresponding calculation results and time information in the system for subsequent data analysis and processing; S2. Cooperative adaptation algorithm design S201. When building a client for pre-scheduling resources, the server needs to pre-configure certain resources according to the system requirements and pre-scheduling algorithm rules for rapid deployment and use; S202. The server also needs to perform dynamic resource scheduling of the client according to the dynamic scheduling algorithm. By monitoring the operating status and resource utilization of the system, the server can perform dynamic resource scheduling to improve the efficiency and performance of the system; S203. On the application side, application scheduling optimizes the application's operating efficiency and user experience by dynamically adjusting the application's task configuration parameters based on the monitoring system's operating status and application performance indicators. Application scheduling can dynamically adjust the number of application instances, task priorities, and task allocation strategies based on the load. By real-time monitoring of the application's health status and performance, application scheduling can automatically restore faulty instances, optimize traffic distribution, and respond quickly when user needs change, ultimately improving the overall system efficiency and optimizing service performance. In conjunction with resource scheduling, this enables coordinated adaptation of the system.

[0010] S204. The server needs to receive the calculation results and integrate and store them. Once the client completes the calculation task and sends the calculation results, the result type message will be received by the server, and the server will store the corresponding calculation results and time information in the system for subsequent data analysis and processing.

[0011] Preferably, in S202, a priority-based greedy algorithm is designed, and the server obtains the client's status information every 1 second. For each client, the server will use to obtain its status information, including the CPU and memory usage, and then use the exponentially weighted moving average (EWMA) algorithm to process the sequence and then use the sliding window average as the evaluation indicator of the current pre-scheduled resource status priority. The EWMA algorithm is specifically: Where EMA is the exponentially weighted moving average at each time, X is the observed value, and α is the smoothing factor that controls the weight of the data point. The closer its value is to 1, the greater the weight of the latest data point.

[0012] Preferably, according to pre-set rules, the server divides the resource status of the current pre-scheduled resources into three different states: idle, available and fully loaded. The window size and the thresholds of each state can be adjusted as parameters to meet the needs of different scenarios. After the priority setting of the algorithm is completed, its allocation cycle process is as follows: first, traverse downward from the client with the highest priority. If the client is not in an idle state, it is necessary to obtain resources from a client with a lower priority than it. In this case, it is necessary to traverse upward from the client with the lowest priority. If the client is in an available or idle state, and the low-priority client is still higher than the minimum resource limit to ensure the normal operation of the client after the resources are given, the resources are given and the client resource configuration list is updated; otherwise, the resources are not given. After completing these two layers of loops, all clients are traversed, and all clients are uniformly updated according to the updated client resource configuration list. Then, after waiting for a period of time, the next allocation is performed, and the loop is repeated until the computing tasks of all clients are completed.

[0013] Preferably, dynamic application scheduling is based on the operating status of the monitoring system, application performance indicators, and future load trends predicted by neural networks. The number of application instances, task priorities, and task allocation strategies are dynamically adjusted according to real-time monitoring and prediction results, while ensuring the normal migration and operation of tasks when underlying resources change. At the same time, by real-time monitoring of the health status and performance of applications, and combining neural network models to predict possible bottlenecks and resource requirements, application scheduling can deploy resources in advance, automatically recover faulty instances, optimize traffic distribution, and respond quickly when user needs change, ultimately achieving overall system efficiency improvement and service performance optimization.

[0014] Preferably, a collaborative adaptation mechanism combining resource pre-scheduling and flexible allocation is implemented: the resource scheduling mechanism proposed in the present invention can be applied to resources, storage and network resources. By combining static resource pre-scheduling and dynamic scheduling, it ensures efficient utilization of resources during task execution and improves the adaptability and flexibility of the system.

[0015] Preferably, an optimization mechanism for application configuration and resource scheduling is implemented: collaborative optimization between application configuration and resource scheduling is achieved, so that resource scheduling is more in line with actual task requirements, scheduling isolation in traditional methods is avoided, and the overall efficiency of the system is improved.

[0016] Preferably, a container-based real-time resource monitoring and scheduling feedback mechanism is implemented: in a containerized environment, the client provides some resource information in real time and transmits this information back to the server. The server can dynamically adjust the resource scheduling strategy based on real-time feedback to achieve refined resource management. For details, see the system design and flow diagram section in the specific implementation method.

[0017] Preferably, when evaluating system performance, four evaluation indicators are mainly used, namely, resource utilization efficiency, computing delay, reliability indicator, and load balancing indicator.

[0018] Preferably, the resource utilization efficiency refers to the product of the number of CPU cores and time, which reflects the resources used by the system when executing computing tasks. The sampling period is 1 second, and the overall resource efficiency of the system is calculated by the following formula: Where N is the number of CPU cores and C is the CPU usage.

[0019] Preferably, the computation delay refers to the time it takes for the server to receive all task results, which depends on the longest completion time of the task.

[0020] Preferably, the load balancing indicator is the standard deviation of the time it takes for each client to complete the task under the same task scale. The smaller the variance, the closer the task execution time of each client is, and the more balanced the load is.

[0021] Preferably, the reliability index refers to the task completion rate within a given time, that is, the completed data processing volume and the total data processing volume. The higher the completion rate, the higher the system reliability can be considered.

[0022] The present invention provides a distributed computing system based on pre-scheduled resource collaborative adaptation. It has the following beneficial effects: 1. The present invention provides a distributed computing system based on collaborative adaptation of pre-scheduled resources. By combining resource pre-scheduling with flexible allocation, the system can dynamically adjust resource scheduling according to changes in task load to avoid resource waste, while ensuring the resource requirements of high-priority tasks and improving the flexibility of resource scheduling. This flexibility significantly improves resource utilization and reduces the cost of idle resources.

[0023] 2. The present invention provides a distributed computing system based on collaborative adaptation of pre-scheduled resources. Through the collaborative optimization of application configuration and resource scheduling, the system can reasonably schedule resources according to the actual needs of the application, avoiding the inefficiency and resource conflict problems caused by the separation of the two in the traditional scheduling mode, and enhancing the stability and operation efficiency of the overall system.

[0024] 3. The present invention provides a distributed computing system based on pre-scheduled resource cooperative adaptation. By acquiring the client's resource information in real time, the system can dynamically adjust the task allocation and execution strategy to ensure that the computing tasks can be completed efficiently and reasonably allocate tasks when resources are tight. This not only improves the execution speed of tasks, but also reduces the waiting time for tasks to queue, thereby improving the efficiency of task execution.

[0025] 4. The present invention provides a distributed computing system based on pre-scheduled resource collaborative adaptation. Through flexible resource scheduling and heuristic collaborative adaptation algorithms, the system can flexibly schedule resources according to different task types and requirements, realize elastic expansion of the system, enable it to cope with sudden computing needs, improve the system's load processing capabilities, and enhance system stability.

[0026] 5. The present invention provides a distributed computing system based on collaborative adaptation of pre-scheduled resources. The client provides part of the client's resource information in real time, so that the server can accurately manage and allocate resources, thereby improving the system's control over computing resources and avoiding resource scheduling errors caused by information lag in traditional methods. It makes computing resources visual and facilitates accurate management.

[0027] 6. The present invention provides a distributed computing system based on collaborative adaptation of pre-scheduled resources. The client provides real-time display of the calculation progress and feedback of the calculation results, so that the system can monitor the execution status of the task in real time and adjust the allocation strategy in time, further improving the reliability and controllability of task execution and realizing real-time feedback and monitoring of the task execution status.

[0028] 7. The present invention provides a distributed computing system based on collaborative adaptation of pre-scheduled resources. Through the mechanism of dynamic scheduling and collaborative allocation, the present invention effectively reduces the idleness and waste of computing resources, reduces hardware costs and energy consumption, and also reduces dependence on manual intervention, thereby reducing the operation and maintenance costs of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of the connection and communication flow between the server and the client of the present invention; Figure 2 It is a pseudo code schematic diagram of the priority-based dynamic scheduling algorithm of the present invention; Figure 3 It is a schematic diagram of the horizontal comparison structure of the resource scheduling algorithm of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Example 1 The embodiment of the present invention provides a distributed computing system based on pre-scheduled resource collaborative adaptation, such as Figure 1As shown, the server mainly consists of two threads: information processing and collaborative adaptation algorithm. The specific workflow is as follows: S1. System design S101. The server needs to obtain some resource information of the client and store this information in the Redis database. The Redis database is a memory-based key-value storage system with the characteristics of fast reading and writing and efficient storage. It is often used for real-time data caching and storage. In the present invention, Redis is used to store real-time client information to help the system make quick decisions and resource scheduling. This resource information includes the client's CPU usage, GPU usage, and memory usage, which are used for subsequent dynamic resource scheduling and monitoring; S102. The server needs to build a Docker container with pre-scheduled resources. Docker container is a lightweight virtualization technology used to run applications in a computing environment. Each container includes the application and its dependent libraries, configurations, etc., to ensure that the application can maintain a consistent operating environment on any platform. Compared with traditional virtual machines, Docker containers have more efficiency and resource utilization advantages. According to system requirements and pre-scheduling algorithm rules, the server will pre-configure a certain number of Docker container resources for rapid deployment and use; S103. The server is responsible for sending different tasks and parameter settings to different clients. Through communication with the client, the server can send the task file to the corresponding client according to the needs of the system for calculation and processing; S104. The server also needs to monitor the system's operating status and resource utilization, and dynamically schedule the container's resources according to the dynamic scheduling algorithm to improve system efficiency and optimize performance; S105. The server needs to receive the calculation results and integrate and store them. Once the client completes the calculation task and sends the calculation results, a result-type message will be received by the server, and the server will store the corresponding calculation results and time information in the system for subsequent data analysis and processing.

[0032] Specifically, the client mainly implements four functions: providing partial resource information of pre-scheduled resources in real time, receiving computing tasks, executing computing tasks and displaying computing progress, and returning computing results. The client first sends a "wave" message to the server to indicate that it is online, and then receives computing tasks and related parameters and data sent by the server. Next, the client uses two threads for processing: one thread is responsible for running computing tasks with given parameters, and displays the computing progress and estimated task completion time in real time by using the tqdm library; the other thread is used to calculate real-time delay information and send the results to the server. Until the computing tasks of all threads are completed, the client integrates the results and sends them to the server. This enables the client to efficiently receive tasks, perform calculations, and communicate with the server in real time, thereby achieving smooth execution of tasks and timely return of results.

[0033] S2. Cooperative adaptation algorithm design Heuristic algorithms are algorithms that solve problems based on empirical rules and heuristic information, and are usually used to solve complex optimization problems. Heuristic collaborative adaptation algorithms combine the characteristics and resource requirements of different tasks, and can intelligently allocate appropriate resources to tasks, thereby improving resource utilization efficiency and system performance.

[0034] S201. When building a client for pre-scheduling resources, the server needs to pre-configure certain resources according to the system requirements and pre-scheduling algorithm rules for rapid deployment and use; S202. The server also needs to perform dynamic resource scheduling of the container according to the dynamic scheduling algorithm. By monitoring the operating status and resource utilization of the system, the server can dynamically schedule container resources to improve system efficiency and optimize performance; S203. On the application side, application scheduling optimizes the application's operating efficiency and user experience by dynamically adjusting the application's task configuration parameters based on the monitoring system's operating status and application performance indicators. Application scheduling can dynamically adjust the number of application instances, task priorities, and task allocation strategies based on the load. By real-time monitoring of the application's health status and performance, application scheduling can automatically restore faulty instances, optimize traffic distribution, and respond quickly when user needs change, ultimately improving the overall system efficiency and optimizing service performance. In conjunction with resource scheduling, this enables coordinated adaptation of the system.

[0035] S204. The server needs to receive the calculation results and integrate and store them. Once the client completes the calculation task and sends the calculation results, the result type message will be received by the server, and the server will store the corresponding calculation results and time information in the system for subsequent data analysis and processing.

[0036] Preferably, in S202, a priority-based greedy algorithm is designed, and the server obtains the client's status information every 1 second. For each client, the server will use to obtain its status information, including the CPU and memory usage, and then use the exponentially weighted moving average (EWMA) algorithm to process the sequence and then use the sliding window average as the evaluation indicator of the current pre-scheduled resource status priority. The EWMA algorithm is specifically: Where EMA is the exponentially weighted moving average at each time, X is the observed value, and α is the smoothing factor that controls the weight of the data point. The closer its value is to 1, the greater the weight of the latest data point.

[0037] Specifically, the algorithm sets the weight of the data to decay exponentially over time, with recent data points having higher weights and older data points having lower weights. The EWMA algorithm is often used to smooth time series data in order to better capture trends and changes in the data while reducing the impact of noise.

[0038] As attached Figure 2 As shown in the figure, according to the pre-set rules, the server divides the resource status of the current pre-scheduled resources into three different states: free (Free, marked as -1), available (Available, marked as 0) and full (Full, marked as 1). The window size and the thresholds of each state can be adjusted as parameters to meet the needs of different scenarios. After the priority setting is completed, the allocation algorithm has an allocation cycle process as follows: First, traverse downward from the client with the highest priority. If the client is not in an idle state, it is necessary to obtain resources from a client with a lower priority than it. In this case, it is necessary to traverse upward from the client with the lowest priority. If the client is in an available or idle state, and the low-priority client is still higher than the minimum resource limit to ensure the normal operation of the pre-scheduled resources after the resources are given, the resources are given and the pre-scheduled resource configuration list is updated; otherwise, the resources are not given. After completing these two layers of loops, all clients are traversed, and the resource scheduling of all clients is uniformly updated according to the updated pre-scheduled resource configuration list. Then, after waiting for a period of time, the next allocation is performed, and the cycle is repeated until the computing tasks of all clients are completed.

[0039] Example 2 The embodiment of the present invention provides a distributed computing system based on pre-scheduled resource collaborative adaptation, and its application fields cover multiple technical fields, mainly including: 1) Cloud computing and edge computing: The dynamic resource scheduling mechanism in this invention is very suitable for application in cloud computing platforms. It can efficiently manage computing resources in cloud servers and ensure the smooth execution of tasks in a multi-tenant environment. At the same time, in edge computing scenarios, the resource utilization of edge devices can also be optimized through pre-scheduling resource technology to improve task execution efficiency and system response speed.

[0040] 2) Data processing and distributed computing: The scheduling mechanism can be used to optimize the resource scheduling of computing tasks, improve the resource utilization in the computing cluster, and ensure the efficient processing of large-scale computing tasks and the optimization of data transmission.

[0041] 3) Artificial Intelligence and Machine Learning: In artificial intelligence training tasks, model training often requires a large amount of computing resources. The collaborative allocation mechanism can dynamically schedule resources according to the needs of the training tasks, ensure efficient use of resources, shorten training time, and adapt to computing needs of different scales and loads.

[0042] 4) Pre-scheduling resource-based application deployment and microservice architecture: By supporting pre-scheduling resource-based technologies such as Docker, efficient resource scheduling and management of applications is achieved. This has significant advantages for dynamic expansion and resource optimization allocation under the microservice architecture, and can support the flexible deployment and expansion of microservices in multiple nodes.

[0043] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A distributed computing system based on collaborative adaptation of pre-scheduled resources, characterized in that: The specific steps include: S1. System design S101. The server needs to obtain some resource information of the pre-scheduled resource client and store this information in the database. This resource information includes the client's CPU, GPU, network and memory usage, which is used for subsequent dynamic resource scheduling and monitoring; S102. The server needs to build multiple clients and pre-schedule the required resources for rapid deployment and use; S103. The server is responsible for sending different tasks and parameter settings to different clients. Through communication with the client, the server can send the application task file to the corresponding client according to the needs of the system for calculation and processing; S104. The server also needs to monitor the system's operating status and resource utilization, and perform dynamic resource scheduling according to the collaborative adaptation algorithm to achieve system efficiency improvement and performance optimization; S105. The server needs to receive the calculation results and integrate and store them. Once the client completes the calculation task and sends the calculation results, the result type message will be received by the server, and the server will store the corresponding calculation results and time information in the system for subsequent data analysis and processing; S2. Cooperative adaptation algorithm design S201. When building a client for pre-scheduling resources, the server needs to pre-configure certain resources according to the system requirements and pre-scheduling algorithm rules for rapid deployment and use; S202. The server also needs to perform dynamic resource scheduling of pre-scheduled resources according to the dynamic scheduling algorithm. By monitoring the system's operating status and resource utilization, the server can dynamically schedule resources to achieve system efficiency improvement and performance optimization; S203. On the application side, application configuration optimizes the application's operating efficiency and user experience by dynamically adjusting the application's task allocation parameters based on the monitoring system's operating status and application performance indicators. Application scheduling can dynamically adjust the number of application instances, task priorities, and task allocation strategies based on the load. By real-time monitoring of the application's health status and performance, application scheduling can automatically restore faulty instances, optimize traffic distribution, and respond quickly when user needs change, ultimately improving the overall system efficiency and optimizing service performance. In conjunction with resource scheduling, this enables coordinated adaptation of the system. S204. The server needs to receive the calculation results and integrate and store them. Once the client completes the calculation task and sends the calculation results, the result type message will be received by the server, and the server will store the corresponding calculation results and time information in the system for subsequent data analysis and processing.

2. The distributed computing system based on pre-scheduled resource collaborative adaptation according to claim 1, characterized in that: In S202, a priority-based greedy algorithm is designed, and the server obtains the client's status information every 1 second. For each client, the server will use its status information, including the CPU and memory usage, and then use the exponentially weighted moving average (EWMA) algorithm to process the sequence and then use the sliding window average as the evaluation indicator of the current pre-scheduled resource status priority. The EWMA algorithm is specifically: Where EMA is the exponentially weighted moving average at each time, X is the observed value, and α is the smoothing factor that controls the weight of the data point. The closer its value is to 1, the greater the weight of the latest data point.

3. The distributed computing system based on pre-scheduled resource collaborative adaptation according to claim 2, characterized in that: According to the pre-set rules, the server divides the resource status of the current client into three different states: idle, available and fully loaded. The window size and the thresholds of each state can be adjusted as parameters to meet the needs of different scenarios. After the priority setting is completed, the allocation algorithm has an allocation cycle process as follows: First, traverse downward from the client with the highest priority. If the client is not in an idle state, it is necessary to obtain resources from clients with lower priorities than it. In this case, it is necessary to traverse upward from the client with the lowest priority. If the client is in an available or idle state, and the low-priority client is still higher than the minimum resource limit to ensure the normal operation of the client after the resources are given, the resources will be given and the client resource configuration list will be updated; otherwise, no resources will be given. After completing these two cycles, all clients are traversed, and the resource scheduling of all clients is uniformly updated according to the updated client resource configuration list.

4. The distributed computing system based on pre-scheduled resource collaborative adaptation according to claim 1, characterized in that: Dynamic application scheduling is based on the operating status of the monitoring system, application performance indicators, and the use of neural networks to predict future load trends. It dynamically adjusts the number of application instances, task priorities, and task allocation strategies based on real-time monitoring and prediction results, while ensuring the normal migration and operation of tasks when underlying resources change. At the same time, by real-time monitoring of the health status and performance of applications, and combining neural network models to predict possible bottlenecks and resource requirements, application scheduling can deploy resources in advance, optimize traffic distribution, and respond quickly when user needs change, ultimately achieving overall system efficiency improvement and service performance optimization.

5. The distributed computing system based on pre-scheduled resource collaborative adaptation according to claim 1, characterized in that: A collaborative adaptation mechanism combining resource pre-scheduling and flexible allocation is realized: the resource scheduling mechanism proposed in the present invention can be applied to computing resources, storage and network resources. By combining static resource pre-scheduling and dynamic scheduling, it ensures efficient utilization of resources during task execution and improves the adaptability and flexibility of the system.

6. The distributed computing system based on pre-scheduled resource collaborative adaptation according to claim 1, characterized in that: Implemented an optimization mechanism for application configuration and resource scheduling: Implemented collaborative optimization between application configuration and resource scheduling, making resource scheduling more in line with actual task requirements, avoiding the scheduling isolation in traditional methods, and improving the overall efficiency of the system.

7. The distributed computing system based on pre-scheduled resource collaborative adaptation according to claim 1, characterized in that: A container-based real-time resource monitoring and scheduling feedback mechanism has been implemented: in a containerized environment, the client provides some resource information in real time and transmits this information back to the server. The server can dynamically adjust the resource scheduling strategy based on real-time feedback to achieve refined resource management.

Images