Container arranging and scheduling system based on domestic Quhua embedded real-time operating system

Abstract

The invention discloses a container arranging and scheduling system based on a domestic Quhua embedded real-time operating system, which is characterized by comprising a group scheduling management platform, a distributed resource and state sensing subsystem; a task collaborative arrangement scheduling subsystem; and arranging and deploying the management subsystem. According to the method, the advantages of a container arrangement scheduling technology in the cloud native field are absorbed and referred, the characteristics of an embedded system are fully considered, a set of container arrangement scheduling framework is designed on a domestic Quhua embedded real-time operating system, and the lightweight, high-reliability and extensible container arrangement scheduling capability of an embedded end is realized. And the arrangement and scheduling requirements of the applications developed based on the container of the embedded terminal are met.

Description

Technical Field

[0001] The present invention relates to an implementation technology of a container orchestration and scheduling system on a domestic Ruihua embedded operating system, belonging to the technical field of embedded operating systems. Background Art

[0002] A container orchestration and scheduling system is a tool or platform for managing and coordinating containerized applications. It is mainly responsible for automating the deployment, expansion, update, rollback, and resource allocation of containers in a cluster environment to achieve high availability of business services, improve cluster utilization, reduce operation and maintenance costs, and quickly respond to business needs. Common container orchestration and scheduling tools include Kubernetes, Mesos, Docker Swarm, Apache Mesos Marathon, etc. For example, Kubernetes is an open source version of Google's large-scale container management system borg. It provides application deployment, maintenance, and expansion mechanisms, and can easily manage containerized applications running across machines. Kubernetes is an open container scheduling and management platform that supports multiple languages ​​and has complete distributed system support functions, such as multi-layer security protection, admission mechanism, multi-tenant application support, transparent service registration and discovery, built-in load balancing, powerful fault detection and self-healing mechanisms, service rolling upgrades and online expansion, scalable resource automatic scheduling mechanisms, and multi-granular resource quota management capabilities. These container orchestration and scheduling systems make the management of containerized applications more efficient, convenient, and reliable, helping enterprises better respond to business changes and expansion needs. Different systems may differ in functions, performance, applicable scenarios, etc. Enterprises can choose a suitable container orchestration and scheduling system based on their own needs and actual conditions.

[0003] Embedded container orchestration and scheduling refers to the process of managing and coordinating containers in embedded systems. It is mainly responsible for implementing automated deployment, expansion, update, rollback, and resource allocation of containers in resource-constrained embedded environments to improve the resource utilization, scalability, and reliability of the system. Compared with traditional container orchestration and scheduling, embedded container orchestration and scheduling need to consider more factors, such as real-time, resource constraints, hardware differences, etc. Therefore, embedded container orchestration and scheduling usually requires the use of lightweight container technology and orchestration tools, and customized development combined with specific embedded system requirements. In practical applications, embedded container orchestration and scheduling can help developers manage and deploy embedded applications more efficiently, improve system development efficiency and operational stability. At the same time, it also provides strong support for the intelligent and interconnected development of embedded systems.

[0004] A container orchestration scheduler usually includes the following functional modules:

[0005] 1. Cluster management: responsible for maintaining cluster status information, such as the number and status of nodes and the distribution of containers.

[0006] 2. Resource management: Schedule and allocate various resources in the cluster (such as CPU, memory, disk, network, etc.) to ensure that the container can obtain the required resources.

[0007] 3. Task scheduling: Allocate application workloads on resources to nodes in the cluster to ensure stable operation of applications.

[0008] 4. Service discovery: helps applications identify the IP addresses and port numbers of each node within the cluster, facilitating communication between applications.

[0009] Common embedded container orchestration and scheduling tools include Docker Swarm, Kubernetes, etc. These tools usually have the following characteristics:

[0010] -Lightweight: It takes up little resources and is suitable for running in embedded systems.

[0011] - Scalability: Supports large-scale container deployment and management.

[0012] -High reliability: It has fault tolerance and recovery mechanisms to ensure the stable operation of the container.

[0013] -Flexibility: Customizable configurations can be made according to different application requirements.

[0014] The traditional container orchestration and scheduling system originated from the Linux system and has been widely used and developed mainly in the field of cloud native technology. Due to the huge gap between the application scenarios and hardware environment of embedded system container orchestration and scheduling and cloud computing, the container orchestration and scheduling technology in the cloud native field has problems such as insufficient real-time performance and too high resource requirements when migrating to embedded real-time systems. Summary of the invention

[0015] The purpose of the present invention is to optimally utilize the computing resources of an embedded computing cluster and take into account the differences between an embedded cluster environment and a cloud computing environment.

[0016] In order to achieve the above-mentioned purpose, the technical solution of the present invention is to disclose a container orchestration and scheduling system based on the domestic Ruihua embedded real-time operating system, which is characterized by comprising:

[0017] Group scheduling management platform: The group scheduling management platform provides resource collaborative management services for embedded clusters. The group scheduling management platform generates and distributes group terminal-side scheduling policy configurations based on user input adjustments through the collected group terminal operation status information and scenario analysis data.

[0018] Distributed resource and status perception subsystem: used to collect the usage status of end-side group system resources, analyze and perceive system scenarios, and provide basic data reporting capabilities;

[0019] Task collaborative scheduling and orchestration subsystem: Based on factors such as user needs, environmental constraints, and resource performance requirements, it conducts modeling and solving to achieve optimal allocation of complex cluster tasks and supports unified scheduling and orchestration of tasks for multiple nodes in a group.

[0020] Orchestration and deployment management subsystem: Provides a unified runtime environment based on architecture neutrality, supports container image deployment and operation; provides basic support capabilities including environmental status perception analysis, dynamic resource management, and application collaborative deployment;

[0021] The group scheduling management platform interacts with users to create container application deployment plans for the cluster, and provides users with resource status and environmental data. In addition, it also provides task scheduling management interfaces and data operation and maintenance analysis services. It issues retrieval strategies to the task collaborative orchestration scheduling subsystem. The task collaborative orchestration scheduling subsystem performs orchestration demand analysis based on scheduling strategies and application deployment plans, generates orchestration tasks, manages and maintains orchestration tasks, and sends scheduling tasks to the orchestration deployment management subsystem to execute container orchestration scheduling. The orchestration deployment management subsystem implements the full life cycle management of container applications, including creation, startup, suspension, shutdown, and deletion, based on the application programming management interface provided by the Ruihua system. The distributed resource and status perception subsystem is responsible for collecting the status of various software and hardware resources in the cluster and providing them to the group scheduling management platform and the task collaborative orchestration scheduling subsystem for scheduling decisions and status display.

[0022] Preferably, the group scheduling management platform builds a visual cloud resource collaborative management platform based on the environmental perception data, resource monitoring information and task execution information transmitted by the cluster network terminals, realizes the task execution information, resource usage, environmental data monitoring and analysis, fault anomaly warning and processing functions of each terminal node, and provides data storage and query engines to support historical data playback and multi-dimensional analysis functions.

[0023] Preferably, the group scheduling management platform includes:

[0024] The resource monitoring and management module provides real-time monitoring and viewing of CPU, memory, storage, network, and power consumption resource information of distributed terminal nodes in the cluster network, realizes cloud-based collaborative resource monitoring and management, and provides visual operation and maintenance and scheduling optimization support capabilities for multi-terminal collaborative resource scheduling;

[0025] The task scheduling management module provides information statistics and display functions for the number of tasks running on distributed terminal nodes, task status, scheduling success rate, task progress, and task completion rate;

[0026] Environmental data management module provides access and management of terminal node time data, meteorological data, image data, and geographic spatial data, and performs cleaning and normalization of environmental data to improve data quality and provide data support capabilities for environmental change trend prediction and environmental early warning analysis;

[0027] The data operation and maintenance analysis module builds data operation and maintenance management and analysis based on terminal resource usage, task scheduling, and environmental monitoring data collection. It supports users to perform data query and cluster analysis based on resource type, time range, and task type, and provides effective data support capabilities for container scheduling optimization and operation and maintenance management.

[0028] Preferably, in the distributed resource and state perception subsystem, a multi-dimensional resource demand matching management strategy is established in view of the characteristics of resource diversity and task complexity in the embedded cluster. Starting from computing, storage, energy consumption, and environmental perception, a multi-dimensional resource comprehensive evaluation model is constructed. Through in-depth analysis of the nonlinear demand characteristics of system tasks for resources of different dimensions, on-demand allocation and efficient utilization of resources can be achieved, providing support for container scheduling decisions.

[0029] Preferably, the distributed resource and state awareness subsystem includes:

[0030] The resource information monitoring module monitors, cleans, and normalizes the hardware parameter specifications, CPU, memory, network, and energy consumption of different computing nodes in the cluster, establishes a structured resource database, and provides data support capabilities for subsequent modeling and analysis and evaluation;

[0031] The resource map analysis module builds a resource evaluation index system model for CPU, memory, storage, energy consumption, and network resource data, dynamically calculates resource evaluation values, and provides resource trend analysis and load overload warning functions;

[0032] The resource matching and scheduling module establishes a task resource intelligent matching strategy model based on the resource requirements of different tasks, mines the feature representation of task resources through deep learning, realizes adaptive optimal task resource matching strategy decision, and makes expected inferences on the task completion rate, resource utilization rate, task execution delay, etc. of each terminal, realizing dynamic task scheduling migration and resource scheduling optimization in a multi-objective task environment.

[0033] Preferably, the task collaborative orchestration and scheduling subsystem performs semantic analysis and feature extraction based on the collaborative task commands given by the user, extracts the execution method, location, time, participants, collected data types, and dependencies, and realizes task object generation and release.

[0034] Preferably, the task collaborative scheduling subsystem includes:

[0035] The orchestration requirement analysis module is used to extract the basic information, constraints, discrete features, timing information and dependency conditions of the orchestration task;

[0036] The task object generation module combines the extracted task information and discrete features to form a high-dimensional task intermediate vector, and maps its information to the task object for recording and storage;

[0037] The task object management module stores and manages the task object data constructed and published according to the command data input by the user, and realizes the task object publishing, query and display functions.

[0038] Preferably, the orchestration deployment management subsystem includes:

[0039] The container lifecycle management module is responsible for the full process management of container applications;

[0040] The task result feedback module keeps real-time status synchronization and feedback with the task scheduling system for task scheduling decisions, reports task execution progress, task execution status, and resource usage, so that the scheduling system can timely grasp the system's operating status and make dynamic adjustments based on feedback information;

[0041] The container migration and fault recovery module establishes a distributed fault-tolerant scheduling mechanism. When the node load is too high, it performs task migration and status synchronization between nodes to improve system resource utilization. When a node fails, it uses a replica mechanism or a background restart mechanism to redistribute the execution of abnormal applications and ensure the reliability of container application execution.

[0042] This paper absorbs and refers to the advantages of container orchestration and scheduling technology in the cloud native field, and fully considers the characteristics of embedded systems. It designs a container orchestration and scheduling framework on top of the domestic Ruihua embedded real-time operating system, and realizes lightweight, highly reliable, and scalable container orchestration and scheduling capabilities on the embedded side. It meets the orchestration and scheduling needs of container-based applications on the embedded side.

[0043] By adapting the container orchestration and scheduling system designed in the present invention to an embedded cluster hardware system composed of different multi-computing nodes, the orchestration and scheduling and fault recovery of container applications on the cluster can be realized, and the real-time performance, high performance and efficient use of various resources of computing tasks can be ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 The embedded container orchestration and scheduling system architecture is illustrated. DETAILED DESCRIPTION

[0045] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0046] Applications in embedded scenarios usually rely on different types of computing tasks, such as data pre-occupancy, model training / refinement, and decision prediction, each of which requires different computing resources. These tasks are not fixed over time, and the same is true for the underlying available computing resources. In order to optimally utilize the computing resources of the embedded computing cluster and taking into account the differences between the embedded cluster environment and the cloud computing environment, the present invention designs a container scheduling system for embedded scenarios, such as Figure 1 As shown in Figure 1, the tasks in the embedded computer cluster can be scheduled by the system to the appropriate computing nodes for execution, ensuring the real-time performance and execution reliability of the system.

[0047] The container scheduling system designed by the present invention includes the following functional modules:

[0048] 1. Group scheduling management platform: The group scheduling management platform provides resource collaborative management services for embedded clusters. The group scheduling management platform generates and distributes group terminal scheduling policy configuration based on user input adjustment through the collected group terminal operation status information and scenario analysis data. Based on the environmental perception data, resource monitoring information and task execution information transmitted by the cluster network terminal, the management platform builds a visual cloud resource collaborative management platform to realize the task execution information, resource usage, environmental data monitoring and analysis, fault abnormality warning and processing functions of each terminal node. At the same time, it provides data storage and query engines, supports historical data playback and multi-dimensional analysis functions. Among them, the resource monitoring management module provides real-time monitoring and viewing functions of CPU, memory, storage, network, power and energy consumption resource information of distributed terminal nodes in the cluster network, realizes cloud collaborative resource monitoring and management, and provides visual operation and maintenance and scheduling optimization support capabilities for multi-terminal collaborative resource scheduling. The task scheduling management module provides statistics and display functions for key task information such as the number of tasks running on distributed terminal nodes, task status, scheduling success rate, task progress, and task completion rate. The environmental data management module provides access and management of terminal node time data, meteorological data, image data, geographic spatial data, etc., and cleans and normalizes environmental data to improve data quality, providing data support capabilities for environmental change trend prediction and environmental early warning analysis. The data operation and maintenance analysis module builds data operation and maintenance management and analysis based on various types of data such as terminal resource usage, task scheduling, and environmental monitoring collection. It supports users to perform data query and cluster analysis based on dimensions such as resource type, time range, and task type, providing effective data support capabilities for container scheduling optimization and operation and maintenance management.

[0049] 2. Distributed resource and status perception subsystem: used to collect the usage status of end-side group system resources, analyze and perceive system scenarios, and provide basic data reporting capabilities. In view of the characteristics of resource diversity and task complexity in embedded clusters, a multi-dimensional resource demand matching management strategy is established. Starting from multiple dimensions such as computing, storage, energy consumption, and environmental perception, a multi-dimensional resource comprehensive evaluation model is constructed. By deeply analyzing the nonlinear demand characteristics of system tasks for resources of different dimensions, on-demand allocation and efficient utilization of resources are achieved, providing support for container scheduling decisions. This subsystem consists of three modules: resource matching scheduling, resource map analysis, and resource information monitoring. Among them, the resource information monitoring module mainly monitors, cleans, and normalizes key data information such as hardware parameter specifications, CPU, memory, network, and energy consumption of different computing nodes in the cluster, establishes a structured resource database, and provides data support capabilities for subsequent modeling and analysis and evaluation. The resource map analysis module constructs a resource evaluation indicator system model for different types of resource data such as CPU, memory, storage, energy consumption, and network, dynamically calculates resource evaluation values, and provides resource trend analysis and load overload warning functions. The resource matching and scheduling module establishes a task resource intelligent matching strategy model based on the resource requirements of different tasks, mines the feature representation of task resources through deep learning, realizes adaptive optimal task resource matching strategy decision, and makes expected inferences on the task completion rate, resource utilization rate, task execution delay, etc. of each terminal, realizing dynamic task scheduling migration and resource scheduling optimization in a multi-objective task environment.

[0050] 3. Task collaborative scheduling and scheduling subsystem: Based on user needs, environmental constraints, resource performance requirements and other factors, modeling and solving are carried out to achieve the optimal allocation of complex cluster tasks and support the unified scheduling and scheduling of tasks for multiple nodes in the group. Based on the collaborative task commands given by the user, this subsystem performs semantic parsing and feature extraction, extracts key information such as execution mode, location, time, participants, data type collected, and dependencies, and realizes the generation and release of task objects. This subsystem consists of three submodules: scheduling demand analysis, task object generation, and task object management. Among them, the scheduling demand analysis module is used to extract the basic information, constraints, discrete features, timing information, and dependency conditions of the scheduling task. The task object generation module splices the extracted task information and discrete features to form a high-dimensional task intermediate vector, and maps its information to the task object for recording and storage. The task object management module stores and manages the constructed and released task object data according to the user input command data, and realizes the task object release, query and display functions.

[0051] 4. Orchestration and deployment management subsystem: The orchestration and deployment management subsystem provides a unified runtime environment based on architecture neutrality, supports container image deployment and operation; provides basic support capabilities such as environmental status perception analysis, dynamic resource management, and application collaborative deployment. This subsystem consists of three submodules: container lifecycle management, task result feedback, and container migration and fault recovery. Among them, the container lifecycle management module is responsible for the full process management of container applications. The task result feedback module maintains real-time status synchronization and feedback with the task scheduling system for task scheduling decisions, reports task execution progress, task execution status, and resource usage, so that the scheduling system can grasp the system's operating status in a timely manner and make dynamic adjustments based on feedback information. The container migration and fault recovery module establishes a distributed fault-tolerant scheduling mechanism. When the node load is too high, it performs task migration and status synchronization between nodes to improve system resource utilization; when a node fails abnormally, it uses a replica mechanism or a background restart mechanism to achieve abnormal application redistribution execution to ensure the reliability of container application execution.

[0052] The execution process of the container orchestration and scheduling system is as follows: The group scheduling management platform interacts with users to create a container application deployment plan for the cluster, and provides users with resource status and environmental data. In addition, it also provides task scheduling management interfaces and data operation and maintenance analysis services. It sends the retrieval strategy to the task collaborative orchestration scheduling subsystem, which performs orchestration demand analysis based on the scheduling strategy and application deployment plan, generates orchestration tasks, manages and maintains orchestration tasks, and sends scheduling tasks to the orchestration deployment management subsystem to execute the container orchestration scheduling. The orchestration deployment management subsystem implements the full life cycle management of container applications such as creation, startup, suspension, shutdown, and deletion based on the application programming management interface provided by the Ruihua system. The distributed resource and status perception subsystem is responsible for collecting the status of various software and hardware resources in the cluster and providing it to the group scheduling management platform and the task collaborative orchestration scheduling subsystem for scheduling decisions and status display.

Claims

1. A container orchestration and scheduling system based on the domestic Ruihua embedded real-time operating system, characterized in that: include: Group scheduling management platform: The group scheduling management platform provides resource collaborative management services for embedded clusters. The group scheduling management platform generates and distributes group terminal-side scheduling policy configurations based on user input adjustments through the collected group terminal operation status information and scenario analysis data. Distributed resource and status perception subsystem: used to collect the usage status of end-side group system resources, analyze and perceive system scenarios, and provide basic data reporting capabilities; Task collaborative scheduling and orchestration subsystem: Based on factors such as user needs, environmental constraints, and resource performance requirements, it performs modeling and solving to achieve optimal allocation of complex cluster tasks and supports unified scheduling and orchestration of tasks for multiple nodes in a group. Orchestration and deployment management subsystem: Provides a unified runtime environment based on architecture neutrality, supports container image deployment and operation; provides basic support capabilities including environmental status perception analysis, dynamic resource management, and application collaborative deployment; The group scheduling management platform interacts with users to create a container application deployment plan for the cluster, and provides users with resource status and environment data. In addition, it also provides task scheduling management interfaces and data operation and maintenance analysis services. It sends the retrieval strategy to the task collaborative orchestration scheduling subsystem. The task collaborative orchestration scheduling subsystem performs orchestration demand analysis based on the scheduling strategy and application deployment plan, generates orchestration tasks, manages and maintains orchestration tasks, and sends scheduling tasks to the orchestration deployment management subsystem to execute container orchestration scheduling. The orchestration and deployment management subsystem implements the full life cycle management of container applications, including creation, startup, suspension, shutdown, and deletion, based on the application programming management interface provided by the Ruihua system; the distributed resource and status perception subsystem is responsible for collecting the status of various software and hardware resources in the cluster and providing it to the group scheduling management platform and the task collaborative orchestration scheduling subsystem for scheduling decisions and status display.

2. A container orchestration and scheduling system based on the domestic Ruihua embedded real-time operating system as described in claim 1, characterized in that: The group scheduling management platform builds a visual cloud resource collaborative management platform based on the environmental perception data, resource monitoring information and task execution information transmitted by the cluster network terminals, realizes the task execution information, resource usage, environmental data monitoring and analysis, fault abnormality warning and processing functions of each terminal node, and provides data storage and query engine to support historical data playback and multi-dimensional analysis functions.

3. A container orchestration and scheduling system based on the domestic Ruihua embedded real-time operating system as described in claim 2, characterized in that: The group scheduling management platform includes: The resource monitoring and management module provides real-time monitoring and viewing of CPU, memory, storage, network, and power consumption resource information of distributed terminal nodes in the cluster network, realizes cloud-based collaborative resource monitoring and management, and provides visual operation and maintenance and scheduling optimization support capabilities for multi-terminal collaborative resource scheduling; The task scheduling management module provides information statistics and display functions for the number of tasks running on distributed terminal nodes, task status, scheduling success rate, task progress, and task completion rate; Environmental data management module provides access and management of terminal node time data, meteorological data, image data, and geographic spatial data, and performs cleaning and normalization of environmental data to improve data quality and provide data support capabilities for environmental change trend prediction and environmental early warning analysis; The data operation and maintenance analysis module builds data operation and maintenance management and analysis based on terminal resource usage, task scheduling, and environmental monitoring data collection. It supports users to perform data query and cluster analysis based on resource type, time range, and task type, and provides effective data support capabilities for container scheduling optimization and operation and maintenance management.

4. A container orchestration and scheduling system based on the domestic Ruihua embedded real-time operating system as described in claim 1, characterized in that: In the distributed resource and state perception subsystem, a multi-dimensional resource demand matching management strategy is established in view of the characteristics of resource diversity and task complexity in embedded clusters. A multi-dimensional resource comprehensive evaluation model is constructed based on computing, storage, energy consumption, and environmental perception. Through in-depth analysis of the nonlinear demand characteristics of system tasks for resources of different dimensions, on-demand allocation and efficient utilization of resources are achieved, providing support for container scheduling decisions.

5. A container orchestration and scheduling system based on the domestic Ruihua embedded real-time operating system as described in claim 4, characterized in that: The distributed resource and state perception subsystem includes: The resource information monitoring module monitors, cleans, and normalizes the hardware parameter specifications, CPU, memory, network, and energy consumption of different computing nodes in the cluster, establishes a structured resource database, and provides data support capabilities for subsequent modeling and analysis and evaluation; The resource map analysis module builds a resource evaluation index system model for CPU, memory, storage, energy consumption, and network resource data, dynamically calculates resource evaluation values, and provides resource trend analysis and load overload warning functions; The resource matching and scheduling module establishes a task resource intelligent matching strategy model based on the resource requirements of different tasks, mines the feature representation of task resources through deep learning, realizes adaptive optimal task resource matching strategy decision, and makes expected inferences on the task completion rate, resource utilization rate, task execution delay, etc. of each terminal, realizing dynamic task scheduling migration and resource scheduling optimization in a multi-objective task environment.

6. A container orchestration and scheduling system based on the domestic Ruihua embedded real-time operating system as described in claim 1, characterized in that: The task collaborative scheduling subsystem performs semantic analysis and feature extraction based on the collaborative task commands given by the user, extracts the execution method, location, time, participants, collected data type, and dependency relationships, and realizes the generation and release of task objects.

7. A container orchestration and scheduling system based on the domestic Ruihua embedded real-time operating system as described in claim 6, characterized in that: The task collaborative scheduling subsystem includes: The orchestration requirement analysis module is used to extract the basic information, constraints, discrete features, timing information and dependency conditions of the orchestration task; The task object generation module combines the extracted task information and discrete features to form a high-dimensional task intermediate vector, and maps its information to the task object for recording and storage; The task object management module stores and manages the task object data constructed and published according to the command data input by the user, and realizes the task object publishing, query and display functions.

8. A container orchestration and scheduling system based on the domestic Ruihua embedded real-time operating system as described in claim 1, characterized in that: The orchestration deployment management subsystem includes: The container lifecycle management module is responsible for the full process management of container applications; The task result feedback module keeps real-time status synchronization and feedback with the task scheduling system for task scheduling decisions, reports task execution progress, task execution status, and resource usage, so that the scheduling system can timely grasp the system's operating status and make dynamic adjustments based on feedback information; The container migration and fault recovery module establishes a distributed fault-tolerant scheduling mechanism. When the node load is too high, it performs task migration and status synchronization between nodes to improve system resource utilization. When a node fails, it uses a replica mechanism or a background restart mechanism to redistribute the execution of abnormal applications and ensure the reliability of container application execution.

Images