Distributed task scheduling method and system applicable to space-based distributed operating system

Abstract

The present invention discloses a distributed task scheduling method and system suitable for a space-based distributed operating system, the method comprising: real-time monitoring and collection of network communication data of the communication link between the satellite-borne computing power unit and other computing power units by a distributed communication tracking system integrated in the space-based system, and synchronization to the ground control system; a distributed task scheduler deployed in the ground control system analyzes the network communication data, selects the best satellite or satellite-borne computing power unit combination for distributed tasks for scheduling; and reschedules the already running distributed tasks in abnormal situations by a rescheduler deployed in the satellite-borne computing power unit. The present invention can intelligently select the computing power unit combination with the best communication quality for distributed tasks in a complex space-based system environment, ensure the efficient circulation of information interaction between distributed tasks, improve the accuracy and reliability of data exchange, and provide strong support for the continuity and stability of space-based computing tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of space-to-ground communication scenarios of space-based systems, and in particular to a distributed task scheduling method and system applicable to space-based distributed operating systems. Background Art

[0002] Cloud computing centers can provide universal and powerful computing resources for different applications and have been widely used in satellite data processing. However, due to the high demand for computing power and energy, it is difficult to deploy cloud computing centers on satellites. Therefore, the data generated on satellites usually needs to be transmitted back to ground-based cloud computing centers for processing.

[0003] In order to solve the problems of cloud computing centers being far away from data and large computing delays, an edge computing technology has been proposed. Edge computing technology deploys computing resources in the place closest to data generation, thereby providing low-latency computing services for applications and reducing the amount of data interacting with cloud computing centers. With the development of edge computing technology, using edge computing technology to build a space-based edge computing platform on satellites and provide low-latency computing services for satellite applications has become one of the important trends in the current aerospace development.

[0004] With the vigorous development of cloud computing, distributed storage and other technologies, the research on Kubernetes cluster architecture system has become a hot direction in the field of Internet communication technology due to its low cost, high efficiency and lightweight testing. Kubernetes container cluster management tools are mainly used to manage large-scale computing-intensive task applications in cloud computing environments. However, container resource requests are dynamically changing. In addition to CPU and memory, there are other resources such as storage and network bandwidth. The types are diverse and the corresponding resource types are also multi-dimensional resource types. However, the static scheduling algorithm of Kubernetes has a single input parameter, and the analysis of resource occupancy is rough. It has very large limitations and cannot fully utilize the various resources in each server (including heterogeneous resources on edge nodes in the architecture), nor can it provide corresponding assistance for container multi-dimensional resource requests. The existing scheduling algorithm cannot meet the needs of scheduling multi-dimensional resource types, so there is an urgent need for a distributed task scheduling system suitable for space-based distributed operating systems. Summary of the invention

[0005] The purpose of the present invention is to provide a distributed task scheduling method and system suitable for a space-based distributed operating system in view of the deficiencies of the prior art. When executing the distributed task of scheduling traffic conversion, the present invention can select the optimal satellite or computing unit combination for the distributed task for scheduling, ensure the real-time and reliability of data collaboration between distributed tasks, and fully meet the stability and reliability requirements of the overall system.

[0006] The objective of the present invention is achieved through the following technical solutions: In a first aspect, an embodiment of the present invention provides a distributed task scheduling method applicable to a space-based distributed operating system, comprising the following steps:

[0007] S1. In the space-based system, a satellite cluster covering the entire earth is built to achieve real-time and efficient satellite operation management and monitoring functions. Through the collaborative work of the satellite cluster and the ground control system, space mission requirements are jointly supported;

[0008] S2. Integrate a distributed communication tracking system in the space-based system, specifically: deploy a communication tracking system server in the ground control system, and deploy a communication tracking system client in each onboard computing unit in the satellite cluster; the communication tracking system client collects network communication data between the current onboard computing unit and other onboard computing units in the satellite cluster in real time, and pushes the collected network communication data to the communication tracking system server of the ground control system;

[0009] S3. Integrate an intelligent distributed task scheduler in the ground control system. During the deployment of distributed tasks, the distributed task scheduler intelligently analyzes the network communication data received by the communication tracking system server and makes the optimal scheduling decision for the distributed tasks based on the needs of the distributed tasks, that is, selects the optimal combination of satellites or onboard computing units for scheduling;

[0010] S4. A rescheduler is run in each onboard computing unit in the satellite cluster to periodically analyze the network communication data collected by the communication tracking system client during the operation of the distributed task to generate a corresponding network topology structure. When it is determined according to the generated network topology structure that the current onboard computing unit is not suitable for the distributed task to continue running, that is, when its communication quality is reduced, the optimal satellite or onboard computing unit is reselected for the distributed task for rescheduling.

[0011] Furthermore, the satellite cluster specifically comprises: utilizing Kubernetes technology to divide the onboard computing units of the space-based system into a plurality of star cluster nodes with different uses or complementary resources according to different satellite networking criteria, and the star cluster nodes are centrally and standardizedly managed through the standard API server of Kubernetes.

[0012] Furthermore, the distributed communication tracking system utilizes a distributed network to perform network monitoring, specifically including:

[0013] Deploy a communication tracking system server in the ground control system, which is used to implement remote inspection network services through its own service protocol;

[0014] A communication tracking system client is run on the satellite computing unit, and the communication tracking system client is used to collect network communication data between the current satellite computing unit and other satellite computing units in real time, and send the collected network communication data to the communication tracking system server;

[0015] The communication tracking system client is used to provide service mesh functions with minimal resource overhead, including service discovery, load balancing, and secure communication;

[0016] The communication tracking system client is used to perceive changes in the network topology in real time to dynamically adjust the service routing strategy; wherein the changes in the network topology specifically include the addition and exit of satellite computing power units and changes in network conditions.

[0017] Furthermore, the network communication data includes port status, ping packet response time, IDC bandwidth network traffic, network inflow and outflow rate, network utilization rate and network packet loss rate.

[0018] Furthermore, the distributed task scheduler intelligently analyzes the network communication data received by the communication tracking system server, and makes optimal scheduling decisions for distributed tasks based on the needs of distributed tasks, that is, selects the optimal combination of satellites or onboard computing units for scheduling, specifically including:

[0019] Preprocessing the network communication data received by the communication tracking system server to obtain preprocessed network communication data; wherein the preprocessing includes data cleaning, feature extraction, aggregate statistics and data analysis;

[0020] Based on the pre-processed network communication data, multiple evaluation indicators are obtained, including response time, traffic utilization, packet loss rate and stability. The response speed is determined according to the response time, and the communication quality score is given according to all the above evaluation indicators;

[0021] Based on the needs of distributed tasks, the optimal combination of satellites or onboard computing units is intelligently recommended for distributed tasks according to communication quality and response speed for scheduling.

[0022] Furthermore, the timing analysis of the network communication data collected by the communication tracking system client to generate a corresponding network topology structure specifically includes:

[0023] The communication tracking system client realizes automatic discovery and connection of satellite computing units through multicast and distributed hash tables;

[0024] The communication tracking system client realizes comprehensive and real-time monitoring of the communication links between satellites or onboard computing units, extracts the network communication data between satellites or onboard computing units, and stores it in the distributed database located on the onboard computing unit, and pushes the network communication data to the communication tracking system server in real time;

[0025] The communication tracking system server realizes comprehensive and real-time monitoring of the communication link between the satellite and the ground, extracts the network communication data between the satellite and the ground, and combines the network communication data between the satellites or onboard computing units pushed by the communication tracking system client to build a complete network topology structure and store it in the distributed database located in the ground control system.

[0026] A second aspect of an embodiment of the present invention provides a scheduling system for implementing the above-mentioned distributed task scheduling method applicable to a space-based distributed operating system, including:

[0027] A ground control system, built on Kubernetes, which is used to control the distributed scheduler to schedule according to the optimal combination of satellites or onboard computing units;

[0028] A distributed communication tracking system, comprising a communication tracking system server deployed in a ground control system and a communication tracking system client deployed on each onboard computing unit in a satellite cluster, wherein the communication tracking system client is used to collect network communication data between the current onboard computing unit and other onboard computing units in the satellite cluster in real time, and send the collected network communication data to the communication tracking system server;

[0029] A distributed scheduler is deployed in the ground control system. The distributed scheduler is used to intelligently analyze the network communication data received by the communication tracking system server during the deployment of distributed tasks, and make optimal scheduling decisions for distributed tasks based on the needs of distributed tasks, that is, to select the optimal combination of satellites or onboard computing units;

[0030] A rescheduler is deployed on an onboard computing unit in a satellite cluster. The rescheduler is used to periodically analyze network communication data collected by a communication tracking system client to generate a corresponding network topology structure during the operation of a distributed task. When it is determined according to the generated network topology structure that the current onboard computing unit is not suitable for the distributed task to continue to operate, that is, when its communication quality decreases, the optimal satellite or onboard computing unit is reselected for the distributed task for rescheduling; and

[0031] A distributed database is deployed in each onboard computing unit of the satellite cluster and the ground control system. The distributed database is used to store network communication data and metadata of distributed tasks.

[0032] Furthermore, the distributed scheduler is responsible for selecting satellites or onboard computing units for deployment of distributed tasks, specifically including: when allocating onboard computing units for distributed tasks, a complete network topology structure is constructed based on the network communication data of the communication links between the satellite and the ground, between satellites, and between onboard computing units summarized by the communication tracking system server, and based on the requirements of the distributed tasks and the complete network topology structure, the optimal combination of satellites or onboard computing units is selected for deployment, scheduling and operation of the distributed tasks.

[0033] Furthermore, the rescheduler is responsible for selecting satellites or onboard computing units when rescheduling an already running distributed task, specifically including: when rescheduling any distributed task, obtaining the current complete network topology structure based on the network communication data of the inter-satellite and inter-onboard computing unit communication links collected by the communication tracking system client, and selecting the optimal combination of satellites or onboard computing units based on the requirements of the distributed task and the current complete network topology structure for the distributed task to be redeployed and scheduled for operation.

[0034] Furthermore, the distributed database specifically includes: a distributed database deployed on the onboard computing unit for local storage of network communication data and metadata of distributed tasks, and a distributed database deployed on the ground control system for local storage of network communication data.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] (1) The present invention improves the task scheduling mechanism of Kubernetes, especially in the distributed task execution scenario, by introducing a distributed task scheduling system based on the communication topology of computing power units, optimizing the task scheduling process and improving the communication quality and resource utilization between tasks.

[0037] (2) The Agent provided by the communication tracking system of the present invention uses kernel-level network interfaces (such as eBPF interfaces, Netfilter interfaces, etc.) or user space tools (such as libpcap) to capture the incoming and outgoing data packets on the links of different communication protocols in real time; through these data packets, the traffic throughput, including upload speed and download speed, is calculated in real time.

[0038] (3) The Agent provided by the communication tracking system of the present invention measures the end-to-end delay by adding a timestamp to the data packet or using the time information during the TCP three-way handshake / QUIC connection establishment process; for WebSocket, the delay can be evaluated by sending a heartbeat packet and recording the round-trip time (RTT).

[0039] (4) The present invention monitors the transport layer of QUIC and WebSocket through a sequence number or confirmation response mechanism to identify packet loss.

[0040] (5) The rescheduling system in the present invention monitors the communication status between distributed tasks in real time and reselects the best computing unit for the distributed tasks, thereby ensuring the optimality of communication between distributed tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of a distributed task scheduling method applicable to a space-based distributed operating system of the present invention;

[0042] Figure 2 It is a schematic diagram of the framework of the distributed task scheduling system applicable to the space-based distributed operating system of the present invention;

[0043] Figure 3 is a component diagram of the distributed task scheduling system of the present invention;

[0044] Figure 4 is a component diagram of the rescheduling system of the present invention;

[0045] Figure 5 It is a data flow diagram of the distributed task scheduling system of the present invention;

[0046] Figure 6 It is the data flow diagram of the rescheduling system of the present invention. DETAILED DESCRIPTION

[0047] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Obviously, the drawings used in the following description are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can also be obtained based on these drawings without paying creative labor. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the present invention. On the contrary, they are only examples of devices and methods consistent with some aspects of the present invention as detailed in the attached claims.

[0048] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0049] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0050] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0051] See also Figure 1 The distributed task scheduling method applicable to the space-based distributed operating system of the present invention specifically comprises the following steps:

[0052] S1. Within the space-based system, a satellite cluster that fully covers the earth is constructed to achieve real-time and efficient satellite operation management and monitoring functions. Through the collaborative work of the satellite cluster and the ground control system, the complex and ever-changing space mission requirements are jointly supported.

[0053] Furthermore, the satellite cluster is specifically as follows: using Kubernetes technology, the onboard computing units of the space-based system are divided into multiple cluster nodes with different uses or complementary resources according to different satellite networking criteria. The cluster nodes are centrally and standardizedly managed through the standard API server (api-server) of Kubernetes, which can greatly improve the flexibility and management efficiency of space-based system resources.

[0054] It should be noted that some onboard computing units are used for telemetry, some are used for navigation, and some are used for weather monitoring. Therefore, the onboard computing units of the space-based system can be divided into multiple star cluster nodes with different specific purposes. In addition, some distributed tasks will be performed on the satellite, and there will be data interaction between each subtask. This is a scenario of resource complementarity. Based on this scenario, the onboard computing units of the space-based system can be divided into multiple star cluster nodes with complementary resources.

[0055] S2. Integrate an advanced distributed communication tracking system into the space-based system. Specifically, deploy a communication tracking system server (Server) in the ground control system, and deploy a communication tracking system client (Agent) in each onboard computing unit in the satellite cluster. The communication tracking system client collects the network communication data between the current onboard computing unit and other onboard computing units in the satellite cluster in real time, and pushes the collected network communication data to the communication tracking system server of the ground control system.

[0056] Furthermore, the distributed communication tracking system utilizes the distributed network to perform network monitoring, specifically including:

[0057] S21. Deploy a communication tracking system server in the ground control system. The communication tracking system server is used to implement remote inspection network services through its own simple service protocol.

[0058] S22. Run the communication tracking system client on the satellite computing unit. The communication tracking system client is used to collect the network communication data between the current satellite computing unit and other satellite computing units in real time, realize the network monitoring between local and remote, and send the collected network communication data to the communication tracking system server for further processing in subsequent steps.

[0059] S23. The communication tracking system client is used to provide service grid functions with minimal resource overhead, including service discovery, load balancing, and secure communication (such as mTLS).

[0060] S24. The communication tracking system client is used to perceive the changes in the network topology in real time to dynamically adjust the service routing strategy to ensure the high availability of the service. The changes in the network topology specifically include the addition and exit of satellite computing units and changes in network conditions.

[0061] Furthermore, network communication data includes, but is not limited to: port status, ping packet response time, IDC bandwidth network traffic, network inflow and outflow rate, network utilization rate and network packet loss rate, etc. Among them, port status: monitor the port status on the network device, including the port open / closed status, traffic, etc. Ping packet response time: measure the network delay by sending ICMP request packets, which is the ping packet response time. IDC bandwidth network traffic: record the network traffic inside and outside the IDC. Network inflow and outflow rate: monitor the upload rate and download rate of the network. Network utilization rate: count the usage of network bandwidth. Network packet loss rate: count the packet loss ratio during network transmission. The above network communication data can be collected and obtained by using tools such as ping, traceroute, netstat, iftop, etc., deploying SNMP probes to monitor network devices, and using traffic analysis tools such as nfdump, wireshark, etc. to capture network traffic data.

[0062] S3. Integrate an intelligent distributed task scheduler in the ground control system. During the deployment of distributed tasks, the distributed task scheduler performs intelligent analysis on the network communication data received by the communication tracking system server, and makes the optimal scheduling decision for the distributed tasks based on the special needs of the distributed tasks, that is, selects the optimal combination of satellites or onboard computing units for scheduling.

[0063] Furthermore, the distributed task scheduler intelligently analyzes the network communication data received by the communication tracking system server, and makes the best scheduling decision for the distributed task based on the special needs of the distributed task, that is, selects the best combination of satellites or onboard computing units for scheduling, including:

[0064] S31. Preprocess the network communication data received by the communication tracking system server to obtain preprocessed network communication data; wherein the preprocessing includes data cleaning, feature extraction, aggregate statistics and data analysis.

[0065] Specifically, the network communication data is first cleaned to remove invalid data and outliers; then feature extraction is performed to extract useful features from the raw data, such as average response time, peak traffic, etc.; the data is then aggregated and statistically analyzed, such as daily / weekly / monthly average response time, traffic, etc.; finally, data analysis is performed, specifically by using statistical analysis methods (such as regression analysis, correlation analysis, etc.) to understand the relationship between data.

[0066] S32. Based on the pre-processed network communication data, a plurality of evaluation indicators are obtained, wherein the evaluation indicators include response time, traffic utilization, packet loss rate and stability, etc. The response speed is determined according to the response time, and a communication quality (QoS) score is given according to all the above evaluation indicators.

[0067] Among them, response time is used to measure network delay, such as the average ping packet response time; traffic utilization is used to evaluate the usage of network bandwidth, such as the average bandwidth utilization; packet loss rate is used to evaluate packet loss in network transmission, such as the average packet loss rate; stability is obtained by evaluating the stability of the network through historical data, such as the standard deviation of response time.

[0068] S33. Based on the special needs of distributed tasks, the optimal combination of satellites or onboard computing units is intelligently recommended for distributed tasks according to communication quality and response speed for scheduling, which can ensure the communication quality and response speed between distributed tasks and ensure the reliability and efficiency of data transmission in complex space-based system network environments.

[0069] It should be understood that all satellite-borne applications corresponding to distributed tasks will run on satellite-borne computing units. The satellite-borne computing units in the optimal combination of satellite-borne computing units may be in the same satellite or on different satellites. Therefore, the optimal scheduling decision is to select the optimal satellite or satellite-borne computing unit combination for scheduling.

[0070] For example, assume that there are three scenarios, and their corresponding task requirements are: ① There will be data interaction between subtask 1 and subtask 2 in the distributed task; ② There will be data interaction between subtask 1, subtask 2, and subtask 3 in the distributed task, but the data interaction frequency and bandwidth requirements are different; ③ There are some resource preemption situations between subtask 1 and subtask 2 in the distributed task, and they need to be deployed on different satellite computing units. The optimal scheduling decision made for the above-mentioned distributed tasks with different requirements specifically includes: allocating two satellite computing units with better communication quality and higher bandwidth to the subtask combination with high data interaction frequency and high bandwidth requirements in the distributed task; if the subtasks do not require exclusive satellite computing units and the communication bandwidth requirements are high, they can be run on one satellite computing unit; if there is not much data interaction between subtask 1 and subtask 2 in the distributed task or real-time communication is not required, they can also be allocated to different satellites.

[0071] S4. A rescheduler is run in each onboard computing unit in the satellite cluster to periodically analyze the network communication data collected by the communication tracking system client during the operation of the distributed task to generate a corresponding network topology structure. When it is determined according to the generated network topology structure that the current onboard computing unit is not suitable for the distributed task to continue running, that is, when its communication quality is reduced, the optimal satellite or onboard computing unit is reselected for the distributed task for rescheduling.

[0072] Furthermore, the functions of the rescheduler specifically include: based on the network communication data collected by the communication tracking system client, generating the corresponding network topology structure through in-depth analysis of the network communication data. When the communication quality between the current distributed task and other distributed tasks decreases, the rescheduler can intelligently reselect the optimal satellite or onboard computing unit for the running task for rescheduling, which can further improve the communication quality and response speed between distributed tasks, and ensure the reliability and efficiency of data transmission in the complex space-based system network environment.

[0073] For example, assuming that subtask 1 and subtask 2 in the distributed task were originally running on satellite computing power unit 1 and satellite computing power unit 2, when the communication tracking system client detects that the communication status between satellite computing power unit 1 and satellite computing power unit 2 is no longer suitable for data interaction between subtask 1 and subtask 2, and there is another satellite computing power unit with better communication quality, the rescheduler makes a new scheduling decision and reschedules it to the new satellite computing power unit.

[0074] Furthermore, the network communication data collected by the communication tracking system client is regularly analyzed to generate a corresponding network topology structure, specifically including:

[0075] S41. The communication tracking system client realizes automatic discovery and connection of satellite computing units through multicast and distributed hash table (DHT).

[0076] Specifically, the communication tracking system client implements the automatic discovery capability of the relay computing unit, and its mechanism is mainly implemented through two methods: multicast and distributed hash table, aiming to ensure efficient discovery and connection establishment between nodes even in complex network environments. Combining these two methods, the communication tracking system client can effectively realize the automatic discovery and connection of satellite-borne computing units, ensuring the stability and efficiency of communication between services in different network environments. Among them, the relay computing unit refers to the satellite-borne computing unit that plays a relay role.

[0077] S42. The communication tracking system client realizes comprehensive and real-time monitoring of the communication links between satellites or onboard computing units, extracts the network communication data between satellites or onboard computing units, and stores it in the distributed database (ETCD) located on the onboard computing unit, and pushes the network communication data to the communication tracking system server in real time.

[0078] Specifically, the communication tracking system client realizes comprehensive and real-time monitoring of the communication links between satellites or onboard computing units, extracts network communication data between satellites or onboard computing units, such as traffic throughput, delay, packet loss rate and other information, stores it in a distributed database located on the onboard computing unit, and pushes the network communication data between satellites or onboard computing units to the communication tracking system server in real time, providing strong data support for building the network topology structure.

[0079] S43. The communication tracking system server realizes comprehensive and real-time monitoring of the communication link between the satellite and the ground, extracts the network communication data between the satellite and the ground, and combines the network communication data between the satellites or onboard computing units pushed by the communication tracking system client to build a complete network topology structure, and stores it in the ETCD located in the ground control system.

[0080] Specifically, the communication tracking system server realizes comprehensive and real-time monitoring of the communication link between the satellite and the ground, extracts the network communication data between the satellite and the ground, such as traffic throughput, delay, packet loss rate and other information, and combines the network communication data between the satellites or onboard computing units pushed by the communication tracking system client to build a complete network topology structure and store it in the ETCD located in the ground control system.

[0081] It should be noted that when the distributed task scheduler located in the ground control system or the rescheduler located on the satellite computing power unit allocates satellite computing power units for distributed tasks, it can select the optimal combination of satellites or satellite computing power units based on the complete network topology structure provided by the communication tracking system server.

[0082] It is worth mentioning that an embodiment of the present invention also provides a scheduling system for implementing the distributed task scheduling method applicable to a space-based distributed operating system in the above embodiment.

[0083] The scheduling system described in this embodiment is deployed on the basis of the KubeEdge cluster to achieve the purpose of selecting the optimal combination of onboard computing units for task operation in the scenario of running distributed tasks. The structure of the corresponding scheduling system is as follows: Figure 2 and Figure 3 As shown, the scheduling system specifically includes a ground control system, a distributed communication tracking system, a distributed scheduler (Distributed-Scheduler), a rescheduler and a distributed database (ETCD).

[0084] The above parts are explained one by one below.

[0085] In this embodiment, a Kubernetes-based ground control system is used to implement control of a distributed scheduler so as to perform scheduling according to the optimal combination of satellites or onboard computing units.

[0086] The ground control system is built on Kubernetes to facilitate the control of KubeEdge. Figure 2As shown. The bottom layer of the ground control system is composed of various Go-Client components, including Kubernetes' API server (Kube-apiserver), edge node controller (EdgeController), controller manager (ControllerManager) and distributed scheduler (DistributedScheduler). Among them, the Kube-apiserver component is the core of Kubernetes cluster management. It is responsible for providing REST API interface, authentication and authorization, managing cluster state changes, realizing data interaction and communication, and providing resource quota control and security mechanisms. Kube-apiserver is the only entrance to the Kubernetes cluster. Through the REST API interface it provides, users and other Kubernetes components can interact with the cluster; in addition, DistributedScheduler is responsible for allocating newly created Pods to the most suitable satellite computing unit in the cluster through intelligent scheduling strategies. Among them, the REST API interface is an application programming interface. Finally, the cloud server (CloudHub) plays the role of an intermediary between the ground control system and the satellite end. Its main functions include downlink distribution messages (these messages encapsulate K8s resource events, such as Pod application creation (add) / update (update) / deletion (delete)), and receiving and sending uplink messages from onboard computing units to controller components (controllers). CloudHub is also responsible for monitoring changes in the ground control system, caching, and sending messages to EdgeHub.

[0087] In addition, in the ground control system, the EdgeController component is a core component of KubeEdge, which is mainly responsible for the lifecycle management of onboard applications. When the Kubernetes Master node receives a request to create, update, or delete a Pod, the EdgeController converts these operations into messages pushed to the onboard computing unit. The EdgeCore running on the onboard computing unit receives these messages and performs corresponding operations, such as creating or destroying Pod instances, updating Pod configurations, etc.

[0088] The EdgeController component also participates in the synchronization of the metadata of the onboard computing power unit. EdgeController collaborates with EdgeCore to ensure that the metadata of the onboard computing power unit (such as onboard computing power unit information, Pod, configuration information (ConfigMap), secret key, etc.) is consistent with the ground control system. When the resource information of the ground control system changes, EdgeController will notify these changes to the corresponding onboard computing power unit, thereby realizing the synchronous update of resource data between the satellite and the ground; and events occurring on the onboard computing power unit, such as the creation, deletion, and update of Pod, will be collected by EdgeCore and synchronized to the ground control system through EdgeController, so that cluster managers can fully grasp the operation of the entire cluster (including the onboard computing power unit) in the ground control system.

[0089] In this embodiment, the distributed communication tracking system includes a communication tracking system server deployed in the ground control system and a communication tracking system client deployed on each satellite-borne computing unit in the satellite cluster. It can comprehensively optimize the efficiency and stability of satellite-to-ground and inter-satellite communications. The communication tracking system client is used to collect network communication data between the current satellite-borne computing unit and other satellite-borne computing units in the satellite cluster in real time, and send the collected network communication data to the communication tracking system server.

[0090] Specifically, the communication tracking system client is deployed on each onboard computing unit in the satellite cluster. First, it monitors and collects the network communication data of all communication links between the current onboard computing unit and other onboard computing units in the satellite cluster in real time, including but not limited to the link throughput, delay and packet loss rate indicators, and pushes these network communication data to the communication tracking system server in real time. On the other hand, the communication tracking system server located in the ground control system plays a central role in data aggregation and analysis. It receives network communication data pushed by each onboard computing unit client, and uses existing intelligent algorithms, such as Long Short-Term Memory (LSTM), Graph Convolutional Network (GCN), Gated Recurrent Units (GRU), etc., to conduct in-depth analysis of these network communication data to construct the network topology between each onboard computing unit. The distributed scheduler will implement the optimal satellite or onboard computing unit combination screening for distributed tasks based on the network topology. Among them, LSTM is suitable for processing sequence data and can be used to predict network traffic, response time and other indicators; GCN is suitable for analyzing network topology, such as identifying bottleneck nodes and optimizing routing; GRU is more efficient than LSTM in some cases and can be used to predict network performance indicators.

[0091] It should be noted that the distributed communication tracking system is composed of the communication tracking system server deployed in the ground control system and the communication tracking system client deployed in each onboard computing unit in the satellite cluster. As a network monitoring component, it can monitor many parameters of the network and the health and integrity of servers, virtual machines, applications, services, databases, websites, clouds, etc. Among them, the communication tracking system client monitors the network status, involves tracking bandwidth usage, packet loss, latency and overall network monitoring status, collects various indicator data of systems and services, and stores the obtained network communication data in the nearest on-board distributed database in real time, and sends it synchronously to the communication tracking system server deployed in the ground control system. The communication tracking system server receives the network communication data from the communication tracking system client. Of course, it can also use the agent program to actively poll the monitoring target (passive monitoring, that is, the communication tracking system server initiates a connection to the communication tracking system client) to obtain network communication data, and aggregate, analyze, store and process these network communication data for use by external components.

[0092] In this embodiment, the distributed scheduler is deployed in the ground control system, such as Figure 3As shown, the distributed scheduler is used to perform intelligent analysis on the network communication data received by the communication tracking system server during the deployment of distributed tasks, and make optimal scheduling decisions for distributed tasks based on the needs of distributed tasks, that is, to select the optimal combination of satellites or onboard computing units.

[0093] Specifically, the distributed scheduler is responsible for selecting satellites or onboard computing units for the deployment of distributed tasks, including: when allocating onboard computing units for distributed tasks, a complete network topology structure is constructed based on the network communication data of the satellite-to-ground, inter-satellite, and inter-onboard computing unit communication links summarized by the communication tracking system server; based on the needs of the distributed tasks and the complete network topology structure, the optimal combination of satellites or onboard computing units is selected for the deployment, scheduling and operation of the distributed tasks.

[0094] In this embodiment, the rescheduler is deployed on the onboard computing unit in the satellite cluster, such as Figure 3 and Figure 4 As shown, the rescheduler is used to periodically analyze the network communication data collected by the communication tracking system client during the operation of the distributed task to generate a corresponding network topology structure. When it is determined according to the generated network topology structure that the current satellite-borne computing power unit is not suitable for the distributed task to continue running, that is, when its communication quality is reduced, the optimal satellite or satellite-borne computing power unit is reselected for the distributed task for rescheduling.

[0095] Specifically, the rescheduler is responsible for selecting satellites or onboard computing units when rescheduling an already running distributed task, including: when rescheduling any distributed task, obtaining the current complete network topology structure based on the network communication data of the inter-satellite and inter-onboard computing unit communication links collected by the communication tracking system client, and selecting the optimal combination of satellites or onboard computing units based on the requirements of the distributed task and the current complete network topology structure for the distributed task to be redeployed and scheduled for operation.

[0096] In this embodiment, the distributed database is deployed in each onboard computing unit of the satellite cluster and the ground control system, such as Figure 3 As shown, the distributed database is used to store network communication data and metadata of distributed tasks.

[0097] Furthermore, the distributed database specifically includes: a distributed database deployed on the onboard computing unit for local storage of network communication data and metadata of distributed tasks, and a distributed database deployed on the ground control system for local storage of network communication data. The distributed database provides local storage and access functions for data, so that the generated data can be stored in a place that is physically closer to the source of the data at the first time, thereby greatly reducing data transmission delays and enhancing real-time performance, which is conducive to accelerating the data processing speed of edge terminals such as Internet of Things (IoT) devices, mobile devices, and satellite computing units.

[0098] In some embodiments, users deploy distributed tasks, and the corresponding scheduling system data flow is as follows: Figure 5 As shown, on the satellite computing unit, the communication tracking system client monitors the network status between satellites / satellite computing units, updates the corresponding network communication data extracted in real time, and pushes it to the communication tracking system server; the distributed task scheduler then updates the original network communication data between satellites / satellite computing units according to the received network communication data between satellites / satellite computing units; the communication tracking system server monitors the network status of the satellite-to-ground link in real time, and then updates the network communication data between satellites and ground. Prepare to deploy distributed tasks and prepare to deploy corresponding satellite applications in batches. Obtain the application list of the nodes to be allocated (satellites or satellite computing units); the distributed scheduler obtains the complete network topology based on the latest network communication data between satellites and ground, between satellites, and satellite computing units, and parses the demand information of distributed tasks, and then selects the satellite or satellite computing unit combination with the best communication quality and response speed based on the complete network topology and the needs of distributed tasks. The distributed scheduler deploys the satellite applications to be deployed in batches to the optimal satellite or satellite computing unit combination to realize the deployment, scheduling and operation of distributed tasks.

[0099] In some other embodiments, when the satellite cloud ground control system is disconnected, that is, when the satellite is not in the ground station communication window period, the distributed tasks currently being run by the satellite will be rescheduled, and the corresponding data flow of the rescheduling system is as follows: Figure 6As shown. The communication tracking system clients running on the satellite computing unit 1 and the satellite computing unit 2 monitor the network communication data between the satellite / satellite computing units in real time and update it in real time; the communication tracking system clients running on the satellite computing unit 1 and the satellite computing unit 2 obtain the network communication data between the satellite / satellite computing units regularly. The rescheduler regularly analyzes the network communication data between the satellite / satellite computing units and generates the corresponding network topology structure; obtains the current application list and detailed data, and each satellite computing unit obtains its own detailed data. At the beginning, the satellite application corresponding to the distributed task is deployed and deployed on the satellite computing unit 1. When the current satellite computing unit is not suitable for the distributed task to continue running according to the latest network topology, that is, when its communication quality decreases, the optimal satellite or satellite computing unit is reselected for the distributed task for rescheduling. Here, it is assumed that the satellite computing unit 2 is the optimal satellite computing unit, then the satellite application running on the satellite computing unit 1 needs to be migrated to the satellite computing unit 2, and it will be cleared after the migration is completed.

[0100] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed task scheduling method applicable to a space-based distributed operating system, characterized in that: The following steps are involved: S1. In the space-based system, a satellite cluster covering the entire earth is built to achieve real-time and efficient satellite operation management and monitoring functions. Through the collaborative work of the satellite cluster and the ground control system, space mission requirements are jointly supported; S2. Integrate a distributed communication tracking system into the space-based system, specifically: deploy the communication tracking system server in the ground control system, and deploy the communication tracking system client in each onboard computing unit in the satellite cluster; The communication tracking system client collects the network communication data between the current onboard computing unit and other onboard computing units in the satellite cluster in real time, and pushes the collected network communication data to the communication tracking system server of the ground control system; S3. Integrate an intelligent distributed task scheduler in the ground control system. During the deployment of distributed tasks, the distributed task scheduler intelligently analyzes the network communication data received by the communication tracking system server and makes the optimal scheduling decision for the distributed tasks based on the needs of the distributed tasks, that is, selects the optimal combination of satellites or onboard computing units for scheduling; The distributed task scheduler intelligently analyzes the network communication data received by the communication tracking system server, and makes the optimal scheduling decision for the distributed task based on the needs of the distributed task, that is, selects the optimal combination of satellites or onboard computing units for scheduling, specifically including: Preprocessing the network communication data received by the communication tracking system server to obtain preprocessed network communication data; wherein the preprocessing includes data cleaning, feature extraction, aggregate statistics and data analysis; Based on the pre-processed network communication data, multiple evaluation indicators are obtained, including response time, traffic utilization, packet loss rate and stability. The response speed is determined according to the response time, and the communication quality score is given according to all the above evaluation indicators; Based on the needs of distributed tasks, the optimal combination of satellites or onboard computing units is intelligently recommended for distributed tasks according to communication quality and response speed for scheduling; S4. A rescheduler is run in each onboard computing unit in the satellite cluster to periodically analyze the network communication data collected by the communication tracking system client during the operation of the distributed task to generate a corresponding network topology structure. When it is determined according to the generated network topology structure that the current onboard computing unit is not suitable for the distributed task to continue running, that is, when its communication quality is reduced, the optimal satellite or onboard computing unit is reselected for the distributed task for rescheduling.

2. The distributed task scheduling method applicable to a space-based distributed operating system according to claim 1, characterized in that: The satellite cluster specifically comprises: using Kubernetes technology, dividing the onboard computing power units of the space-based system into multiple cluster nodes with different uses or complementary resources according to different satellite networking criteria, and the cluster nodes are centrally and standardizedly managed through the standard API server of Kubernetes.

3. The distributed task scheduling method applicable to a space-based distributed operating system according to claim 1, characterized in that: The distributed communication tracking system uses a distributed network to perform network monitoring, specifically including: Deploy a communication tracking system server in the ground control system, which is used to implement remote inspection network services through its own service protocol; A communication tracking system client is run on the satellite computing unit, and the communication tracking system client is used to collect network communication data between the current satellite computing unit and other satellite computing units in real time, and send the collected network communication data to the communication tracking system server; The communication tracking system client is used to provide service mesh functions with minimal resource overhead, including service discovery, load balancing, and secure communication; The communication tracking system client is used to perceive changes in the network topology in real time to dynamically adjust the service routing strategy; wherein the changes in the network topology specifically include the addition and exit of satellite computing power units and changes in network conditions.

4. The distributed task scheduling method applicable to a space-based distributed operating system according to claim 1, characterized in that: The network communication data includes port status, ping packet response time, IDC bandwidth network traffic, network inflow and outflow rate, network utilization rate and network packet loss rate.

5. The distributed task scheduling method applicable to a space-based distributed operating system according to claim 1, characterized in that: The timing analysis of the network communication data collected by the communication tracking system client to generate a corresponding network topology structure specifically includes: The communication tracking system client realizes automatic discovery and connection of satellite computing units through multicast and distributed hash tables; The communication tracking system client realizes comprehensive and real-time monitoring of the communication links between satellites or onboard computing units, extracts the network communication data between satellites or onboard computing units, and stores it in the distributed database located on the onboard computing unit, and pushes the network communication data to the communication tracking system server in real time; The communication tracking system server realizes comprehensive and real-time monitoring of the communication link between the satellite and the ground, extracts the network communication data between the satellite and the ground, and combines the network communication data between the satellites or onboard computing units pushed by the communication tracking system client to build a complete network topology structure and store it in the distributed database located in the ground control system.

6. A scheduling system for implementing the distributed task scheduling method applicable to a space-based distributed operating system as described in any one of claims 1 to 5, characterized in that: include: A ground control system, built on Kubernetes, which is used to control the distributed scheduler to schedule according to the optimal combination of satellites or onboard computing units; A distributed communication tracking system, comprising a communication tracking system server deployed in a ground control system and a communication tracking system client deployed on each onboard computing unit in a satellite cluster, wherein the communication tracking system client is used to collect network communication data between the current onboard computing unit and other onboard computing units in the satellite cluster in real time, and send the collected network communication data to the communication tracking system server; A distributed scheduler is deployed in the ground control system. The distributed scheduler is used to intelligently analyze the network communication data received by the communication tracking system server during the deployment of distributed tasks, and make optimal scheduling decisions for distributed tasks based on the needs of distributed tasks, that is, to select the optimal combination of satellites or onboard computing units; A rescheduler is deployed on the onboard computing unit in the satellite cluster. The rescheduler is used to periodically analyze the network communication data collected by the communication tracking system client to generate a corresponding network topology structure during the operation of the distributed task. When it is determined according to the generated network topology structure that the current onboard computing unit is not suitable for the distributed task to continue running, that is, when its communication quality decreases, the optimal satellite or onboard computing unit is reselected for the distributed task for rescheduling. and A distributed database is deployed in each onboard computing unit of the satellite cluster and the ground control system. The distributed database is used to store network communication data and metadata of distributed tasks.

7. The dispatching system according to claim 6, characterized in that: The distributed scheduler is responsible for selecting satellites or onboard computing units for deployment of distributed tasks, specifically including: when allocating onboard computing units for distributed tasks, a complete network topology is constructed based on the network communication data of the communication links between the satellite and the ground, between satellites, and between onboard computing units summarized by the communication tracking system server, and based on the requirements of the distributed tasks and the complete network topology, the optimal combination of satellites or onboard computing units is selected for deployment, scheduling and operation of the distributed tasks.

8. The dispatching system according to claim 6, characterized in that: The rescheduler is responsible for selecting satellites or onboard computing units when rescheduling an already running distributed task, specifically including: when rescheduling any distributed task, obtaining the current complete network topology structure based on the network communication data of the inter-satellite and inter-onboard computing unit communication links collected by the communication tracking system client, and selecting the optimal combination of satellites or onboard computing units based on the requirements of the distributed task and the current complete network topology structure for the distributed task to be redeployed and scheduled for operation.

9. The dispatching system according to claim 6, characterized in that: The distributed database specifically includes: a distributed database deployed on the satellite computing unit for local storage of network communication data and metadata of distributed tasks, and a distributed database deployed on the ground control system for local storage of network communication data.

Images