A satellite communication terminal simulation task scheduling system and device based on intelligent scheduling

Abstract

This invention discloses a satellite communication terminal simulation task orchestration system and device based on intelligent scheduling. It adopts a microservice architecture, with five modules working collaboratively via RESTful APIs or gRPC. The system constructs a multi-dimensional global resource view, combining historical data to drive a regression prediction model to quantify resource requirements. The intelligent scheduling decision module models complex task allocation as a multi-dimensional optimization problem, using the PSO algorithm combined with a multi-objective fitness function to find the optimal solution, supplemented by a task preprocessing mechanism to improve efficiency and effectiveness. The system manages task dependencies using a DAG and deeply integrates with Kubernetes to achieve automated task orchestration, status monitoring, and fault recovery. This effectively solves the pain points of traditional scheduling, such as resource fragmentation and static configuration waste, achieving dual-dimensional global dynamic scheduling of business and resources. This significantly improves cluster resource utilization and simulation execution efficiency, adapting to the needs of large-scale, high-concurrency simulation scenarios.

Description

Technical Field

[0001] This invention relates to the field of digital simulation technology, and more particularly to the field of task orchestration and intelligent resource scheduling in satellite communication terminal simulation scenarios. Background Technology

[0002] In the field of digital simulation, large-scale simulation scenarios involving multiple users and multiple tasks have become widespread. Efficient resource management and scheduling under high concurrency has become a critical issue that urgently needs to be addressed. In resource-constrained, dynamic, and complex simulation environments, traditional task scheduling and resource allocation schemes are prone to problems such as resource contention, task delays, and system overload, making them difficult to adapt to actual needs. Therefore, there is an urgent need to use intelligent algorithms to achieve dynamic orchestration and optimized resource scheduling of simulation tasks, thereby improving resource utilization efficiency.

[0003] The field of digital simulation task orchestration and scheduling has formed some well-known technical foundations: using the YAML standardized language combined with Docker container technology to realize simulation task modeling and resource isolation; using tools such as Grafana and Prometheus to monitor resource status in real time, and relying on systems such as Hadoop YARN and Mesos to complete the global management of resource and task metadata; using directed acyclic graphs (DAGs) to describe task dependencies, and attempting to apply particle swarm optimization algorithms to resource allocation.

[0004] The closest technical solution to this invention is a Kubernetes-based semi-dynamic scheduling and orchestration method for simulation tasks. Its core is as follows: the Docker image, CPU, GPU and other resource requirements of the simulation task are defined by YAML file, and the dependencies between tasks are described by DAG; the scheduler parses the DAG structure, and when the task reaches the executable state, it matches the task with a node with sufficient resources in the cluster for execution according to the random allocation or minimum load priority strategy, and schedules subsequent tasks in sequence after completion; at the same time, it monitors the task execution status and automatically retryes failed tasks.

[0005] While the aforementioned existing technologies can achieve basic orchestration and scheduling of simulation tasks, they have significant shortcomings for large-scale, high-concurrency simulation scenarios, failing to address issues such as resource fragmentation, waste, and task congestion. Specifically: 1. No business-level optimization, business-level objectives are suboptimal: Scheduling decisions are based solely on hardware resource availability, without incorporating business-level objectives such as task completion time and business priority. It only ensures task operation, resulting in a suboptimal scheduling solution at the business level.

[0006] 2. Lack of global forward planning can easily lead to resource fragmentation: Real-time scheduling only perceives the current resource status of the cluster, without a complete picture of the task queue, only matches a single task node, and lacks batch global planning and resource prediction, resulting in scattered resource occupation, low utilization, and increased total task duration.

[0007] 3. Static resource allocation and manual application lead to resource waste: Resource requirements are pre-submitted by users and statically fixed, making it impossible to dynamically adjust according to the real-time status of the cluster and the actual consumption of tasks. Excessive user reservation leads to idle resources and intensifies competition in resource-constrained scenarios.

[0008] 4. Poor adaptability to complex dynamic scenarios and weak business awareness: The simple scheduling strategy is adopted without customized optimization for simulation scenarios. It is not adapted to heterogeneous resources, complex task dependencies, and high concurrency impacts, which can easily lead to task congestion and system overload.

[0009] The aforementioned shortcomings of existing technologies prevent them from meeting the demands of large-scale batch processing, AI training, and other simulation scenarios. Therefore, there is an urgent need for an intelligent scheduling method that can achieve global resource planning and integrate business dimensions to address the deficiencies of existing technologies. Summary of the Invention

[0010] In view of this, this invention proposes a simulation task orchestration method, system, device, and storage medium based on intelligent scheduling. It constructs a unified, real-time, multi-dimensional global resource state model and combines it with historical data to predict resource demand. The particle swarm optimization algorithm or its improved variants are applied to simulation task orchestration and resource scheduling. The task allocation problem is modeled as a multi-dimensional optimization problem to find the globally optimal solution. Simultaneously, the scheduling decision logic is extended based on Kubernetes orchestration capabilities, incorporating intelligent algorithms to achieve global dynamic scheduling in both business and resource dimensions. This solves the technical problems of existing technologies in large-scale, high-concurrency simulation scenarios, such as lack of business-dimensional optimization, suboptimal business-level objectives, resource fragmentation due to lack of global forward planning, resource waste caused by static resource configuration, and poor adaptability to complex dynamic scenarios. It achieves efficient global planning and dynamic optimization scheduling of cluster resources, improving overall resource utilization and simulation execution efficiency, achieving the optimal solution for business-level scheduling objectives, and adapting to the high-concurrency task requirements of complex simulation scenarios such as large-scale batch processing and AI training.

[0011] The specific technical solution is as follows: A satellite communication terminal simulation task orchestration system based on intelligent scheduling adopts a microservice architecture, with each module communicating via RESTful API or gRPC. The system includes: a user interface module, a task management module, a resource monitoring and metadata management module, an intelligent scheduling decision module, and a task execution and control module. The user interface module receives user-configured scenario parameters, verifies them, and then sends structured scenario model data. The task management module parses the scenario model data, breaks down task units, constructs a Directed Acyclic Graph (DAG) dependency relationship, generates a queue to be scheduled, and persists it. The resource monitoring and metadata management module collects real-time resource data from the cluster and tasks, manages historical resource consumption data, and constructs and maintains a global resource view; the historical consumption data drives a prediction model. The intelligent scheduling decision module obtains the queue to be scheduled and the global resource view, and outputs the optimal scheduling scheme through the prediction model and the PSO algorithm. The task execution and control module receives the scheduling scheme, interacts with the Kubernetes API to execute tasks, monitors status, and releases resources, achieving global dynamic scheduling across both business and resource dimensions.

[0012] Furthermore, the user interface module provides a WebUI and API. The received scenario parameters include basic metadata, satellite constellation and single-satellite parameters, terminal parameters, business parameters, resource requirements and evaluation indicators. After the parameters are verified for completeness and logic, they are stored in the database and sent to the task management module in JSON format via a message queue. The basic metadata is used to identify the simulation scenario and support basic management information for task scheduling and lifecycle management.

[0013] Furthermore, the task management module receives scenario model data via message queue or REST API, extracts scenario IDs, and queries complete structured parameter configurations. It breaks down simulation requests into pre-tasks, core simulation sub-tasks, and post-tasks, constructs a directed acyclic graph (DAG) based on task logic to clarify execution order and dependencies, and generates a queue of tasks to be scheduled after marking. For each task unit, it creates an internal task object containing a unique identifier, scenario, task type, initial state, resource requirements, command parameters, and pre-task ID. The task object, DAG relationship, and queue of tasks to be scheduled are persistently stored in the database. Whenever a task state changes, the corresponding task state in the database is updated in real time, and a status update event is published via message queue, synchronously notifying the intelligent scheduling decision module and the task execution and control module. A RESTful API is provided externally, supporting queries for task details and real-time status by scenario ID, task ID, and task state.

[0014] Furthermore, the resource monitoring and metadata management module implements its core functions through four core steps executed sequentially. Specifically, the process is as follows: First, through a multi-dimensional data collection step, raw data is collected from cluster nodes and container Pods and categorized into three types of monitoring metrics, which are then pushed to Prometheus for centralized storage in real time. Next, through a data aggregation and storage step, the three types of monitoring metrics are preprocessed according to their purpose and access frequency, transforming them into real-time monitoring data, node basic configuration data, historical raw data, and prediction model input data, which are then stored separately. Subsequently, through a global resource view construction step, the standardized data is integrated at a preset cycle, correlated to form a single node's full-dimensional resource status, aggregated to generate a global resource view in JSON format, and written to a Redis cache for periodic updates. Finally, through a historical data archiving step, task completion notifications are received, the peak resource consumption of completed Pods is extracted, task metadata is correlated, the prediction deviation rate is calculated, and the archived data is written to the resource consumption history database to supplement the global resource view sample size and update the prediction model training set, forming an optimized closed loop of data, model, and view.

[0015] Furthermore, the specific implementation of the multi-dimensional data collection is as follows: The data collection tool Node Exporter is deployed on each Kubernetes node, in conjunction with the Prometheus monitoring system, while simultaneously listening to the Kubernetes API to obtain Pod and Node status changes. The collected raw data is categorized into three types of monitoring metrics: node resource metrics, container Pod resource metrics, and status event correlation metrics. Node resource metrics refer to basic configuration data reflecting the inherent attributes of cluster nodes and real-time data reflecting the dynamics of current resource usage. Basic configuration data includes the node's unique identifier, node type label, network topology label, storage mount path, and total capacity. Real-time data includes the node's current CPU utilization, remaining available CPU cores, GPU utilization, remaining available GPUs and GPU memory capacity, and used and available memory capacity. Container Pod resource metrics refer to data recording the CPU, GPU, and memory resource allocation quotas for scheduled but not yet started Pod tasks, as well as the peak CPU, GPU, and memory resource consumption throughout the entire lifecycle of completed Pod tasks. Status event correlation metrics refer to data capturing the node's health status, Pod running status, and creation, deletion, and update event timestamps. All collected monitoring metrics are pushed to Prometheus in real time for centralized storage.

[0016] Furthermore, the core content of the global resource view includes basic cluster node information, real-time node resource status, pre-occupied resource statistics, historical task statistics, and resource demand prediction data. Basic cluster node information refers to the inherent attributes and basic configuration data of each node in the cluster, used to uniquely identify the node's identity, clarify its hardware capabilities and deployment location, and serve as the basic positioning basis for task scheduling. Real-time node resource status refers to the dynamic data of resource usage at the current moment for each node, including the utilization rate and remaining availability of core computing and storage resources, as well as the node's operational health status, intuitively reflecting the node's current resource carrying capacity. Pre-occupied resource statistics refer to the pre-request quota of core resources for cluster nodes by Pod tasks that have been scheduled but not yet started, used to avoid the risk of resource over-allocation and ensure the accuracy of resource reserve calculations for scheduling decisions. Historical task statistics refer to historical sample data formed during the past execution of similar tasks on each node, including peak resource consumption patterns and task execution success rates, providing historical reference for scheduling decisions. Resource demand prediction data refers to the resource demand prediction results output by the regression prediction model trained based on historical task consumption data for the tasks to be scheduled, providing forward-looking resource demand references for scheduling decisions and improving scheduling matching accuracy.

[0017] Furthermore, the intelligent scheduling decision module employs a particle swarm optimization algorithm to achieve simulation task orchestration and resource scheduling. Specifically, the node allocation problem of the tasks to be scheduled is modeled as a multi-dimensional optimization problem. Let the number of tasks to be scheduled be M and the number of nodes that can be allocated in the cluster be N. Each particle is defined by an M-dimensional vector, where each element corresponds to the target node number of a single task to be scheduled, and each particle represents a complete scheduling scheme. By iteratively calculating the multi-objective weighted fitness function value, the individual optimal value and the global optimal value of the population are dynamically updated until the iteration termination condition is met, and the globally optimal scheduling scheme is output, thereby achieving the optimal matching between simulation tasks and cluster resources.

[0018] Furthermore, the multi-objective weighted fitness function determines the merits of a scheduling scheme by comprehensively evaluating its resource utilization, load balancing, and constraint satisfaction. A higher fitness value indicates a better scheduling scheme. Resource utilization, based on global resource view data, measures the effective utilization of core cluster resources, reflecting the effectiveness of resource activation and idle avoidance. Load balancing, based on resource usage status data of each node in the cluster, measures the balance of resource load among nodes, avoiding overload and idle imbalance. Constraint satisfaction, reflected by constraint violation penalties, is based on preset resource usage rules and measures the scheduling scheme's adherence to resource usage boundaries, preventing over-allocation risks. Finally, the evaluation results of resource utilization and load balancing are combined using a weighted method, and the constraint violation penalties are subtracted to form a comprehensive evaluation result of the scheduling scheme.

[0019] Furthermore, the intelligent scheduling decision module performs scheduling based on a global resource view, wherein the resource demand prediction data is generated by a prediction model; the prediction model uses the peak resource consumption of completed tasks and metadata as training data, and outputs estimated resource demand values ​​for new tasks to be scheduled, replacing the static demand declared by the user, as the core input for scheduling matching and scheme calculation.

[0020] A computer device includes a processor, a readable storage medium, and a communication interface, wherein the processor, the readable storage medium, and the communication interface are connected via a bus; the readable storage medium includes a computer-integrated storage medium and a computer-supported extended storage medium, used to store computer programs and a database, the database containing scenario parameters, task information, resource data, and historical statistical data related to the simulation task; the processor is used to provide computing and control capabilities, and by loading and executing one or more instructions stored in the readable storage medium, supports the collaborative operation of the entire process of simulation task reception, parameter verification, parsing and splitting, dependency management, resource status monitoring, intelligent scheduling decision-making, and execution control; the communication interface is used to establish a connection with external devices through a network and conduct data communication, supporting information interaction between modules during simulation task processing and API interaction with a Kubernetes cluster.

[0021] Beneficial effects 1. Relying on the global optimization capabilities of the PSO intelligent algorithm and the forward-looking planning of the global resource view, resources are allocated from the global perspective of batch tasks, effectively avoiding the problem of resource fragmentation, significantly improving the overall utilization of core resources such as cluster CPU, GPU, and memory, and optimizing allocation efficiency in resource-constrained scenarios.

[0022] 2. By using data-driven predictive models, the resource requirements of tasks to be scheduled can be accurately quantified based on historical task consumption data, replacing the static user reporting model, avoiding resource waste caused by excessive reservation, and improving the rationality and accuracy of resource allocation.

[0023] 3. Construct a dynamic automated workflow of "parameter verification - task splitting - DAG dependency construction - automatic scheduling - status monitoring - resource release" without manual intervention, reducing manual configuration and scheduling errors, and improving the automation level and reliability of simulation task orchestration.

[0024] 4. Create a unified, real-time, and multi-dimensional global resource view, integrating basic information of cluster nodes, real-time resource status, historical data, and predictive data to achieve comprehensive perception of resource and task information, completely break down information silos, and provide comprehensive data support for scheduling decisions.

[0025] 5. Deeply integrates Kubernetes orchestration capabilities, combining the optimization decisions of the PSO intelligent algorithm with engineering implementation. Through interaction with the Kubernetes API, it completes task execution, status monitoring, and resource release, efficiently solving the resource management challenges brought about by high-concurrency tasks in large-scale simulation scenarios.

[0026] 6. By integrating business-dimensional objectives such as business priority, task type, and timeout urgency, it achieves global dynamic scheduling of both business and resources, breaking through the limitations of traditional scheduling that only focuses on hardware resources, achieving the optimal solution for business-level scheduling, and meeting the core business needs of large-scale simulation scenarios.

[0027] 7. Adapt to dynamic simulation scenarios with high concurrency, heterogeneous resources, and complex task dependencies. Through differentiated scheduling paths, node affinity configuration, and automatic fault recovery mechanisms, it effectively avoids task congestion and system overload, and improves the system's adaptability and stability to complex scenarios.

[0028] 8. Establish a closed-loop optimization system encompassing data acquisition, model training, scheduling decisions, data archiving, and model iteration to continuously improve the accuracy of resource demand prediction and the optimization level of scheduling schemes. At the same time, reduce the task failure rate and ensure the long-term efficient and stable operation of the system through full lifecycle status management and fault handling mechanisms. Attached Figure Description

[0029] Figure 1 The system structure diagram described in this invention; Figure 2 Flowchart of the user interface module implementation; Figure 3 Flowchart of the task management module implementation; Figure 4 Flowchart of the resource monitoring and metadata management module; Figure 5 Implementation process of the intelligent scheduling decision module; Figure 6 Implementation process of the task execution and control module; Figure 7 A structural diagram of the computer device described in this invention. Detailed Implementation

[0030] I. System Structure The system modules are deployed as microservices and communicate with each other via RESTful APIs or gRPC. The system architecture of this method is as follows: Figure 1 As shown. 1. The user interface module provides a WebUI and API, receives tasks submitted by users, and validates and stores the parameters of the user-created scenarios in a structured manner.

[0031] 2. The task management module parses scene parameters, obtains the system's task execution plan, parses task descriptions, maintains dependencies between tasks, and manages the lifecycle of tasks, including creation, querying, and updating. It also performs persistent storage for all task elements and their dependencies.

[0032] 3. Resource monitoring and metadata management module: collects resource data from nodes and task execution in the cluster, manages historical consumption data of CPU, GPU, memory and other resources of simulation tasks, uses it to drive the prediction model, and builds and maintains a global resource view.

[0033] 4. Intelligent scheduling decision module: This module is the core of the system. It receives the queue of tasks to be scheduled from the task management module, accesses the global view of the resource monitoring module and the metadata management module, predicts resource demand by loading machine learning models, and calls the PSO algorithm engine to perform global optimization scheduling calculations.

[0034] 5. The task execution and management module is responsible for interacting with the Kubernetes API, performing the final scheduling and binding operations, and monitoring the task running status. II. User Interface Module.

[0035] The execution flow of the user interface module includes: 1. User Login and Scene Initialization: Users log in through the Web portal and complete authentication to enter the simulation task creation page. They then input basic metadata, which is used to identify the simulation scene and support basic management information for task scheduling and lifecycle management, including but not limited to the simulation scene name, scene description, and business priority tag. 2. Scenario Parameter Configuration: On the parameter configuration page, users configure satellite constellation and individual satellite parameters, such as orbital parameters, payload parameters, and on-board resources; configure terminal parameters, such as terminal type, antenna parameters, mobility parameters, and service access capabilities; configure service parameters, such as service type, traffic model, and QoS requirements, defining which terminals communicate with the satellites for the service; and determine the configuration scheme of computing and storage resources required for this simulation task, the simulation runtime, and task evaluation metrics through manual specification or system intelligent recommendation. 3. System Verification: The user interface module performs integrity and logical verification on the parameters submitted by the user. After the verification is successful, the parameters are stored in the local database in a structured manner. 4. Task Issuance: Send a JSON-formatted scene model object to the task management module via a message queue to trigger the subsequent task parsing process.

[0036] III. Task Management Module The implementation process of the task management module is as follows: Figure 3 As shown, the core revolves around four stages: task parsing, maintenance of inter-task dependencies, task state persistence, and task lifecycle management. The specific implementation process is as follows: 1. Task Analysis The task management module receives events from the user interface module via a message queue or REST API. First, it extracts the scene ID from the event, and then queries the metadata database based on the scene ID to obtain the complete structured parameter configuration of the simulation scene. The acquired parameter configurations were then analyzed in depth, and a single simulation request was broken down into multiple independent executable task units, including: pre-processing tasks (such as basic data preparation tasks such as satellite orbit ephemeris generation and visibility calculation between satellite and terminal), core simulation tasks (broken down into multiple parallel sub-tasks according to time slice and geographical region dimensions to improve simulation efficiency), and post-processing tasks (such as simulation result data aggregation, data analysis, and visualization).

[0037] 2. Maintaining inter-task dependencies After the task units are decomposed, a corresponding directed acyclic graph (DAG) is constructed based on the logical relationships between each task unit (e.g., the output of the preceding task is the input of the core simulation task, and the result of the core simulation task is the basis for the subsequent task). This graph clarifies the execution order and interdependencies of all task units. After the dependencies are sorted out, each task unit is marked with a corresponding identifier (including a "pending scheduling" status marker and a DAG node association identifier) ​​to ensure that the scheduling system can identify the current status and dependency relationships of the task. Then, they are all placed into the queue of tasks to be scheduled, waiting for the scheduling instructions from the intelligent scheduling decision module.

[0038] 3. Task state persistence For each split task unit, a dedicated internal task object is created. This object contains key information such as a unique task identifier, its associated scenario ID, task type (pre-task / core / post-task), initial state, resource requirements, command parameters, and the IDs of the pre-tasks it depends on. The resource requirements are the specific specifications of CPU, GPU, memory, and storage required for task execution, while the command parameters are the script path, runtime parameters, and output path that drive the task execution. Subsequently, all internal task objects and the constructed DAG dependencies are persistently stored in the database, achieving stable storage of detailed task information and inter-task relationships, providing data support for subsequent task scheduling and status tracking.

[0039] 4. Task lifecycle management The system provides full lifecycle management for all task units (creation, pending scheduling, execution, completion, failure, termination, etc.): Whenever a task status changes, the system updates the corresponding task status in the database in real time and publishes the status update event through a message queue, synchronously notifying relevant modules such as the intelligent scheduling decision module and the task execution and management module; at the same time, it provides a standardized RESTful API to support users in querying detailed task information and real-time running status by multiple dimensions such as scenario ID, task ID, and task status, meeting the needs of task monitoring and management.

[0040] IV. Resource Monitoring and Metadata Management Module The core function of the resource monitoring and metadata management module is to collect full-dimensional resource and task data from the Kubernetes cluster, aggregate and store the data, build and maintain a unified, real-time, and multi-dimensional global resource view, and manage historical consumption data of task resources to drive regression prediction models. Its final output is a global resource view data that can be directly called by the intelligent scheduling decision module, supporting task selection, PSO algorithm parameter initialization, and fitness function calculation in scheduling decisions.

[0041] The module achieves the above functions through four core steps: multi-dimensional data collection, data aggregation and storage, global resource view construction, and historical data archiving. The specific implementation process is as follows: 1. Multi-dimensional data collection to obtain raw data from the global resource view - three types of monitoring metrics; This step uses standardized tools to collect raw data from cluster nodes and container Pods. All data is categorized into three types of monitoring metrics: node resource metrics, container Pod resource metrics, and status event correlation metrics. This data is used to build a global resource view later. The specific data collected is as follows: Data collection tools and deployment: Deploy the data collection tool NodeExporter on each Kubernetes node, in conjunction with the monitoring system Prometheus, and listen to the Kubernetes API to obtain Pod / Node status changes to ensure full data coverage; The node resource metrics include basic attribute data and real-time resource data. The basic attribute data includes, but is not limited to, a unique node identifier, node type labels (e.g., CPU node, GPU node, GPU node includes model label), network topology labels (e.g., rack number, network segment address), storage mount path, and total capacity. The real-time resource data includes, but is not limited to, the node's current CPU utilization, remaining available CPU cores, GPU utilization, remaining available GPUs and video memory capacity, and used and available memory capacity. The container Pod resource metrics include pre-request data and peak consumption data. The pre-request data includes, but is not limited to, the pre-requested CPU, GPU, and memory resources of Pods that have been scheduled but not yet started. The peak consumption data includes, but is not limited to, the peak CPU, GPU, and memory resource consumption of Pods that have been completed and whose entire lifecycle has been completed. The status event-related metrics include node status data and Pod status data. Node status data includes, but is not limited to, the health status of the Node (Ready / NotReady, quantified as 1 / 0); Pod status data includes, but is not limited to, the running status of the Pod and the timestamps of Pod creation / deletion / update events. The three types of monitoring indicators collected are pushed to Prometheus in real time, completing the centralized storage of raw data and preparing for the next stage of data processing.

[0042] 2. Data aggregation and storage, classification and preprocessing of three types of monitoring indicators, and adaptation to view construction requirements; This step uses the specific breakdown of data from three types of monitoring indicators as a foundation, and processes and stores the data according to its purpose and access frequency. The raw data is transformed into standardized data that can directly support the construction of a global resource view. This includes real-time monitoring data, basic node configuration data, historical raw data, and predictive model input data. The real-time monitoring data includes real-time resource data for node resource metrics and pre-request data for container Pod resource metrics. The node resource metrics extract the current CPU utilization, remaining available CPU cores, GPU utilization, remaining available GPUs and GPU memory capacity, and used and available memory capacity. Simultaneously, the container Pod resource metrics extract the pre-request amounts of CPU, GPU, and memory resources declared by scheduled but not yet started Pods. Both types of data are standardized into a (node / Pod, time, resource value) format, such as node-01_20241002_16:00_CPU_available=8 cores, and written to a time-series database for short-term storage. This provides frequently updated real-time data for the real-time resource status and pre-occupied resource statistics of nodes in the global resource view.

[0043] The basic configuration data of a node corresponds to the basic attribute data of the node resource indicators. The unique identifier of the node, the node type label (such as CPU node / GPU node, GPU node includes model label), the network topology label (rack number, network segment address), the storage mount path and the total capacity are extracted from the node resource indicators. The above data is associated with the scenario model data issued by the user interface module (such as the maximum number of available CPU cores in scenario A), and aggregated to form structured data, which is stored in the Redis cache and directly used as the core content of the cluster node basic information in the global resource view.

[0044] Historical raw data corresponds to the peak consumption data of container Pod resource metrics. Extract the peak CPU, GPU, and memory resource consumption of completed Pods throughout their entire lifecycle from the container Pod resource metrics. Compress and aggregate the data according to two dimensions: task type and execution node. For example, the peak CPU value of the core simulation task - node-01 = 6 cores is stored in the time series database for a short period of time. After the expiration period, the data is compressed and transferred to object storage to provide historical samples for the historical task statistics of the global resource view.

[0045] The input data for the prediction model is derived from the peak consumption data of container Pod resource metrics. The peak CPU, GPU, and memory resource consumption data of the entire lifecycle of the completed Pod are extracted from the object storage and compressed. This data is then associated with the task metadata provided by the task management module, such as task type and scenario ID, to form a training dataset of task metadata and historical consumption peaks. This dataset is stored in the metadata database to train the prediction model and provide model support for the resource demand prediction data of the global resource view.

[0046] 3. Global resource view construction: The preprocessed data is aggregated into five categories to generate the final view; This step, based on the standardized data preprocessed above, integrates and generates a global resource view at a preset cycle, including basic cluster node information, real-time node resource status, pre-occupied resource statistics, historical task statistics, and resource demand prediction data. It reads the basic cluster node information from the "Node Basic Configuration Data" cache in Redis, the real-time node resources and Pod pre-request volume from the "Real-time Monitoring Data" cache corresponding to the real-time node resource status and pre-occupied resource statistics, respectively, from the time-series database, and the peak consumption and execution success rate of similar tasks from the "Historical Raw Data" cache corresponding to historical task statistics. It then calls a regression prediction model trained on the "Prediction Model Input Data" from the metadata database, inputs the metadata features of the task to be scheduled, and outputs the estimated resource demand value corresponding to the resource demand prediction data. Simultaneously, it reads the node health status from the "Status Event Correlation Indicators" cache from the time-series database to supplement the real-time node resource status. Finally, it associates the five types of data according to the "Node Unique Identifier" to form a "Single Node Full-Dimensional Resource Status." Then, it integrates the full-dimensional resource status of all nodes into JSON format, writes it to a specified location in the Redis cache and records the timestamp, repeating the process every 10 seconds to ensure the view is synchronized with the cluster status.

[0047] The final view contains five core categories of information: basic cluster node information (unique node identifier, CPU / GPU type, network topology, storage mount path and available capacity), real-time node resource status (current CPU utilization, remaining available CPU cores, GPU utilization, remaining available GPUs and video memory capacity, used and available memory capacity, and node health status), pre-occupied resource statistics (pre-requested resources for scheduled but not yet started Pods), historical task statistics (peak resource consumption and execution success rate of similar tasks on each node), and resource demand prediction data (estimated resource demand values ​​for tasks to be scheduled, output by the regression prediction model).

[0048] 4. Historical data archiving (supplementing historical data in the view, optimizing the prediction model, and forming a closed loop) This step supplements the historical task statistics sample size of the global resource view by archiving the entire lifecycle consumption data of tasks, while simultaneously optimizing the accuracy of the prediction model, forming an optimization loop of data, model, and view. When the task execution and management module detects the completion of a Pod task, it sends a task completion notification (including the task's unique identifier, type, and execution node) via a message queue. Upon receiving the notification, the resource monitoring and metadata management module calls the Prometheus interface to extract the peak CPU / GPU / memory resource consumption of the Pod throughout its lifecycle. Simultaneously, it retrieves the task metadata from the task management module, retrieves the estimated resource requirement value of the task, calculates the prediction deviation rate, and writes the task metadata, actual peak consumption, and prediction deviation data into the resource consumption history database. This data, on the one hand, supplements the sample size of the historical task statistics in the global resource view, and on the other hand, is periodically used to update the prediction model training set, retraining the model parameters to improve the accuracy of the estimated resource requirement value, thus inversely optimizing the accuracy of the resource requirement prediction data in the global resource view.

[0049] V. Intelligent Scheduling Decision Module The intelligent scheduling decision module, as the core unit for the system to achieve global dynamic scheduling of both business and resources, completes the entire process of data input, task selection, algorithm optimization, instruction output, and data accumulation in a closed loop. Combining resource demand prediction and particle swarm optimization (PSO) algorithms, it outputs the optimal scheduling scheme adapted to Kubernetes cluster execution. The specific implementation process is as follows: 1. Obtain the queue of tasks to be scheduled and the global resource view. The task query interface of the task management module is called via a RESTful API. Based on filtering conditions such as scenario ID and scheduling status, structured task queue data is retrieved in batches. This queue contains key information such as unique task identifier, scenario ID, task type, resource requirements, business priority tag, and completion status of prerequisite dependencies, providing basic data support for subsequent modeling. At the same time, the resource query API of the resource monitoring and metadata management module is called to obtain unified, real-time, and multi-dimensional global resource view data. Finally, the two types of data are cross-validated, focusing on verifying the matching of task resource requirements with cluster-supported resource types and scenario resource quotas with total available cluster resources. If data inconsistencies or resource mismatches are found, an exception notification is triggered and the scheduling process is paused. Data is retrieved again after the problem is fixed.

[0050] 2. Task Preprocessing The task preprocessing stage processes task units in the queue of tasks to be scheduled in a logically layered manner according to feasibility screening, priority sorting, and scheduling path classification, thereby reducing invalid calculations and improving scheduling efficiency.

[0051] The infeasible task filtering is based on global resource view data. Tasks that meet any of the following conditions are marked as infeasible and moved to the abnormal task queue. The reason for filtering is recorded (for subsequent user queries and problem investigation): 1) The resource requirement exceeds the total available resources of the cluster (e.g., the task requires 8 GPU cores, but the cluster currently only has 5 cores available); 2) The special resources that the task depends on (e.g., a specific model of FPGA, high-bandwidth storage) have no matching nodes in the cluster; 3) The resource quota of the scenario to which the task belongs has been exhausted (e.g., scenario A has a preset maximum of 100 CPU cores, and 98 cores are currently occupied, and the task requires 5 CPU cores).

[0052] Task priority ranking is based on the business priority tags associated with the task (high, medium, low), task type (core simulation tasks have higher priority than preceding and subsequent tasks), and task timeout (the shorter the remaining timeout, the higher the priority). A multi-dimensional priority scoring model is constructed, with the following formula: Priority Score = Business Priority Weight (0.5) × Business Score + Task Type Weight (0.3) × Type Score + Timeout Urgency Weight (0.2) × Urgency Score. Where "High," "Medium," and "Low" business priorities correspond to scores of 3, 2, and 1 respectively; core tasks, pre-tasks, and post-tasks correspond to scores of 3, 2, and 1 respectively; and timeout urgency (remaining time / total duration) ≤ 0.3 earns 3 points, 0.3-0.6 earns 2 points, and > 0.6 earns 1 point. Tasks are sorted in descending order of score to ensure high-priority tasks are scheduled first.

[0053] Based on task complexity and resource requirements, tasks are categorized into two types and assigned different scheduling paths: Simple tasks that meet three conditions—single-node executable (resource requirements ≤ remaining resources on a single node), no complex dependencies (depending on only one completed prerequisite task), and high business priority—skip the PSO algorithm calculation and generate a scheduling scheme based on the minimum load priority strategy (selecting the node with the lowest combined CPU, GPU, and memory utilization) to shorten scheduling latency; Complex tasks that do not meet the simple task conditions (such as batch core simulation tasks, cross-node dependent tasks, and tasks with resource requirements exceeding the remaining resources on a single node) are marked as requiring PSO optimization and enter the subsequent PSO scheduling algorithm stage.

[0054] 3. Execution of the PSO scheduling algorithm For complex tasks, the scheduling problem is modeled as a multi-objective resource optimization and allocation problem. The globally optimal scheduling scheme is calculated iteratively using the PSO algorithm. The specific steps are as follows: Step 1: Algorithm Parameter Initialization Population and Particle Definitions: Let M be the number of complex tasks to be scheduled (i.e., the total number of tasks requiring PSO optimization), and N be the number of nodes that can be allocated in the cluster (i.e., the total number of nodes in the cluster capable of handling complex tasks). The population size (i.e., the total number of particles) is set to 50 (which can be dynamically adjusted according to the number of tasks, increasing to 80 when the number of tasks > 100). Each particle represents a complete scheduling scheme (i.e., one particle corresponds to the node allocation results of all complex tasks to be scheduled), represented by an M-dimensional vector. Where i ∈ [1, population size], i is the particle number, such as i = 1 to 50, j ∈ [1, M], j is the number of the complex task to be scheduled, such as j = 1 to 30), vector elements ∈ [1,N], The node number represents the assignment of the j-th complex task to the node number. Each cluster can be assigned nodes, enabling the mapping of task allocation problems to a multi-dimensional vector model.

[0055] Velocity vector initialization: the velocity vector of each particle The initial velocity range is set to [-2, 2] to ensure that the particle searches reasonably within the solution space; Iteration parameter settings: the maximum number of iterations K is set to 50, and the fitness convergence threshold ε is set to 0.001; Other weighting parameter settings: Set parameters such as inertia weight, individual learning factor, and social learning factor to balance the algorithm's global search and local optimization capabilities.

[0056] Step 2: Fitness Function Calculation A multi-objective weighted fitness function is constructed to comprehensively evaluate the resource utilization, load balancing, and constraint satisfaction of the scheduling scheme. The higher the fitness value, the better the scheme. The specific formula is as follows:

[0057] in, and These are the weighting coefficients; To determine resource utilization, based on global resource view data, we first calculate the CPU utilization, GPU utilization, and memory utilization (all of which are "node allocated resources / node total resources") of each allocable node, and take the average of the three as the comprehensive resource utilization of a single node; then we calculate the average comprehensive resource utilization of all nodes, which is the resource utilization, with a value ranging from 0 to 1. The closer the value is to 1, the more fully the cluster resources are used. To achieve load balancing, we first calculate the standard deviation of the utilization of all allocable nodes in terms of CPU, GPU, and memory (reflecting the differences in resource usage among nodes); then we take the average of the standard deviations of the three types of resource utilization as the comprehensive standard deviation of cluster resource utilization; and finally, we divide 1 by this comprehensive standard deviation to obtain the load balance. The larger the value, the more balanced the resource usage pressure among the nodes. To constrain the violation penalty, the scheduling scheme is verified node by node: if the total resource demand of all tasks on a node does not exceed the total resources of the node and does not exceed the resource quota of the scenario to which the task belongs, the penalty is 0; if there is resource over-allocation (such as task demand exceeding node capacity), the penalty value of a single node is calculated as the over-allocated resource amount / total node resources × 10, and the maximum penalty value of all over-allocated nodes is taken as the final constraint violation penalty. The more severe the over-allocation, the larger the penalty.

[0058] Third step iterative optimization Update individual optimal value Compare the fitness of the current position with the global optimum. Its individual historical best fitness The individual with the highest fitness function value is updated to the individual optimal value. After all particles have updated their individual optimal values, the individual optimal value with the highest fitness is selected as the global optimal value G(t+1) of the population. This realizes the dynamic update of individual and global optimal values ​​during the particle swarm iteration process and gradually approaches the global optimal scheduling scheme.

[0059]

[0060]

[0061] The fourth step is to terminate the iteration. Set the iteration termination conditions according to the system requirements, such as the maximum number of iterations K, for example, 50 times, or the fitness convergence threshold. ,for example Final output The optimal scheduling scheme is the globally optimal scheduling scheme found through particle swarm iteration.

[0062] 4. Scheduling Method Output: The task and node allocation vectors (i.e., the globally optimal scheduling scheme) output by the PSO algorithm are mapped to actual discrete nodes in the cluster, transforming them into a task-node-resource binding table. This table clearly defines the target node and required resources for each task (resource requirements are based on the estimated resource requirements output by the prediction model, replacing static requirements submitted solely by the user). Based on this binding relationship, Kubernetes Pod instructions are generated. These instructions include the name and namespace from the basic configuration, the image and environment variables from the container configuration, resource constraints, and node affinity configuration. After generating the instructions, they are validated using the Kubernetes dryRun interface. If errors are found, they are corrected promptly. Once confirmed to be error-free, the instructions are output to ensure the normal execution of subsequent tasks. Node affinity configuration ensures that tasks are only scheduled to nodes that meet the attribute requirements (e.g., GPU tasks are scheduled to GPU nodes), ensuring the feasibility of the scheduling scheme.

[0063] 5. Data storage for scheduling: The input data (queue of tasks to be scheduled, global resource view data), algorithm process data (PSO algorithm parameters, individual optimal and global optimal values ​​during iteration), output data (Pod instructions, verification results), and the final resource consumption peak and metadata characteristics of the task are archived together in the metadata database. This is used for subsequent scheduling review and strategy optimization, and also provides data support for the iterative training of the prediction model, realizing closed-loop optimization of data, model and scheduling.

[0064] VI. Task Execution and Control Module The task execution and management module, as the core execution unit for implementing scheduling instructions, is responsible for receiving and parsing scheduling instructions from the intelligent scheduling decision module, creating Pod resources through interaction with the Kubernetes API, monitoring the entire lifecycle status of tasks in real time, handling exceptions such as node failures and task timeouts and providing feedback to the scheduling module, and releasing resources promptly after the task ends to ensure the efficient and stable execution of simulation tasks. The specific implementation process includes: 1. Receiving Scheduling Instructions: The task execution and management module receives the scheduling decision results from the intelligent scheduling decision module from the message queue, parses the task identifier, target computing node, CPU, memory, GPU and other resource requirements information in the instruction; then, according to the task identifier, it retrieves the detailed task information from the relevant storage, prepares the container image and environment variables required for task execution in sync, and then generates the YAML definition of the Pod by combining the scheduling instructions and task details. The Pod resource is created by calling the APIServer through the cluster's client library, thus completing the execution of the scheduling instructions.

[0065] 2. Task Status Capture: The module continuously monitors the running status of Pod resources and the status information of target nodes, capturing the entire lifecycle status of tasks from startup, execution to termination in real time; at the same time, it sets a timeout for each task, dynamically tracks the task runtime, and ensures that it can promptly grasp whether the task is progressing normally within the specified time. After the task is completed, it automatically releases the cluster resources such as CPU, GPU, and memory occupied by the task, ensuring the recycling of resources.

[0066] 3. Fault Handling and Response: If the monitoring detects that the target node is in the NotReady state, all running tasks on that node are immediately marked as Failed, and the intelligent scheduling decision module is notified through the message queue to reschedule these tasks; if the task runtime exceeds the preset timeout, the task is automatically and forcibly terminated to avoid long-term resource occupation; all fault conditions and handling results are synchronously recorded and archived to provide data support for subsequent problem investigation and scheduling strategy optimization.

[0067] VII. Computer Equipment Computer equipment includes a processor, readable storage media, and a communication interface, which are connected via a bus. The readable storage media includes the computer's built-in storage media and computer-supported extended storage media, used to store computer programs and databases; the processor provides computing and control capabilities, loading one or more instructions to implement methods for simulating task orchestration; the communication interface is used to communicate with external interrupts via a network connection.

Claims

1. A satellite communication terminal simulation task orchestration system based on intelligent scheduling, characterized in that, Employing a microservice architecture, the modules communicate via RESTful APIs or gRPC, including: a user interface module, a task management module, a resource monitoring and metadata management module, an intelligent scheduling decision module, and a task execution and control module. The user interface module receives user-configured scenario parameters, verifies them, and then distributes structured scenario model data. The task management module parses the scenario model data, breaks down task units, constructs a Directed Acyclic Graph (DAG) dependency relationship, generates a queue to be scheduled, and persists it. The resource monitoring and metadata management module collects real-time cluster and task resource data, manages historical resource consumption data, and constructs and maintains a global resource view; the historical consumption data drives a prediction model. The intelligent scheduling decision module obtains the queue to be scheduled and the global resource view, and outputs the optimal scheduling scheme using the prediction model and the PSO algorithm. The task execution and control module receives the scheduling scheme, interacts with the Kubernetes API to execute tasks, monitor status, and release resources, achieving global dynamic scheduling across both business and resource dimensions.

2. The system according to claim 1, characterized in that, The user interface module provides a WebUI and API. The received scenario parameters include basic metadata, satellite constellation and single-satellite parameters, terminal parameters, business parameters, resource requirements and evaluation indicators. After the parameters are verified for completeness and logic, they are stored in the database. The module sends a JSON-formatted scenario model object to the task management module through a message queue. The basic metadata is used to identify the simulation scenario and support basic management information for task scheduling and lifecycle management.

3. The system according to claim 1, characterized in that, The task management module receives scenario model data through a message queue or REST API, extracts the scenario ID, and queries the complete structured parameter configuration; it splits the simulation request into pre-tasks, core simulation sub-tasks, and post-tasks, constructs a directed acyclic graph based on the task logic association to clarify the execution order and dependencies, and generates a queue of tasks to be scheduled after marking them. For each task unit, an internal task object is created, containing a unique identifier, its associated scenario, task type, initial state, resource requirements, command parameters, and a preceding task ID. The task object, DAG relationship, and queue of tasks to be scheduled are persistently stored in the database. Whenever the task state changes, the corresponding task state in the database is updated in real time, and a status update event is published through a message queue to synchronously notify the intelligent scheduling decision module and the task execution and control module. A RESTful API is provided externally to support querying task details and real-time status by scenario ID, task ID, and task state.

4. The system according to claim 1, characterized in that, The resource monitoring and metadata management module achieves its core functions through four core steps executed sequentially. Specifically, the process is as follows: First, a multi-dimensional data collection step gathers raw data from cluster nodes and container Pods, categorizing it into three types of monitoring metrics, and pushes it to Prometheus for centralized storage in real time. Next, a data aggregation and storage step preprocesses the three types of monitoring metrics according to their purpose and access frequency, transforming them into real-time monitoring data, node basic configuration data, historical raw data, and prediction model input data, and stores them accordingly. Then, a global resource view construction step integrates the standardized data according to a preset cycle, correlates them to form a single node's full-dimensional resource status, aggregates and generates a global resource view in JSON format, and writes it to a Redis cache for periodic updates. Finally, a historical data archiving step receives task completion notifications, extracts the peak resource consumption of completed Pods, correlates them with task metadata, calculates the prediction deviation rate, and writes the archived data to a historical resource consumption database to supplement the global resource view sample size and update the prediction model training set, forming an optimized closed loop of data, model, and view.

5. The system according to claim 4, characterized in that, The specific implementation of the multi-dimensional data collection is as follows: Node Exporter, a data collection tool, is deployed on each Kubernetes node, in conjunction with the Prometheus monitoring system, while listening to the Kubernetes API to obtain changes in the status of Pods and Nodes. The collected raw data is classified into three types of monitoring metrics: node resource metrics, container Pod resource metrics, and status event correlation metrics. Among them, node resource metrics refer to basic configuration data that reflects the inherent attributes of cluster nodes and real-time data that reflects the dynamics of current resource usage. Basic configuration data includes unique node identifiers, node type labels, network topology labels, storage mount paths, and total capacity. Real-time data includes the node's current CPU utilization, number of remaining available CPU cores, GPU utilization, number of remaining available GPUs and video memory capacity, and used and available memory capacity. Container Pod resource metrics refer to the data that records the CPU, GPU, and memory resource allocation of scheduled but not started Pod tasks, as well as the peak CPU, GPU, and memory resource consumption throughout the entire lifecycle of completed Pod tasks; status event correlation metrics refer to the data that captures the node health status and Pod running status, as well as the timestamps of creation, deletion, and update events. All collected monitoring metrics are pushed to Prometheus in real time for centralized storage.

6. The system according to claim 1, characterized in that, The core content of the global resource view includes basic cluster node information, real-time node resource status, pre-occupied resource statistics, historical task statistics, and resource demand prediction data. Basic cluster node information refers to the inherent attributes and basic configuration data of each node in the cluster, used to uniquely identify the node's identity, clarify its hardware capabilities and deployment location, and serve as the basic positioning basis for task scheduling. Real-time node resource status refers to the dynamic data of resource usage at each node at the current moment, including the utilization rate and remaining availability of core computing and storage resources, as well as the node's operational health status, intuitively reflecting the node's current resource carrying capacity. Pre-occupied resource statistics refer to the pre-request quota of core resources for cluster nodes by Pod tasks that have been scheduled but not yet started, used to avoid the risk of resource over-allocation and ensure the accuracy of resource reserve calculations for scheduling decisions. Historical task statistics refer to historical sample data formed during the past execution of similar tasks on each node, including peak resource consumption patterns and task execution success rates, providing historical reference for scheduling decisions. Resource demand prediction data refers to the resource demand prediction results output by the regression prediction model trained based on historical task consumption data for the tasks to be scheduled, providing forward-looking resource demand references for scheduling decisions and improving scheduling matching accuracy.

7. The system according to any one of claims 1-6, characterized in that, The intelligent scheduling decision module uses particle swarm optimization (PSO) to orchestrate simulation tasks and schedule resources. Specifically, it models the node allocation problem of the tasks to be scheduled as a multi-dimensional optimization problem. Let the number of tasks to be scheduled be M and the number of nodes that can be allocated in the cluster be N. Each particle is defined by an M-dimensional vector, where each element corresponds to the target node number of a single task to be scheduled, and each particle represents a complete scheduling scheme. By iteratively calculating the multi-objective weighted fitness function value, the individual optimal value and the global optimal value of the population are dynamically updated until the iteration termination condition is met, and the globally optimal scheduling scheme is output to achieve the optimal matching between simulation tasks and cluster resources.

8. The system according to claim 7, characterized in that, The multi-objective weighted fitness function determines the merits of a scheduling scheme by comprehensively evaluating its resource utilization, load balancing, and constraint satisfaction. A higher fitness value indicates a better scheduling scheme. Resource utilization, based on global resource view data, measures the effective utilization of core cluster resources, reflecting the effectiveness of resource activation and idle avoidance. Load balancing, based on resource usage status data of each node in the cluster, measures the balance of resource load among nodes, avoiding node overload and idle imbalance. Constraint satisfaction, reflected by constraint violation penalties, is based on preset resource usage rules and measures the scheduling scheme's adherence to resource usage boundaries, preventing over-allocation risks. Finally, the evaluation results of resource utilization and load balancing are combined using a weighted method, and the constraint violation penalties are subtracted to form a comprehensive evaluation result of the scheduling scheme.

9. The system according to claim 8, characterized in that, The intelligent scheduling decision module performs scheduling based on a global resource view. Resource demand prediction data is generated by a prediction model. The prediction model uses the peak resource consumption of completed tasks and metadata as training data, and outputs estimated resource demand values ​​for new tasks to be scheduled, replacing the static demand declared by the user, as the core input for scheduling matching and scheme calculation.

10. A computer device, characterized in that, The system includes a processor, a readable storage medium, and a communication interface, which are connected via a bus. The readable storage medium includes built-in computer storage and computer-supported extended storage, used to store computer programs and a database. The database contains scenario parameters, task information, resource data, and historical statistical data related to the simulation task. The processor provides computing and control capabilities, supporting the collaborative operation of the entire process of simulation task reception, parameter verification, parsing and splitting, dependency management, resource status monitoring, intelligent scheduling decision-making, and execution control by loading and executing one or more instructions stored in the readable storage medium. The communication interface is used to establish a connection with external devices through a network and conduct data communication, supporting information interaction between modules during simulation task processing and API interaction with the Kubernetes cluster.

Images