Distributed simulation method and system based on containerization deployment and elastic scaling

Abstract

The application relates to the technical field of distributed simulation based on containerization deployment and elasticity, and discloses a distributed simulation method and system based on containerization deployment and elasticity, which comprises the following steps: collecting real-time resource monitoring and historical load trend data of a simulation task, and combining simulation calculation complexity parameters and I / O density parameters to perform unified load evaluation, so that fine modeling of the simulation task and resource demand prediction are realized, and the accuracy of task scheduling and distribution is improved; through subtask segmentation based on the minimum executable unit and a standard containerization packaging mechanism, and in cooperation with a container arrangement platform, modular deployment of the simulation task is realized, and then concurrent execution and elastic scheduling in a multi-node environment are supported.

Description

Technical Field

[0001] This invention relates to the field of distributed simulation technology based on containerized deployment and elastic scaling, specifically to a distributed simulation method and system based on containerized deployment and elastic scaling. Background Technology

[0002] In existing technologies, distributed simulation engines typically employ a centralized or fixed-resource-allocation architecture to support applications with high computational demands. However, this traditional architecture has many limitations in practical applications;

[0003] Traditional simulation platforms employ static resource allocation, leading to significant resource waste under low computational loads and potential resource shortages under high loads, impacting simulation task execution efficiency. Most simulation systems lack flexible scalability mechanisms, making it difficult to dynamically adjust computing power based on task requirements and adapt to simulation needs of varying scales. In high-concurrency simulation environments, resource contention can cause task queuing, resulting in simulation computation delays and reduced real-time performance. Traditional distributed simulation systems struggle to quickly recover from failed nodes, meaning a single point of failure can cause the entire simulation task to fail, affecting system stability and reliability.

[0004] Currently, some research attempts to improve the flexibility of simulation systems through cloud computing and virtualization technologies. For example, virtual machine-based simulation platforms can provide a certain degree of resource isolation and elastic scaling capabilities. However, due to their large resource overhead and slow startup speed, virtual machines are difficult to meet the requirements of simulation tasks with high real-time and high concurrency demands.

[0005] In contrast, distributed simulation engine systems based on containerized deployment and elastic scaling are better able to adapt to the complexity and dynamic changes of high-concurrency simulation environments. By accurately analyzing the computational load of simulation tasks and combining containerization technology with elastic scaling mechanisms, this system achieves efficient allocation and dynamic expansion of computing resources, ensuring that simulation tasks can run stably under different load conditions.

[0006] Therefore, to address the shortcomings of existing technologies, this invention proposes a distributed simulation engine system based on containerized deployment and elastic scaling. This system, based on container orchestration management and dynamic resource scheduling optimization, integrates automatic scaling strategies, load balancing, and computing resource pool management. This enables rapid and efficient scheduling of simulation tasks and dynamically adjusts computing resources to adapt to different load scenarios, significantly improving the simulation engine's concurrent processing capabilities and resource utilization efficiency. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides the following technical solution: a distributed simulation method based on containerized deployment and elastic scaling, comprising: collecting real-time resource monitoring and historical load trends of simulation tasks, and evaluating the load by calculating complexity parameters and I / O density parameters.

[0008] The simulation task is divided into subtasks based on the minimum executable unit and containerized separately. Based on real-time resource monitoring and historical load trends, a two-factor mechanism is used to determine whether to perform the scaling up or down operation of the container instance, and the scheduling node is selected in combination with the node scoring rules.

[0009] Based on periodic heartbeat detection and resource probe status feedback, the node's operating status is determined. When the node failure condition is met, task migration is performed and task node resources are reallocated through snapshot data.

[0010] The two-factor mechanism involves normalizing the computational complexity parameter and the I / O density parameter and using them as inputs to the load evaluation function. The load evaluation value obtained by the load evaluation function is used to dynamically adjust two trigger threshold parameters in the scaling up and down operation.

[0011] The two trigger threshold parameters include the current CPU utilization threshold and the resource prediction threshold, supporting a dual threshold mechanism for subsequent capacity expansion trigger judgment.

[0012] As a preferred embodiment of the distributed simulation method based on containerized deployment and elastic scaling described in this invention, the real-time resource monitoring and historical load trend data collection for the simulation task includes:

[0013] Monitoring probes deployed on the node side collect CPU utilization, memory usage, disk read / write frequency, and network bandwidth utilization of each computing node as real-time resource indicators, and aggregate the data to the scheduling center at fixed time intervals.

[0014] The dispatch center constructs a sliding time window model based on historical resource data, extracts the average value, variance, and trend direction of each node over N time periods, and characterizes the current load trend.

[0015] The submission time, estimated resource requirements, task duration, and historical execution performance of simulation tasks are stored in the task history database, which serves as the basis for load estimation and scaling strategy judgment.

[0016] As a preferred embodiment of the distributed simulation method based on containerized deployment and elastic scaling described in this invention, the evaluation load includes:

[0017] Extract the total number of simulation entities, the interaction dimension between each entity, the simulation time domain length, and the simulation step size from the simulation task description file, and calculate the computational complexity of the task.

[0018] The interaction dimension is the product of the interaction frequency and the amount of interaction data between simulated entities. The interaction frequency represents the number of logical communication events that occur per unit of simulation time, and the amount of interaction data represents the size of the data packet or the number of parameters involved in each communication.

[0019] The frequency of external data reading required for the monitoring task and the size of the data packets read are used to calculate the average number of I / O interactions per second and the amount of data transmitted as I / O density parameters.

[0020] The computational complexity and I / O density parameters of the task are substituted into a unified load evaluation function, and the function output is used as the basis for simulation task partitioning and initial resource allocation.

[0021] As a preferred embodiment of the distributed simulation method based on containerized deployment and elastic scaling described in this invention, the step of dividing into subtasks and containerizing them separately includes:

[0022] The output value of the load evaluation function is segmented, and the degree of coupling between entities in the simulation model is used to determine whether the task is divisible.

[0023] When it is determined that the simulation task can be split, it is divided into the smallest executable unit subtask with independent input and output interfaces.

[0024] Each subtask builds a standard container image, pulls the image, binds it to the network, and registers its metadata through container orchestration. After starting, it enters the task scheduling waiting queue, waiting for the scheduler to assign it to a specific node for execution.

[0025] A standard container image includes a basic simulation framework, configuration parameter files, a model library, and a list of dependency libraries.

[0026] The dependency library index list includes the name, version number, source address, and integrity verification hash value fields of each dependency.

[0027] As a preferred embodiment of the distributed simulation method based on containerized deployment and elastic scaling described in this invention, the scaling operations of the container instance include:

[0028] Within each fixed resource sampling period, the scheduler obtains real-time resource metrics for all currently running simulation container nodes.

[0029] Real-time resource metrics include CPU utilization, memory usage, and network bandwidth utilization.

[0030] A historical load forecasting model is constructed based on a sliding time window mechanism combined with time series forecasting methods. The historical load forecasting model is used to calculate the resource change trend of each node within a future preset time window, and obtain the predicted value of resource utilization rate for the next period.

[0031] Based on the output of the load assessment function, the current CPU utilization threshold and the resource prediction threshold are dynamically adjusted, and the dynamic threshold is used to determine the scaling conditions. The current CPU utilization threshold and the resource prediction threshold constitute a dual-threshold judgment mechanism for scaling triggering, corresponding to a dual-factor mechanism.

[0032] In the scaling down decision, nodes that have been assigned tasks in the previous scheduling cycle are excluded.

[0033] The expansion trigger condition is met when the current CPU utilization of a node exceeds the first threshold and the corresponding predicted value exceeds the second threshold.

[0034] The scheduler calculates the number of container replicas required for expansion based on the number of subtasks to be executed in the current simulation task queue, and calls the container orchestration platform interface to create an equal number of new container replicas according to the task type, which are then added to the current scheduling pool. The container replicas are derived from the container replica images.

[0035] When a node meets the following conditions for two consecutive scheduling cycles: the load continuously decreases to the stable range set by the experimenter, the node is not in the task receiving state, the CPU utilization and memory usage are both lower than the corresponding lower limit threshold and the network bandwidth utilization lower limit, and the predicted load does not exceed the average line for two consecutive cycles, the scaling-down mechanism is triggered.

[0036] The scheduler identifies currently running idle or completed but not reclaimed container instances and performs release operations according to task priority, calling container orchestration instructions to remove no longer needed container replicas, releasing node resources, and recording scaling actions in the scheduling log.

[0037] As a preferred embodiment of the distributed simulation method based on containerized deployment and elastic scaling described in this invention, the node scoring rules include:

[0038] After completing the creation of container replicas and adding them to the current scheduling pool, the scheduler collects CPU utilization, memory usage, and network bandwidth usage data for candidate nodes in the current schedulable state in the cluster, and uses these data as input parameters for the scoring function.

[0039] After the scheduler calculates the scores of all candidate nodes, it prioritizes assigning the created simulation subtask container replicas to the node with the highest score.

[0040] When multiple nodes have the same score, the node with the lower priority in the task scheduling history is scheduled first to maintain the overall scheduling balance.

[0041] As a preferred embodiment of the distributed simulation method based on containerized deployment and elastic scaling described in this invention, the selection of scheduling nodes includes:

[0042] After obtaining the score results of each candidate node, the scheduler constructs a node score mapping table, adds each container replica to the list to be assigned, and sorts and processes them according to the task type, resource requirement type, and creation order of the container replica.

[0043] The overall score is calculated by weighting the current CPU, memory, and network utilization. The weight coefficients of the resource items meet the normalization constraint conditions, and the weight values ​​can be strategically set according to the simulation task type.

[0044] The scheduler sequentially retrieves a container replica to be allocated from the container replica list, traverses all candidate nodes in the node rating mapping table whose ratings are higher than the set scheduling score threshold, and judges the resource conditions in turn.

[0045] Resource conditions include whether the node's current remaining CPU resources meet the CPU quota configuration required for container operation, whether the node's remaining memory resources meet the memory requirements of the container image, and whether the node's network port and bandwidth status allow access to simulated traffic connections.

[0046] When all resource conditions are met, the current container copy is allocated to the node, and the node's resource status table and the current period container reception record are updated.

[0047] If all qualified nodes fail to meet the allocation criteria, the container replica is temporarily added to the delayed scheduling queue, waiting for the next scheduling cycle to retry the allocation.

[0048] After scheduling is completed, the allocation relationships from this scheduling operation are written into the node task mapping table and the task execution status cache.

[0049] Node scoring is only performed on nodes that have not been marked as shrinkage candidates, thus avoiding scheduling nodes that are in a critical state of resource release for receiving new tasks.

[0050] As a preferred embodiment of the distributed simulation method based on containerized deployment and elastic scaling described in this invention, the step of determining the node's running status includes:

[0051] During operation, a heartbeat monitoring agent is deployed for each node participating in the simulation task. The agent sends heartbeat packets to the scheduler at fixed intervals.

[0052] The heartbeat packet includes a node identifier, a system timestamp, a snapshot of the resource status for the current period, and a summary of the probe verification results.

[0053] The scheduler maintains a node status table and performs periodic statistics and consistency checks on all received heartbeat packets. When a node continuously misses a predetermined number of heartbeat packets, a preliminary anomaly flag is triggered.

[0054] An independent resource probe task is run on each node. The probe periodically checks the underlying operating system response status, disk I / O normality, whether core services in the process table are online, and network interface connectivity. The probe results are synchronously uploaded to the status check server via REST.

[0055] A node is marked as an abnormal node when it meets one of the following two conditions.

[0056] Condition 1 is that the number of consecutive missing heartbeat packets reaches a preset number; condition 2 is that any item in the probe detection returns an abnormal status and has not recovered for two or more consecutive detection cycles.

[0057] When a node is marked as running abnormally, immediately stop assigning new simulation subtasks to the node, add it to the task migration pending queue, and write a fault event record in the node status table for use by subsequent scheduling and recovery strategies.

[0058] As a preferred embodiment of the distributed simulation method based on containerized deployment and elastic scaling described in this invention, the step of migrating and reallocating task node resources includes:

[0059] Once a node is marked as an abnormal running node, the scheduler immediately retrieves all simulation subtasks running on the node and loads the most recently available state snapshot corresponding to each subtask from the distributed object storage system.

[0060] A state snapshot is automatically generated at a fixed time interval based on the task type. When a sudden change in a key simulation variable, a change in the state of a subtask, or a scheduling event is detected, the snapshot is immediately triggered.

[0061] The state snapshot includes simulation timestamps, key variable values, intermediate calculation results, and container runtime configuration parameters.

[0062] Snapshot generation is handled asynchronously and stored in distributed object storage. The snapshot record contains the snapshot version number, generation timestamp, and unique task identifier. During task migration, the scheduler performs reliable recovery based on the latest snapshot data consistency flag.

[0063] The scheduler performs a reallocation process for each subtask to be migrated based on the latest ratings and resource status of all schedulable nodes in the current cluster, combined with the original running configuration of the task.

[0064] The reallocation process includes screening a list of candidate nodes with scores higher than the minimum scheduling threshold, prioritizing them according to the ratio of remaining node resources to subtask resource requirements, checking each candidate node to see if it meets the requirements for CPU cores, memory capacity, and image environment dependencies for snapshot recovery, starting a new container copy on a node that meets the conditions, and restoring the simulation execution context using snapshot data.

[0065] When a migration task competes for resources with a regular task in the current delayed scheduling queue, the scheduler prioritizes the allocation request of the migration task. The migration task has a higher scheduling priority than regular tasks and all tasks in the delayed scheduling queue.

[0066] When all nodes with scores above the scheduling threshold fail to meet the migration resource conditions, it is permissible to continue filtering among nodes with scores below the scheduling threshold to ensure the continuity of task migration and system fault tolerance. Migration priority takes precedence over regular scheduling rules in this scenario.

[0067] After recovery, the running status information of the subtask is synchronously written to the task status table, and the corresponding record in the original fault node resource mapping table is marked as invalid. The scheduler also records this task migration event and updates the migration log, resource reallocation table and container replica lifecycle management table.

[0068] After all subtasks have been migrated and reassigned, a new scheduling cycle begins, and the subsequent simulation task scheduling process continues.

[0069] A distributed simulation system based on containerized deployment and elastic scaling is characterized by including: a data acquisition and task evaluation module, an elastic scheduling module, and a fault migration module.

[0070] The data acquisition and task evaluation module is used to collect real-time resource monitoring and historical load trends of simulation tasks, and evaluate the load by calculating complexity parameters and I / O density parameters.

[0071] The elastic scheduling module is used to divide the simulation task into subtasks based on the minimum executable unit and containerize them separately. Based on real-time resource monitoring and historical load trends, it uses a two-factor mechanism to determine whether to perform the scaling up and down operation of the container instance, and selects the scheduling node in combination with the node scoring rules.

[0072] The fault migration module is used to determine the node's operating status based on periodic heartbeat detection and resource probe status feedback. When the node fault condition is met, it performs task migration and reallocates task node resources through snapshot data.

[0073] The beneficial effects of this invention are as follows: By collecting real-time resource monitoring and historical load trend data of simulation tasks, and combining simulation computation complexity parameters and I / O density parameters for unified load assessment, this invention achieves refined modeling of simulation tasks and prediction of resource requirements, thereby improving the accuracy of task scheduling and allocation.

[0074] By using a subtask splitting and standard containerization encapsulation mechanism based on the minimum executable unit, and in conjunction with a container orchestration platform, the modular deployment of simulation tasks is achieved, thereby supporting concurrent execution and elastic scheduling in a multi-node environment.

[0075] By employing a two-factor mechanism based on sliding time windows and predictive models to perform dynamic scaling operations on container replicas, combined with real-time node scoring and resource probe status feedback, dynamic management and intelligent selection of running nodes are achieved, effectively reducing resource waste and scheduling congestion risks.

[0076] By constructing a fault detection mechanism that links periodic heartbeat monitoring with resource status probes, the migration and recovery of simulation subtasks are completed based on snapshot data when a node malfunctions, ensuring the high availability and stability of the distributed simulation environment under abnormal conditions.

[0077] This invention enables high concurrency, high stability, and high resource utilization in simulation tasks within a multi-node dynamic heterogeneous resource environment, and possesses good horizontal scalability and engineering practical value. Attached Figure Description

[0078] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0079] Figure 1 The overall flowchart of the distributed simulation method and system based on containerized deployment and elastic scaling provided in the first embodiment of the present invention is shown. Detailed Implementation

[0080] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0081] Example 1, referring to Figure 1As one embodiment of the present invention, a distributed simulation method based on containerized deployment and elastic scaling is provided, comprising:

[0082] S1: Collect real-time resource monitoring and historical load trends of simulation tasks, and evaluate the load by calculating complexity parameters and I / O density parameters.

[0083] Monitoring probes deployed on the node side collect CPU utilization, memory usage, disk read / write frequency, and network bandwidth utilization of each computing node as real-time resource indicators, and aggregate the data to the scheduling center at fixed time intervals.

[0084] The dispatch center constructs a sliding time window model based on historical resource data, extracts the average value, variance, and trend direction of each node over N time periods, and characterizes the current load trend.

[0085] A preferred approach to characterizing the current load trend is:

[0086]

[0087] in, This represents the value of the i-th resource indicator within the tk-th period. This represents the mean. Let represent the standard deviation, i represent the node resource indicator type number, k represent the time window sliding index, and N represent the number of sliding window periods.

[0088] The submission time, estimated resource requirements, task duration, and historical execution performance of simulation tasks are stored in the task history database, which serves as the basis for load estimation and scaling strategy judgment.

[0089] Furthermore, based on the simulation task description file, the total number of simulation entities, the interaction dimension between each entity, the simulation time domain length, and the simulation step size are extracted to calculate the computational complexity of the task.

[0090] An optimal solution for reducing the computational complexity of a computational task is as follows:

[0091]

[0092] Among them, C task The simulation task computational complexity is represented by E, the total number of simulation entities is represented by D, the interaction dimension between entities is represented by T, the simulation time domain length is represented by Δt, and the simulation step size is represented by Δt.

[0093] The frequency of external data reading required for the monitoring task and the size of the data packets read are used to calculate the average number of I / O interactions per second and the amount of data transmitted as I / O density parameters.

[0094] A preferred method for calculating the average number of I / O interactions per second and the amount of data transferred is as follows:

[0095] D io =R f ×B s

[0096] Among them, R f Indicates the frequency of external data reading, B s D represents the size of each data packet read. io This represents the I / O density parameter.

[0097] The computational complexity and I / O density parameters of the task are substituted into a unified load evaluation function, and the function output is used as the basis for simulation task partitioning and initial resource allocation.

[0098] A preferred approach for substituting into the unified load evaluation function is:

[0099]

[0100] in, This represents the normalized computational complexity. The parameter represents the normalized I / O density, L represents the comprehensive load assessment value, and α and β represent the weighting coefficients, where α+β=1.

[0101] S2: The simulation task is divided into subtasks based on the minimum executable unit and containerized separately. Based on real-time resource monitoring and historical load trends, a two-factor mechanism is used to determine whether to perform the scaling up and down operation of the container instance, and the scheduling node is selected in combination with the node scoring rules.

[0102] The output value of the load evaluation function is segmented, and the degree of coupling between entities in the simulation model is used to determine whether the task is divisible.

[0103] When it is determined that the simulation task can be split, it is divided into the smallest executable unit subtask with independent input and output interfaces.

[0104] Each subtask builds a standard container image, pulls the image, binds it to the network, and registers its metadata through container orchestration. After starting, it enters the task scheduling waiting queue, waiting for the scheduler to assign it to a specific node for execution.

[0105] A standard container image includes a basic simulation framework, configuration parameter files, a model library, and a list of dependency libraries.

[0106] Within each fixed resource sampling period, the scheduler obtains real-time resource metrics for all currently running simulation container nodes.

[0107] Real-time resource metrics include CPU utilization, memory usage, and network bandwidth utilization.

[0108] A historical load forecasting model is constructed based on a sliding time window mechanism combined with time series forecasting methods.

[0109] A preferred approach for constructing a historical load forecasting model is:

[0110]

[0111] in, R represents the predicted load for the next cycle. (t-j+1) Representing historical periodic loads, λ0, λ j denoted by , j represents the historical time series window index, and H represents the historical review window length.

[0112] By using a historical load prediction model to calculate the resource change trend of each node within a preset time window, the predicted value of resource utilization rate for the next period is obtained.

[0113] A preferred method for calculating resource change trends within a future preset time window is:

[0114]

[0115] in, This represents the predicted value of the i-th indicator in period t+1, where θ0 and θ are... k Represents the regression coefficient. Let i represent the value of the i-th indicator at time k in history, and K represent the length of the regression window.

[0116] Based on the output of the load assessment function, the current CPU utilization threshold and the resource prediction threshold are dynamically adjusted, and the dynamic threshold is used to determine the scaling conditions.

[0117] A preferred approach for using dynamic thresholds in scaling decisions is as follows:

[0118]

[0119] in, This represents the initial CPU utilization threshold. This represents the initial predicted CPU utilization threshold. and The value range is [0.5, 0.95], δ1 and δ2 represent dynamic adjustment coefficients, γ1 represents the CPU utilization threshold, and γ2 represents the predicted CPU utilization threshold.

[0120] L∈[0,1] represents the current simulation task load level. A higher value indicates a higher task computation intensity and I / O density.

[0121] The expansion trigger condition is met when the current CPU utilization of a node exceeds the first threshold and the corresponding predicted value exceeds the second threshold.

[0122] A preferred scheme for determining the expansion trigger condition is as follows:

[0123]

[0124] in, This indicates the current CPU utilization. This indicates the predicted CPU utilization for the next cycle.

[0125] The scheduler calculates the number of container replicas required for expansion based on the number of subtasks to be executed in the current simulation task queue, and calls the container orchestration platform interface to create an equal number of new container replicas according to the task type, which are then added to the current scheduling pool. The container replicas are derived from the container replica images.

[0126] When the load continues to decrease to a stable range, and the node is not in the task receiving state, and the CPU utilization and memory usage are both below the corresponding lower limit threshold and the network bandwidth utilization lower limit, and the predicted load does not exceed the average line in two consecutive periods, the scaling-down mechanism is triggered.

[0127] The scheduler identifies currently running idle or completed but not reclaimed container instances and performs release operations according to task priority, calling container orchestration instructions to remove no longer needed container replicas, releasing node resources, and recording scaling actions in the scheduling log.

[0128] After creating the container replica and adding it to the current scheduling pool, the scheduler collects CPU utilization, memory usage, and network bandwidth usage data for candidate nodes in the current schedulable state in the cluster, and uses these data as input parameters for the scoring function to calculate the node score.

[0129] A preferred scheme for calculating node scores is as follows:

[0130] S node =w c ·(1-U c )+w m ·(1-U m )+w n ·(1-U n )

[0131] Among them, w c w m w n U represents the resource scoring weight. c U m U n This represents the current CPU, memory, and network utilization, S nodeThis indicates the overall score result.

[0132] The overall score is calculated by weighting the current CPU, memory, and network utilization. The weight coefficients of the resource items meet the normalization constraint conditions, and the weight values ​​can be strategically set according to the simulation task type.

[0133] After the scheduler calculates the scores of all candidate nodes, it prioritizes assigning the created simulation subtask container replicas to the node with the highest score.

[0134] When multiple nodes have the same score, the node with the lower priority in the task scheduling history is scheduled first to maintain the overall scheduling balance.

[0135] After obtaining the score results of each candidate node, the scheduler constructs a node score mapping table, adds each container replica to the list to be assigned, and sorts and processes them according to the task type, resource requirement type, and creation order of the container replica.

[0136] The scheduler sequentially retrieves a container replica to be allocated from the container replica list, traverses all candidate nodes in the node rating mapping table whose ratings are higher than the set scheduling score threshold, and judges the resource conditions in turn.

[0137] Resource conditions include whether the node's current remaining CPU resources meet the CPU quota configuration required for container operation, whether the node's remaining memory resources meet the memory requirements of the container image, and whether the node's network port and bandwidth status allow access to simulated traffic connections.

[0138] One optimal solution for judging resource conditions sequentially is:

[0139]

[0140] Among them, C avail Indicates available CPU resources, M avail N represents available memory resources. avail Indicates available network bandwidth, C need M need N need P represents the resource requirements of the container. sched This indicates the resource scheduling suitability score.

[0141] When all resource conditions are met, the current container copy is allocated to the node, and the node's resource status table and the current period container reception record are updated.

[0142] If all qualified nodes fail to meet the allocation criteria, the container replica is temporarily added to the delayed scheduling queue, waiting for the next scheduling cycle to retry the allocation.

[0143] After scheduling is completed, the allocation relationships from this scheduling operation are written into the node task mapping table and the task execution status cache.

[0144] S3: Based on periodic heartbeat detection and resource probe status feedback, the node's running status is determined. When the node failure condition is met, the task migration is performed and the task node resources are reallocated through snapshot data.

[0145] During operation, a heartbeat monitoring agent is deployed for each node participating in the simulation task. The agent sends heartbeat packets to the scheduler at fixed intervals.

[0146] The heartbeat packet includes a node identifier, a system timestamp, a snapshot of the resource status for the current period, and a summary of the probe verification results.

[0147] The scheduler maintains a node status table and performs periodic statistics and consistency checks on all received heartbeat packets. When a node continuously misses a predetermined number of heartbeat packets, a preliminary anomaly flag is triggered.

[0148] An independent resource probe task is run on each node. The probe periodically checks the underlying operating system response status, disk I / O normality, whether core services in the process table are online, and network interface connectivity. The probe results are synchronously uploaded to the status check server via REST.

[0149] A node is marked as an abnormal node when it meets one of the following two conditions.

[0150] Condition 1 is that the number of consecutive missing heartbeat packets reaches a preset number; condition 2 is that any item in the probe detection returns an abnormal status and has not recovered for two or more consecutive detection cycles.

[0151] When a node is marked as running abnormally, immediately stop assigning new simulation subtasks to the node, add it to the task migration pending queue, and write a fault event record in the node status table for use by subsequent scheduling and recovery strategies.

[0152] Furthermore, when a node is marked as an abnormal running node, the scheduler immediately retrieves all simulation subtasks running on the node and loads the most recently available state snapshot corresponding to each subtask from the distributed object storage system.

[0153] The state snapshot includes simulation timestamps, key variable values, intermediate calculation results, and container runtime configuration parameters.

[0154] The scheduler performs a reallocation process for each subtask to be migrated based on the latest ratings and resource status of all schedulable nodes in the current cluster, combined with the original running configuration of the task.

[0155] The reallocation process includes screening a list of candidate nodes with scores higher than the minimum scheduling threshold, prioritizing them according to the ratio of remaining node resources to subtask resource requirements, checking each candidate node to see if it meets the requirements for CPU cores, memory capacity, and image environment dependencies for snapshot recovery, starting a new container copy on a node that meets the conditions, and restoring the simulation execution context using snapshot data.

[0156] A preferred redistribution scheme is:

[0157]

[0158] Among them, C avail This indicates that the snapshot records CPU resource requirements, M snap The snapshot records memory resource requirements, δ(E) indicates whether the image environment has been deployed, and S reassign This indicates the node migration adaptation score.

[0159] After recovery, the running status information of the subtask is synchronously written to the task status table, and the corresponding record in the original fault node resource mapping table is marked as invalid. The scheduler also records this task migration event and updates the migration log, resource reallocation table and container replica lifecycle management table.

[0160] After all subtasks have been migrated and reassigned, a new scheduling cycle begins, and the subsequent simulation task scheduling process continues.

[0161] The above embodiments also include a distributed simulation system based on containerized deployment and elastic scaling, specifically: a data acquisition and task evaluation module, an elastic scheduling module, and a fault migration module.

[0162] The data acquisition and task evaluation module is used to collect real-time resource monitoring and historical load trends of simulation tasks, and evaluate the load by calculating complexity parameters and I / O density parameters.

[0163] The elastic scheduling module is used to divide the simulation task into subtasks based on the minimum executable unit and containerize them separately. Based on real-time resource monitoring and historical load trends, it uses a two-factor mechanism to determine whether to perform the scaling up and down operation of the container instance, and selects the scheduling node in combination with the node scoring rules.

[0164] The fault migration module is used to determine the node's operating status based on periodic heartbeat detection and resource probe status feedback. When the node fault condition is met, it performs task migration and reallocates task node resources through snapshot data.

[0165] Example 2, referring to the distributed simulation based on containerized deployment and elastic scaling, is an embodiment of the present invention, providing a distributed simulation method and system based on containerized deployment and elastic scaling. To verify the beneficial effects of the present invention, a simulation experiment is conducted for scientific demonstration.

[0166] A distributed simulation platform with four edge nodes was constructed. The experimental platform was uniformly configured, with each node having an 8-core CPU, 16GB of memory, and a network bandwidth of 1Gbps. By deploying lightweight simulation container tasks, phased concurrent task scheduling operations were performed, and the performance differences between conventional strategies and the strategy of this invention in terms of dynamic load perception, predictive decision-making, and scheduling response were compared.

[0167] The experimental tasks focused on meteorological simulation and vehicle traffic flow simulation as core scenarios. The number of simulation entities ranged from 500 to 1000, with tasks submitted in batches every 20 seconds, each batch containing 5 tasks. Throughout the experimental process, resource probe collectors were deployed on all nodes to collect real-time data on CPU utilization, memory usage, and network bandwidth usage, and a sliding time window was constructed to characterize load trends.

[0168] The strategy of this invention further introduces a simulation task complexity and I / O density evaluation mechanism. During the task submission stage, parameters such as interaction dimension, step size, and read frequency are extracted, and a unified load function is calculated to form a standardized task score.

[0169] During scheduling and execution, a scaling logic based on a two-factor prediction mechanism is adopted, which combines a regression prediction model to generate future resource utilization trends. When both the current utilization rate and the predicted value exceed the threshold, the system automatically scales up.

[0170] When the duration is below the lower limit and the predicted value is below the historical average, a scaling-down mechanism is triggered to dynamically control the number of container replicas. This invention also introduces a node score mapping and scheduling priority mechanism to achieve high-priority task allocation under the premise that both resource fit ratio and node score are satisfied.

[0171] All experimental tasks were run under the same basic configuration for comparative testing. The traditional static scheduling strategy and the dynamic container scheduling mechanism proposed in this invention were used respectively. Two sets of comparative tests were carried out under the same task scale and task interval conditions, and the tests were run continuously for 45 minutes. The key resource indicators and scheduling results were recorded. The experimental data are shown in Table 1.

[0172] Table 1 Experimental Data

[0173]

[0174]

[0175] As can be seen from the experimental data in the table, when using the conventional strategy, the average CPU utilization and memory utilization are 59.85% and 71.20%, respectively, the network bandwidth is close to saturation, the average task waiting time reaches 0.795 seconds, the number of scaling up and down operations is as high as 2.2 times, and the overall simulation completion time exceeds 126 seconds. In contrast, the CPU utilization and memory usage of the nodes using the strategy of this invention are significantly reduced, at 49.95% and 60.95%, respectively, indicating that this invention can more effectively allocate and predict and control resources. At the same time, the average network bandwidth is reduced to 64.55%, indicating that the coupling conflict between simulation tasks is alleviated and the I / O distribution is more balanced.

[0176] The average latency of tasks using the strategy of this invention is 0.585 seconds, significantly lower than the 0.795 seconds of the traditional method, a reduction of approximately 26%. The number of scaling up and down operations is reduced to 1.55, a decrease of approximately 29.5%, indicating that the present invention has stronger stability and robustness in terms of load awareness and replica control strategies.

[0177] Regarding the total simulation completion time, it decreased from an average of 126.5 seconds under the conventional strategy to 105.65 seconds under the strategy of this invention, representing an efficiency improvement of approximately 16.4%. This result confirms the optimization capability of this invention in key aspects such as task decomposition, dynamic scoring, and node matching for resource utilization.

[0178] Most existing scheduling strategies only adjust based on the current load or the static number of containers, making it difficult to cope with sudden increases in resources or instantaneous bottlenecks in high-concurrency scenarios. This invention, however, effectively improves task-granularity scheduling capabilities and global load balancing performance by introducing computational complexity modeling, I / O density evaluation, and load prediction mechanisms. Experimental results clearly demonstrate the innovative effects of this invention in scheduling timeliness, resource adaptability, and system load control, exhibiting significant novelty and practical value.

[0179] Example 3, referring to the distributed simulation based on containerized deployment and elastic scaling, is an embodiment of the present invention. It provides a distributed simulation system based on containerized deployment and elastic scaling, including: a data acquisition and task evaluation module 100, an elastic scheduling module 200, and a fault migration module 300.

[0180] Among them, S4: Data Acquisition and Task Evaluation Module 100 is used to collect real-time resource monitoring and historical load trends of simulation tasks, and evaluate the load by calculating complexity parameters and I / O density parameters.

[0181] It should also be noted that the data acquisition and task evaluation module 100 transmits the real-time resource indicator matrix, load trend sequence, and load evaluation value of the simulation task as input parameters to the elastic scheduling module 200, which is used to determine whether to perform task splitting and container instance scaling.

[0182] S5: The elastic scheduling module 200 is used to divide the simulation task into subtasks based on the minimum executable unit and containerize them separately. Based on real-time resource monitoring and historical load trends, it uses a two-factor mechanism to determine whether to perform the scaling up and down operation of the container instance, and selects the scheduling node in combination with the node scoring rules.

[0183] It should also be noted that the node task load mapping relationship and resource status snapshot formed by the elastic scheduling module 200 during the scheduling process will be synchronously transmitted to the fault migration module 300, which is used by the latter to perform task snapshot recovery and migration rescheduling when a node is abnormal.

[0184] S6: The fault migration module 300 is used to determine the node's operating status based on periodic heartbeat detection and resource probe status feedback. When the node fault condition is met, it performs task migration and reallocates task node resources through snapshot data.

[0185] It should also be noted that the correction results of the fault migration module 300 on the node scheduling status will affect the node scoring, task allocation priority and load balancing judgment in the subsequent scheduling strategy of module 200, thus forming a dynamic closed-loop control.

[0186] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0187] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0188] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0189] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc. It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0190] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A distributed simulation method based on containerized deployment and elastic scaling, characterized in that, include: Collect real-time resource monitoring information and historical load trends of simulation tasks, and evaluate the load by calculating complexity parameters and I / O density parameters; The simulation task is divided into subtasks based on the minimum executable unit and containerized separately. Based on real-time resource monitoring information and historical load trends, a two-factor mechanism is used to determine whether to perform the scaling up and down operation of the container instance, and the scheduling node is selected in combination with the node scoring rules. Based on periodic heartbeat detection and resource probe status feedback, the node running status is determined. When the node failure condition is met, task migration is performed and task node resources are reallocated through snapshot data. The two-factor mechanism includes: normalizing the computational complexity parameter and the I / O density parameter and using them as inputs to the load evaluation function; the load evaluation value obtained by the load evaluation function is used to dynamically adjust the two trigger threshold parameters in the scaling up and down operation. The two trigger threshold parameters include: the current CPU utilization threshold and the predicted resource utilization threshold, supporting a dual threshold mechanism for subsequent capacity expansion trigger judgment; The threshold value for predicted resource utilization is obtained by using CPU utilization, memory usage, and network bandwidth usage. A historical load prediction model is constructed based on a sliding time window mechanism combined with time series prediction methods. The historical load prediction model is used to calculate the resource change trend of each node within a future preset time window. The computational complexity parameter is obtained through the following formula: in, This represents the computational complexity of the simulation task. Indicates the total number of simulated entities. Indicates the dimension of interaction between entities. Indicates the simulation time domain length. Indicates the simulation step size; The interaction dimension is the product of the interaction frequency and the interaction data volume between simulated entities. The interaction frequency represents the number of logical communication events that occur per unit simulation time, and the interaction data volume represents the size of the data packet or the number of parameters involved in each communication.

2. The distributed simulation method based on containerized deployment and elastic scaling as described in claim 1, characterized in that: The real-time resource monitoring information and historical load trends of the simulation task collected include: Monitoring probes deployed on the node side collect CPU utilization, memory usage, disk read / write frequency and network bandwidth utilization of each computing node as real-time resource indicators, and aggregate the data to the scheduling center at fixed time intervals. The dispatch center constructs a sliding time window model based on historical resource data, extracts the average value, variance and trend direction of each node within N time periods, and characterizes the current load trend. The submission time, estimated resource requirements, task duration, and historical execution performance of simulation tasks are stored in the task history database, which serves as the basis for load estimation and scaling strategy judgment.

3. The distributed simulation method based on containerized deployment and elastic scaling as described in claim 2, characterized in that: The evaluation load includes: Extract the total number of simulation entities, the interaction dimension between each entity, the simulation time domain length, and the simulation step size index from the simulation task description file, and calculate the computational complexity of the task. The frequency and size of external data reads required for the monitoring task are used to calculate the average number of I / O interactions and data transfer volume per second as I / O density parameters. The computational complexity and I / O density parameters of the task are substituted into a unified load evaluation function, and the function output is used as the basis for simulation task partitioning and initial resource allocation.

4. The distributed simulation method based on containerized deployment and elastic scaling as described in claim 3, characterized in that: The process of dividing tasks into subtasks and containerizing them separately includes: The output value of the load assessment function is segmented, and the degree of coupling between entities in the simulation model is combined to determine whether the task is divisible. When it is determined that the simulation task can be decomposed, it is divided into the smallest executable unit subtask with independent input and output interfaces. Each subtask builds a standard container image, pulls the image, binds it to the network, and registers its metadata through container orchestration. After starting, it enters the task scheduling waiting queue, waiting for the scheduler to assign it to a specific node for execution. A standard container image includes: a basic simulation framework, configuration parameter files, a model library, and a list of dependency libraries; The dependency library index list includes: the name, version number, source address, and integrity verification hash value field of each dependency.

5. The distributed simulation method based on containerized deployment and elastic scaling as described in claim 4, characterized in that: The scaling operations of the container instance include: Within each fixed resource sampling period, the scheduler obtains real-time resource metrics for all currently running simulation container nodes; Real-time resource metrics include CPU utilization, memory usage, and network bandwidth utilization. A historical load forecasting model is constructed based on a sliding time window mechanism combined with time series forecasting methods. The historical load forecasting model is used to calculate the resource change trend of each node within a future preset time window, and obtain the predicted value of resource utilization rate for the next period. Based on the output of the load assessment function, the current CPU utilization threshold and the predicted resource utilization threshold are dynamically adjusted, and the dynamic threshold is used to determine the scaling up and down conditions. The current CPU utilization threshold and the predicted resource utilization threshold constitute a dual-threshold judgment mechanism for scaling up and down, corresponding to a dual-factor mechanism. In the scaling down decision, nodes that have been assigned tasks in the previous scheduling cycle are excluded; The expansion trigger condition is met when the current CPU utilization of a node exceeds the CPU utilization threshold and the corresponding predicted value exceeds the resource utilization predicted value threshold. The scheduler calculates the number of container replicas required for expansion based on the number of subtasks to be executed in the current simulation task queue, and calls the container orchestration platform interface to create an equal number of new container replicas according to the task type, which are then added to the current scheduling pool. The container replicas are derived from the container replica images. When a node meets the following conditions within two consecutive scheduling cycles: the load continuously decreases to the stable range set by the experimenter, the node is not in the task receiving state, the CPU utilization and memory usage are both lower than the corresponding lower limit threshold and the network bandwidth utilization lower limit, and the predicted load does not exceed the average line within two consecutive cycles, the scaling-down mechanism is triggered. The scheduler identifies currently running idle or completed but not reclaimed container instances and performs release operations according to task priority, calling container orchestration instructions to remove no longer needed container replicas, releasing node resources, and recording scaling actions in the scheduling log.

6. The distributed simulation method based on containerized deployment and elastic scaling as described in claim 5, characterized in that: The node scoring rules include: After completing the creation of container replicas and adding them to the current scheduling pool, the scheduler collects CPU utilization, memory usage, and network bandwidth usage data for candidate nodes in the current schedulable state in the cluster, and uses these data as input parameters for the scoring function. After the scheduler calculates the scores of all candidate nodes, it prioritizes assigning the created simulation subtask container replicas to the node with the highest score. When multiple nodes have the same score, the node with the lower priority in the task scheduling history is scheduled first to maintain the overall scheduling balance.

7. The distributed simulation method based on containerized deployment and elastic scaling as described in claim 6, characterized in that: The selection of scheduling nodes includes: After obtaining the score results of each candidate node, the scheduler constructs a node score mapping table, adds each container replica to be scheduled to the allocation list, and sorts and processes them according to the task type, resource requirement type and creation order of the container replicas. The overall score is calculated by weighting the current CPU, memory, and network utilization. The weight coefficients of the resource items meet the normalization constraint conditions, and the weight values ​​can be strategically set according to the simulation task type. The scheduler sequentially retrieves one container replica to be allocated from the container replica list, traverses all candidate nodes in the node rating mapping table whose ratings are higher than the set scheduling score threshold, and judges the resource conditions in turn. Resource conditions include: whether the node's current remaining CPU resources meet the CPU quota configuration required for container operation, whether the node's remaining memory resources meet the container image's memory requirements, and whether the node's network port and bandwidth status allow access to simulated traffic connections; When all resource conditions are met, the current container copy is allocated to the node, and the node's resource status table and the current period container reception record are updated. If all qualified nodes fail to meet the allocation conditions, the container replica will be temporarily added to the delayed scheduling queue and re-attempted in the next scheduling cycle. After scheduling is completed, the allocation relationship in this scheduling operation is written into the node task mapping table and the task execution status cache. Node scoring is only performed on nodes that have not been marked as shrinkage candidates, thus avoiding scheduling nodes that are in a critical state of resource release for receiving new tasks.

8. The distributed simulation method based on containerized deployment and elastic scaling as described in claim 7, characterized in that: The determination of the node's operating status includes: During operation, a heartbeat monitoring agent is deployed for each node participating in the simulation task. The agent sends heartbeat packets to the scheduler at fixed intervals. The heartbeat packet includes: node identifier, system timestamp, current period resource status snapshot and probe verification result summary; The scheduler maintains a node status table and performs periodic statistics and consistency checks on all received heartbeat packets. When a node continuously misses a predetermined number of heartbeat packets, it triggers a preliminary anomaly flag. Run an independent resource probe task for each node. The probe periodically checks the underlying operating system response status, disk I / O normality, whether the core services in the process table are online, and the network interface connectivity status of the node. The probe results are synchronously uploaded to the status check server via REST. A node is marked as an abnormal node when it meets one of the following two conditions simultaneously; Condition 1 is that the number of consecutive missing heartbeats reaches a preset number; Condition 2 is that any item in the probe detection returns an abnormal status and has not recovered for two or more consecutive detection cycles. When a node is marked as running abnormally, immediately stop assigning new simulation subtasks to the node, add it to the task migration pending queue, and write a fault event record in the node status table for use by subsequent scheduling and recovery strategies.

9. The distributed simulation method based on containerized deployment and elastic scaling as described in claim 8, characterized in that: The process of migrating and reallocating task node resources includes: Once a node is marked as an abnormal running node, the scheduler immediately retrieves all simulation subtasks running on the node and loads the most recently available state snapshot corresponding to each subtask from the distributed object storage system. A status snapshot is automatically generated at a fixed time interval based on the task type. When a sudden change in a key simulation variable, a change in the state of a subtask, or a scheduling event is detected, the snapshot is immediately triggered. The state snapshot includes simulation timestamps, key variable values, intermediate calculation results, and container runtime configuration parameters; Snapshot generation is handled asynchronously and stored in distributed object storage. The snapshot record contains the snapshot version number, generation timestamp, and unique task identifier. During task migration, the scheduler performs reliable recovery based on the latest snapshot data consistency flag. The scheduler performs a reallocation process for each subtask to be migrated based on the latest ratings and resource status of all schedulable nodes in the current cluster, combined with the original running configuration of the task. The reallocation process includes: screening a list of candidate nodes with scores higher than the minimum scheduling threshold, prioritizing them according to the ratio of remaining node resources to subtask resource requirements, checking whether each candidate node meets the requirements for CPU cores, memory capacity, and image environment dependencies for snapshot recovery, starting a new container copy on a node that meets the conditions, and restoring the simulation execution context through snapshot data. When a migration task competes for resources with a regular task in the current delayed scheduling queue, the scheduler prioritizes the allocation request of the migration task. The migration task has a higher scheduling priority than regular tasks and all tasks in the delayed scheduling queue. When all nodes with scores above the scheduling threshold fail to meet the migration resource conditions, it is permissible to continue filtering among nodes with scores below the scheduling threshold to find nodes that meet all resource conditions, thus ensuring the continuity of task migration and system fault tolerance. After recovery, the running status information of the subtask is synchronously written to the task status table, the corresponding record in the original fault node resource mapping table is marked as invalid, the scheduler records this task migration event at the same time, and updates the migration log, resource reallocation table and container replica lifecycle management table. After all subtasks have been migrated and reassigned, a new scheduling cycle begins, and the subsequent simulation task scheduling process continues.

10. A distributed simulation system based on containerized deployment and elastic scaling, using the method as described in any one of claims 1-9, characterized in that: Includes a data acquisition and task evaluation module (100), an elastic scheduling module (200), and a fault migration module (300). The data acquisition and task evaluation module (100) is used to collect real-time resource monitoring information and historical load trends of simulation tasks, and evaluate the load by calculating complexity parameters and I / O density parameters. The elastic scheduling module (200) is used to divide the simulation task into subtasks based on the minimum executable unit and containerize them separately. Based on real-time resource monitoring information and historical load trends, a two-factor mechanism is used to determine whether to perform the scaling up and down operation of the container instance, and the scheduling node is selected in combination with the node scoring rules. The fault migration module (300) is used to determine the node's operating status based on periodic heartbeat detection and resource probe status feedback, and to perform task migration and reallocate task node resources through snapshot data when the node fault condition is met.

Images