Simulation deduction method and system based on distributed parallel scheduling and storage medium

Abstract

The invention provides a simulation deduction method and system based on distributed parallel scheduling and a storage medium, and is applied to the technical field of data processing.The method comprises the steps that proxy services are deployed at a plurality of preset computing nodes, resource state information of all the nodes is dynamically collected, and a computing resource pool is obtained; arranging a pre-registered plug-in based on the data dependency relationship to obtain a simulation task process; in response to simulation scene configuration submitted by a user, generating a distributed scheduling task containing a fragmentation strategy; determining a plurality of sub-tasks corresponding to the distributed scheduling task based on the simulation task process; and based on the real-time load of the computing resource pool and the task priorities of the plurality of sub-tasks, respectively distributing the plurality of sub-tasks to a plurality of target computing nodes for task execution, and obtaining task execution result information. According to the method and the device, the time delay performance of distributed scheduling and the high availability performance of scheduling simulation tasks can be improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a simulation deduction method, system and storage medium based on distributed parallel scheduling. Background Art

[0002] Distributed simulation technology integrates scattered simulation resources (such as simulators, computing nodes, models, etc.) into a unified virtual simulation environment through computer networks, supports multi-node collaborative computing, and has become an important development direction of simulation technology.

[0003] Currently, there are a variety of technical solutions in the field of distributed simulation. For example, a distributed scheduling center generates pre-processing data for simulation units based on preset scenarios and user operations, distributes tasks to computing nodes using a directed acyclic graph model, and aggregates the results to iteratively update the simulation scenario.

[0004] However, the distributed simulation approach in related technologies still has some drawbacks. The aforementioned distributed scheduling relies on fixed resource allocation, fails to implement resource pooling and dynamic management, and is therefore difficult to adapt to the low-latency requirements of high-concurrency scenarios. Summary of the Invention

[0005] The present invention provides a simulation deduction method, system and storage medium based on distributed parallel scheduling, which are used to solve the defect of the distributed simulation method in the prior art that it is difficult to adapt to the low latency requirements in high-concurrency scenarios, and to improve the latency performance of distributed scheduling and the high availability of scheduling simulation tasks.

[0006] The present invention provides a simulation deduction method based on distributed parallel scheduling, comprising the following steps: deploying an agent service on multiple preset computing nodes, dynamically collecting resource status information of each node, and obtaining a computing resource pool; orchestrating pre-registered plug-ins based on data dependencies to obtain a simulation task flow; generating a distributed scheduling task including a sharding strategy in response to a simulation scenario configuration submitted by a user; determining multiple subtasks corresponding to the distributed scheduling task based on the simulation task flow; and distributing the multiple subtasks to multiple target computing nodes for task execution based on the real-time load of the computing resource pool and the task priorities of the multiple subtasks, thereby obtaining task execution result information.

[0007] According to a simulation deduction method based on distributed parallel scheduling provided by the present invention, pre-registered plug-ins are arranged based on data dependencies to obtain a simulation task flow, including: in response to receiving a directed acyclic graph between multiple plug-ins constructed by a user through a visual interface, the data dependencies of the multiple plug-ins are determined based on the directed acyclic graph; and the multiple plug-ins are arranged according to the data dependencies to obtain a simulation task flow.

[0008] According to a simulation and deduction method based on distributed parallel scheduling provided by the present invention, the simulation scenario configuration includes: a real scenario configuration, a joint training scenario configuration and a deduction scenario configuration. In response to the simulation scenario configuration submitted by the user, a distributed scheduling task including a sharding strategy is generated, including: when the simulation scenario configuration is the real scenario configuration, obtaining real equipment information at the current moment; sharding the real scene based on the real equipment information to obtain multiple real scenario subtasks; when the simulation scenario configuration is the joint training scenario configuration, obtaining joint training scenario information; performing task scheduling based on the joint training scenario information to obtain multi-party scenario tasks and real-time scenario tasks; when the simulation scenario configuration is the deduction scenario configuration, obtaining a deduction scenario task; performing distributed scheduling based on the deduction scenario task to obtain multiple deduction scenario subtasks.

[0009] According to a simulation and deduction method based on distributed parallel scheduling provided by the present invention, the multiple subtasks are respectively allocated to multiple target computing nodes for task execution based on the real-time load of the computing resource pool and the task priority of the multiple subtasks, and task execution result information is obtained, including: sorting the multiple subtasks according to a preset task configuration priority to obtain the task priority of the multiple subtasks; determining the pre-occupied computing resources of each subtask in the multiple subtasks and the allocable computing resources of the computing resource pool; according to the task priority, matching the pre-occupied computing resources with the allocable computing resources to obtain the target computing node corresponding to each subtask; performing distributed scheduling and task execution on each subtask based on the target computing node to obtain task execution result information.

[0010] According to a simulation and deduction method based on distributed parallel scheduling provided by the present invention, the method also includes: determining the execution task information, scenario information and execution computing node information of the multiple subtasks; and performing distributed task caching on the execution task information, the scenario information, the execution computing node information and the task execution result information.

[0011] According to a simulation deduction method based on distributed parallel scheduling provided by the present invention, the method further includes: obtaining execution log information corresponding to the task execution result information; and visually displaying the task execution result information and the execution log information.

[0012] The present invention also provides a simulation and deduction system based on distributed parallel scheduling, comprising the following modules: an acquisition module, used to deploy proxy services on multiple preset computing nodes, dynamically collect resource status information of each node, and obtain a computing resource pool; an orchestration module, used to orchestrate pre-registered plug-ins based on data dependencies, and obtain a simulation task flow; a configuration module, used to generate a distributed scheduling task containing a sharding strategy in response to a simulation scenario configuration submitted by a user; a scheduling module, used to determine, based on the simulation task flow, multiple subtasks corresponding to the distributed scheduling task; an execution module, used to distribute the multiple subtasks to multiple target computing nodes for task execution based on the real-time load of the computing resource pool and the task priority of the multiple subtasks, and obtain task execution result information.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the computer program, it implements a simulation deduction method based on distributed parallel scheduling as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the simulation and deduction method based on distributed parallel scheduling as described above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-described simulation and deduction methods based on distributed parallel scheduling.

[0016] The simulation deduction method, system and storage medium based on distributed parallel scheduling provided by the present invention dynamically collect resource status by deploying proxy services on multiple computing nodes, build a globally visualized computing resource pool, and realize flexible management and efficient utilization of resources; generate simulation task processes based on data dependency orchestration of pre-registered plug-ins, optimize task execution order and collaborative logic, and reduce the complexity of manual orchestration; generate sharding strategies according to user scenario configurations and decompose them into parallel subtasks, accurately match task granularity and computing power, and avoid uneven load; combine the real-time load of the resource pool with the dynamic scheduling allocation of subtask priorities, give priority to guaranteeing the preemption of key task resources, and reduce node congestion through load balancing, ultimately achieving stable execution of large-scale simulation deduction in a high-concurrency, low-latency environment, taking into account both system scalability and real-time requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 It is a flow chart of the simulation deduction method based on distributed parallel scheduling provided by the present invention.

[0019] Figure 2 This is a diagram of the distributed scheduling architecture provided by the present invention.

[0020] Figure 3 This is a schematic diagram of resource pool division provided by the present invention.

[0021] Figure 4 This is a schematic diagram of the simulation task process arrangement provided by the present invention.

[0022] Figure 5 It is a schematic diagram of the distributed parallel scheduling simulation deduction process provided by the present invention.

[0023] Figure 6 This is a real scene scheduling flow chart provided by the present invention.

[0024] Figure 7 This is a flowchart of the joint training scenario scheduling provided by the present invention.

[0025] Figure 8 This is a flowchart of the deduction scenario scheduling provided by the present invention.

[0026] Figure 9 This is a schematic diagram of the distributed scheduling simulation deduction process provided by the present invention.

[0027] Figure 10 It is a structural diagram of the simulation and deduction system based on distributed parallel scheduling provided by the present invention.

[0028] Figure 11 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0029] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0030] As simulation applications expand and deepen, the computational workload is increasing. Traditional stand-alone simulation systems can no longer meet real-world needs. Therefore, research has focused on distributed simulation systems. Distributed simulation technology utilizes computer networks to organically connect geographically dispersed simulators, computer simulation models, computing resources, and other simulation equipment into a unified whole, creating a highly concurrent, low-latency, and highly available virtual simulation environment for human-computer interaction.

[0031] Distributed simulation technology, as a key development direction in modeling and simulation technology, has made significant progress. Currently, distributed simulation systems can support the simulation of large-scale, complex systems and have been widely applied in multiple fields. A related technology has been published, which describes a distributed simulation system based on a wide area network (WAN) environment that implements cross-segment, real-time / super real-time simulation operations. Based on the preset simulation scenario and user operation data, the distributed scheduling center calculates and generates pre-processing data for each simulation unit in real time. This data is distributed to the distributed computing nodes providing services according to a directed acyclic graph model, and the calculation results from each computing node are collected. The results from each computing node are then aggregated and applied to objects in the simulation scenario to form pre-processing data for the next frame of scheduling. The data integration unit receives computing tasks assigned by the distributed scheduling center and sends them to different computing service units in a unified data format. It also receives feedback from the computing service units, encodes it in the format required by the distributed scheduling center, and then feeds it back to the distributed scheduling center.

[0032] However, this method is not applicable in high-concurrency, low-latency scheduling simulation environments, and it does not implement dynamic resource management through resource pooling.

[0033] Related technologies provide a distributed task scheduling method that loads business function plug-ins, deploys and starts business services, and simultaneously loads task configuration information. After acquiring a distributed lock, the corresponding task information is loaded from the database and cached. Executor services are distributedly configured and deployed, orchestrating task execution information according to the corresponding links. Task information is loaded periodically through the scheduler service. According to the preset scheduling strategy, the corresponding executor service is requested through asynchronous load, and execution logs are uniformly recorded. The overall execution status of the task scheduling is monitored.

[0034] This method does not explain how to schedule subtasks in task scheduling. The scheduling tasks are obtained from the database using a timer. The scheduling delay performance is not ideal and cannot meet the performance requirements of high concurrency and low latency in large-scale simulation deduction.

[0035] Therefore, based on the above problems, the main technical problems solved by the present invention are as follows: When faced with different types of simulation tasks, the simulation tasks can be split into subtasks and tasks can be dynamically orchestrated through visual plug-ins, making the task processing flow configurable to meet the scheduling requirements in different environments.

[0036] In the face of multi-user and multi-simulation environments, resources are registered and managed, and resources are pooled to achieve dynamic management of computing resources and flexible horizontal expansion of computing resources.

[0037] It can meet the system's performance requirements for low latency, high concurrency, and high reliability in multi-user, multi-simulation, and concurrent computing scenarios.

[0038] The purpose of the present invention is to provide a distributed parallel scheduling simulation deduction method, which can realize plug-in registration of simulation software, dynamic orchestration and distributed scheduling of simulation processes, register and manage computing nodes and pool resources, realize dynamic management of computing resources and flexible horizontal expansion of computing resources, and meet the performance requirements of high concurrency, low latency and high reliability in multi-user and multi-simulation scenarios.

[0039] refer to Figure 2 , Figure 2 This is a diagram of the distributed scheduling architecture provided by the present invention, which includes a scheduling center and an executor.

[0040] The scheduling center includes: task management (including: executor, task module, task management), resource registration (including: resource registration, computing node information, resource pool management), plug-in registration (including: plug-in registration, process orchestration management, version management), log management (including: scheduling log, task execution log, plug-in log), scheduler, log service, status management and registration service.

[0041] The executor includes: executor service, log service, scheduling results, agent service, scheduling request, task processing and task thread.

[0042] In order to achieve the above object, the present invention provides a distributed parallel scheduling simulation method, which mainly includes a scheduling center and an actuator module such as Figure 2As shown, the scheduling center includes a task management module, a resource registration module, a plug-in registration module, and a log management module. The task management module is mainly responsible for the startup, cancellation, deletion, configuration, and task scheduling of tasks. The resource registration module is mainly responsible for the registration management of computing nodes and the management of resource pooling. The plug-in registration module is mainly responsible for the registration management of simulation deduction software and other processing software, as well as the dynamic orchestration of simulation processes. The log management module is mainly responsible for the collection and management of scheduling logs, the collection and management of plug-in logs, and the management of task execution logs. The executor module is mainly responsible for receiving scheduling request tasks sent by the scheduling center and executing scheduling tasks, while reporting computing node resource information, scheduling result information, scheduling log information, etc.

[0043] Figure 1 This is a flow chart of the simulation deduction method based on distributed parallel scheduling provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 101: deploy an agent service on multiple preset computing nodes, dynamically collect resource status information of each node, and obtain a computing resource pool.

[0044] In an embodiment of the present invention, an agent service is deployed on a computing node to dynamically obtain and report computing node resource information.

[0045] An agent service is deployed on each compute node that needs to perform computing tasks. The agent service primarily obtains computing resource information, including CPU usage, GPU usage, memory usage, host name, and IP address. This information is published in real time through the distributed cache middleware (Redis). The scheduling center subscribes to this node information and uses it as the basis for distributed scheduling. The scheduling center obtains the latest resource usage of the compute node and selects the appropriate compute node for distributed scheduling based on the preset resource usage of the plug-in.

[0046] Step 102 : orchestrate pre-registered plug-ins based on data dependency to obtain a simulation task flow.

[0047] refer to Figure 3 , Figure 3 This is a schematic diagram of resource pool division provided by the present invention.

[0048] In an embodiment of the present invention, computing resource information is registered, computing resource IP and memory information are obtained, and a computing resource pool is configured.

[0049] Subscribe to the computing resource information published to the distributed cache middleware, obtain the resource information of each computing node, customize the resource pool name on the front-end interface and add the computing node to the computing resource pool to complete the registration of each computing node. Figure 3As shown, according to different service requirements, a resource pool with a resource pool name of resource pool 1 is created, and a computing node is added to the resource pool 1. Similarly, another resource pool with a resource pool name of resource pool 2 is created, and a computing node is added to the resource pool 2. The two resource pools have shared computing nodes, such as Figure 3 As shown, a total of four computing nodes are provided, a computing resource pool 1 is created, and a computing node 1, a computing node 2 and a computing node 3 are added to the computing resource pool 1, and a computing resource pool 2 is created, and a computing node 3 and a computing node 4 are added to the computing resource pool 2. The resource pool management supports dynamic expansion, and according to changes in load or service requirements, the scale of computing resources (CPU, GPU, memory and storage) is automatically or manually adjusted to ensure the performance and stability of the system.

[0050] According to the simulation deduction method based on distributed parallel scheduling provided by the application, the pre-registered plug-in is arranged based on the data dependency relationship to obtain a simulation task flow, comprising: In response to receiving a directed acyclic graph between a plurality of plug-ins constructed by a user through a visual interface, determining the data dependency relationship of the plurality of plug-ins based on the directed acyclic graph; The plurality of plug-ins are arranged according to the data dependency relationship to obtain a simulation task flow.

[0051] In the embodiment of the application, the simulation software plug-in is registered, and the flow arrangement is performed according to the simulation deduction task, and the resource pool information and the scheduling strategy information are configured.

[0052] The simulation software plug-in and other data service plug-in are registered, and the registration information mainly includes the identification, name, version number, owner, execution file, retry number, resource pool name, task type and parameter template of the plug-in. The resource pool name is the resource pool name created by the resource pool, and the resource pool contains the computing node information required. The parameter template contains the plug-in basic communication parameter, the plug-in identification information, and the parameter information required by the plug-in itself, and adopts a general and easily extended XML format. The registration information is stored in a mysql relational database, and the plug-in identification is used as a unique key value.

[0053] Reference Figure 4 , Figure 4 is a simulation task flow arrangement schematic diagram provided by the application.

[0054] According to the simulation task business flow requirement, the software plug-in with processing sequence is arranged, and is placed in the arrangement interface through the front-end interface dragging mode, the data dependency relationship is determined according to different connection lines, the plug-in is connected to form a directed acyclic graph (Directed acyclic graph, DAG), and a simulation task flow is constructed. The simulation task flow arrangement is as shown in Figure 4As shown in the figure, a simulation task flow is constructed. The flow consists of five plug-ins: plug-in a, plug-in b, plug-in c, plug-in d, and plug-in e. The plug-ins are connected by directed segments according to the data processing flow relationship to form a directed acyclic graph (DAG). Each plug-in is the smallest unit of task scheduling. When executing distributed scheduling, it is allocated to the computing nodes in the resource pool according to the resource optimization algorithm for execution.

[0055] Through the embodiments of the present invention, intuitive design and automatic analysis of data dependencies between plug-ins are achieved through a user interactive interface, which significantly improves the flexibility and accuracy of simulation task scheduling.

[0056] Step 103: In response to the simulation scenario configuration submitted by the user, a distributed scheduling task including a sharding strategy is generated.

[0057] In an embodiment of the present invention, scenario information is constructed in a simulation interface and a computing task is initiated.

[0058] On the simulation interface, build the simulation scenario by adding the simulation equipment, simulation payload, equipment initial parameters, simulation start time, simulation end time, and simulation step size based on the actual simulation scenario. Click Calculate on the front-end interface to send the scenario file information, including the scenario identifier, start time, and end time, to the scheduler, which then performs distributed scheduling.

[0059] Simulation scenarios are primarily categorized as real-world scenarios, joint training scenarios, and simulation scenarios. The construction methods for these scenarios are the same across all modes, differing only in the initial parameter settings and scheduling methods. Real-world scenarios are primarily designed to monitor real-world equipment situational information and simulate future situations. Initial parameters are those of real equipment. Joint training scenarios simulate joint training confrontations between the Red and Blue sides. Each side assesses the current situation and proposes feasible strategies to complete a simulated Red-Blue confrontation exercise. Simulation scenarios are primarily designed to verify hypothetical scenarios, conduct theoretical simulations, and assist in the analysis and verification of real-world and joint training scenarios.

[0060] According to a simulation deduction method based on distributed parallel scheduling provided by the present invention, the simulation scenario configuration includes: real scenario configuration, joint training scenario configuration, and deduction scenario configuration. In response to the simulation scenario configuration submitted by the user, a distributed scheduling task including a sharding strategy is generated, including: When the simulation scene configuration is the real scene configuration, obtain the real equipment information at the current moment; Segment the real scene based on real equipment information to obtain multiple real scene subtasks; When the simulation scenario is configured as a joint training scenario, obtaining joint training scenario information; Task scheduling is performed based on joint training scene information, and multi-party scene tasks and real-time scene tasks are obtained; When the simulation scene configuration is configured as a deduction scene configuration, a deduction scene task is obtained; Distributed scheduling is performed based on the deduction scene task, and a plurality of deduction scene sub-tasks are obtained.

[0061] Reference Figure 5 , Figure 5 The distributed parallel scheduling simulation deduction process schematic diagram provided by the application.

[0062] Multiple users (user 1, user 2, user 3, user N) perform simulation deduction calculation in real scene, joint training scene and simulation deduction scene three same modes respectively. User 1 uses in real scene, joint training scene and simulation deduction scene three modes; user 2 and user 3 use in joint training scene and simulation deduction scene mode; user N only uses in simulation deduction scene.

[0063] In real scene mode, considering that the number of simulation equipment is relatively large, the real scene needs to be fragmented, N sub-tasks are generated for scheduling, and each sub-task corresponds to a real scene processing flow.

[0064] In joint training scene, white, red, blue and real-time four types of scenes need to be deduced and calculated, respectively corresponding to white task, red task, blue task and real-time task, distributed parallel scheduling is performed, the initialization of joint training scene and the simulation deduction calculation in the process of drill training are completed.

[0065] The simulation deduction scene is to perform simulation deduction calculation for a certain intended scene, mainly for theoretical research and experimental verification.

[0066] The real-world scenario simulation process consists of plugins a, b, c, d, and e connected by directed lines to form a directed acyclic graph. The resource pool used by these plugins is configured as resource pool 1. The joint training scenario simulation process consists of plugins a, f, g, and h connected by directed lines to form a directed acyclic graph. The resource pool used by these plugins is configured as resource pool 2. The simulation scenario simulation process consists of plugins a, i, and j connected by directed lines to form a directed acyclic graph. The resource pool used by these plugins is configured as resource pool 2. Resource pool 1 consists of compute nodes 1, 2, and 3, while resource pool 2 consists of compute nodes 3 through N. Compute node 3 serves as a shared compute node for resource pools 1 and 2. Resource pools support the removal and addition of compute nodes, enabling dynamic management and horizontal expansion of computing resources. When executing distributed scheduling, computing tasks for multiple users and scenarios are stored in a cache queue. Tasks are prioritized based on their configured priorities, with higher-priority tasks being scheduled first. The real-world scenario simulation processing scheduling calculates the computing nodes that meet the conditions based on the resource utilization and plug-in resource pre-occupancy in resource pool 1, and schedules execution on the matching computing nodes. Finally, after distributed scheduling, plug-ins a and e are executed on computing node 1, plug-ins b and c are executed on computing node 2, and plug-in d is executed on computing node 3. Similarly, the joint training scenario processing is scheduled on the computing nodes in resource pool 2, plug-ins a and h are distributed on computing node 3, and plug-ins f and g are distributed on computing node N. Similarly, the simulation simulation computing processing is scheduled on the computing nodes in resource pool 2, plug-ins a and j are distributed on computing node N, and plug-in i is distributed on computing node 3. The scheduling methods under the three response modes are also different. The specific scheduling methods are as follows: refer to Figure 6 , Figure 6 This is a real scene scheduling flow chart provided by the present invention.

[0067] Real-world scheduling mainly involves obtaining the current equipment parameter information, performing scheduled scheduling calculations, and calculating the equipment status information for a period of time in the future. Figure 6As shown, first, a real-world scenario is constructed. Based on the real-world scenario, the corresponding equipment is selected to construct a scenario consistent with the real-world scenario. Secondly, the real-world scenario information is obtained, along with the latest equipment parameter information at the current moment. The latest scenario file based on the current moment is generated, with the default scenario time being the current time, the end time being one week later, and the step length being 10 minutes. Scheduled task scheduling is then performed, and the scenario deduction calculation task is sent to the scheduler for scheduled scheduling. The scheduler performs scheduled distributed scheduling on the simulation deduction engine at a fixed time every day. When executing task scheduling calculations, the calculation tasks are sliced ​​and divided into different subtasks. Distributed scheduling calculations are then performed. Once all subtasks are calculated, the real-world scenario deduction calculation is complete. Finally, the deduction calculation result information is obtained and pushed to the front end via websocket to view the situation information for the next week and analyze the future situation.

[0068] refer to Figure 7 , Figure 7 This is a flowchart of the joint training scenario scheduling provided by the present invention.

[0069] The joint training scenario is mainly based on the assumed training mission scenario, and the four scenarios of red team, blue team, white team and real time are scheduled and simulated respectively. The joint training scenario scheduling process is as follows: Figure 7 As shown, a joint training scenario is first constructed based on the hypothetical joint training mission. Next, the joint training mission scenario information is obtained and task scheduling is performed. The scheduler generates scenario task information for the Red, Blue, and White teams, respectively, and performs distributed scheduling. Upon receiving the scheduling request, the executor performs simulation task calculations to complete the simulation of the joint training scenario. Based on the training mission, the Red and Blue teams send policy information to the simulation calculation engine, which then executes policy control to control the posture and trajectory of the equipment in the scenario. The White team controls the addition and deletion of equipment to the Red and Blue scenarios based on the situational information, achieving global control of the joint confrontation scenario. The Red, Blue, and White teams push real-time simulation result data to their respective front-ends via websockets for display. After obtaining the simulation result information, the Red, Blue, and White teams analyze the situation and execute control actions.

[0070] refer to Figure 8 , Figure 8 This is a flowchart of the deduction scenario scheduling provided by the present invention.

[0071] The deduction scenario mainly performs simulation deduction calculations on the assumed scenario. The user builds the assumed scenario and sends the deduction scenario calculation task to the scheduler to perform the deduction calculation and complete the functional verification of the assumed scenario. The deduction scenario scheduling process is as follows: Figure 8As shown, the user first constructs a hypothetical scenario based on their needs. The dispatch center then obtains the hypothetical scenario information, generates simulation task information, and sends it to the dispatch center for task scheduling. The scheduler then performs distributed scheduling of the simulation scenario tasks, and the executor performs simulation calculations on the corresponding compute nodes, completing the simulation. Finally, the simulation results are pushed to the front end via a websocket. The situation front end then conducts theoretical research and experimental verification of the experiment. Simulation-verified policy information can also be transmitted to real-world scenarios and joint training scenarios to facilitate joint training and real-world experimental verification.

[0072] Through the embodiments of the present invention, the flexibility and processing efficiency of the simulation and deduction system are significantly improved through a sharding strategy that dynamically adapts to different scenarios: in real scenarios, based on sharding processing of real-time equipment data, high-concurrency deduction and calculation of large-scale equipment status are achieved; in joint training scenarios, by decomposing the red / blue / white sides and real-time tasks, multi-party confrontation deduction and dynamic strategy interaction are supported; in deduction scenarios, parallel subtasks are generated based on assumption parameters to quickly verify complex assumption logic.

[0073] Step 104 : Based on the simulation task flow, determine a plurality of subtasks corresponding to the distributed scheduling task.

[0074] In some embodiments, a simulation task flow (directed acyclic graph) is pre-orchestrated through a visual interface, where each node represents a plug-in (such as data processing, model calculation, situation generation, etc.), and the lines between nodes represent data dependencies.

[0075] Each plug-in node is used as the minimum scheduling unit to generate a subtask list. If the amount of data processed by the plug-in exceeds a threshold (such as the memory limit of a single node), it is further split into multiple parallel subtasks according to preset rules (such as the number of devices, geographical region, or time slice). Predestinations are annotated for each subtask to ensure that the execution order conforms to the DAG logic.

[0076] Step 105 : Based on the real-time load of the computing resource pool and the task priorities of the multiple subtasks, the multiple subtasks are respectively assigned to multiple target computing nodes for task execution, and task execution result information is obtained.

[0077] According to a simulation deduction method based on distributed parallel scheduling provided by the present invention, multiple subtasks are respectively assigned to multiple target computing nodes for task execution based on the real-time load of the computing resource pool and the task priorities of multiple subtasks, and task execution result information is obtained, including: Sort multiple subtasks according to the preset task configuration priority to obtain the task priorities of the multiple subtasks; Determining pre-occupied computing resources for each of the multiple subtasks and allocatable computing resources in the computing resource pool; According to the task priority, the target computing node corresponding to each subtask is obtained by matching the pre-occupied computing resources with the allocable computing resources; Distributed scheduling and task execution are performed on each subtask based on the target computing node to obtain task execution result information.

[0078] In an embodiment of the present invention, the scheduler service receives a scheduling request, performs distributed scheduling according to resource information in the computing resource pool, and sends execution task information to the distributed cache.

[0079] In an embodiment of the present invention, the executor monitors the execution task information in the distributed cache and executes the computing task.

[0080] The scheduler service receives a scheduling request and schedules multiple subtasks to be stored in a cache queue. It sorts the task queues according to the configured priority, with higher-priority tasks being scheduled first. Distributed scheduling matches the compute resources pre-allocated by the task with the available compute resources in the computing resource pool. Based on the principle of optimal computing resources, matching compute nodes are selected from the computing resource pool and distributed scheduling is performed. The scheduler pushes information about the tasks to be executed, the scenario, and the executing compute nodes to the distributed task cache.

[0081] The executor monitors the execution task information in the distributed cache, obtains the computing node information, execution task information, startup task script information, etc. of the execution task, and schedules the execution script to perform simulation calculations on the corresponding computing node.

[0082] Through the embodiments of the present invention, task priority sorting is used to ensure that critical tasks (such as real-time adversarial simulations) are given priority in preempting resources, thereby reducing the end-to-end processing delay of high-priority scenarios; based on the precise matching of pre-occupied resources and real-time available resources, computing node overload or idleness is avoided, thereby maximizing resource utilization.

[0083] According to a simulation deduction method based on distributed parallel scheduling provided by the present invention, the method further includes: Determine the execution task information, scenario information, and execution computing node information of multiple subtasks; Distributed task caching is performed on the execution task information, scene information, execution computing node information, and task execution result information.

[0084] In an embodiment of the present invention, the executor sends task execution result information and log information to the scheduling center through the distributed cache.

[0085] The executor collects the status information of the task execution results on the computing node and pushes it to the distributed cache in real time. At the same time, the simulation execution log information is pushed to the distributed cache in real time.

[0086] Through the embodiment of the present invention, by caching task information (parameters, scenario configuration, node allocation) and result data in a centralized manner, efficient collaboration between the dispatch center and the executors is achieved, ensuring low latency (millisecond level) in task distribution and real-time status updates. According to a simulation deduction method based on distributed parallel scheduling provided by the present invention, the method further includes: Get the execution log information corresponding to the task execution result information; Visually display the task execution result information and execution log information.

[0087] In the embodiment of the present invention, the execution of the scheduled tasks is monitored and log information is viewed through the management interface.

[0088] Subscribe to execution task information and log information from the distributed cache, push them to the front end in real time through websocket, monitor the execution status of scheduled tasks and view log information in the management interface.

[0089] refer to Figure 9 , Figure 9 The following is a schematic diagram of the distributed scheduling simulation process provided by the present invention, which includes the following steps: Step S1: deploy the Agent service and obtain computing resource node information.

[0090] Step S2: register resource information and configure the computing resource pool.

[0091] Step S3: register the simulation software plug-in and perform process arrangement according to the simulation deduction task.

[0092] Step S4: construct scene information and initiate computing tasks.

[0093] Step S5: The scheduler service receives the scheduling request, performs distributed scheduling, and sends the scheduling task information to the distributed cache.

[0094] Step S6: The executor monitors the execution task information in the distributed cache and executes the computing task.

[0095] Step S7: The executor pushes the task execution result information and log information to the distributed cache.

[0096] Step S8: The scheduling center pushes the task execution result information and log information to the management interface.

[0097] In the embodiment of the present invention, the simulation tasks are dynamically arranged in the form of visual plug-ins, such as Figure 4 The system uses a directed acyclic graph to represent the dependencies between processing plug-ins. A visual interface allows users to intuitively drag and drop components, set parameters, and define dependencies to build simulation task flows. Integrated into the simulation system, this plug-in offers the advantages of dynamic task orchestration, computational task decomposition, modularization, and process visualization.

[0098] A dynamic management method for computing resource pools is implemented. By registering and managing computing resources, a resource pool is constructed that can meet the needs of flexible scheduling of different resources and resource level expansion. Figure 3 As shown, this approach enables dynamic management of resource pools, allowing for the dynamic addition and removal of compute nodes, facilitating horizontal scaling of computing resources. This approach includes resource registration and scheduling, resource pool monitoring and reporting, elastic scaling, and horizontal scaling to adapt to changing business needs.

[0099] High-availability distributed cache middleware is used to provide high-speed cache services for the system. This distributed cache middleware supports fast writing / reading of scheduling task data, improving the latency performance of distributed scheduling and the high availability of scheduling simulation tasks. Figure 2 In the distributed scheduling architecture shown, a distributed cache middleware is used between the scheduling center and the executor to achieve fast writing and reading of scheduling task data, and to ensure high availability of scheduling task information through a distributed cluster.

[0100] Through the above-described embodiments of the present invention, simulation tasks are dynamically orchestrated using a visual plug-in. The dependencies between simulation tasks are represented using a directed acyclic graph (DAG). A visual interface allows users to intuitively drag and drop components, set parameters, and define dependencies to construct simulation task flows. This plug-in, integrated into the simulation deduction system, achieves the advantages of dynamic task orchestration, ease of adjustment, and process visualization. It also implements a dynamic management method for a computing resource pool. By registering and managing computing resources, a resource pool is constructed that supports flexible scheduling and horizontal resource expansion for diverse resources. This method includes technical means such as resource registration and scheduling, resource pool monitoring and reporting, and elastic scaling and horizontal expansion to adapt to changing business needs. High-availability distributed cache middleware is used to provide cache services for the system. This distributed cache middleware supports fast data write / read, improving the latency performance of distributed scheduling and the high availability of scheduled simulation tasks.

[0101] The simulation and deduction system based on distributed parallel scheduling provided by the present invention is described below. The simulation and deduction system based on distributed parallel scheduling described below and the simulation and deduction method based on distributed parallel scheduling described above can refer to each other.

[0102] refer to Figure 10 , Figure 10 It is a structural diagram of the simulation and deduction system based on distributed parallel scheduling provided by the present invention.

[0103] The collection module 1001 is used to deploy agent services on multiple preset computing nodes, dynamically collect resource status information of each node, and obtain a computing resource pool; The orchestration module 1002 is used to orchestrate pre-registered plug-ins based on data dependencies to obtain a simulation task flow; Configuration module 1003, for generating a distributed scheduling task including a sharding strategy in response to a simulation scenario configuration submitted by a user; The scheduling module 1004 is configured to determine a plurality of subtasks corresponding to the distributed scheduling task based on the simulation task process; The execution module 1005 is used to allocate the multiple subtasks to multiple target computing nodes for task execution based on the real-time load of the computing resource pool and the task priorities of the multiple subtasks, and obtain task execution result information.

[0104] The simulation and deduction system based on distributed parallel scheduling provided by this solution can fully implement the various operating steps in the above method embodiment and achieve the corresponding technical effects. To avoid repetition, the contents and beneficial effects that are the same as those in the method embodiment will not be repeated here.

[0105] Figure 11 This is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as Figure 11 As shown, the electronic device may include: a processor 1110, a communications interface 1120, a memory 1130, and a communication bus 1140, wherein the processor 1110, the communications interface 1120, and the memory 1130 communicate with each other via the communication bus 1140. The processor 1110 may call logic instructions in the memory 1130 to execute a simulation deduction method based on distributed parallel scheduling, the method comprising: deploying an agent service on a plurality of preset computing nodes, dynamically collecting resource status information of each node, and obtaining a computing resource pool; orchestrating pre-registered plug-ins based on data dependencies to obtain a simulation task flow; generating a distributed scheduling task including a sharding strategy in response to a simulation scenario configuration submitted by a user; determining a plurality of subtasks corresponding to the distributed scheduling task based on the simulation task flow; and allocating the plurality of subtasks to a plurality of target computing nodes for task execution based on the real-time load of the computing resource pool and the task priorities of the plurality of subtasks, and obtaining task execution result information.

[0106] Furthermore, the logic instructions in the aforementioned memory 1130 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0107] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the simulation deduction method based on distributed parallel scheduling provided by the above methods, the method including: deploying agent services on multiple preset computing nodes, dynamically collecting resource status information of each node, and obtaining a computing resource pool; orchestrating pre-registered plug-ins based on data dependencies to obtain a simulation task flow; generating a distributed scheduling task including a sharding strategy in response to a simulation scenario configuration submitted by a user; determining multiple subtasks corresponding to the distributed scheduling task based on the simulation task flow; and distributing the multiple subtasks to multiple target computing nodes for task execution based on the real-time load of the computing resource pool and the task priority of the multiple subtasks, and obtaining task execution result information.

[0108] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the simulation deduction method based on distributed parallel scheduling provided by the above-mentioned methods, the method comprising: deploying an agent service on multiple preset computing nodes, dynamically collecting resource status information of each node, and obtaining a computing resource pool; orchestrating pre-registered plug-ins based on data dependencies to obtain a simulation task flow; generating a distributed scheduling task including a sharding strategy in response to a simulation scenario configuration submitted by a user; determining multiple subtasks corresponding to the distributed scheduling task based on the simulation task flow; and distributing the multiple subtasks to multiple target computing nodes for task execution based on the real-time load of the computing resource pool and the task priority of the multiple subtasks, and obtaining task execution result information.

[0109] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0110] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A simulation deduction method based on distributed parallel scheduling, characterized in that: include: Deploy proxy services on multiple preset computing nodes, dynamically collect resource status information of each node, and obtain a computing resource pool; Based on data dependencies, pre-registered plug-ins are orchestrated to obtain simulation task flows. In response to the simulation scenario configuration submitted by the user, a distributed scheduling task including a sharding strategy is generated; Based on the simulation task process, determining a plurality of subtasks corresponding to the distributed scheduling task; Based on the real-time load of the computing resource pool and the task priorities of the multiple subtasks, the multiple subtasks are respectively allocated to multiple target computing nodes for task execution, and task execution result information is obtained.

2. The simulation deduction method based on distributed parallel scheduling according to claim 1 is characterized in that: The pre-registered plug-ins are arranged based on the data dependency to obtain a simulation task flow, including: In response to receiving a directed acyclic graph between a plurality of plug-ins constructed by a user through a visual interface, determining data dependency relationships among the plurality of plug-ins based on the directed acyclic graph; The multiple plug-ins are arranged according to the data dependency to obtain a simulation task flow.

3. The simulation deduction method based on distributed parallel scheduling according to claim 1 is characterized in that: The simulation scenario configuration includes: real scenario configuration, joint training scenario configuration and deduction scenario configuration. The distributed scheduling task including the sharding strategy is generated in response to the simulation scenario configuration submitted by the user, including: When the simulation scene configuration is the real scene configuration, obtaining real equipment information at the current moment; Segmenting the real scene based on the real equipment information to obtain multiple real scene subtasks; When the simulation scenario configuration is the joint training scenario configuration, obtaining joint training scenario information; Perform task scheduling based on the joint training scenario information to obtain multi-party scenario tasks and real-time scenario tasks; When the simulation scenario configuration is the deduction scenario configuration, obtaining a deduction scenario task; Distributed scheduling is performed based on the deduction scenario task to obtain multiple deduction scenario subtasks.

4. The simulation deduction method based on distributed parallel scheduling according to claim 1 is characterized in that: The step of allocating the multiple subtasks to multiple target computing nodes for task execution based on the real-time load of the computing resource pool and the task priorities of the multiple subtasks, and obtaining task execution result information includes: Sorting the plurality of subtasks according to a preset task configuration priority to obtain task priorities of the plurality of subtasks; Determining pre-occupied computing resources of each subtask in the plurality of subtasks and allocatable computing resources in the computing resource pool; According to the task priority, based on matching the pre-occupied computing resources with the allocatable computing resources, obtaining a target computing node corresponding to each subtask; Distributed scheduling and task execution are performed on each subtask based on the target computing node to obtain task execution result information.

5. The simulation deduction method based on distributed parallel scheduling according to claim 4 is characterized in that: The method further comprises: Determining execution task information, scenario information, and execution computing node information of the multiple subtasks; Distributed task caching is performed on the execution task information, the scenario information, the execution computing node information, and the task execution result information.

6. The simulation deduction method based on distributed parallel scheduling according to claim 1 is characterized in that: The method further comprises: Obtaining the execution log information corresponding to the task execution result information; Visually display the task execution result information and the execution log information.

7. A simulation deduction system based on distributed parallel scheduling, characterized in that: include: The acquisition module is used to deploy agent services on multiple preset computing nodes, dynamically collect resource status information of each node, and obtain a computing resource pool; The orchestration module is used to orchestrate pre-registered plug-ins based on data dependencies to obtain the simulation task flow; A configuration module, configured to generate a distributed scheduling task including a sharding strategy in response to a simulation scenario configuration submitted by a user; A scheduling module, configured to determine a plurality of subtasks corresponding to the distributed scheduling task based on the simulation task process; The execution module is used to allocate the multiple subtasks to multiple target computing nodes for task execution based on the real-time load of the computing resource pool and the task priorities of the multiple subtasks, and obtain task execution result information.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the simulation and deduction method based on distributed parallel scheduling as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the simulation and deduction method based on distributed parallel scheduling as described in any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the simulation and deduction method based on distributed parallel scheduling as described in any one of claims 1 to 6 is implemented.

Images