CAE (Computer Aided Engineering) elastic simulation architecture based on containerization arrangement

Abstract

The invention relates to the technical field of computer-aided engineering, and discloses a CAE elastic simulation architecture based on containerization orchestration, which packages each micro-service of CAE software and an operating environment thereof into an independent Docker container mirror image through a containerization method, realizes decoupling of the CAE software and the operating environment, is convenient to realize rapid deployment and dynamic adjustment of each micro-service of the CAE, and improves the simulation efficiency of the CAE software. And the portability and the deployment efficiency of the CAE software micro-service are improved. Meanwhile, the operation and maintenance management module can dynamically increase Worker nodes according to the load condition of the current computing cluster and a preset elastic strategy and match the Worker nodes for the to-be-executed tasks, resources can be accurately allocated to the to-be-executed tasks, unbalanced resource allocation is avoided, the waiting time of the to-be-executed tasks can be shortened, and the overall simulation efficiency can be improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of computer aided engineering, and discloses a CAE elastic simulation architecture based on containerization arrangement. BACKGROUND

[0002] There are two kinds of underlying computing resource architecture modes of existing CAE simulation systems. The first mode is a traditional desktop local software mode, and the expansibility of the mode is poor. With the continuous increase of simulation scale and fineness, the hardware resources and efficiency required for simulation are difficult to guarantee. The second mode is a multi-server or HPC cluster deployment mode, and although the mode improves the hardware computing capacity through multiple servers or HPC clusters, the hardware resources need to be planned in advance. In addition, there are many problems in task scheduling and resource management, such as unbalanced resource allocation, long task waiting time, etc. SUMMARY

[0003] The application aims to provide a CAE elastic simulation architecture based on containerization arrangement, which can avoid unbalanced resource allocation, reduce the waiting time of to-be-executed tasks, improve the expansibility of the architecture, and improve the overall simulation efficiency.

[0004] In order to achieve the above technical effects, the technical scheme adopted by the application is: A CAE elastic simulation architecture based on containerization arrangement, comprising: a private image warehouse, wherein a plurality of Docker container images are stored in the private image warehouse, and the Docker container images are packaged by each microservice of CAE software and the running environment of the microservice; an elastic simulation computing cluster, wherein the elastic simulation computing cluster comprises a Master node and a plurality of Worker nodes, the Master node is used for controlling each Worker node, and each Worker node is used for pulling a corresponding Docker container image from the private image warehouse and executing a to-be-executed task according to the instruction of the Master node; an operation and maintenance management module, wherein the operation and maintenance management module comprises a scheduling engine, the scheduling engine is used for receiving a simulation job request submitted by a user, decoupling the simulation job request into a plurality of to-be-executed tasks, adding a Worker node according to a preset elastic strategy according to the state information of the Worker nodes in the current elastic simulation computing cluster, and determining a scheduling scheme for each to-be-executed task; the scheduling scheme comprises a Docker container image corresponding to each to-be-executed task and a Worker node; The scheduling engine sends the scheduling scheme of each to-be-executed task to the Master node in the form of scheduling instructions, and the Master node controls the Worker node to pull the corresponding Docker container image from the private image warehouse according to the scheduling instructions, and then mounts data to execute the corresponding to-be-executed task.

[0005] Further, the microservices include a geometry microservice, a mesh microservice, a solver microservice, a data processing microservice, a numerical filter microservice, and a report generation microservice.

[0006] Further, the step of increasing the Worker node according to the preset elasticity policy comprises: The scheduling engine determines whether there is an idle Worker node according to the collected current Worker node state information, if there is an idle Worker node, the scheduling engine does not issue an instruction to the Master node to increase the Worker node, otherwise, the scheduling engine determines the number of increased Worker nodes according to the number of current to-be-scheduled tasks and resource requirements, and issues an instruction to the Master node to increase the Worker node, and then the Master node increases the corresponding Worker node.

[0007] Further, the method for the scheduling engine to determine the scheduling scheme for each to-be-executed task comprises: The scheduling engine collects the performance indicators of each Worker node and the constraint conditions of the to-be-scheduled to-be-executed tasks in real time, and the scheduling engine matches the Worker nodes for each to-be-executed task according to the performance indicators of each Worker node and the constraint conditions of the to-be-executed tasks, while comprehensively considering the GPU memory requirements of each to-be-executed task, the GPU memory idle conditions of each Worker node, and the satisfaction degree of the constraint conditions of the to-be-executed tasks.

[0008] Further, the performance indicators of the Worker node include CPU / GPU utilization, memory occupancy, and storage IOPS.

[0009] Further, the constraint conditions of the to-be-executed tasks include the front-back order relationship between the to-be-executed tasks and the priority of the to-be-executed tasks.

[0010] Further, the operation and maintenance management module further comprises a monitoring unit, and the monitoring unit is configured to collect the performance indicators of each Worker node and the task execution progress.

[0011] Further, a resource recycling unit is arranged in the scheduling engine, and is configured to recycle the computing resources allocated to the Worker node after the Worker node executes a task.

[0012] Compared with the prior art, the application has the beneficial effects of: The application encapsulates each micro-service of the CAE software and its running environment into an independent Docker container image through a containerization method, realizes decoupling of the CAE software and the running environment, facilitates rapid deployment and dynamic adjustment of each micro-service of the CAE, and improves the portability and deployment efficiency of the CAE software micro-service. Meanwhile, the operation and maintenance module can dynamically increase a Worker node according to the load condition of the current computing cluster and a preset elastic strategy, match the Worker node for each to-be-executed task, accurately allocate resources for each to-be-executed task, avoid resource allocation imbalance, reduce to-be-executed task waiting time, and improve overall simulation efficiency. Moreover, the application realizes the extensibility of the architecture on computing resources through the preset elastic strategy, enables the architecture to be deployed in multiple environments through the Docker container image, realizes the extensibility of the architecture on the environment, and thus improves the extensibility of the architecture. The architecture of the application not only solves the poor extensibility problem of the traditional desktop local software mode, but also overcomes the deficiencies of the multi-server or HPC cluster deployment mode in task scheduling and resource management, and can realize efficient utilization and flexible scheduling of simulation computing resources. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 FIG. 1 is a first framework schematic diagram of a CAE elastic simulation architecture based on containerization arrangement of the application; Figure 2 FIG. 2 is a second framework schematic diagram of a CAE elastic simulation architecture based on containerization arrangement of the application; Figure 3 FIG. 3 is a schematic diagram of allocation of a Worker node of the application. DETAILED DESCRIPTION

[0014] The application will be further described in detail below in combination with embodiments and drawings. However, this should not be understood as limiting the scope of the above-mentioned subject matter of the application to the following embodiments. Any technology realized based on the content of the application belongs to the scope of the application.

[0015] Referring to Figure 1 The application provides a CAE elastic simulation architecture based on containerization arrangement, which comprises: A private image warehouse, wherein a plurality of Docker container images are stored in the private image warehouse, and the Docker container images are encapsulated by each micro-service of CAE software and a running environment of the micro-service; An elastic simulation computing cluster, the elastic simulation computing cluster comprising a Master node and a plurality of Worker nodes, the Master node being configured to control the respective Worker nodes, and the Worker nodes being configured to pull corresponding Docker container images from the private image repository and execute the to-be-executed tasks according to the instructions of the Master node; An operation and maintenance module, the operation and maintenance module comprising a scheduling engine, the scheduling engine being configured to receive a simulation job request submitted by a user, decouple the simulation job request into a plurality of to-be-executed tasks, add Worker nodes according to a preset elastic strategy according to the state information of the Worker nodes in the current elastic simulation computing cluster, and determine a scheduling scheme for each to-be-executed task; the scheduling scheme comprising a Docker container image corresponding to each to-be-executed task and a Worker node; The scheduling engine sends the scheduling scheme of each to-be-executed task to the Master node in the form of a scheduling instruction, and the Master node controls the Worker nodes to pull corresponding Docker container images from the private image repository according to the scheduling instruction, and then mounts data to execute the corresponding to-be-executed task.

[0016] The present application encapsulates each microservice of the CAE software and its running environment into an independent Docker container image through a containerization method, realizes decoupling of the CAE software and the running environment, facilitates rapid deployment and dynamic adjustment of each microservice of the CAE, and improves the portability and deployment efficiency of the CAE software microservice. Meanwhile, the operation and maintenance module can dynamically add Worker nodes according to the load condition of the current computing cluster and a preset elastic strategy, match Worker nodes for each to-be-executed task, accurately allocate resources for each to-be-executed task, avoid resource allocation imbalance, reduce the waiting time of the to-be-executed task, and improve the overall simulation efficiency. Moreover, the present application realizes the scalability of the architecture on computing resources through a preset elastic strategy, enables the architecture to be deployed in multiple environments through Docker container images, realizes the scalability of the architecture on the environment, and thus improves the scalability of the architecture. The architecture of the present application not only solves the problem of poor scalability of the traditional desktop local software mode, but also overcomes the deficiencies of the multi-server or HPC cluster deployment mode in task scheduling and resource management, and can realize efficient utilization and flexible scheduling of simulation computing resources.

[0017] Reference Figures 1 to 3 To further illustrate the scheme of the present application, the following is an embodiment of a CAE elastic simulation architecture based on containerization orchestration, comprising: A private image warehouse in which a plurality of Docker container images are stored, the Docker container images being encapsulated by respective microservices of CAE software and running environments of the microservices. Specifically, first, a Dockerfile is written according to requirements of the respective microservices of the CAE software and the running environments thereof, to define a building process of a container image corresponding to each microservice. The microservices include a geometry microservice, a mesh microservice, a solver microservice, a data processing microservice, a numerical filter microservice, and a report generation microservice, etc. The geometry microservice is used for rendering of a simulation model and extracting line elements, surface elements, and volume elements in the model; the mesh microservice is used for mesh discretization of the simulation model; the solver microservice corresponds to a solving calculation module in the CAE software; and the data processing microservice is used for extracting and processing results after simulation calculation. The Dockerfile contains selection of a base image, software installation commands, environment variable settings, etc. Then, a docker build command is executed by using a Docker command line tool, to build the container image of each microservice of the CAE software according to the Dockerfile. Finally, the built container image is pushed into the private image warehouse, for subsequent quick deployment and expansion.

[0018] An elastic simulation calculation cluster including a Master node and a plurality of Worker nodes. The Master node is used for controlling the respective Worker nodes, and the Worker nodes are used for pulling corresponding Docker container images from the private image warehouse and executing tasks to be executed according to instructions of the Master node. Specifically, first, a Kubernetes cluster including a Master node and a Worker node pool is deployed on a cloud environment or a physical server, as shown in Figure 2 . Second, Kubernetes control plane components such as an API server, a scheduler, a controller manager, etc. are installed on the Master node, and access permissions of the private image warehouse are configured. Then, a preset number of Worker nodes are initially configured in the Worker node pool according to expected requirements and expected loads of CAE simulation jobs, and Kubernetes node components such as a kubelet component and a kube-proxy component, etc. are installed on each Worker node, and each Worker node is configured with a GPU having a preset memory size. Finally, a network plugin of the Kubernetes cluster is configured, to realize network communication of containers between the Worker nodes.

[0019] The operation and maintenance management module includes a scheduling engine, the scheduling engine is used for receiving a simulation job request submitted by a user, decoupling the simulation job request into a plurality of to-be-executed tasks, increasing a Worker node according to a preset elasticity strategy according to state information of the Worker node in the current elastic simulation computing cluster, and determining a scheduling scheme for each to-be-executed task; the scheduling scheme includes a Docker container image and a Worker node corresponding to each to-be-executed task. Specifically, the scheduling engine is constructed based on a scheduling mechanism of Kubernetes, is used for receiving a simulation job request submitted by a user, decoupling the simulation job request into a plurality of to-be-executed tasks, and putting all to-be-executed tasks into a scheduling queue in a sequence according to a priority of the simulation job request and a sequence relationship between the to-be-executed tasks. For example, when a user submits a simulation job request, the simulation job request is decoupled into six to-be-executed tasks including a geometry task, a grid task, a solving task, a data processing task, a numerical filtering task and a report generation task, wherein the solving task needs to be executed after the grid task is completed, so according to the dependency relationship between the solving task and the grid task, the grid task needs to be arranged in front of the solving task in the scheduling queue. When a user submits a plurality of simulation job requests, a priority is set in a configuration file of each simulation job request, a to-be-executed task decoupled from each simulation job request inherits the corresponding priority, and the arrangement order of all to-be-executed tasks in the scheduling queue needs to be determined in combination with the priority of the to-be-executed task and the sequence relationship between the to-be-executed tasks, wherein the to-be-executed task with a high priority is arranged in front of the to-be-executed task with a low priority.

[0020] Meanwhile, the scheduling engine increases the Worker node according to the state information of the Worker node in the current elastic simulation computing cluster according to a preset elasticity strategy, so as to adapt to the resource requirement of the current task. Referring to Figure 3 , the step of increasing the Worker node by the scheduling engine according to the preset elasticity strategy includes: the scheduling engine determines whether there is an idle Worker node according to the collected current Worker node state information, if there is an idle Worker node, the scheduling engine does not issue an instruction to the Master node to increase the Worker node, otherwise, the scheduling engine determines the number of increased Worker nodes according to the number and resource requirement of the current to-be-scheduled task, and issues an instruction to the Master node to increase the Worker node, and then the Master node increases the corresponding Worker node.

[0021] When there are enough idle Worker nodes in the elastic simulation computing cluster, the scheduling engine determines the corresponding Docker container image and allocates Worker nodes for the tasks to be scheduled in the scheduling queue, and sends them to the Master node in the form of scheduling instructions. The Master node, according to the scheduling instructions, instructs the corresponding Worker nodes to pull the corresponding Docker container images from the private image repository, and then mounts the data to execute the tasks. It should be noted that Worker nodes added through a preset elastic policy are automatically deleted after the task is completed, and can be elastically added again in subsequent scheduling based on real-time needs.

[0022] The scheduling engine also includes a resource reclamation unit. After a worker node completes its task, the scheduling engine reclaims the computing resources allocated to that worker node through this unit. Upon completion of the simulation job, a report generation module is triggered to generate a multi-dimensional performance analysis report based on data from the monitoring system. This performance analysis report includes information such as task execution time, resource utilization, and bottleneck analysis, allowing users to gain a deeper understanding and optimize the simulation process and results.

[0023] The monitoring unit is deployed in a Kubernetes cluster. It collects performance metrics and task execution progress information of each Worker node to continuously track the resource consumption and task execution status of each Worker node, and provides a visual interface for users to view.

[0024] In some embodiments, the scheduling engine allocates Worker nodes to tasks awaiting scheduling in the scheduling queue as follows: Based on the Kubernetes cluster's resource monitoring API, the scheduling engine collects real-time performance metrics such as CPU / GPU utilization, memory usage, and storage IOPS of each Worker node, as well as the constraints of the tasks awaiting execution in the scheduling queue. Based on the performance metrics of each Worker node and the constraints of the tasks awaiting execution, the scheduling engine matches Worker nodes to each task, taking into account the GPU memory requirements of each task, the GPU memory availability of each Worker node, and the degree to which the constraints of the tasks are met. The constraints include the sequential relationship between the tasks and their priorities.

[0025] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A CAE elastic simulation architecture based on containerized orchestration, characterized in that, include: A private image repository, which stores several Docker container images, wherein the Docker container images are encapsulated from the various microservices of the CAE software and the runtime environment of the microservices; The elastic simulation computing cluster includes a Master node and several Worker nodes. The Master node is used to control each Worker node, and the Worker nodes are used to pull the corresponding Docker container images from the private image repository and execute the tasks to be executed according to the instructions of the Master node. The operation and maintenance management module includes a scheduling engine, which is used to receive simulation job requests submitted by users, decouple the simulation job requests into several tasks to be executed, add Worker nodes according to the current status information of Worker nodes in the elastic simulation computing cluster, and determine a scheduling scheme for each task to be executed. The scheduling scheme includes the Docker container image and Worker node corresponding to each task to be executed; The scheduling engine sends the scheduling scheme of each task to be executed to the Master node in the form of scheduling instructions. The Master node controls the Worker nodes to pull the corresponding Docker container images from the private image repository according to the scheduling instructions, and then mounts the data to execute the corresponding tasks.

2. The CAE elastic simulation architecture based on containerized orchestration according to claim 1, characterized in that, The microservices include geometry microservices, mesh microservices, solver microservices, data processing microservices, numerical filter microservices, and report generation microservices.

3. The CAE elastic simulation architecture based on containerized orchestration according to claim 2, characterized in that, The steps for adding Worker nodes according to the preset elastic strategy include: The scheduling engine determines whether there are any idle worker nodes based on the collected current worker node status information. If there are idle worker nodes, the scheduling engine does not issue an instruction to the master node to add worker nodes. Otherwise, the scheduling engine determines the number of worker nodes to be added based on the number of tasks to be scheduled and resource requirements, and issues an instruction to the master node to add worker nodes. Then the master node adds the corresponding worker nodes.

4. The CAE elastic simulation architecture based on containerized orchestration according to claim 3, characterized in that, The method by which the scheduling engine determines a scheduling scheme for each task to be executed includes: The scheduling engine collects the performance metrics of each Worker node and the constraints of the currently scheduled tasks in real time. Based on the performance metrics of each Worker node and the constraints of the tasks to be executed, and taking into account the GPU memory requirements of each task to be executed, the GPU memory availability of each Worker node, and the degree to which the constraints of the tasks to be executed are met, the scheduling engine matches Worker nodes to each task to be executed.

5. The CAE elastic simulation architecture based on containerized orchestration according to claim 4, characterized in that, The performance metrics of the Worker node include CPU / GPU utilization, memory usage, and storage IOPS.

6. The CAE elastic simulation architecture based on containerized orchestration according to claim 5, characterized in that, The constraints on the tasks to be executed include the sequential relationship between the tasks and the priority of the tasks.

7. The CAE elastic simulation architecture based on containerized orchestration according to claim 6, characterized in that, The operation and maintenance management module also includes a monitoring unit, which is used to collect the performance indicators and task execution progress of each Worker node.

8. The CAE elastic simulation architecture based on containerized orchestration according to claim 7, characterized in that, The scheduling engine also includes a resource reclamation unit, which is used to reclaim the computing resources allocated to the Worker node after the Worker node has completed its task.

Images