Parallel rendering method and system for molecular dynamics trajectory based on supercomputing platform

Abstract

The invention relates to the technical field of computer aiding, and discloses a parallel rendering method and system for a molecular dynamics trajectory based on a supercomputing platform so as to improve the rendering efficiency. The method comprises the following steps: acquiring deployment of corresponding molecular dynamics visualization related components matched by a user aiming at a current supercomputing environment; the main node splits the big data track file by using Gromacs, generates sub-task track files related to task requirements, and distributes the sub-task track files to corresponding nodes; executing a non-interactive VMD task in parallel by each node which obtains the sub-task track file, calling a renderer to perform rendering operation, and storing a rendering result in a file folder corresponding to each node task in a shared memory in a tga file format; and the main node re-numbering the rendering subtask generation tga file, and then carrying out picture processing on the rendering tga through FFmpeg to generate an animation video file.

Description

Technical Field

[0001] The present invention relates to the field of computer-aided technology, and in particular to a method and system for parallel rendering of molecular dynamics trajectories based on a supercomputing platform. Background Art

[0002] With the rapid development of biomedicine and drug development, traditional experimental methods have problems such as long cycle, high cost and low success rate. Computer-aided drug design has become one of the important means of drug research and development. Molecular dynamics simulation visualization, as a user-oriented window, can intuitively show the movement of molecular microenvironment and is used to analyze biomolecular structure and simulate data. With the continuous advancement of experimental imaging, structural refinement and simulation technology of large biomolecular complexes such as viruses and photosynthetic organelles, higher and higher requirements are placed on the efficiency and data volume of visualization. In actual work, the animation generated by molecules based on trajectory data is 10 -9 s-10 -12 s is the number of frames. In order to ensure that the observation scale is long enough, the trajectory file data is tens of gigabytes (GB), and the total data volume often reaches hundreds of terabytes (TB) or even petabytes (PB). The increase in trajectory data will lead to an increase in image rendering consumption, affecting the entire drug design cycle. This is a huge challenge for the conventional visualization of the structure and dynamics of biomolecular systems. The use of high-performance graphics hardware and software for parallel rendering is a development trend. However, in existing computer-aided drug design applications, there are still technical problems such as insufficient compatibility between different architectures, long local rendering time and low efficiency, and low degree of parallelism. Summary of the invention

[0003] The present invention aims to disclose a parallel rendering method and system for molecular dynamics trajectories based on a supercomputing platform to improve rendering efficiency.

[0004] To achieve the above-mentioned purpose, the molecular dynamics trajectory parallel rendering method based on a supercomputing platform disclosed in the present invention includes:

[0005] Step S1, obtaining the deployment of molecular dynamics visualization related components adapted to the current supercomputing environment by the user, wherein the visualization application components include Gromacs, VMD and FFmpeg;

[0006] Step S2: The master node uses Gromacs to split the big data trajectory file, generates subtask trajectory files related to task requirements, and distributes each subtask trajectory file to the corresponding node;

[0007] Step S3, each node that obtains the subtask trajectory file executes the non-interactive VMD task in parallel, and calls the renderer to perform the rendering operation, and then stores the rendering result in the tga file format into the folder corresponding to each node task in the shared memory;

[0008] Step S4: The master node renumbers the tga file generated by the rendering subtask, and then processes the rendered tga through FFmpeg to generate an animation video file.

[0009] Preferably, in the step S2, it specifically includes:

[0010] Step S21, the master node obtains the idle node number through the Slurm management system, applies for computing resources using salloc, and performs computing command tests on the nodes that have successfully applied, and puts the computing node numbers that have successfully passed the test into the input parameters; one node corresponds to one server in the server cluster deployed by the supercomputing platform;

[0011] Step S22, the master node reads the TRR trajectory file and inputs preset parameters, which include at least the total number of trajectory frames and the number of computing node resources, and then distributes the task to each computing node that passes the test by task number to execute the Gromacs split trajectory task, and generates a trajectory file that is sorted by the trajectory frame number segmentation number.

[0012] Preferably, in step S2, the splitting strategy includes:

[0013] The master node takes every m+1 frames as a group according to the frame sequence, and the remaining part corresponding to the remainder is taken as another group; and starting from the second group, the first frame in any group is the same as the last frame of the previous group, so as to restore the rendering results corresponding to each group to the repeated frames of the original frame sequence in the TRR trajectory file.

[0014] Preferably, in step S4, the master node implements frame skipping processing for the rendering results corresponding to each repeated frame after each group.

[0015] To achieve the above-mentioned purpose, the present invention discloses a parallel rendering system of molecular dynamics trajectories based on a supercomputing platform, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps corresponding to the above-mentioned method for parallel rendering of large-scale molecular dynamics trajectories based on a supercomputing platform.

[0016] Optionally, in the supercomputing environment constructed by the supercomputing platform, each node used to collaboratively realize parallel rendering of molecular dynamics trajectories is based on an ARM architecture or an X86 architecture.

[0017] The present invention has the following beneficial effects:

[0018] By deeply optimizing the task calculation process and making full use of the parallel processing capabilities of the supercomputing platform, the rendering tasks can be automatically assigned to each computing node, significantly improving the efficiency of the molecular dynamics visualization process; the designed task resource allocation strategy can adjust the allocation of computing resources according to the real-time needs of the task, ensuring that each computing resource can be effectively utilized, avoiding resource waste and ensuring efficient parallel rendering. Ultimately, an efficient and reliable molecular dynamics large-scale trajectory parallel rendering solution is provided for the supercomputing platform to support and guide efficient drug discovery.

[0019] The present invention will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0021] Figure 1 It is a schematic diagram of a parallel rendering method of molecular dynamics trajectories based on a supercomputing platform disclosed in an embodiment of the present invention.

[0022] Figure 2 It is a flow chart of a method for parallel rendering of molecular dynamics trajectories based on a supercomputing platform disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The embodiments of the present invention are described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0024] Example 1

[0025] This embodiment will design the overall application process according to the application requirements of molecular dynamics simulation parallel rendering to adapt to different simulation scenarios and user needs. The process includes key steps such as application deployment, trajectory splitting, VMD (Visual Molecular Dynamics, a computer molecular observation software) parallel rendering, animation generation, etc. According to the preset application type, the whole process operation is completed through the following technical means:

[0026] (1) Task refinement: Big data tasks are divided into small data tasks, and the corresponding small data tasks can be generated by simply changing the relevant parameters.

[0027] (2) Task modularization: The system provides preset task templates. Users only need to modify the parameters to generate a specific rendering task script and submit it to each computing node of the supercomputing platform.

[0028] (3) The computing subtasks are automatically merged, and the system merges the rendering generation results. The user only needs to input parameters to generate a visualization video of the corresponding case.

[0029] Specifically, first, consider how to optimize the calculation process and improve the efficiency of the algorithm so as to efficiently and parallelly render the molecular dynamics simulation results in a short time; second, consider how to make full use of computing resources in large-scale calculations; and finally, automatically generate visualization for the rendered images. In order to process massive trajectory data in a limited time, this embodiment deeply optimizes the calculation process, fully utilizes the storage and parallel processing capabilities of the supercomputing platform, and automatically allocates computing tasks to each computing node, which significantly improves the efficiency of the rendering process. At the same time, this embodiment sets up a task submission process, adjusts the steps of the task according to the requirements of the task, ensures that resources are fully used, avoids resource waste, and ensures that the molecular dynamics simulation parallel rendering task is completed. By presetting parameters, trajectory splitting, VMD trajectory rendering, and animation generation can be performed, which reduces the situation of manual intervention and improves the reliability of the rendering process and the repeatability of the results. Finally, an efficient and reliable molecular dynamics large-scale trajectory parallel rendering solution is provided for the supercomputing platform to support and guide the efficient discovery of drugs.

[0030] The method in this embodiment is mainly used for trajectory file visualization in biomedical research and drug development process, and evaluates and predicts potential drug candidate molecules and binding affinity and selectivity through visualization results. In order to improve computing efficiency and optimize resource utilization, the system can allocate computing resources, automatically plan and adjust rendering tasks and optimize resource utilization to ensure that as many simulation tasks as possible are completed in the shortest time, accelerating the drug design and development process.

[0031] The molecular dynamics large-scale trajectory parallel rendering method based on a supercomputing platform provided in this embodiment is referred to Figure 1 and Figure 2 , the corresponding processing steps are application deployment, trajectory splitting, VMD parallel rendering, and animation generation:

[0032] 1. Application deployment: build and deploy visualization application components such as Gromacs, VMD, POV-ray (Persistence of Vision Raytracer), FFmpeg (Fast Forward Mpeg) based on the supercomputing platform, and adapt them to the current computer architecture environment. The above software already has adapted versions in the x86 and arm frameworks.

[0033] Among them, Gromacs is a molecular dynamics package used to study biomolecular systems, and the product has the advantage of more powerful computing capabilities. POV-ray is the first ray tracing software that renders images in space orbits. FFmpeg is an open source computer program that can be used to record and convert digital audio and video and convert them into streams.

[0034] Second, trajectory splitting, preprocessing the input trajectory files in the drug research and development task scenario, and splitting the large-scale trajectory files through Gromacs. Specifically, first read the number of frames of the TRR trajectory file, input the total number of trajectory frames, the number of computing node resources and other preset parameters, and then obtain the idle node number through the Slurm management system, use salloc to apply for computing resources, and perform a simple computing command test on the successfully applied nodes, and put the computing node number that successfully passed the test into the input parameter. Among them, the Slurm management system is an open source, scalable cluster management and job scheduling system, which is widely used in high-performance computing clusters; salloc is a command line tool in the job scheduling system, which is used to allocate computing resources to the user's tasks and start an interactive session.

[0035] In this embodiment, the trajectory splitting task and the computing node are managed and matched with each other, and the task is distributed to each computing node through the task number to execute the Gromacs trajectory splitting task, and the generated trajectory file is sorted by the segmentation number of the trajectory frame number.

[0036] 3. VMD parallel rendering, based on the VMD after application deployment, performs non-interactive work, and calls the renderer POV-Ray through the tclsh script to perform trajectory batch rendering. Specifically, by loading the VMD configuration parameters, matching the TRR trajectory files generated by trajectory splitting with the computing resource node numbers, and submitting the VMD non-interactive tasks to the computing nodes for execution, the image set corresponding to the POV-Ray rendering generation task can be generated.

[0037] 4. Animation generation: Video generation of rendered images based on FFmpeg after application deployment. By renumbering and aggregating the parallel rendering generated image sets, FFmpeg is called to generate mp4 video files.

[0038] In some implementations, in order to better illustrate the purpose and advantages of this embodiment, the present embodiment is further described below in conjunction with some calculation examples. This example is a small molecule rendering process generated during a drug development process based on a domestic new generation supercomputer platform, specifically including:

[0039] After the application is deployed, the trajectory is split first, and a 16GB trajectory file (10,000 frames) is used for testing, using a protein of a certain biological molecule and its small molecule ligand. The reading time of a 16G trajectory file is nearly 30-40 minutes. If the intermediate frame number is selected for reading, the reading is also processed from the zeroth frame. In order to ensure the efficiency of the operation and speed up the rendering process, it is necessary to split the TRR file (i.e., gromacs output trajectory file). Through the data division of the computing task, the data reading process is greatly accelerated. There are nearly 10,000 frames of trajectory data in the 16GTRR file. In order to ensure that the trajectory can be split into any number of frames, the starting frame number is reasonably planned, the reading is set according to the split frame number, the starting frame number size of the sub-file is configured, and finally the tail is processed to implement the functional algorithm. The specific trajectory splitting strategy can be: take every 40 frames as a group according to the frame sequence, and the remaining part corresponding to the remainder as another group; and starting from the second group, the first frame in any group is the same as the last frame of the previous group, so that the rendering results corresponding to each group can be restored to the repeated frames of the original frame sequence in the TRR trajectory file.

[0040] The entire process of trajectory splitting includes four steps: input parameter setting, calculation of interpolation frames, task division, and parallel execution:

[0041] (a) Input parameter setting: Through the setting interface, configure the parameters, including the input TRR file, output file prefix, total number of frames, number of interpolated frames, time per frame, processing directory and computing nodes.

[0042] (b) Calculate the number of interpolation frames: Calculate the number of interpolation frames according to the input parameters, and calculate the starting coordinates of the real-time interpolation. The algorithm is implemented using trajectory splitting pseudocode.

[0043] (c) Task division: The starting frame number is obtained by calculation, and the task assignment is written into the corresponding node execution task.

[0044] (d) Parallel execution: By submitting tasks in parallel using the new generation of domestic supercomputers, the multi-node TRR file splitting function can be realized.

[0045] Then VMD parallel rendering is performed, using the shared storage function in the domestic new generation supercomputer, the trajectory file TRR involved in the rendering is distributed to 256 computing nodes (cn nodes) as the input of VMD non-interactive jobs for calculation, and the rendering results of each computing node are generated in the shared storage, which can accelerate the entire rendering process. In the visualization interface of VMD, tasks cannot be submitted to the computing resource pool, so tasks need to be distributed in the form of non-interactive jobs, in which each subtask reads the corresponding TRR file on the computing node according to the task number, and finally completes the parallel rendering function.

[0046] The processes of VMD parallel rendering are as follows:

[0047] (e) Parameter setting: VMD non-interactive task submission is passed through environment variables, including file path, case path, protein name, TRR file name, number of TRR files and output directory.

[0048] (f) Trajectory file sorting: By parsing the TRR file number, the TRR files are sorted to generate a file array.

[0049] (g) Task division: Associate the computing node number with the TRR file number to form a computing sub-job.

[0050] (h) Multi-task parallel execution: For multiple parallel rendering cases, open the multi-task interface, allocate nodes, and submit them in parallel on high-performance computers.

[0051] Finally, the animation is generated. The data generated by parallel rendering will be stored in the folder of each node task, and the name number of the rendered image will start from 0. The demonstration animation generated by FFmpeg can only be identified by continuous file numbers, and when the number of rendered images is large, it will directly lead to a surge in numbering work. In view of the above situation, we have improved the animation generation process, specifically:

[0052] The rendering results are sorted twice. The first sorting is to sort the rendering file numbers into an array by the number of the generated results, from low to high. The second sorting is to sort by the tga (tagged graphics, an image file format) number in the generated results. Since the generated results are related to the number of trajectory file frames, the sorting number follows the following rules:

[0053] (1) Only one tga file of the 0th frame is needed in each folder.

[0054] (2) The tga file number of the first frame in each file is consistent with the last frame number of the folder with the previous number, and the first frame image in the first file is retained.

[0055] (3) The order of the entire folder number is consistent with the order of the total frame number, both in ascending order.

[0056] Animation generation is gradually disassembled according to the above principles, forming the following three processes:

[0057] (i) Parameter setting: Enter the required parameters and information, such as tga directory prefix, rendered image subdirectory, number of merged files, output directory, etc.

[0058] (j) File sorting: Based on the pseudocode, write a sorting algorithm to ensure that the tga file numbers are renumbered starting from 0.

[0059] (k) FFmpeg animation generation: Generate an mp4 format animation video file through FFmpeg. Preferably, in the process of generating the animation video file, a frame skipping process is implemented for the rendering results corresponding to each repeated frame after each group.

[0060] The demonstration video can be generated through the above steps.

[0061] Example 2

[0062] Corresponding to the above embodiment, this embodiment discloses a molecular dynamics trajectory parallel rendering system based on a supercomputing platform, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps corresponding to the above molecular dynamics large-scale trajectory parallel rendering method based on a supercomputing platform are implemented:

[0063] Step S1, obtaining the deployment of molecular dynamics visualization related components adapted to the current supercomputing environment by the user, wherein the visualization application components include Gromacs, VMD and FFmpeg.

[0064] Step S2: The master node uses Gromacs to split the big data trajectory file, generates subtask trajectory files related to task requirements, and distributes each subtask trajectory file to the corresponding node.

[0065] Step S3: Each node that obtains the subtask trajectory file executes the non-interactive VMD task in parallel, calls the renderer to perform rendering operations, and then stores the rendering results in the tga file format in the folder corresponding to each node task in the shared memory.

[0066] Step S4: The master node renumbers the tga file generated by the rendering subtask, and then processes the rendered tga through FFmpeg to generate an animation video file.

[0067] The specific implementation of the above steps can refer to Example 1 and will not be described in detail.

[0068] Optionally, in the supercomputing environment constructed by the supercomputing platform, each node used to collaboratively realize parallel rendering of molecular dynamics trajectories is based on an ARM architecture or an X86 architecture.

[0069] In summary, the methods and systems disclosed in the above embodiments of the present invention respectively have at least the following beneficial effects:

[0070] By deeply optimizing the task calculation process and making full use of the parallel processing capabilities of the supercomputing platform, the rendering tasks can be automatically assigned to each computing node, significantly improving the efficiency of the molecular dynamics visualization process; the designed task resource allocation strategy can adjust the allocation of computing resources according to the real-time needs of the task, ensuring that each computing resource can be effectively utilized, avoiding resource waste and ensuring efficient parallel rendering. Ultimately, an efficient and reliable molecular dynamics large-scale trajectory parallel rendering solution is provided for the supercomputing platform to support and guide efficient drug discovery.

[0071] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A parallel rendering method for molecular dynamics trajectories based on a supercomputing platform, characterized in that: include: Step S1, obtaining the deployment of molecular dynamics visualization related components adapted to the current supercomputing environment by the user, wherein the visualization application components include Gromacs, VMD and FFmpeg; Step S2: The master node uses Gromacs to split the big data trajectory file, generates subtask trajectory files related to task requirements, and distributes each subtask trajectory file to the corresponding node; Step S3, each node that obtains the subtask trajectory file executes the non-interactive VMD task in parallel, and calls the renderer to perform the rendering operation, and then stores the rendering result in the tga file format into the folder corresponding to each node task in the shared memory; Step S4: The master node renumbers the tga file generated by the rendering subtask, and then processes the rendered tga through FFmpeg to generate an animation video file.

2. The method according to claim 1, characterized in that: In the step S2, it specifically includes: Step S21, the master node obtains the idle node number through the Slurm management system, applies for computing resources using salloc, and performs computing command tests on the nodes that have successfully applied, and puts the computing node numbers that have successfully passed the test into the input parameters; one node corresponds to one server in the server cluster deployed by the supercomputing platform; Step S22, the master node reads the TRR trajectory file and inputs preset parameters, which include at least the total number of trajectory frames and the number of computing node resources, and then distributes the task to each computing node that passes the test by task number to execute the Gromacs split trajectory task, and generates a trajectory file that is sorted by the trajectory frame number segmentation number.

3. The method according to claim 2, characterized in that In step S2, the splitting strategy includes: The master node takes every m+1 frames as a group according to the frame sequence, and the remaining part corresponding to the remainder is taken as another group; and starting from the second group, the first frame in any group is the same as the last frame of the previous group, so as to restore the rendering results corresponding to each group to the repeated frames of the original frame sequence in the TRR trajectory file.

4. The method according to claim 3, characterized in that In step S4, the master node implements frame skipping processing on the rendering results corresponding to each repeated frame after each group.

5. A molecular dynamics trajectory parallel rendering system based on a supercomputing platform, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps corresponding to the method for parallel rendering of large-scale trajectories of molecular dynamics based on a supercomputing platform according to any one of claims 1 to 4 are implemented.

6. The system according to claim 5, characterized in that In the supercomputing environment constructed by the supercomputing platform, each node used to collaboratively realize parallel rendering of molecular dynamics trajectories is based on the ARM architecture or the X86 architecture.

Images