Lightweight high-throughput calculation mode and fault tolerance method thereof

Abstract

The invention discloses a lightweight high-throughput calculation mode and a fault tolerance method thereof, and particularly relates to the technical field of computer application, and the method comprises the following steps: 1, providing a runtime high-throughput calculation mode; 2, providing an error discrimination method based on a threshold value; and 3, designing a high-flux calculation fault-tolerant technical framework which can achieve efficient fault tolerance of thousands of flux calculation. A father-child task relation model is designed, and the management overhead of super-large-scale concurrent computing tasks is reduced; According to the method, a runtime high-throughput task model is provided for thousands of throughput concurrent calculation modes, a father-child task relation model is designed in a dynamic task expansion mode, and the management overhead of super-large-scale concurrent calculation tasks is reduced; and through the task packaging technology of the job array, the interaction frequency with the supercomputing scheduling system is reduced, and the overhead of job delivery and job state monitoring is further reduced.

Description

technical field [0001] The invention relates to the technical field of computer applications, in particular to a lightweight high-throughput computing mode and a fault-tolerant method thereof. Background technique [0002] In the field of computational materials science, the traditional material research methods relying on scientific intuition and trial and error have increasingly become the bottleneck of social development and technological progress. In the engineering design simulation process based on numerical simulation, sensitivity analysis, uncertainty quantification and other processes also need to run the same software multiple times with different parameters. These multiple software execution processes in the same mode can be attributed to high-throughput calculations model; [0003] High Throughput Computing (High Throughput Computing) widely exists in new material design, design optimization and other fields, such as material molecular screening, large-scale ran...

AI Technical Summary

Problems solved by technology

[0006] 1. The existing science-based workflow technology expresses the business process in the form of a directed graph, and develops an automatic process execution engine to form a workflow platform is an important auxiliary means to accelerate scientific research. However, in the high-throughput computing mode, it has A directed graph with tens of thousands of nodes will seriously affect the execution efficiency of the process engine. In addition;

[0007] 2. The existing high-throughput computing mode needs to dynamically adjust the computing throughput according to the runtime data as needed

[0008] It is difficult to meet the actual needs by completing the high-throughput design in the design period. Therefore, we propose a lightweight high-throughput computing model and its fault-tolerant method to solve the above problems

Images