Method for quickly collecting large-scale parallel program performance data based on layer tree network

Abstract

The invention discloses a method for quickly collecting large-scale parallel program performance data based on a layer tree network, which comprises the following steps of: S1, determining the numberm of sub-nodes needing to be responsible for sub-collection nodes according to the number limit M of port numbers which can be used at most by a process started by a calculation node; S2, according tothe m value obtained in S1, dividing the total process number N of the performance data needing to be collected into a topological network with the same parallelism degree of each tree layer; s3, analyzing a data transmission efficiency ratio of 1: eta 1, eta 2,..., eta<h-1> of each tree layer, wherein h is the depth of the tree; S4, optimizing the topological network according to the data transmission efficiency ratio of each tree layer in S3; and S5, establishing a layer tree collection network through socket to realize collection of performance data. Aiming at the structural characteristics of the heterogeneous many-core, in combination with the characteristics of the heterogeneous many-core processor, part of the collection processes sink to the computing nodes, and topological optimization is performed on the tree-type collection network in combination with the data transmission efficiency of each tree layer, so that quick collection of large-scale parallel program performance data can be realized.

Description

Technical field [0001] The present invention relates to a rapid collection method based on a large-scale parallel program performance data based on a layer tree network, which belongs to the technical field of computer. Background technique [0002] In order to obtain higher computing power, the parallel scale of the high-performance computing platform is increasing, and the supercomputer system with tens of millions of calculated core has begun to serve research applications, and performs large-scale application running performance on such platforms. Monitor analysis, the rapid processing of a large distributed performance data generated during the period faces challenges. [0003] The existing performance data collection method can be divided into two, one is based on the collection method of the forward process, generates each process or thread performance data file, and the client is responsible for subsequent processing; the other is based on the program Collection method, t...

AI Technical Summary

Problems solved by technology

When there is a lot of data to be collected, the collection nodes often need to be deployed on dedicated servers, and the amount of performance data collected on a single node is small. If the above scheme is adopted, it will cause a certain degree of waste of resources and will Affect transmission efficiency

Images