Construction method and device of function call tree, equipment, medium and product

Abstract

The invention provides a function call tree construction method and device, equipment, a medium and a product. The method comprises the following steps: performing source code instrumentation on a plurality of target functions of the distributed file system to generate performance track data of the plurality of target functions during operation; sampling the performance trajectory data to obtain sampling data, the sampling data including at least one-order function call information of each objective function; function call information of the target functions is completed according to a function call stack, function call data are obtained, and the function call stack is generated based on the sampling data; and constructing a function call tree according to the function call data and a preset function hash table. According to the embodiment of the invention, the constructed function call tree can meet the requirement of performing function-level fine-grained performance analysis on the distributed file system, and the system performance and the resource utilization rate are improved.

Description

Technical Field

[0001] The present application belongs to the field of big data technology, and in particular relates to a method, device, equipment, medium and product for constructing a function call tree. Background Art

[0002] Distributed file system is the main component of distributed storage system, and it is the bridge between big data application and underlying storage. Its performance level directly affects the operation efficiency and service quality of big data application. Therefore, it is of great research value to analyze the performance of distributed file system to improve its performance.

[0003] Related technologies usually perform performance analysis on distributed file systems based on function call relationships, including static analysis methods and traditional function call tree construction algorithms. However, there are limitations in using these methods to construct function call trees in distributed large-scale systems, and it is difficult to meet the requirements of function-level fine-grained performance analysis of distributed file systems. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, device, medium and product for constructing a function call tree to solve the technical problem in the related art that it is difficult to perform function-level fine-grained performance analysis on a distributed file system.

[0005] In a first aspect, an embodiment of the present application provides a method for constructing a function call tree, the method comprising:

[0006] Perform source code instrumentation on multiple target functions of the distributed file system to generate performance trace data of multiple target functions at runtime;

[0007] Sampling the performance trace data to obtain sampled data, wherein the sampled data includes at least one function call information of each target function;

[0008] Complete the function call information of multiple target functions according to the function call stack to obtain function call data, wherein the function call stack is generated based on the sampling data;

[0009] Construct a function call tree based on the function call data and the preset function hash table.

[0010] In a second aspect, an embodiment of the present application provides a device for constructing a function call tree, the device comprising:

[0011] The plug-in module is used to perform source code plug-in on multiple target functions of the distributed file system to generate performance trace data of the multiple target functions at runtime;

[0012] A sampling module, used to sample the performance trace data to obtain sampled data, wherein the sampled data includes at least one function call information of each target function;

[0013] A completion module is used to complete the function call information of multiple target functions according to the function call stack to obtain function call data, and the function call stack is generated based on the sampling data;

[0014] The construction module is used to construct a function call tree according to function call data and a preset function hash table.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising:

[0016] A processor and a memory storing programs or instructions;

[0017] The above method is implemented when the processor executes the program or instruction.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored, and the program or instruction implements the above method when executed by a processor.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the above method.

[0020] An embodiment of the present application provides a method for constructing a function call tree, including: performing source code instrumentation on multiple target functions of a distributed file system to generate performance trace data of the multiple target functions at runtime; sampling the performance trace data to obtain sampled data, wherein the sampled data includes at least one function call information of each target function; completing the function call information of the multiple target functions according to a function call stack to obtain function call data, wherein the function call stack is generated based on the sampled data; and constructing a function call tree according to the function call data and a preset function hash table.

[0021] In an embodiment of the present application, a method for constructing a function call tree first obtains performance trace data of the target function by instrumenting the target function of the distributed file system, and then completes the function call information of the target function based on the sampled performance trace data according to the function call stack, so that the data for constructing the function call tree includes the direct call relationship and indirect call relationship of the target function, thereby providing complete and sufficient data information for the construction of the function call tree, so that the constructed function call tree can meet the requirements of function-level fine-grained performance analysis of the distributed file system, thereby improving system performance and resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 is a flow chart of a method for constructing a function call tree according to an embodiment of the present invention;

[0024] Figure 2 It is a schematic diagram of the system architecture of a performance analysis tool for a distributed file system according to a method for constructing a function call tree in an embodiment of the present invention;

[0025] Figure 3 It is a flowchart of a specific scenario embodiment of a method for constructing a function call tree according to an embodiment of the present invention;

[0026] Figure 4 is a structural block diagram of a device for constructing a function call tree according to an embodiment of the present invention;

[0027] Figure 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0029] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0030] In addition, it should be noted that the acquisition, storage, use, and processing of data in the embodiments of this application are in compliance with the relevant provisions of national laws and regulations. It should be noted that in the embodiments of this application, some existing solutions in the industry such as certain software, components, and models may be mentioned, and they should be considered as exemplary. Their purpose is only to illustrate the feasibility of the implementation of the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0031] With the advent of the big data era, the processing of large amounts of data cannot be separated from the support of big data technology. The core of big data technology is distributed computing, which can use the computing and storage capabilities of multiple nodes to achieve parallel processing of large-scale data. As the basis of distributed computing, the distributed storage system mainly provides data storage services for upper-level applications. The distributed file system is the main component of the distributed storage system. It is responsible for managing and accessing file-level data and is a bridge between big data applications and underlying storage. Its performance level directly affects the operating efficiency and service quality of big data applications. Nowadays, the widely used big data application platforms all use Hadoop Distributed File System (HDFS) as the default distributed file system. Therefore, it is of great research value to analyze the performance and locate the performance bottleneck of distributed file systems represented by HDFS.

[0032] Most methods for performance analysis and performance bottleneck analysis are based on function call relationships, so constructing a function call tree is of great significance for performance analysis of distributed file systems. Through the review and summary of related technologies, the existing performance collection and function call tree construction methods are roughly as follows: (1) Static analysis method (2) Traditional function call tree construction algorithm.

[0033] Although the above methods can provide certain solutions for the performance analysis of distributed file systems, the static analysis method can only construct a static function call tree, so it is difficult to correspond to the trace data obtained by instrumentation. If you want to analyze the mutual influence relationship between functions, you must know which call of the parent function affects which call of the child function. The traditional function call tree construction algorithm is designed based on the function call relationship of a single machine scenario, not a distributed function call relationship. Therefore, the traditional function call tree construction algorithm is also difficult to meet the needs of performance analysis.

[0034] In order to solve the problems of the prior art, an embodiment of the present application provides a method for constructing a function call tree, which can be applied in an application scenario of performing performance analysis on a distributed file system based on a function call tree. Figure 1 FIG. 1 is a flow chart of a method for constructing a function call tree according to an embodiment of the present invention. Figure 1As shown, the method for constructing the function call tree may include the following steps:

[0035] S110, source code instrumentation is performed on multiple target functions of the distributed file system to generate performance trace data of the multiple target functions when they are running.

[0036] In S110, the basis for function-level fine-grained performance analysis of the distributed file system HDFS is to obtain effective performance trace data. In order to obtain effective performance trace data, the target function can be instrumented, so that the performance trace data of multiple target functions at runtime can be obtained. Among them, the target function can be instrumented using a probe to insert the code for performance trace data collection into the target function in the HDFS source code. When instrumenting, the function interface can be encapsulated, and the encapsulated function interface can be TraceScope newPathTraceScope(String description, String path). The performance trace data can include the timestamps of the entry and exit of the target function, the data volume characteristics of the target function read and write, etc.

[0037] In this embodiment, as shown in Table 1, the target function may be any function in HDFS required when performing performance analysis on HDFS.

[0038] Table 1 shows the objective function of the instrumentation.

[0039]

[0040]

[0041]

[0042] S120, sampling the performance trace data to obtain sampled data, wherein the sampled data includes at least one function call information of each target function.

[0043] In S120, when the performance trace data obtained by the instrumentation is too much, if all the performance trace data is output, the system overhead will increase. At the same time, if the probe code used by the instrumentation is inserted into some code segments with short execution time but frequent calls, the instrumentation code may have a serious impact on the performance of HDFS. Therefore, the present application can reduce the system overhead by sampling the performance trace data.

[0044] In this embodiment, the performance trace data may be sampled using the HTrace distributed tracing framework and the token bucket algorithm. In one example, when the token bucket algorithm is used for sampling, the sampling parameters may be set to a token bucket size of 1000 and a sampling time interval of 180 ms.

[0045] In this embodiment, since each target function is called a different number of times during the execution of the distributed file system program, during the sampling process, the sampling probability of the target function with a smaller number of calls can be increased, and the sampling probability of the target function with a larger number of calls can be reduced. At the same time, in order to ensure the integrity of the data when performing subsequent function-level fine-grained performance analysis on HDFS, the function call information of the target function can be sampled at least once during sampling.

[0046] S130, completing function call information of multiple target functions according to the function call stack to obtain function call data, wherein the function call stack is generated based on the sampling data.

[0047] In this embodiment, the function call stack may be a static data structure in a class, and the data structure of the function call stack is not limited here. Since the function call information of multiple target functions does not include all functions in the distributed file system and is obtained through sampling, the function call information of multiple target functions may be missing, and the function call information of multiple target functions needs to be supplemented according to the function call stack. For example, the missing function call information may be the missing parent function information that calls the target function.

[0048] S140, constructing a function call tree according to the function call data and a preset function hash table.

[0049] In this embodiment, since the function call data is already the completed function call information, in order to perform function-level fine-grained performance analysis on HDFS, a function call tree constructed according to the function call data and the preset function hash table can be used. The preset function hash table can be composed of several key-value pairs, where the key is a unique identifier and the value is data associated with the key. The structure of the preset function hash table is not limited here.

[0050] In an embodiment of the present application, a method for constructing a function call tree first obtains performance trace data of the target function by instrumenting the target function of the distributed file system, and then completes the function call information of the target function based on the sampled performance trace data according to the function call stack, so that the data for constructing the function call tree includes the direct call relationship and indirect call relationship of the target function, thereby providing complete and sufficient data information for the construction of the function call tree, so that the constructed function call tree can meet the requirements of function-level fine-grained performance analysis of the distributed file system, thereby improving system performance and resource utilization.

[0051] In some embodiments, the preset function hash table includes a correspondence between the target function and the nodes of the function call tree, and constructing the function call tree according to the function call data and the preset function hash table includes:

[0052] Constructing a function call tree framework, wherein the function call tree framework includes a plurality of nodes, each node including: a connection relationship with a parent node, a connection relationship with a child node, and function call information of a target function corresponding to the node;

[0053] According to the corresponding relationship between function call data and nodes, a function call tree is constructed according to the function call tree framework.

[0054] In this embodiment, if the HTrace framework is used to output function call data, the obtained function call data may be a trace file of multiple modules, and the multiple modules may include a client node, a name node, and a data node. When constructing a function call tree according to the function call data, the trace files of multiple modules may be merged into one trace file.

[0055] In this embodiment, in order to construct a function call tree, a function call tree framework may be constructed first. The function call tree framework may include multiple nodes, and each node may include: a connection relationship with a parent node, a connection relationship with a child node, and function call information of a target function corresponding to the node. Among them, the connection relationship with the parent node may be a pointer to the parent node, the parent node may be a node corresponding to a parent function, and the parent function may be a function that calls the target function; the connection relationship with the child node may be a pointer to the child node, and the child node may be a node corresponding to a function called by the target function; the function call information may include a function name, a function ID (Identifier) ​​value, a parent function ID value, a function execution time, a module that executes the function, and possible I / O data size.

[0056] The preset function hash table includes the correspondence between the target function and the nodes of the function call tree. Therefore, the preset function hash table can be a hash table with the function ID value of the target function as the key and the pointer of the node of the function call tree as the value.

[0057] It is understandable that since the function call data includes the complete function call information of the target function, and the preset function hash table includes the node correspondence between the target function and the function call tree, the function call tree can be constructed within the function call tree framework based on this information.

[0058] In some embodiments, according to the function call data and the node correspondence, a function call tree is constructed according to a function call tree framework, including:

[0059] Traversing the function call data, creating a first node of a function call tree for each target function in the function call data according to the function call tree framework, and obtaining a call tree node set;

[0060] For each first node in the call tree node set, determine whether the first parent node of the first node is in a preset function hash table according to the node correspondence relationship;

[0061] When the first parent node of the first node is in the preset function hash table, establishing a connection between the first node and the first parent node;

[0062] Determine that a tree structure formed by connecting the first node and the first parent node is a function call tree.

[0063] In this embodiment, in order to obtain the function call tree, a first node may be created for each target function in the function call data according to the function call tree framework, and the first node may be initialized. Then, the preset function hash table may be searched according to the parent function of the target function, and if the parent node of the parent function already exists in the preset function hash table, a connection relationship is directly established between the first node and the parent node.

[0064] In this way, the function call tree can be obtained by first constructing the nodes of the function call tree and then establishing the connection relationship between the nodes according to the preset function hash table.

[0065] In some embodiments, after determining, for each node in the call tree node set, according to the node correspondence, whether the first parent node of the node is in the preset function hash table, the method further includes:

[0066] When the first parent node of the first node is not in the preset function hash table, adding the first node to the root node linked list;

[0067] Add each node in the call tree node set to the preset function hash table, and update the node correspondence in the preset function hash table;

[0068] Construct a function call tree based on the root node linked list and the updated node correspondence.

[0069] In this embodiment, the root node linked list can be a blank linked list, which is used to store the first node whose parent node is not in the preset function hash table; by adding each node in the call tree node set to the preset function hash table and updating the node correspondence in the preset function hash table, it can be ensured that the node correspondence of all nodes is recorded in the updated preset function hash table.

[0070] In this way, through the updated node correspondence relationship, the corresponding parent node can be found for the first node whose parent node is not in the preset function hash table to construct a function call tree.

[0071] In some embodiments, constructing a function call tree according to the root node linked list and the updated node correspondence relationship includes:

[0072] Traverse the root node linked list, and search for the second parent node of the second node according to the updated node correspondence relationship, where the second node is any node in the root node linked list;

[0073] If the second node has a second parent node, establish a connection between the second node and the second parent node, and delete the second node from the root node linked list;

[0074] When the root node linked list is deleted until there is only one second node, and the second node does not have a second parent node, determine that the second node is the root node;

[0075] A function call tree is obtained according to the root node and the tree structure formed by the connection between the second node and the second parent node.

[0076] In this embodiment, the second node is determined to be the root node, and the root node of the function call tree can be constructed according to the pointer of the second node. According to the root node and the tree structure formed by the connection between the second node and the second parent node, the function call tree is obtained, and the root node can be used as a starting point to connect the root node with the already formed tree structure to obtain the function call tree.

[0077] In this way, the root node of the function call tree is determined in the root node linked list through the updated node correspondence, and the complete function call tree is obtained based on the connection between the root node and the already formed tree structure.

[0078] In some embodiments, after constructing the function call tree according to the function call data and the node correspondence according to the function call tree framework, the method further includes:

[0079] Traversing the function call tree, determining successively whether each third node has multiple target child nodes, wherein the third node is any node in the function call tree, and the target child nodes are used to represent that the target functions corresponding to the child nodes of the third node are the same;

[0080] When there are multiple target child nodes of the third node, the multiple target child nodes of the third node are merged into one child node.

[0081] In this embodiment, since the performance trace data obtained by instrumentation may be very large, the function call tree constructed based on the performance trace data will also be large and may contain a lot of repeated information and the same substructure. In order to compress the function call tree generated in series, the function call tree generated in series can be compressed by determining whether there are multiple target child nodes for each third node in the function call tree.

[0082] Among them, compressing the function call tree generated in series can specifically include: sorting the child nodes of the third node currently traversed according to the function names corresponding to the child nodes, traversing the child nodes of the third node and comparing two adjacent child nodes, if the function names of two adjacent child nodes are the same, merging the two child nodes, and merging the function call information and the function call tree structure information into one child node, and clearing the other child node, and continuing to repeat the execution until all third nodes are traversed.

[0083] In this way, multiple target subnodes are merged into one subnode through the isomorphic subtree determination algorithm, which can effectively reduce the function call tree constructed according to the performance trajectory data, making it easier to perform performance analysis and visualization of the distributed file system based on the function call tree.

[0084] In some embodiments, constructing a function call tree according to the function call data and a preset function hash table includes:

[0085] Process the function call data according to the MapReduce framework to obtain multiple data pairs including the corresponding relationship between the function name and the function call information, wherein each data pair corresponds to a target function;

[0086] Construct a fourth node of the function call tree for each data pair;

[0087] According to the data pair, search the third parent node corresponding to the fourth node in the preset function hash table;

[0088] When there is a third parent node corresponding to the fourth node in the preset function hash table, the fourth node is connected to the third parent node using a set data structure;

[0089] It is determined that the tree structure formed by connecting the fourth node and the third parent node is a function call tree.

[0090] In this embodiment, in a large-scale distributed system, the performance trajectory data generated may far exceed the memory of HDFS, so a parallel method can also be used to construct a function call tree to avoid insufficient memory of HDFS. The parallel method can be used to construct a distributed function call tree based on the MapReduce computing framework.

[0091] According to the MapReduce computing framework, the basis for building a distributed function call tree is to build a preset function hash table. The preset function hash table can be a distributed hash table, which can be stored in various modules of HDFS in a dispersed manner. However, in specific use, it can be considered that each module operates the same complete hash table. In an example, the preset function hash table can be two distributed hash tables, namely the nameTable distributed hash table and the tree distributed hash table; the nameTable distributed hash table can use the function ID value in the function call information as the key and the function name as the value; because in the MapReduce computing framework, a large amount of data needs to be processed and transmitted efficiently. Therefore, the nameTable distributed hash table performs a simple mapping, which is used to quickly find the corresponding function name through the function ID value in the subsequent process, which can simplify and speed up the search operation; the tree distributed hash table can use the function name as the key and the function call tree node corresponding to the function name as the value. The tree stores a more complex data structure, which contains the node information of the entire function call tree.

[0092] In the Map phase of MapReduce, when processing each row of records, it is necessary to quickly record the function call ID and function name. This process is frequent and the amount of data is large, so using a dedicated nameTable can improve the efficiency of search and storage.

[0093] In the Reduce phase, all call records need to be merged and processed according to the function name, and the structure of the call tree node needs to be updated. Using tree hash tables to specifically handle these more complex structures helps improve data organization and search efficiency.

[0094] Since the MapReduce computing framework mainly includes the Map process and the Reduce process, the Map process mainly processes the function call data according to the MapReduce framework, writes the function ID value and the corresponding function name in the function call information into the nameTable distributed hash table, and obtains multiple data pairs including the corresponding relationship between the function name and the function call information. The Reduce process can mainly include building a node, finding the parent node of the node, establishing the connection between the node and the parent node, and building a function call tree according to the connection relationship.

[0095] The creation node may be a fourth node for constructing a function call tree for each data pair. In the process of creating the node, the information in the data pair may be analyzed and counted to obtain statistical indicators including the number of items, sum, average value, three quartiles, etc., and these statistical indicators may be recorded in the fourth node.

[0096] It is understandable that the data pair includes the parent function ID of the target function corresponding to the fourth node. Therefore, the parent function ID can be used as a key to search the nameTable table to obtain the function name of the parent function. However, although the data pair sometimes corresponds to the same target function, the parent function corresponding to the target function may not be the same. There may be multiple different functions that directly or indirectly call the same target function. Therefore, the function name of the parent function can be used as a key to search the tree table to obtain the function call tree node corresponding to the parent function, that is, the third parent node.

[0097] In the case where there is a third parent node corresponding to the fourth node in the preset function hash table, in order to prevent the third parent node from being connected to redundant child nodes, a set data structure can be used to establish a connection between the fourth node and the third parent node, so that the same fourth node is only inserted once, and there will be no redundant nodes in the final function call tree. For example, for the third parent node, a child node set is set. Before adding the fourth node to the child node list of the third parent node, check whether the node already exists in the child node set. If it does not exist, add it to the set, and if it exists, do not add it to the set.

[0098] In some embodiments, after searching the preset function hash table for the third parent node corresponding to the fourth node according to the data pair, the method further includes:

[0099] When the third parent node corresponding to the fourth node does not exist in the preset function hash table, the fourth node is determined to be the root node of the function call tree.

[0100] In this embodiment, when the third parent node corresponding to the fourth node does not exist in the preset function hash table, the fourth node can be confirmed as the root node, and then the function name of the function corresponding to the fourth node is output, and then the tree table is searched according to the function name to obtain the root node data structure of the function call tree and obtain the root node of the function call tree.

[0101] In some embodiments, the function call information of multiple target functions is completed according to the function call stack to obtain the function call data, including:

[0102] According to the order of multiple target functions in the sampled data, the multiple target functions are pushed into the function call stack in sequence;

[0103] Determine whether the function call information of each target function in the sampled data includes parent function information, where the parent function information is used to characterize the call relationship between multiple target functions;

[0104] In the case that the function call information of the target function does not include the parent function information, the top function of the function call stack is determined as the parent function of the target function;

[0105] The function information of the top function of the stack is added to the function call information of the target function to complete the function call information of the target function and obtain the function call data.

[0106] In this embodiment, since the function call information of the current multiple target functions is only sampling data, the function call information may be missing. In this regard, the function call information of the target function can be completed according to the function call stack to obtain the function call data to generate a complete function call tree.

[0107] For example, if the HTrace framework is used to output function call data, a static member can be added to the Tracer class in the native HTrace framework as a function call stack.

[0108] After the function call stack is constructed, multiple target functions are pushed into the function call stack in sequence according to the order of multiple target functions in the sampled data; wherein, the order of multiple target functions can be obtained by setting the Tracer.newScope() function, and the return value of Tracer.newScope() can be a Scope object, and the Scope object can directly obtain the function ID of the target function. Therefore, Tracer.newScope() is encapsulated, and based on the target function calling Tracer.newScope(), the target function corresponding to the return value is pushed into the function call stack, so that a function call stack containing complete function call information is obtained.

[0109] Furthermore, in order to complete the function call information of the target function, when the function call information of the target function does not include the parent function information, the top function of the function call stack can be determined as the parent function of the target function; the function information of the top function is added to the function call information of the target function to complete the function call information of the target function and obtain function call data. The principle is that in recursion, ancestor functions and descendant functions are defined, and the parent function of a function and the ancestor function of the parent function are collectively referred to as the ancestor function of this function; due to sampling, the parent function of the next target function pushed into the stack may be missing between the top function of the function call stack and the next target function pushed into the stack. Therefore, the current top function is the ancestor function closest to the next target function pushed into the stack, so the top function at this time is determined as the parent function of the next target function pushed into the stack.

[0110] In this way, by determining the top element of the function call stack as the parent function of the target function, the function call information of the target function is completed, and the technical problem of missing function call information in the function call tree is solved.

[0111] Figure 2is a schematic diagram of the system architecture of a distributed file system performance analysis tool according to a method for constructing a function call tree according to an embodiment of the present invention. Figure 2 As shown, the system may include a system layer 204, a collection layer 203, an analysis layer 202, and a presentation layer 201. Among them, the system layer 204 may direct the load generator to the plug-in function module of the HDFS system, wherein the plug-in function module may include a name node namenode and a data node datanode after plugging, which respectively correspond to the name module and the data module of the present application, and a trace collection module. In the collection layer 203, the user can configure the reading module, and the trace collection module can collect function call data and merge the function call data. The function call tree construction module of the analysis layer 202 can construct a function call tree according to the configuration reading module and the merged trace data, and then the function call tree compression module can compress the constructed function call tree. The visualization module of the presentation layer 201 can visualize the compressed function call tree to obtain a visualization result.

[0112] The load generator can not only call existing applications, but also generate the required workload as needed. The instrumentation function module can also provide the instrumented HDFS source code and the extended HTrace framework. The trace collection module can be mainly divided into two parts: one is the part extended in the HTrace framework, which is used to collect instrumentation information and output it to the sampling data; the other is the part that collects and merges function call data from each node.

[0113] In order to facilitate understanding of the method for constructing a function call tree provided in the above embodiment, the method for constructing a function call tree is described below using a specific scenario embodiment. Figure 3 is a flowchart of a specific scenario embodiment of a method for constructing a function call tree according to an embodiment of the present invention, such as Figure 3 As shown, this scenario embodiment may include:

[0114] S301, HDFS function instrumentation, using HTrace and extending it;

[0115] Among them, the HDFS function instrumentation can correspond to the source code instrumentation of multiple target functions of HDFS in this application. During the instrumentation process, the HTrace framework can be used.

[0116] S302, sampling the performance data obtained by the instrumentation;

[0117] The performance data may correspond to the performance trajectory data of the present application, and the performance data may be obtained by sampling the performance trajectory data.

[0118] S303, maintaining the function call stack during program execution;

[0119] Among them, the maintenance of the function call stack during program execution can correspond to the present application of completing the function call information of multiple target functions according to the function call stack to obtain function call data.

[0120] S304, determine whether to execute the serial construction method, if yes, proceed to S305, otherwise proceed to S307;

[0121] Among them, the serial construction method can correspond to the serial construction function call tree of this application.

[0122] S305, executing a serial construction method for constructing an HDFS function call tree;

[0123] S306, traversing the call tree and compressing the performance trace data using the tree structure;

[0124] Among them, compressing the performance trace data may correspond to compressing the function call tree in the present application.

[0125] S307, executing the Map process of constructing the HDFS function call tree parallel construction method;

[0126] S308, executing the Reduce process of the parallel construction method for constructing the HDFS function call tree.

[0127] Figure 4 A schematic diagram of the structure of a function call tree construction device provided in another embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0128] Reference Figure 4 , the function call tree construction device 400 may include:

[0129] The plugging module 401 is used to perform source code plugging on multiple target functions of the distributed file system to generate performance trace data of the multiple target functions when they are running;

[0130] A sampling module 402 is used to sample the performance trace data to obtain sampled data, wherein the sampled data includes at least one function call information of each target function;

[0131] A completion module 403 is used to complete the function call information of multiple target functions according to the function call stack to obtain function call data, and the function call stack is generated based on the sampled data;

[0132] The construction module 404 is used to construct a function call tree according to the function call data and the preset function hash table.

[0133] In some embodiments, the function call tree construction device 400 may further include the following modules:

[0134] The construction module 404 is further used to construct a function call tree framework, wherein the function call tree framework includes a plurality of nodes, each of which includes: a connection relationship with a parent node, a connection relationship with a child node, and function call information of a target function corresponding to the node;

[0135] The construction module 404 is further used to construct a function call tree according to the function call data and the node correspondence relationship and in accordance with the function call tree framework.

[0136] In some embodiments, the function call tree construction device 400 may further include the following modules:

[0137] Creating a module, used for traversing the function call data, creating a first node of a function call tree for each target function in the function call data according to the function call tree framework, and obtaining a call tree node set;

[0138] A determination module, configured to determine, for each first node in the call tree node set, whether a first parent node of the first node is in a preset function hash table according to a node correspondence relationship;

[0139] A connection module, used for establishing a connection between the first node and the first parent node when the first parent node of the first node is in a preset function hash table;

[0140] The adjustment module is used to determine that the tree structure formed by connecting the first node and the first parent node is a function call tree.

[0141] In some embodiments, the function call tree construction device 400 may further include the following modules:

[0142] An adding module, used for adding the first node to the root node linked list when the first parent node of the first node is not in the preset function hash table;

[0143] An updating module, used to add each node in the call tree node set to a preset function hash table, and update the node correspondence in the preset function hash table;

[0144] The construction module 404 is further used to construct a function call tree according to the root node linked list and the updated node correspondence relationship.

[0145] In some embodiments, the function call tree construction device 400 may further include the following modules:

[0146] A search module, used for traversing the root node linked list, and searching for a second parent node of a second node according to the updated node correspondence relationship, where the second node is any node in the root node linked list;

[0147] The connection module is further used to establish a connection between the second node and the second parent node when the second node has a second parent node, and delete the second node from the root node linked list;

[0148] A determination module, used for determining that the second node is a root node when the root node linked list is deleted to have only one second node and the second node does not have a second parent node;

[0149] The generating module is used to obtain a function call tree according to a root node and a tree structure formed by a connection between a second node and a second parent node.

[0150] In some embodiments, the function call tree construction device 400 may further include the following modules:

[0151] A judgment module is used for, after constructing a function call tree according to the function call data and the node correspondence and the function call tree framework, traversing the function call tree and successively judging whether there are multiple target sub-nodes for each third node, wherein the third node is any node in the function call tree, and the target sub-node is used to represent that the target function corresponding to the sub-node of the third node is the same;

[0152] The merging module is used to merge the multiple target sub-nodes of the third node into one sub-node when there are multiple target sub-nodes of the third node.

[0153] In some embodiments, the function call tree construction device 400 may further include the following modules:

[0154] A processing module is used to process the function call data according to the MapReduce framework to obtain a plurality of data pairs including a corresponding relationship between a function name and function call information, wherein each data pair corresponds to a target function;

[0155] The construction module 404 is further used to construct a fourth node of the function call tree for each data pair;

[0156] The search module is further used to search the third parent node corresponding to the fourth node in the preset function hash table according to the data pair;

[0157] The connection module is further used to establish a connection between the fourth node and the third parent node using a set data structure when there is a third parent node corresponding to the fourth node in the preset function hash table;

[0158] The determination module is further used to determine that the tree structure formed by connecting the fourth node and the third parent node is a function call tree.

[0159] In some embodiments, the function call tree construction device 400 may further include the following modules:

[0160] The determination module is also used to determine that the fourth node is the root node of the function call tree after searching the third parent node corresponding to the fourth node in the preset function hash table according to the data pair, if the third parent node corresponding to the fourth node does not exist in the preset function hash table.

[0161] In some embodiments, the function call tree construction device 400 may further include the following modules:

[0162] A stack push module is used to push multiple target functions into a function call stack in sequence according to the order of multiple target functions in the sampled data;

[0163] The judgment module is also used to judge whether the function call information of each target function in the sampled data includes the parent function information, and the parent function information is used to characterize the calling relationship between multiple target functions;

[0164] The determination module is further used to determine the top function of the function call stack as the parent function of the target function when the function call information of the target function does not include the parent function information;

[0165] The completion module 403 is further used to complete the function information of the top stack function into the function call information of the target function to complete the function call information of the target function and obtain function call data.

[0166] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application, and are devices corresponding to the method for constructing the above-mentioned function call tree. All implementation methods in the above-mentioned method embodiment are applicable to the embodiment of the device. Its specific functions and technical effects brought about can be found in the method embodiment part, which will not be repeated here.

[0167] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0168] Figure 5 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Figure 5 As shown:

[0169] The device may include a processor 501 and a memory 502 storing programs or instructions.

[0170] When the processor 501 executes the program, the steps in any of the above method embodiments are implemented.

[0171] Exemplarily, the program may be divided into one or more modules / units, one or more modules / units are stored in the memory 502, and executed by the processor 501 to complete the present application. One or more modules / units may be a series of program instruction segments capable of completing a specific function, and the instruction segments are used to describe the execution process of the program in the device.

[0172] Specifically, the processor 501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0173] The memory 502 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 502 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 502 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 502 is a non-volatile solid-state memory.

[0174] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) machine-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0175] The processor 501 implements any one of the methods in the above embodiments by reading and executing the program or instruction stored in the memory 502 .

[0176] In one example, the electronic device may further include a communication interface 503 and a bus 504. The processor 501, the memory 502, and the communication interface 503 are connected via the bus 504 and communicate with each other.

[0177] The communication interface 503 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0178] Bus 504 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 504 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0179] In addition, in combination with the method in the above embodiment, the embodiment of the present application may provide a machine-readable storage medium for implementation. The machine-readable storage medium stores a program or instruction; when the program or instruction is executed by a processor, any one of the methods in the above embodiment is implemented. The machine-readable storage medium can be read by a machine such as a computer.

[0180] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0181] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0182] An embodiment of the present application provides a computer program product, which is stored in a machine-readable storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0183] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0184] The functional modules shown in the above block diagram can be implemented as hardware, software, firmware or their combination. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), suitable firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer grids such as the Internet, intranets, etc.

[0185] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0186] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by a computer program or instruction. These programs or instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0187] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A method for constructing a function call tree, characterized in that: The method comprises: Performing source code instrumentation on multiple target functions of the distributed file system to generate performance trace data of the multiple target functions when they are running; Sampling the performance trajectory data to obtain sampled data, wherein the sampled data includes at least one function call information of each target function; Completing the function call information of the plurality of target functions according to a function call stack to obtain function call data, wherein the function call stack is generated based on the sampling data; A function call tree is constructed according to the function call data and a preset function hash table.

2. The method according to claim 1, characterized in that The preset function hash table includes the correspondence between the target function and the nodes of the function call tree, and constructing the function call tree according to the function call data and the preset function hash table includes: Constructing a function call tree framework, wherein the function call tree framework includes a plurality of nodes, each of the nodes including: a connection relationship with a parent node, a connection relationship with a child node, and function call information of a target function corresponding to the node; The function call tree is constructed according to the function call data and the node correspondence and the function call tree framework.

3. The method according to claim 2, characterized in that According to the function call data and the node correspondence, the function call tree is constructed according to the function call tree framework, including: Traversing the function call data, creating a first node of the function call tree for each target function in the function call data according to the function call tree framework, and obtaining a call tree node set; For each first node in the call tree node set, determining, according to the node correspondence, whether a first parent node of the first node is in the preset function hash table; In a case where the first parent node of the first node is in the preset function hash table, establishing a connection between the first node and the first parent node; It is determined that a tree structure formed by connecting the first node and the first parent node is the function call tree.

4. The method according to claim 3, characterized in that: After determining, for each node in the call tree node set, according to the node correspondence relationship, whether the first parent node of the node is in the preset function hash table, the method further includes: If the first parent node of the first node is not in the preset function hash table, adding the first node to the root node linked list; Add each node in the call tree node set to the preset function hash table, and update the node correspondence in the preset function hash table; The function call tree is constructed according to the root node linked list and the updated node correspondence relationship.

5. The method according to claim 4, characterized in that The step of constructing the function call tree according to the root node linked list and the updated node correspondence relationship includes: Traversing the root node linked list, searching for a second parent node of a second node according to the updated node correspondence relationship, where the second node is any node in the root node linked list; If the second node has the second parent node, establish a connection between the second node and the second parent node, and delete the second node from the root node linked list; When the root node linked list is deleted until there is only one second node, and the second node does not have the second parent node, determining the second node as the root node; The function call tree is obtained according to the root node and the tree structure formed by the connection between the second node and the second parent node.

6. The method according to claim 2, characterized in that After constructing the function call tree according to the function call data and the node correspondence according to the function call tree framework, the method further includes: Traversing the function call tree, determining successively whether each third node has multiple target child nodes, wherein the third node is any node in the function call tree, and the target child nodes are used to represent that the target functions corresponding to the child nodes of the third node are the same; In the case that the third node has the multiple target child nodes, the multiple target child nodes of the third node are merged into one child node.

7. The method according to claim 1, characterized in that The step of constructing a function call tree according to the function call data and a preset function hash table includes: Processing the function call data according to a map-reduce framework to obtain a plurality of data pairs including a corresponding relationship between a function name and the function call information, wherein each data pair corresponds to a target function; Constructing a fourth node of the function call tree for each data pair; According to the data pair, searching the preset function hash table for a third parent node corresponding to the fourth node; When there is a third parent node corresponding to the fourth node in the preset function hash table, establishing a connection between the fourth node and the third parent node using a set data structure; It is determined that the tree structure formed by connecting the fourth node and the third parent node is the function call tree.

8. The method according to claim 7, characterized in that After searching the third parent node corresponding to the fourth node in the preset function hash table according to the data pair, the method further includes: When the third parent node corresponding to the fourth node does not exist in the preset function hash table, the fourth node is determined to be the root node of the function call tree.

9. The method according to claim 1, characterized in that: The step of completing the function call information of the plurality of target functions according to the function call stack to obtain the function call data includes: According to the order of the multiple target functions in the sampled data, the multiple target functions are sequentially pushed into the function call stack; Determine whether the function call information of each target function in the sampled data includes parent function information, where the parent function information is used to characterize the call relationship between the multiple target functions; In a case where the function call information of the target function does not include the parent function information, determining the top function of the function call stack as the parent function of the target function; The function information of the top function of the stack is added to the function call information of the target function to complete the function call information of the target function and obtain function call data.

10. A device for constructing a function call tree, characterized in that: The device comprises: The plugging module is used to perform source code plugging on multiple target functions of the distributed file system to generate performance trace data of the multiple target functions when running; A sampling module, used for sampling the performance trajectory data to obtain sampled data, wherein the sampled data includes at least one function call information of each target function; A completion module, used to complete the function call information of the multiple target functions according to the function call stack to obtain function call data, wherein the function call stack is generated based on the sampling data; A construction module is used to construct a function call tree according to the function call data and a preset function hash table.

11. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the method according to any one of claims 1 to 9 is implemented.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the method according to any one of claims 1 to 9.

13. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the method according to any one of claims 1 to 9.

Images