Method and device for determining response duration of operation command execution of distributed file system

Abstract

The invention provides a response duration determination method and device for executing an operation command by a distributed file system. The method comprises the following steps: acquiring log data of a metadata node in the distributed file system within a preset duration; inputting the log data into a log data analysis model to obtain each operation command in the log data, a corresponding timestamp and an operation file name; dividing the operation command and the timestamp corresponding to the same operation file name into the same group; sorting the operation commands in each group according to the timestamps, and querying the timestamps corresponding to the operation commands for acquiring the operation file information during execution and the timestamps corresponding to the next operation commands for acquiring the operation file information during execution; and according to the two timestamps, determining the response duration of the distributed file system for executing the operation command on each operation file.

Description

Technical Field

[0001] The present invention relates to the technical field of big data, and particularly to a method and device for determining the response duration of an operation command executed by a distributed file system. Background Art

[0002] This section aims to provide background or context for the embodiments of the present invention described in the claims. The description herein is not admitted to be prior art merely by virtue of being included in this section.

[0003] In a distributed file system (HDFS), all operation commands need to send requests from the client to the server. After the server executes the operation command, the processing result is returned to the client. During this process, if the response duration of the operation command is too long, it will directly affect the performance and user experience of the distributed file system. It can be seen that the response duration of executing the operation command has a crucial impact on the performance of the distributed file system. However, in the prior art, there is no method for calculating the response duration of the operation command executed by the distributed file system. Summary of the Invention

[0004] In an embodiment of the present invention, a method for determining the response duration of an operation command executed by a distributed file system is proposed, which is used to calculate the response duration of the operation command executed by the distributed file system and provide data support for analyzing the impact of the operation file and the operation command on the response duration, including:

[0005] Obtain the log data of the metadata node in the distributed file system within a preset duration;

[0006] Input the log data into a log data analysis model to obtain each operation command in the log data, and the corresponding timestamp and operation file name; wherein, the log data analysis model is trained on a machine learning model according to the historical log data, and the corresponding historical operation command, historical timestamp, and historical operation file name.

[0007] Divide the operation commands and timestamps corresponding to the same operation file name into the same group;

[0008] Sort the operation commands in each group according to the timestamp, and query the timestamp corresponding to the operation command for obtaining operation file information during execution, and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution;

[0009] Determine the response duration of the distributed file system for executing the operation command on each operation file according to the timestamp corresponding to the operation command for obtaining operation file information during execution in each group, and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution.

[0010] In an embodiment of the present invention, a device for determining the response duration of a distributed file system to execute an operation command is proposed, which is used to calculate the response duration of the distributed file system to execute the operation command, and provide data support for analyzing the influence of the operation file and the operation command on the response duration, including:

[0011] A log acquisition module, configured to acquire log data of a metadata node in the distributed file system within a preset duration;

[0012] A log analysis module, configured to input the log data into a log data analysis model to obtain each operation command, corresponding timestamp, and operation file name in the log data; wherein, the log data analysis model is obtained by training a machine learning model based on historical log data, corresponding historical operation commands, historical timestamps, and historical operation file names;

[0013] A data grouping module, configured to divide the operation commands and timestamps corresponding to the same operation file name into the same group;

[0014] A timestamp query module, configured to sort the operation commands in each group according to the timestamps, and query the timestamp corresponding to the operation command for obtaining operation file information during execution, and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution;

[0015] A response duration determination module, configured to determine the response duration of the distributed file system to execute the operation command for each operation file according to the timestamp corresponding to the operation command for obtaining operation file information during execution in each group, and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution.

[0016] In an embodiment of the present invention, a computer device is proposed, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the method for determining the response duration of the distributed file system to execute the operation command is implemented.

[0017] In an embodiment of the present invention, a computer-readable storage medium is proposed, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for determining the response duration of the distributed file system to execute the operation command is implemented.

[0018] In an embodiment of the present invention, a computer program product is proposed, where the computer program product includes a computer program, and when the computer program is executed by a processor, the method for determining the response duration of the distributed file system to execute the operation command is implemented.

[0019] The method and device for determining the response duration of an operation command executed by a distributed file system proposed in the embodiments of the present invention can solve the problem that there is no calculation of the response duration of an operation command executed by a distributed file system in the prior art; in the embodiments of the present invention, by obtaining the log data of the metadata node in the distributed file system within a preset duration; inputting the log data into a log data analysis model to obtain each operation command in the log data, and the corresponding timestamp and operation file name; wherein, the log data analysis model is trained for a machine learning model according to historical log data, and the corresponding historical operation command, historical timestamp, and historical operation file name; dividing the operation commands and timestamps corresponding to the same operation file name into the same group; sorting the operation commands in each group according to the timestamp, querying the timestamp corresponding to the operation command for obtaining operation file information during execution, and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution; according to the timestamp corresponding to the operation command for obtaining operation file information during execution in each group, and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution, determining the response duration of the distributed file system for executing the operation command on each operation file. The embodiments of the present invention can calculate the response duration of the distributed file system for executing the operation command, and provide data support for analyzing the influence of operation files and operation commands on the response duration. Brief Description of the Drawings

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other accompanying drawings can be obtained without creative efforts based on these drawings.

[0021] Figure 1 It is a flowchart of the method for determining the response duration of an operation command executed by a distributed file system in the embodiments of the present invention;

[0022] Figure 2 It is a specific example diagram of the method for determining the response duration of an operation command executed by a distributed file system in the embodiments of the present invention;

[0023] Figure 3 It is a specific example diagram of the method for determining the response duration of an operation command executed by a distributed file system in the embodiments of the present invention;

[0024] Figure 4 It is a schematic diagram of the device for determining the response duration of an operation command executed by a distributed file system in the embodiments of the present invention;

[0025] Figure 5 It is a schematic diagram of a computer device in the embodiments of the present invention. Detailed Embodiments

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer and more understandable, the following further elaborates on the embodiments of the present invention in conjunction with the accompanying drawings. Herein, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but do not limit the present invention.

[0027] The term "and / or" in this document merely describes an association relationship, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, the term "at least one" in this document means any one of multiple or any combination of at least two of multiple. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set composed of A, B, and C.

[0028] In the description of this specification, the terms "comprising", "including", "having", "containing", etc. are all open-ended terms, meaning including but not limited to. The description with reference to terms such as "an embodiment", "a specific embodiment", "some embodiments", "for example", etc. means that the specific features, structures, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. The step sequences involved in each embodiment are used to schematically illustrate the implementation of this application, and the step sequences are not limited and can be adjusted appropriately as needed.

[0029] The term explanations of the present invention are as follows:

[0030] Hadoop: An open-source framework for efficiently storing and processing large datasets ranging from GB to PB levels;

[0031] HDFS (Distributed File System): The distributed file system of Hadoop;

[0032] NameNode (Metadata Node): In the HDFS architecture, the NameNode is the metadata node; the NameNode is used to store metadata. These metadata are stored in the memory of the metadata node. To obtain data at the fastest speed, the NameNode is responsible for maintaining and managing the slave nodes and assigning tasks to the slave nodes.

[0033] The following elaborates on the principles and spirit of the present invention in detail with reference to several representative embodiments of the present invention.

[0034] Figure 1It is a schematic flowchart of a method for determining the response duration of an operation command executed by a distributed file system in an embodiment of the present invention. As Figure 1 shown, the method includes:

[0035] Step 101, obtain the log data of the metadata node in the distributed file system within a preset duration;

[0036] Step 102, input the log data into a log data analysis model to obtain each operation command, corresponding timestamp, and operation file name in the log data; wherein, the log data analysis model is trained for a machine learning model according to historical log data, corresponding historical operation commands, historical timestamps, and historical operation file names;

[0037] Step 103, divide the operation commands and timestamps corresponding to the same operation file name into the same group;

[0038] Step 104, sort the operation commands in each group according to the timestamp, and query the timestamp corresponding to the operation command for obtaining operation file information during execution, and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution;

[0039] Step 105, determine the response duration of the distributed file system for executing the operation command on each operation file according to the timestamp corresponding to the operation command for obtaining operation file information during execution in each group, and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution.

[0040] By Figure 1As can be seen from the process shown, in the embodiment of the present invention, the log data of the metadata node in the distributed file system within a preset duration is obtained; the log data is input into a log data analysis model to obtain each operation command, corresponding timestamp, and operation file name in the log data; among them, the log data analysis model is trained on a machine learning model according to historical log data, corresponding historical operation commands, historical timestamps, and historical operation file names; the operation commands corresponding to the same operation file name and the timestamps are divided into the same group; the operation commands in each group are sorted according to the timestamps, and the timestamps corresponding to the operation commands for obtaining operation file information during execution are queried, and the timestamps corresponding to the next operation commands of the operation commands for obtaining operation file information during execution; according to the timestamps corresponding to the operation commands for obtaining operation file information during execution in each group and the timestamps corresponding to the next operation commands of the operation commands for obtaining operation file information during execution, the response duration of the distributed file system for executing each operation command on each operation file is determined. The embodiment of the present invention can calculate the response duration of the distributed file system for executing operation commands, providing data support for analyzing the influence of operation files and operation commands on the response duration.

[0041] In order to more clearly explain the method for determining the response duration of the above-mentioned distributed file system for executing operation commands, the following will be described in detail in combination with each step.

[0042] In an embodiment of the present invention, after obtaining the log data of the metadata node (NameNode) in the distributed file system (HDFS) within a preset duration, the log data is input into a log data analysis model to obtain each operation command, corresponding timestamp, and operation file name in the log data; among them, the log data analysis model is trained on a machine learning model according to historical log data, corresponding historical operation commands, historical timestamps, and historical operation file names; specifically, the machine learning model can select a keyword extraction model, such as: TextRank, Bert, etc.

[0043] During specific implementation, the embodiment of the present invention can also pre-edit a regular expression according to the obtained log data form, and parse the log data according to the edited regular expression to obtain each operation command, corresponding timestamp, and operation file name in the log data.

[0044] In an embodiment of the present invention, before sorting the operation commands in each group according to the timestamps, it further includes:

[0045] An operation process record table corresponding one-to-one with the operation file name is established in the database.

[0046] Figure 2It is a specific example diagram of the method for determining the response duration of the operation command executed by the distributed file system in the embodiment of the present invention.

[0047] In an embodiment of the present invention, with reference to Figure 2 , the operation commands in each group are sorted according to the time stamp, and the detailed process of querying the time stamp corresponding to the operation command for obtaining operation file information during execution and the time stamp corresponding to the next operation command of the operation command for obtaining operation file information during execution is as follows:

[0048] Step 201, sort the operation commands in each group according to the time stamp, and store the sorted operation commands and the corresponding time stamps line by line into the corresponding operation process record table;

[0049] Step 202, in the operation process record table, query the time stamp corresponding to the operation command for obtaining operation file information (getfileinfo command) during execution and the time stamp corresponding to the next operation command of the operation command for obtaining operation file information during execution; among them, the line where the operation command for obtaining operation file information during execution is located is adjacent to the line where the next operation command of the operation command for obtaining operation file information during execution is located.

[0050] In specific implementation, the operation commands in each group are sorted according to the time stamp, the sorted operation commands and the corresponding time stamps are stored line by line into the corresponding operation process record table, and a number is generated for each line, and the numbers increase line by line; in the operation process record table, identify the line where the operation command for obtaining operation file information (getfileinfo command) during execution is located and the line where the next operation command for obtaining operation file information is located, and the number interval between the two lines is 1, to prevent inaccurate statistical results due to long-time repeated operations on the same directory due to business reasons; record the time stamp corresponding to the operation command for obtaining operation file information (getfileinfo command) during execution and the time stamp corresponding to the next operation command for obtaining operation file information.

[0051] In an embodiment of the present invention, according to the time stamp corresponding to the operation command for obtaining operation file information during execution in each group and the time stamp corresponding to the next operation command of the operation command for obtaining operation file information during execution, determining the response duration of the distributed file system for executing the operation command on each operation file includes:

[0052] The time difference between the time stamp corresponding to the operation command for obtaining operation file information during execution in each group and the time stamp corresponding to the next operation command of the operation command for obtaining operation file information during execution is determined as the response duration of the distributed file system for executing the operation command on each operation file.

[0053] In one embodiment of the present invention, before determining the response time of the distributed file system to execute the operation command on each operation file, the method further includes:

[0054] The timestamp corresponding to the operation command for obtaining the operation file information when executed and the timestamp corresponding to the next operation command of the operation command for obtaining the operation file information when executed in each group are converted into millisecond format.

[0055] During specific implementation, the timestamp corresponding to the operation command for obtaining the operation file information during execution and the timestamp corresponding to the next operation command of the operation command for obtaining the operation file information during execution are converted into milliseconds, and the time difference between the two rows of log data in the operation process record table is calculated, that is, the time consumed between getfileinfo and the next operation command.

[0056] In one embodiment of the present invention, after determining the response time of the distributed file system to execute the operation command on each operation file, the method further includes:

[0057] Displays the operation commands executed by the distributed file system whose response time exceeds the preset threshold and the corresponding operation file names.

[0058] During specific implementation, the operation commands executed by the distributed file system whose response time is longer than a preset threshold and the corresponding operation file names are displayed to the staff, so that the staff can conduct targeted analysis and optimization.

[0059] Figure 3 This is a specific example diagram of a method for determining the response time of a distributed file system executing an operation command in an embodiment of the present invention.

[0060] In one embodiment of the present invention, referring to Figure 3 , parse the log data of the metadata node (NameNode), use the pre-edited regular expression to extract each operation command, the corresponding timestamp, and the operation file name in the log data, group them according to the operation file name, sort the data of each group according to the timestamp, and number each line of the data in each group, with the number increasing line by line; query the timestamp of the line where the operation command for obtaining the operation file information (getfileinfo) is located and the timestamp of the line where the next operation command is located in each group, convert the two timestamps into millisecond format, and determine the time difference between the two timestamps as the response time of the distributed file system (HDFS) to execute the operation command.

[0061] It should be noted that although the operations of the method of the present invention are described in a specific order in the above embodiments and accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution.

[0062] For the implementation of the response duration determination device of the distributed file system for executing operation commands, reference can be made to the implementation of the above method, and the repeated parts will not be elaborated here. The terms "module" or "unit" used hereinafter may be a combination of software and / or hardware for implementing a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0063] Based on the same inventive concept, the present invention also proposes a response duration determination device for a distributed file system to execute operation commands, as Figure 4 shown, the device includes:

[0064] A log acquisition module 401, configured to acquire log data of the metadata node in the distributed file system within a preset duration;

[0065] A log analysis module 402, configured to input the log data into a log data analysis model to obtain each operation command, corresponding timestamp, and operation file name in the log data; wherein, the log data analysis model is obtained by training a machine learning model based on historical log data, corresponding historical operation commands, historical timestamps, and historical operation file names;

[0066] A data grouping module 403, configured to divide the operation commands and timestamps corresponding to the same operation file name into the same group;

[0067] A timestamp query module 404, configured to sort the operation commands in each group according to the timestamps, and query the timestamp corresponding to the operation command for obtaining operation file information during execution, and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution;

[0068] A response duration determination module 405, configured to determine the response duration of the distributed file system for executing operation commands on each operation file according to the timestamp corresponding to the operation command for obtaining operation file information during execution in each group, and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution.

[0069] In an embodiment of the present invention, it further includes:

[0070] A record table creation module, which is used to create an operation process record table in the database that corresponds one-to-one with the operation file name before sorting the operation commands in each group according to the time stamp;

[0071] The time stamp query module is specifically used for:

[0072] Sort the operation commands in each group according to the time stamp, and store the sorted operation commands and the corresponding time stamps line by line into the corresponding operation process record table;

[0073] In the operation process record table, query the time stamp corresponding to the operation command for obtaining operation file information during execution, and the time stamp corresponding to the next operation command of the operation command for obtaining operation file information during execution; wherein, the line where the operation command for obtaining operation file information during execution is located is adjacent to the line where the next operation command of the operation command for obtaining operation file information during execution is located.

[0074] In an embodiment of the present invention, the response duration determination module is specifically used for:

[0075] Determine the time difference between the time stamp corresponding to the operation command for obtaining operation file information during execution in each group and the time stamp corresponding to the next operation command of the operation command for obtaining operation file information during execution as the response duration of the distributed file system for executing the operation command on each operation file.

[0076] In an embodiment of the present invention, it further includes:

[0077] A format conversion module, which is used to convert the time stamp corresponding to the operation command for obtaining operation file information during execution in each group and the time stamp corresponding to the next operation command of the operation command for obtaining operation file information during execution into millisecond format before determining the response duration of the distributed file system for executing the operation command on each operation file.

[0078] In an embodiment of the present invention, it further includes:

[0079] A display module, which displays the operation commands with a response duration greater than a preset threshold and the corresponding operation file names in the operation commands executed by the distributed file system after determining the response duration of the distributed file system for executing the operation command on each operation file.

[0080] It should be noted that although several modules of the response duration determination device for the distributed file system to execute operation commands are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present invention, the features and functions of two or more of the above-described modules can be embodied in one module. Conversely, the features and functions of one module described above can be further divided and embodied by multiple modules.

[0081] Based on the foregoing inventive concept, as Figure 5 shown, the present invention further provides a computer device 500, including a memory 501, a processor 502, and a computer program 503 stored on the memory 501 and executable on the processor 502. When the processor 502 executes the computer program 503, the method for determining the response duration of the distributed file system for executing an operation command as described above is implemented.

[0082] Based on the foregoing inventive concept, the present invention provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the method for determining the response duration of the distributed file system for executing an operation command as described above is implemented.

[0083] Based on the foregoing inventive concept, the present invention provides a computer program product including a computer program, and when the computer program is executed by a processor, the method for determining the response duration of the distributed file system for executing an operation command is implemented.

[0084] The method and apparatus for determining the response duration of the distributed file system for executing an operation command provided by the embodiments of the present invention can solve the problem that there is no way to calculate the response duration of the distributed file system for executing an operation command in the prior art. The embodiments of the present invention obtain the log data of the metadata node in the distributed file system within a preset duration; input the log data into a log data analysis model to obtain each operation command, corresponding timestamp, and operation file name in the log data; wherein, the log data analysis model is trained from a machine learning model according to historical log data, corresponding historical operation commands, historical timestamps, and historical operation file names; group the operation commands and timestamps corresponding to the same operation file name into the same group; sort the operation commands in each group according to the timestamps, and query the timestamp corresponding to the operation command for obtaining operation file information during execution and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution; determine the response duration of the distributed file system for executing an operation command on each operation file according to the timestamp corresponding to the operation command for obtaining operation file information during execution and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution in each group. The embodiments of the present invention can calculate the response duration of the distributed file system for executing an operation command, providing data support for analyzing the influence of operation files and operation commands on the response duration.

[0085] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0086] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or the combination of blocks.

[0087] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing devices to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means realizes the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or the combination of blocks.

[0088] These computer program instructions can also be loaded onto a computer or other programmable data processing devices, so that a series of operation steps are executed on the computer or other programmable devices to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable devices provide steps for realizing the functions specified in Figure 1 one or more of the flows Figure 1

[0089] The specific embodiments described above further elaborate on the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only specific embodiments of the present invention and is not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.​

Claims

1. A method for determining the response duration of a distributed file system to execute an operation command, characterized in that Including: Obtain the log data of the metadata node in the distributed file system within a preset duration; Input the log data into a log data analysis model to obtain each operation command, corresponding timestamp, and operation file name in the log data; wherein, the log data analysis model is trained for a machine learning model based on historical log data, corresponding historical operation commands, historical timestamps, and historical operation file names; Divide the operation commands corresponding to the same operation file name and the timestamps into the same group; Sort the operation commands in each group by timestamp, and query the timestamp corresponding to the operation command for obtaining operation file information during execution, and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution; Determine the response duration of the distributed file system for executing the operation command on each operation file according to the timestamp corresponding to the operation command for obtaining operation file information during execution in each group, and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution; 2. The method according to claim 1, characterized in that Before sorting the operation commands in each group by timestamp, it further includes: Establish an operation process record table in the database that corresponds one-to-one with the operation file name; Sort the operation commands in each group by timestamp, and query the timestamp corresponding to the operation command for obtaining operation file information during execution, and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution, including: Sort the operation commands in each group by timestamp, and store the sorted operation commands and corresponding timestamps line by line into the corresponding operation process record table; In the operation process record table, query the timestamp corresponding to the operation command for obtaining operation file information during execution, and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution; wherein, the row where the operation command for obtaining operation file information during execution is located is adjacent to the row where the next operation command of the operation command for obtaining operation file information during execution is located.

3. The method according to claim 1, wherein Determine the response duration of the distributed file system for executing the operation command on each operation file according to the timestamp corresponding to the operation command for obtaining operation file information during execution in each group, and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution, including: Determine the time difference between the timestamp corresponding to the operation command for obtaining operation file information during execution in each group and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution as the response duration of the distributed file system for executing the operation command on each operation file.

4. The method according to claim 1, wherein Before determining the response duration of the distributed file system for executing the operation command on each operation file, it further includes: Convert the timestamp corresponding to the operation command for obtaining operation file information during execution in each group and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution into millisecond format.

5. The method according to claim 1, wherein After determining the response duration of the distributed file system for each operation file to execute the operation command, it further includes: Displaying the operation commands with a response duration greater than the preset threshold in the operation commands executed by the distributed file system and the corresponding operation file names.

6. A device for determining the response duration of an operation command executed by a distributed file system, characterized in that, It includes: A log acquisition module for acquiring the log data of the metadata node in the distributed file system within a preset duration; A log analysis module for inputting the log data into a log data analysis model to obtain each operation command, the corresponding timestamp, and the operation file name in the log data; among them, the log data analysis model is trained from a machine learning model based on historical log data, and the corresponding historical operation commands, historical timestamps, and historical operation file names. A data grouping module for grouping the operation commands and timestamps corresponding to the same operation file name into the same group; A timestamp query module for sorting the operation commands in each group by timestamp and querying the timestamp corresponding to the operation command for obtaining operation file information during execution and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution; A response duration determination module for determining the response duration of the distributed file system for each operation file to execute the operation command according to the timestamp corresponding to the operation command for obtaining operation file information during execution in each group and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution.

7. The device according to claim 6, characterized in that, It further includes: A record table establishment module for establishing an operation process record table corresponding one-to-one with the operation file name in the database before sorting the operation commands in each group by timestamp; The timestamp query module is specifically used for: Sorting the operation commands in each group by timestamp and storing the sorted operation commands and the corresponding timestamps row by row into the corresponding operation process record table; In the operation process record table, querying the timestamp corresponding to the operation command for obtaining operation file information during execution and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution; among them, the row where the operation command for obtaining operation file information during execution is located is adjacent to the row where the next operation command of the operation command for obtaining operation file information during execution is located.

8. The device according to claim 6, characterized in that, The response duration determination module is specifically used for: Determining the time difference between the timestamp corresponding to the operation command for obtaining operation file information during execution in each group and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution as the response duration of the distributed file system for each operation file to execute the operation command.

9. The device according to claim 6, characterized in that, It further includes: A format conversion module for converting the timestamp corresponding to the operation command for obtaining operation file information during execution in each group and the timestamp corresponding to the next operation command of the operation command for obtaining operation file information during execution into millisecond format before determining the response duration of the distributed file system for each operation file to execute the operation command.

10. The device according to claim 6, characterized in that, It further includes: A display module, after determining the response time of the distributed file system to execute the operation commands for each operating file, displays the operation commands with response times greater than a preset threshold among the operation commands executed by the distributed file system and the corresponding names of the operating files.

11. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the method according to any one of claims 1 to 5.

13. A computer program product, characterized in that, The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the method according to any one of claims 1 to 5.

Images