Method and device for diagnosing abnormal RPC request

Abstract

The invention discloses a method and device for diagnosing an abnormal RPC request, and relates to the technical field of computers. A specific embodiment of the method comprises the steps of collecting log data of a distributed system, and extracting a field influencing abnormal RPC request diagnosis from the log data as a key index field; counting the occurrence frequency of the target index field in the log data within a set window time to obtain an index aggregation result; wherein the target index field comprises at least one index field selected from the key index fields; and matching the index aggregation result with a set exception rule, and if the index aggregation result meets a triggering condition of the exception rule, determining that the RPC request corresponding to the index aggregation result is an exception RPC request. According to the method, the key index fields in the log data are subjected to index aggregation statistics within the set window time, and the indexaggregation result is matched with the exception rule, so that the exception RPC request can be quickly and accurately positioned.

Description

technical field [0001] The invention relates to the field of computers, in particular to a method and device for diagnosing abnormal RPC requests. Background technique [0002] HDFS (Hadoop Distributed File System, Hadoop Distributed File System) is the core sub-project of the Hadoop project. It is a highly fault-tolerant system that can provide high-throughput data access and is very suitable for applications on large-scale data sets. Users of each business line can request to access data stored on HDFS through RPC (Remote Procedure Call Protocol, remote procedure call protocol). [0003] Since HDFS itself has a limit on the number of accesses to RPC requests, and users access data on HDFS with great randomness, in some cases, user access behaviors, such as when users frequently read the historical data of a very large table, will As a result, the HDFS service is in a busy state and cannot provide stable services. [0004] In the existing technology, when the HDFS service...

AI Technical Summary

Problems solved by technology

[0006] (1) Locating the problem of excessive RPC request access by finding the abnormal task submission method requires R&D personnel to have the ability to identify related abnormal tasks; and using this method to locate the problem has low accuracy;

[0007] (2) By manually counting audit logs, other key indicators such as table information of the path operated by the user cannot be associated with results in real time; and using this method to locate problems has poor timeliness

Images