Method for detecting cause of access anomaly, system, device, medium, and program product

Abstract

Embodiments of the present application provide a method for detecting a cause of an access anomaly, a system, a device, a medium, and a program product. The method comprises: a log storage system receives access requests for a database, and determines from among the access requests target requests having the same attribute information, wherein these target requests can be considered as a class of requests which can constitute a target request set; then, the log storage system can analyze the target request set, so as to determine whether a server where the database is deployed has a performance index value anomaly when responding to the target request set. The log storage system can use an access request set as the granularity, that is, the target request set is used as a whole to detect whether a response to the set is a cause of the performance index value anomaly of the server.

Description

[0001] TECHNICAL FIELD: METHOD, SYSTEM, DEVICE, MEDIUM AND PROGRAM PRODUCT FOR DETECTING ACCESS ABNORMALITY CAUSES

[0002]

[0001] The present application relates to the field of database technology, and more particularly to a method, system, device, medium, and program product for detecting the cause of access anomalies.

[0003]

[0002] The database may store a large amount of data required by users. For example, in an online shopping scenario, the database may store product information, transaction information, and the like.

[0004] In practice, a database can be deployed on a server. When the database receives a large number of access requests from different users, it can utilize the computing resources provided by the server to respond to the access requests. However, the process of responding to access requests may also cause abnormal performance indicators of the server. The performance indicators of the server may include central processing unit (CPU) utilization, memory usage, etc.

[0005]

[0004] The cause of abnormal performance indicators may be caused by the server responding to a single access request or a group of access requests. Based on this, how to detect whether responding to a group of access requests will cause abnormal server performance indicators has become an urgent problem to be solved.

[0006]

[0005] In view of this, embodiments of the present application provide a method, system, device, medium, and program product for detecting the cause of an access anomaly, so as to detect whether responding to a group of access requests will cause an anomaly in a server performance indicator.

[0007]

[0006] In a first aspect, an embodiment of the present application provides a method for detecting the cause of an access anomaly, comprising: receiving an access request to a database; determining target requests with identical attribute information in the access request as a target request set; and determining whether a performance indicator value of a server deployed with the database is abnormal when responding to the target request set.

[0007] In a second aspect, an embodiment of the present application provides a detection system, comprising: a database, a log storage system, and a monitoring platform; the database is configured to receive an access request; the monitoring platform is configured to send a detection request; and a detection result reflecting whether a performance indicator value of a server deployed with the database is abnormal when responding to the target request set is displayed; the log storage system is configured to receive the access request; and in response to receiving the detection request, determining target requests with identical attribute information in the access request as the target request set; and determining the detection result.

[0008] In a third aspect, embodiments of the present application provide an electronic device, including a processor and a memory, wherein the memory is configured to store one or more computer instructions. When executed by the processor, the one or more computer instructions implement the method for detecting the cause of an access anomaly described in the first aspect. The electronic device may further include a communication interface for communicating with other devices or communication systems.

[0008]

[0009] In a fourth aspect, an embodiment of the present application provides a non-transitory machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor can at least implement the method for detecting the cause of an access anomaly as described in the first aspect.

[0009]

[0010] In a fifth aspect, embodiments of the present application provide a computer program product. The computer program product includes a computer program or instructions. When the computer program or instructions are executed by a processor, the processor is enabled to implement the method for detecting the cause of an access anomaly as described in the first aspect.

[0010]

[0011] In the access anomaly cause detection method provided by embodiments of the present application, a log storage system receives access requests to a database and identifies target requests with identical attribute information from the access requests. These target requests can be considered to be of a class and constitute a target request set. The log storage system then analyzes this target request set to determine whether a server hosting the database experiences abnormal performance indicator values ​​while responding to the target request set. Specifically, the system detects whether the abnormal server performance indicator is caused by responding to the target request set.

[0011]

[0012] In the above method, the log storage system can use the access request set as the granularity, treat the target request set as a whole, and detect whether the response to the set will cause an abnormality in the server performance indicator value, that is, detect whether the response to the set is the cause of the abnormality in the server performance indicator value.

[0012]

[0013] To more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. Those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0013]

[0014] FIG1 is a flow chart of a method for detecting the cause of an access anomaly provided in an embodiment of the present application;

[0014]

[0015] FIG2 is a flow chart of another method for detecting the cause of an access anomaly provided in an embodiment of the present application;

[0015]

[0016] FIG3 is a schematic diagram of a base table provided in an embodiment of the present application;

[0016]

[0017] FIG4 is a schematic diagram of a first materialized view provided in an embodiment of the present application;

[0017]

[0018] FIG5 is a flowchart of a method for determining a preset indicator threshold value provided in an embodiment of the present application;

[0018]

[0019] FIG6 is a schematic diagram of the size relationship of an aggregation result provided in an embodiment of the present application;

[0019]

[0020] FIG7 is a flowchart of a method for determining a preset indicator threshold value provided in an embodiment of the present application;

[0020]

[0021] FIG8 is a schematic diagram of an unprocessed SQL statement provided in an embodiment of the present application;

[0021]

[0022] FIG9 is a schematic diagram of a processed SQL statement provided in an embodiment of the present application;

[0022]

[0023] FIG10 is a flowchart of another method for detecting the cause of an access anomaly provided in an embodiment of the present application;

[0023]

[0024] FIG11 is a schematic diagram of a second materialized view provided in an embodiment of the present application;

[0024]

[0025] FIG12 is a flowchart of another method for detecting the cause of an access anomaly provided in an embodiment of the present application;

[0025]

[0026] FIG13 is a schematic structural diagram of a detection system provided in an embodiment of the present application;

[0026]

[0027] FIG14 is a schematic diagram of the structure of a device for detecting the cause of an access anomaly provided in an embodiment of the present application;

[0027]

[0028] FIG15 is a schematic diagram of the structure of an electronic device corresponding to the device for detecting the cause of access anomaly provided by the embodiment shown in FIG14.

[0028]

[0029] To further clarify the objectives, technical solutions, and advantages of the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings. It should be understood that the described embodiments are only a portion of the embodiments of this application, and are not exhaustive. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of this application without inventive effort are intended to fall within the scope of protection of this application.

[0029]

[0030] The terms used in the examples of this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in the examples of this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms. Unless the context clearly indicates otherwise, "a plurality" generally includes at least two, but does not exclude the inclusion of at least one.

[0030]

[0031] It should be understood that the term "and / or" as used herein is merely a description of an association relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0031]

[0032] Depending on the context, the words "if" and "if" as used herein may be interpreted as "when" or "when..." or "in response to determining" or "in response to identifying." Similarly, depending on the context, the phrases "if it is determined" or "if (a stated condition or event) is identified" may be interpreted as "when it is determined" or "in response to determining" or "when identifying (a stated condition or event)" or "in response to identifying (a stated condition or event)."

[0032]

[0033] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0033]

[0034] It should also be noted that the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such product or system. In the absence of further limitations, the phrase "comprising a..." does not preclude the presence of additional identical elements in the product or system comprising the elements.

[0034]

[0035] The following detailed description of some embodiments of the present application is provided in conjunction with the accompanying drawings. The following embodiments and their features may be combined unless they conflict with each other. Furthermore, the sequence of steps in the following method embodiments is provided for illustrative purposes only and is not intended to be a strict limitation.

[0035]

[0036] FIG1 is a flowchart of a method for detecting the cause of an access anomaly provided in an embodiment of the present application. The method provided in an embodiment of the present application can be executed by a log storage system. Optionally, the log storage system can specifically represent a detection database. As shown in FIG1 , the method may include the following steps S101 to S103.

[0036]

[0037] S101, receiving a database access request.

[0037]

[0038] Users can issue access requests to the database based on actual needs. This access request, while being received by the database, can also be received by the log storage system. The access request can be a request to add, delete, modify, or query data in the database. Optionally, the access request can include a Structured Query Language (SQL) statement and attribute information.

[0038]

[0039] It should be noted that, to distinguish it from the detection database serving as the execution entity, the database that users need to access, i.e., the database where users need to add, delete, modify, and query data, as mentioned in the various embodiments provided herein, can be referred to as a service database. This service database is used to provide database services to users. The subsequent description of this embodiment and the following embodiments of this application will also use this term "service database."

[0039]

[0040] Optionally, the service database can be a distributed database, that is, servers dispersed across different physical locations can constitute a distributed database. Multiple servers located in one physical location are referred to as a regional cluster, and a server within a regional cluster is referred to as an instance. Therefore, the server referred to in each embodiment of this application is a server on which a database is deployed, and access requests are requests issued by users to access the database deployed on this server.

[0040]

[0041] Alternatively, the distributed database may include an analytical database or a transactional database. For an analytical database, user access requests are often data query requests. In an online shopping scenario, for example, the analytical database may be a database that maintains product information, and user access requests are used to query product information from the database.

[0041]

[0042] For transactional databases, user access requests are mostly requests to read, write, or modify data. Continuing with the above scenario, a transactional database might be one that maintains commodity transaction information. User access requests are used to write or modify commodity transaction information in the database.

[0042]

[0043] S102: Determine target requests with the same attribute information in the access requests as a target request set.

[0043]

[0044] Furthermore, after receiving the access request, the log storage system may divide the access request according to the attribute information of the access request, so as to form a target request set consisting of target requests having the same attribute information.

[0044]

[0045] Optionally, the attribute information may include a format identifier for the access request, which indicates the category of the access request and the action content of the access request, such as access requests generated within a certain time period to query a certain data page in a database. The log storage system may first analyze the SQL statement in the access request, perform a hash calculation on the analysis result, and use the hash calculation result as the format identifier. The log storage system may then identify target requests with the same hash calculation result as a target request set. The specific process for generating the format identifier can be found in the relevant description of the following embodiments.

[0045]

[0046] S103: Determine whether a performance indicator value of the server where the database is deployed is abnormal when responding to a target request.

[0046]

[0047] Ultimately, the log storage system can also determine whether the performance indicator value of the server hosting the service database is abnormal while responding to the target request set. In other words, the system can determine, at the granularity of the set, whether responding to the set is the cause of the abnormal server performance indicator value. Optionally, the server performance indicator value can include the server's central processing unit (CPU) utilization, the server's memory usage, and so on. If the CPU utilization and / or memory usage is greater than a preset threshold, the server performance indicator value can be considered abnormal.

[0047]

[0048] Optionally, the log storage system may receive access requests to the service database in real time, that is, execute step S10L in real time. The log storage system may execute steps S102 and S103 in response to receipt of a detection request. The detection request may be sent by the monitoring platform. Optionally, the detection request may include a detection period. The log storage system may first obtain access requests generated within the detection period and classify these access requests to obtain the target request set described above. Optionally, the detection period may be manually set.

[0048]

[0049] Optionally, the log storage system and monitoring platform can be deployed on hardware devices that are not instances of the distributed database. The monitoring platform can also automatically generate detection requests or generate detection requests in response to manual operation. Regarding the automatic generation of detection requests, one option is for the monitoring platform to periodically generate detection requests according to a preset detection cycle. Another option is for the monitoring platform to continuously obtain performance indicator values ​​of servers deployed with the service database and generate detection requests when such performance indicator values ​​are abnormal. Optionally, the detection request can also include a detection period. The log storage system can then group access requests generated within this detection period into sets and further determine whether any of these sets of requests cause abnormal server performance indicator values.

[0049]

[0050] As can be seen from the above description, the steps of this embodiment do not have a strict execution order. That is, the log storage system can execute step S101 in real time, but the execution of steps S102 through S103 requires the access request obtained in step S101. However, after executing step S101, steps S102 through S103 can only be executed when the execution time is met.

[0050]

[0051] In this embodiment, the log storage system receives access requests to a service database and identifies target requests with the same attribute information from the access requests. These target requests can be considered to be of the same type and constitute a target request set. The log storage system then analyzes this target request set to determine whether the server hosting the service database experiences abnormal performance indicator values ​​while responding to the target request set. Specifically, the system detects whether the abnormal server performance indicator is caused by responding to the target request set.

[0051]

[0052] In the above method, the log storage system can use the access request set as the granularity, treat the target request set as a whole, and detect whether the server performance indicator value will be abnormal in the process of responding to the set, that is, detect whether responding to the set is the cause of the abnormal server performance indicator value.

[0052]

[0053] Furthermore, compared to testing at the granularity of a single access request, specifically detecting whether a response to a single access request causes an abnormal server performance indicator value, this embodiment innovatively proposes testing at the granularity of a set of access requests, specifically detecting whether responding to that set of requests is the cause of the abnormal server performance indicator value. Furthermore, testing at this granularity consumes fewer computing resources in the log storage system, and therefore, this testing process does not affect the normal operation of other functions of the log storage system.

[0053]

[0054] Optionally, after executing step S103, the log storage system can also obtain a detection result indicating whether the server's performance indicator value is abnormal when responding to the target request set. The log storage system can also display the detection result to maintenance personnel of the service database to provide a basis for the maintenance personnel's next processing. Optionally, the detection result can also be fed back to the user who generated the access request.

[0054]

[0055] Based on the above description, the practical application significance of the methods provided in each embodiment of the present application can be understood as follows.

[0055]

[0056] In practice, the most common service database access process is a user querying an analytical database that stores massive amounts of data. Furthermore, for the massive amounts of data stored in an analytical database, the same or different users can generate a wide variety of query requests against the database. A single type of query request is referred to in various embodiments of this application. When a large number of query requests of the same type occur, server performance indicator values ​​often become abnormal. In this case, the methods provided in various embodiments of this application can be used to accurately locate the target request set causing the abnormal performance indicator values, identifying the user and type of query request generated during the specified time period. Based on accurate location results, maintenance personnel can address the issue in a targeted and timely manner, minimizing the impact on query requests generated by other users and other types of query requests generated by the user. Furthermore, in practice, the types of query requests generated by the same or different users against an analytical database are often relatively stable. Therefore, analyzing whether a particular type of query request is the cause of the abnormal server performance indicator values ​​is also of practical significance.

[0056]

[0057] Optionally, the number of target requests in the target request set and / or the response index value of each target request can serve as a key basis for analyzing whether the target request set causes abnormalities in the server's own performance index value. The response index of the target request can include parameters reflecting the service database's response to the request, such as at least one of execution time (wall time), table scan data size, total planning time, queued time, and total wall time (including execution time and queued time). The response index value is the parameter value. In addition to the access request format identifier and query statement, the response index values ​​corresponding to each of the above-mentioned response indicators can also be included in the access request and sent to the log storage system.

[0057]

[0058] Based on this, FIG2 is a flowchart of another method for detecting the cause of access anomalies provided by an embodiment of the present application. As shown in FIG2 , by analyzing the response index values ​​of target requests in a target request set, it is possible to determine whether the responses to the target request set will cause abnormalities in the server's own performance index values. The method may include the following steps S201 to S203.

[0058]

[0059] S201, receiving a database access request.

[0059]

[0060] The specific implementation process of the above step S201 can refer to the specific description of the relevant steps in the embodiment shown in Figure 1, and will not be repeated here.

[0060]

[0061] S202: Determine target requests with the same attribute information in the access requests as a target request set.

[0061]

[0062] For any access request received at any time, the log storage system can store the attribute information and response index value of the access request as a record in the base table of the log database. The log storage system maintains the log database.

[0062]

[0063] Based on this base table, after receiving a detection request, the log storage system can filter out at least one record from this base table. The filtered record is the target request with the same attribute information, so as to obtain a target request set.

[0063]

[0064] Specifically, when the detection request does not include a detection period, records corresponding to target requests with the same attribute information can be filtered from all records in the base table to obtain a target request set. When the detection request includes a detection period, records corresponding to target requests generated within the detection period with the same attribute information can be filtered from the base table to obtain a target request set.

[0064]

[0065] S203: Determine, based on the response index value of the target request set, whether the performance index value of the server in responding to the target request set is abnormal.

[0065]

[0066] The log storage system may further obtain the response index value of the target request set, and determine whether the performance index value of the server is abnormal when responding to the target request set based on the response index value of the target request set.

[0066]

[0067] Specifically, the log storage system may use the response index value of the target request set as the value to be detected and compare it with a preset index threshold. If the response index value of the target request set is greater than the preset index threshold, the log storage system may determine that the server's performance index value when responding to the target request set is abnormal. If the response index value of the target request set is less than the preset index threshold, the log storage system may determine that the server's performance index value when responding to the target request set is normal. Optionally, the preset index threshold can be manually set based on actual needs or obtained through statistical methods. The specific process for determining the preset index threshold can be found in the relevant description of the following embodiments.

[0067]

[0068] The response index value of the target request set can be considered the aggregated result obtained by aggregating the response index values ​​of the target requests in the target request set. The log storage system can obtain the response index values ​​of the target requests from the base table and perform aggregation processing. Optionally, the aggregation result may include the maximum, quantile, median, and other values ​​of the response index values.

[0068]

[0069] For example, the response metric values ​​of the target request may include the aforementioned execution time, scanned data size, queue time, and so on. Correspondingly, the aggregation result may include the maximum, quantile, median, and so on of the execution time, scanned data size, and queue time. Optionally, the maximum value in the aggregation result can generally be used as the value to be tested for comparison with a preset metric threshold. Other values ​​in the aggregation result can be used to determine the preset metric threshold.

[0069]

[0070] When the response index value of the target request set, that is, the aggregation result, can be expressed as a maximum value, the aggregation result can reflect the extreme response of the service database when responding to different target requests in the target request set. Using this maximum value as the value to be detected can accurately determine whether the target request set will cause abnormalities in the server performance index value.

[0070]

[0071] Optionally, the log storage system can further generate a first materialized view by aggregating the response indicator values ​​of access requests in the base table. The first materialized view is a virtual table generated based on the base table. The attribute information of the target request and the response indicator value of the target request set are used as a record in the first materialized view.

[0071]

[0072] In other words, the first materialized view can be considered an aggregation of the records in the base table. The difference between the base table and the first materialized view lies in the following: the base table includes access request attribute information and response indicator values ​​for access requests, while the first materialized view includes access request attribute information and response indicator values ​​for a set of requests. The difference between the two tables can also be understood in conjunction with Figures 3 and 4 below.

[0072]

[0073] In this embodiment, the log storage system can compare the response index value of the target request set with a preset index threshold to determine whether the server's performance index value is abnormal when responding to the target request set. In other words, the system can detect whether the abnormal server performance index value is caused by responding to a certain type of request through numerical comparison.

[0073]

[0074] Compared to using an anomaly detection model represented by an artificial intelligence model, the numerical comparison method provided in this embodiment is simpler and faster, thereby reducing the computational pressure on the log storage system. Furthermore, since an anomaly detection model is not used, time-consuming training and optimization of the anomaly detection model is not required, thus simplifying the implementation of anomaly detection.

[0074]

[0075] Optionally, in addition to including the format identifier mentioned in the embodiment shown in FIG. 1 , the attribute information of the access request may further include at least one of a first identifier of the user generating the access request, a second identifier of the server where the database is located, a type of operation performed by the access request on data in the service database, and a preset time period to which the access request is generated. Data operation types may include querying data, writing data, modifying data, deleting data, and the like. The log storage system may divide the time periods according to the preset time length, thereby obtaining at least one preset time period. The preset time period may be a relatively short time period, such as 5 minutes.

[0075]

[0076] Therefore, the following explanation can be given for the partitioning of the set.

[0076]

[0077] On the one hand, when the attribute information includes a format identifier, the format identifier can be used as a basis for grouping access requests in the base table into sets, thereby grouping all access requests with the same format identifier into the same set. Access requests in this set may include access requests from different users at different times to the service database deployed on the same server. Therefore, grouping records in the base table using the format identifier yields a small number of sets and high grouping efficiency. Furthermore, the log storage system can quickly determine whether an abnormal server performance indicator value is caused by this set.

[0077]

[0078] In addition to the format identifier, if the attribute information also includes other items, such as the first identifier and the second identifier, then these multiple items can be used as the basis for grouping, thereby grouping access requests generated by the same user to the service database on the same server within the same preset time period into the same group. As can be seen, the more items included in the attribute information used in the grouping process, the greater the number of groups generated, the more targeted the groups, and the more accurately the log storage system can determine the cause of abnormal server performance indicator values.

[0078]

[0079] On the other hand, when the attribute information includes the aforementioned format identifier, first identifier, second identifier, the type of operation performed by the access request on the data in the service database, and the preset time period to which the access request was generated, the base table may be as shown in FIG3. In FIG3, a record of an access request, namely, access request 1, includes: the first identifier of the user is user 1; the type of operation performed by access request 1 on the data in the service database is data query; the execution duration is 561 ms; the generation time of access request 1 is 12:00 on August 1, 2023; the scanned data size is 1571588993 bytes; the second identifier of the server is server a; and the format identifier of access request 1 is 2465376594264356412.

[0080] The record of another access request, Access Request 2, includes: the user's first identifier is User 2; the operation type of Access Request 2 on the data in the service database is query data; the execution time is 4250ms; the generation time of Access Request 2 is 12:05 on August 1, 2023; the scanned data size is 297567493 bytes; the server's second identifier is Server B; and the format identifier of Access Request 2 is 164859231372452643 L.

[0079]

[0081] Furthermore, based on the base table shown in FIG3 , when the log storage system performs set partitioning based on the user's first identifier, the server's second identifier, the format identifier, and the preset time period to which the access request was generated, a first materialized view as shown in FIG4 can be obtained. In FIG4 , a record in the first materialized view includes: the first identifier is user 1, the second identifier is server a, the format identifier of the access request is 2465376594264356412, the preset time period to which the access request was generated is 12:00-12:05 on August 1, 2023; and the quantiles are respectively:

[0080] {561,561,561,561}, where the four values ​​are the 25th percentile, 50th percentile, 75th percentile, and 90th percentile respectively; the maximum value to be detected is 56L

[0081]

[0082] Another record includes: the first identifier is user 2, the second identifier is server b, the format identifier of the access request is 164859231372452643K, the time of generation of the access request belongs to the preset time period of 12:05-12:10 on August 1, 2023; the percentiles are {4250,4250,4250,4250}, where these four values ​​are also the 25th percentile, 50th percentile, 75th percentile, and 90th percentile; and the maximum value of the value to be detected is 4250.

[0082]

[0083] In this embodiment, the attribute information of an access request may include multiple items. The log storage system can select one or more items as the basis for grouping based on actual detection requirements. Furthermore, the more items in the attribute information, the more targeted and refined the groupings. This allows the log storage system to more precisely determine which users and time periods caused the abnormal server performance indicator values.

[0083]

[0084] In addition, as mentioned in the embodiment shown in FIG. 2 , the response index value of the target request set can be compared with a preset index threshold to determine whether the performance index value of the server in responding to the target request set is abnormal. An optional method for determining the preset index threshold is shown in FIG. 5 .

[0084]

[0085] FIG5 is a flow chart of a method for determining a preset indicator threshold value provided in an embodiment of the present application. As shown in FIG5 , the method may include the following steps S301 to S302.

[0085]

[0086] S301: Aggregate response indicator values ​​of a historical request set in a first materialized view, wherein the historical requests in the historical request set have the same attribute information as the target request, and the generation time of the historical requests is within a preset historical period.

[0086]

[0087] For the base table shown in FIG3 , when a detection request includes a detection period, the data stored in the base table may include attribute information and response index values ​​for access requests generated within the detection period and in historical periods prior to the detection period. It will be readily understood that the target requests in the aforementioned embodiments are clearly generated within the detection period.

[0087]

[0088] Similarly, for the first materialized view shown in FIG4 , when the detection request includes a detection period, the data in the first materialized view may include attribute information of target requests generated during the detection period and response index values ​​of the target request set. It may also include attribute information of historical requests generated during the historical period and response index values ​​of the historical request set. The historical requests and the target request have the same attribute information.

[0088]

[0089] E) The log storage system can filter out at least one historical request set corresponding to a preset historical period from the first materialized view shown in Figure 4. The preset historical period can be included in the detection request. The preset historical period can be a period of time before the detection period. As shown in Figure 4 , the preset period in the attribute information can be 5 minutes. The detection period and the historical period can also be divided into multiple sub-periods based on the length of the preset period. The preset historical period can be at least one continuous sub-period within the historical period. In other words, the first materialized view filters out historical requests generated within each sub-period within the preset historical period. These requests constitute at least one historical request set.

[0090] For example, if the preset period is 5 minutes long, the detection period can be set to 12:00 PM - 2:00 PM on December 6, 2023. The historical period starts at the time the access request corresponding to the earliest record in the base table is generated, and ends at 12:00 PM on December 6, 2023. The preset historical period can be a portion of the historical period, for example, November 6, 2023 - December 5, 2023. Both the detection period and the preset historical period can be divided into at least one continuous sub-period based on the preset period. Historical requests generated during the sub-period from 12:00 AM to 12:05 AM on November 6, 2023, can be filtered out to form a historical request set. The response indicator value for this historical request set can then be obtained in the first materialized view. Similarly, historical requests generated during the sub-period from 00:05 to 00:10 on November 6, 2023, can be filtered out to form another historical request set. The response index value for this set can also be determined from the first materialized view. Similarly, the log storage system can obtain the response index value corresponding to multiple historical request sets corresponding to the preset historical period.

[0089]

[0091] S302: Determine a preset indicator threshold based on the aggregation result.

[0090]

[0092] Afterwards, the log storage system may further aggregate the response indicator values ​​of the respective historical request sets filtered out from the first materialized view to obtain an aggregation result, and determine a preset indicator threshold value according to the aggregation result.

[0091]

[0093] The aggregation of the response index values ​​of the historical request set in this step is also a secondary aggregation of the response index values ​​of the historical requests. The response index values ​​of the historical request set are the result of a primary aggregation of the response index values ​​of the historical requests. Continuing with the example in step S301, the reason for this secondary aggregation is that if a primary aggregation is used, the aggregation step during detection would take a long time due to the large number of requests to be aggregated. The aggregation algorithm used in this step is an approximate algorithm. The shorter the initial aggregation period, the more accurate the resulting aggregated value and the longer the time required. Conversely, the less accurate the resulting aggregated value, the shorter the time required. Therefore, the period length used for the primary aggregation, such as 5 minutes, is a practical value.

[0092]

[0094] In addition, the embodiments provided herein involve two aggregations. The first aggregation is performed on the response index values ​​of multiple target requests. The aggregation results may include quantiles, medians, and maximum values ​​of the response index of the target requests, and may also serve as the response index value of the target request set. The second aggregation is performed on the response index values ​​of the historical request set. The aggregation results may include quantiles, maximum values, and medians of the response index values ​​of the historical request set. The aggregation results obtained after the second aggregation can reflect the service database's response to historical requests with the same attribute information as the target request within a preset historical period. This aggregation result also serves as the basis for determining the preset index threshold. Furthermore, based on the aggregation results obtained after the second aggregation, the quantile range can also be obtained.

[0093]

[0095] In this embodiment, the log storage system can determine the preset indicator threshold value by aggregating the response indicator values ​​of the historical request set in the first materialized view to obtain an aggregated result. Because this aggregated result can reflect the service database's response to historical requests with the same attribute information as the target request within a preset historical period, the preset indicator threshold value can be determined more accurately.

[0094]

[0096] As mentioned in the embodiment shown in FIG. 5 , the log storage system can determine a preset indicator threshold based on the aggregation results (i.e., the results after secondary aggregation) of the response indicator values ​​of a set of historical requests. The aggregation results can reflect the service database's response to historical requests during a historical period. This response can also be considered the distribution of response indicator values ​​for historical requests, such as the distribution of execution times or scanned data sizes for different historical requests during the historical period. Furthermore, these aggregation results can include types such as quantiles, maximums, and quantile ranges. The log storage system can use different methods to determine the preset indicator threshold based on different types of aggregation results.

[0095]

[0097] When the distribution of response index values ​​of historical requests with the same attribute information as the target request is relatively balanced, for example, when the distribution of the index values ​​conforms to a normal distribution, an optional threshold determination method is that the log storage system may determine the preset index threshold based on the quantile of the response index values ​​of the historical request set, or may determine the preset index threshold based on both the quantile and the quantile interval.

[0096]

[0098] For example, quantiles can specifically include quartiles, with the upper quartile represented as Q3 and the lower quartile represented as QL. The quantile interval is also known as the interquartile range (IQR). The physical meaning of a quantile is also the response index value of an access request. Considering the actual database situation, a smaller response index value indicates a faster access speed to the service database, while a larger response index value is more likely to cause an abnormal server performance index value. Therefore, the log storage system may optionally ignore Q1, the lower inner limit Q1m*IQR, and the lower outer limit Q1-n*IQR in Figure 6. Alternatively, the log storage system may directly use Q3 as the preset index threshold, or may determine the preset index threshold based on Q3 and IQR, for example, determining the upper inner limit Q3+m*IQR as the preset index threshold. Optionally, m is greater than 1, such as m=1.5. n is greater than m, such as n=3. The aforementioned relationship between quantiles, inner and outer limits can also be understood in conjunction with Figure 6.

[0097]

[0099] Considering actual database usage, the distribution of response index values ​​for historical requests with the same attribute information as the target request is often uneven, that is, it does not conform to a normal distribution. To address this situation, after obtaining the response index values ​​of the target request set as the values ​​to be tested, the log storage system can first determine a reasonable preset index threshold through segmented judgment. Then, by comparing the response index values ​​of the target request set with this reasonable preset index threshold, it determines whether executing the target request set will cause anomalies in the server performance index value. The above process can also be understood in conjunction with the flowchart shown in Figure 7.

[0098]

[0100] In another optional threshold determination method, the log storage system may first determine an alternative threshold based on the quantile, maximum value, and preset coefficient in the aggregation result, and then determine the preset indicator threshold corresponding to the target request set from the alternative thresholds based on the quantile. Optionally, the quantiles may include the upper quartile represented by Q1, the lower quartile represented by Q3, and the 90th quartile represented by p90. The maximum value may be represented by M. The preset coefficients may include a1, a2, a4, and b. A1, a2, and b may all be greater than or equal to 1. Typically, a1, a2, and b may be set to 1.5, and a3 may be set to 1.0. Alternative thresholds may include oc1*M, a2*P90, max(P90, a3*Q3), and max(P90, oc4*Q3+b*IQR).

[0099]

[0101] The specific segmented judgment process is as follows: first determine whether Q1 is equal to M. If Q1 is equal to M, then a 1*M in the alternative threshold can be determined as the preset indicator threshold.

[0100]

[0102] Furthermore, if Q1 is not equal to M, it can be determined whether Q1 is equal to p90. If Q1 is equal to p90, 2*p90 among the candidate thresholds can be determined as the preset indicator threshold.

[0101]

[0103] Furthermore, if Q1 is not equal to p90, it can be determined whether Q1 is equal to Q3. If Q1 is equal to Q3, the preset indicator threshold is determined as max(P90, a3*Q3) among the candidate thresholds.

[0102]

[0104] Finally, if Q1 is not equal to Q3, max(P90, a 4*Q3+b*IQR) among the alternative thresholds is determined as the preset indicator threshold.

[0103]

[0105] In this embodiment, the log storage system can understand the server's response to historical requests with the same attribute information as the target request through segmented judgment, that is, the distribution of response index values ​​of historical requests, and thus determine a reasonable preset index threshold based on the distribution of response index values.

[0104]

[0106] Optionally, in order to raise the detection threshold, that is, to more accurately detect the cause of the abnormal server performance indicator value, the above Q1 (i.e., the 25th percentile) may be replaced with a lower percentile, such as the 20th percentile represented by p20; the above Q3 (i.e., the 75th percentile) may be replaced with a higher percentile, such as the 80th percentile represented by p80; and the above p90 may be replaced with a higher percentile, such as the 95th percentile represented by p95.

[0107] In the embodiment shown in FIG. 1 , it is mentioned that the request set can be divided based on the format identifier in the attribute information, and the format identifier can be obtained by analyzing the SQL statement in the access request and performing hash calculation on the analysis result.

[0105]

[0108] To determine the format identifier, the log storage system may optionally first perform syntax analysis on the SQL statement in the access request to obtain a syntax tree. The original string in the access request, which serves as a node in the syntax tree, may then be formatted uniformly. This formatting uniformity may include standardizing the string writing style and eliminating parameters. This standardization may include standardizing indentation and spacing. Optionally, for service databases that require case sensitivity, standardization may further include standardizing the case of letters in the string. The process of eliminating parameters may involve replacing field values ​​with default values.

[0106]

[0109] For example, before formatting is standardized for a service database requiring case sensitivity, the SQL statement generated for this database might look like the one shown in Figure 8. In this SQL statement, SELECT, FROM, WHERE, AND, INTERVAL, DAY, and LIMIT are keywords; live_time, date_format, date_add, and now are functions; online_num, room_id, and live_time are fields; and numbers are field values. As Figure 8 shows, the case of the online_num field is not standardized. In this case, the log storage system can standardize the string format, changing "online_num" to "online_num."

[0107] [110 19: Figure 9

[0108]

[0111] Ultimately, the log storage system may concatenate the strings that have undergone format unification to obtain a concatenated string. A hash calculation may then be performed on the concatenated string to determine the hash calculation result as the format identifier of the access request. Optionally, the hash calculation method may include an FNV-1a hash algorithm.

[0109]

[0112] In this embodiment, the SQL statements are first syntactically analyzed, and then the formats are unified based on the analysis results. A format identifier is generated according to the access requests after the format unification. This enables access requests containing SQL statements with the same semantics but different writing formats and parameter values ​​to be classified into the same set, thereby improving the accuracy of set division.

[0110] Based on the methods provided in the above embodiments, the log storage system can ultimately obtain a detection result. When responses to multiple request sets all cause abnormal server performance index values, the degree of impact of the abnormal performance index values ​​can be optionally sorted so that maintenance personnel can prioritize request sets that have a greater impact on the performance index values.

[0111]

[0114] In an optional sorting method, the log storage system may compare the response index value of the target request set with the quantiles of the response index value of the historical request set, and determine the abnormality level corresponding to the target request set based on the comparison result. The abnormality level may reflect the correlation between the server's response to the target request set and the abnormality of the server's performance index value.

[0112]

[0115] Considering that the alternative thresholds provided in the embodiment shown in FIG7 all include the 90th percentile, the 90th percentile of the response index of the historical request set can be used as a benchmark for determining the degree of abnormality, thereby ensuring the accuracy of the determination of the degree of abnormality.

[0113]

[0116] In this embodiment, the abnormality levels corresponding to the request sets are sorted to obtain the request set with the greatest impact on the server performance index value and process it first, so as to quickly restore the normal operation of the service database.

[0114] FIG10 is a flow chart of another method for detecting the cause of an access anomaly provided in an embodiment of the present application. As shown in FIG10 , whether a response to a target request set will cause an anomaly in the server's own performance indicator value can be determined by analyzing the number of target requests in the target request set. The method may include the following steps S401 to S403. S401: Receive a database access request.

[0115]

[0119] S402, determine the target requests with the same attribute information in the access requests as a target request set.

[0116]

[0120] The specific implementation process of the above steps S401-S402 can refer to the specific description of the relevant steps in the embodiment shown in Figure 1, which will not be repeated here.

[0117]

[0121] S403, according to the number of target requests in the target request set, determining whether the performance indicator value of the server is abnormal when responding to the target request set.

[0118] The log storage system can further obtain the number of target requests in the target request set and use this number as a value to be detected. If the number of target requests in the target request set is greater than a preset number threshold, the log storage system can determine that the performance index value of the server is abnormal when responding to the target request set. If the number of target requests in the target request set is less than a preset number threshold, the log storage system can determine that the performance index value of the server is normal when responding to the target request. Optionally, the preset number threshold can be artificially set according to actual needs.

[0119]

[0123] Optionally, the log storage system may aggregate the number of target requests based on the records in the base table to obtain a second materialized view, and determine the number of target requests in the target request set based on the second materialized view. Furthermore, in terms of fields, the base table and the second materialized view differ in that: the base table includes attribute information of access requests and response indicator values ​​of the access requests, while the second materialized view includes attribute information of access requests and the number of access requests in the request set.

[0120] In the present embodiment, the log storage system determines whether the performance index value of the server is abnormal when responding to the target request set by analyzing the number of target requests in the target request set. That is, by detecting whether the abnormality of the server performance index value is caused by responding to a class of requests in a relatively simple and fast numerical comparison mode, the computing pressure of the log storage system can be alleviated.

[0121]

[0125] In addition, for the contents not described in detail in this embodiment and the technical effects that can be achieved, please refer to the relevant descriptions in the above embodiments, and will not be repeated here.

[0122]

[0126] Optionally, when the attribute information of the access request includes the user's first identifier, the server's second identifier, the format identifier, and the preset time period to which the access request was generated, that is, when the set division is performed based on the above four fields, the second materialized view shown in FIG11 can be generated based on the base table shown in FIG3. The second materialized view can be considered a virtual table generated based on the base table. The attribute information of the target request and the number of target request sets can be used as a record in the second materialized view.

[0123]

[0127] As shown in FIG11 , a record in the second materialized view includes: a first identifier of user 1, a second identifier of server a, an access request format identifier of 2465376594264356412, a preset time period of 12:00-12:05 on August 1, 2023, and a number of access requests of 5.

[0124]

[0128] Another record includes: the first identifier is user 2, the second identifier is server b, the format identifier of the access request is 164859231372452643K, the preset time period of the generation time of the access request is 12:05-12:10 on August 1, 2023, and the number of access requests is 3.

[0125]

[0129] As shown in the second materialized view of FIG11, one record corresponds to one set and also corresponds to one preset time period. Based on the preset historical time period exemplified in the embodiment shown in FIG5, the time period obviously corresponds to multiple records in the second materialized view. Therefore, optionally, the preset quantity threshold can also be determined by statistics.

[0126] Specifically, the log storage system may aggregate the number of historical requests in the multiple historical request sets selected from the second materialized view shown in FIG. 11 to obtain an aggregation result, and determine a preset number threshold based on the aggregation result. The aggregation result may include the quantile, maximum, and median of the number of historical requests in the historical request set. The definition of the historical request set and the method for determining the preset number threshold based on the aggregation result can be found in the methods shown in FIG. 6 and FIG. 7 , and will not be further described here.

[0127]

[0131] In addition, when the attribute information of the access request includes a format identifier, that is, for the target request set obtained by grouping based on this field, in order to more finely detect whether the number of target requests will cause abnormal server performance indicator values, the log storage system can also execute the following method.

[0128]

[0132] The present application embodiment provides a flow chart of another access abnormality cause detection method. As shown in FIG12, the method may include the following steps S501 to S505.

[0129]

[0133] S501, receiving a database access request.

[0130]

[0134] S502, target requests with the same attribute information in the access requests are determined as a target request set.

[0131]

[0135] The specific implementation process of the above steps S501-S502 can refer to the specific description of the relevant steps in the embodiment shown in Figure 1, which will not be repeated here.

[0132]

[0136] S503, according to the time window and the generation time of the target request, the target request set is divided into at least one subset.

[0133]

[0137] S504, according to the number of target requests in any subset, determine the detection result corresponding to any subset, and the detection result reflects whether the performance indicator value of the server is abnormal when responding to the target request in any subset.

[0134]

[0138] The log storage system may divide the detection period in the detection instruction into multiple sub-periods according to the time window. Optionally, the length of the time window may be manually set according to actual needs. Then, the log storage system may divide the target request set into at least one subset according to the generation time of the target request and the time window.

[0135] For example, assuming the detection period is 12:00-14:00, the log storage system can divide these two hours into 24 sub-periods based on a time window of, for example, 5 minutes. The sub-periods may include 12:00-12:05, 12:05-12:10, and so on. The log storage system can further divide the target request set obtained by the format identifier into at least one subset based on this time window and the generation time of the target request. Continuing with the above example, the target requests generated between 12:00-12:05 in the target request set are divided into one subset, and the target requests generated between 12:05-12:10 in the target request set are divided into another subset.

[0136]

[0140] Afterwards, the log storage system may further obtain the number of target requests in each subset, and determine the detection result corresponding to any subset based on the number of requests in the subset. The detection result may reflect whether the performance indicator value of the server is abnormal when responding to any subset.

[0137]

[0141] Optionally, the log storage system may compare the number of target requests in any subset with a preset number threshold. If the number of target requests in any subset is greater than the preset number threshold, it may be determined that the performance indicator value of the server deployed with the service database is abnormal when responding to the any subset; otherwise, it may be determined that the performance indicator value of the server deployed with the service database is normal when responding to the any subset.

[0138]

[0142] The preset number threshold may also aggregate the number of requests in the historical request set and determine the number based on the aggregated result. The specific determination process is similar to the process of the preset indicator threshold, and will not be repeated here.

[0139] To determine the number of target requests in any subset, the log storage system can optionally aggregate the contents of the base table records to generate a second materialized view containing the format identifier, the preset time period to which the access request generation time belongs, and the number of target requests. Specifically, the records in the second materialized view include the preset time period to which the access request generation time belongs, and the number of requests with the same format identifier generated within this preset time period. Furthermore, since the duration of the preset time period to which the access request generation time belongs is the same as the window duration, for example, 5 minutes, the preset time periods can also include 12:00-12:05, 12:05-12:10, and so on. Therefore, the log storage system can directly use this second materialized view to determine the number of target requests in each subset. The specific process of generating the base table can be found in the description of the above-mentioned related embodiments and will not be repeated here.

[0140]

[0144] S505: Determine whether the performance indicator value of the server is abnormal when responding to the target request set based on the detection results corresponding to different subsets.

[0141]

[0145] Ultimately, the log storage system can determine whether the performance indicator value of the server is abnormal when responding to the target request set based on the detection results corresponding to different subsets.

[0142]

[0146] In this embodiment, by dividing the target request set into at least one subset, the log storage system can use the subset as the granularity, treat any subset contained in the target request set as a whole, and detect whether the server performance indicator value will be abnormal in the process of responding to the subset, that is, detect whether responding to the subset is the cause of the server performance indicator value abnormality.

[0143]

[0147] Based on the embodiment shown in Figure 12, in order to more accurately determine the access request that causes the abnormal server performance indicator value and its generation time, different time windows can be used to divide the target request set generated within the detection period multiple times, and each division can obtain at least one subset.

[0144]

[0148] Following the example in the embodiment shown in FIG12 , for target requests generated within the detection period, according to multiple time windows included in a set of time windows, such as 12:00-12:05, 12:05-12:10, etc., the target requests generated between 12:00-12:05 in the target request set can be divided into one subset, the target requests generated between 12:05-12:10 can be divided into another subset, and so on.

[0145]

[0149] At this time, the log storage system can detect at least one subset divided according to the above time window to obtain the detection results of each subset, that is, to obtain the subset generated in which time window the abnormal server performance indicator value is caused.

[0146]

[0150] Meanwhile, another set of time windows may include 12:02:00-12:07:00, 12:07:00-12:12:00, and so on. The target requests generated between 12:02:00-12:07:00 may be divided into a subset, and the target requests generated between 12:07:00-12:12:00 may be divided into a subset, and so on.

[0147]

[0151] At this time, the log storage system can also determine the subset generated in which time window the abnormal server performance indicator value is caused.

[0148]

[0152] It should be noted that, according to the above description, the lengths of the time windows in different groups of time windows are the same, for example, they are all 5 minutes, but the start times of the time windows at the same position in different groups of time windows are different, for example, 12:00-12:05 and 12:02:00-12:07:00 are the time windows at the same position (i.e., the first one) in two groups of time windows, but their start times are different.

[0149] In practice, if a large number of target requests are received at the junction of two adjacent time windows within a set of time windows, such as at 12:05 in the above example, the number of target requests received at the junction of the two time windows may be inaccurate, further affecting the detection of abnormal causes. However, in this embodiment, the target request set can be divided into multiple groups of time windows, which can improve the above situation, thereby ensuring the accuracy of the number of requests counted and ultimately the accuracy of the abnormal cause detection.

[0150]

[0154] The above embodiments have described the process of detecting the cause of abnormal database access from the perspective of methods. For ease of understanding, the specific implementation process of detecting the cause of abnormal database access can be described below using a practical example.

[0151]

[0155] When the monitoring platform detects that an abnormality in the server performance indicator value occurs between 12:00 and 14:00 on December 8, 2023, the monitoring platform can generate a detection request, which includes the detection period 12:00 to 14:00 and the historical period November 8, 2023 to December 7, 2023.

[0156] The log storage system can first obtain access requests generated during the detection period based on the base table, and divide them into multiple sets to be detected based on the attribute information of the access requests. Furthermore, the log storage system can determine which of the multiple sets obtained by the above divisions are the causes of the abnormality in the server performance indicator from the perspectives of response performance indicators and the number of requests.

[0152]

[0157] For any set to be detected, from the perspective of the response index value, the log storage system can aggregate the response index values ​​of each access request in the set to be detected, and the aggregation result is the response index value of the set to be detected. At the same time, the log storage system can also aggregate the response index values ​​of historical requests generated within a historical period, and the aggregation result can be used as the response index value of the historical request set. An appropriate preset index threshold can also be determined based on the response index value of the historical request set. Ultimately, by comparing whether the response index value of any set to be detected exceeds the preset index threshold, it is determined whether any set to be detected causes the abnormality of the server's new performance index. Of course, the historical requests and the access requests in any set to be detected have the same attribute information. Optionally, the above-mentioned aggregation process can be performed based on the base table, and the aggregation result can be stored in a materialized view.

[0153] From a quantitative perspective, the log storage system can count the number of access requests in any set to be detected. It can also aggregate the number of historical requests within a historical period and determine an appropriate preset threshold based on the aggregation result. Ultimately, by comparing the number of access requests in any set to be detected with the preset threshold, it can be determined whether any set to be detected causes abnormal server performance indicators.

[0154]

[0159] In this embodiment, the method for determining the preset indicator threshold, the preset quantity threshold, and the detection method for judging whether the request set is the cause of the abnormal server performance indicator value can all be described in the relevant descriptions in the above embodiments and will not be repeated here.

[0155]

[0160] Figure 13 is a structural diagram of a detection system provided in an embodiment of the present application. As shown in Figure 13, the detection system includes: a database, a log storage system and a monitoring platform.

[0156]

[0161] The database may receive access requests from users based on actual needs. The access request may be a request to perform operations such as adding, deleting, modifying, and querying data in the database. Optionally, the access request may include an SQL statement and may also include attribute information.

[0157] The monitoring platform can automatically generate a detection request or generate a detection request in response to a human operation, and send the detection request to the log storage system, so that the log storage system can determine whether the performance index value of the server deployed with the database is abnormal when responding to the target request set based on the detection request. If the detection result of the log storage system is that the performance index value of the server is abnormal when responding to the target request set, the monitoring platform can display this abnormal detection result.

[0158]

[0163] To determine the detection result, the log storage system may, in response to receiving the detection request, identify target requests with the same attribute information in the access requests as a target request set. Optionally, the access request may be received by the log storage system at the same time as it is received by the database. Optionally, the attribute information may include a format identifier of the access request, which is used to indicate the category to which the access request belongs and the action content of the access request, such as querying access requests generated within a certain time period in a certain data page in the database.

[0159]

[0164] Thereafter, the log storage system may determine a detection result of whether the performance indicator value of the server deployed with the database is abnormal when responding to the target request set, that is, determining whether responding to the set is the cause of the abnormal performance indicator value of the server at the set granularity. Optionally, the performance indicator value of the server may include the CPU utilization rate of the server, the memory occupancy rate of the server, etc. If the CPU utilization rate and / or the memory occupancy rate is greater than a preset threshold value, it can be considered that the performance indicator value of the server is abnormal.

[0160]

[0165] In addition, the database in this embodiment is the service database in the above-described method embodiment.

[0166] In this embodiment, in response to receiving a detection request, the log storage system can identify target requests with the same attribute information from access requests to the database. These target requests can be considered to be a class of requests, which can constitute a target request set. The log storage system can then analyze this target request set to determine whether the performance indicator values ​​of the server deployed with the database are abnormal when responding to the target request set. If the log storage system determines that the performance indicator values ​​of the server are abnormal when responding to the target request set, the monitoring platform can display this anomaly detection result.

[0161]

[0167] In the above process, the log storage system can use the access request set as the granularity and treat the target request set as a whole to detect whether the server performance index value will be abnormal in the process of responding to the set, that is, to detect whether responding to the set is the cause of the abnormal server performance index value.

[0162]

[0168] In addition, the contents not described in detail in this embodiment and the technical effects that can be achieved can be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0163]

[0169] The following will describe in detail the detection device for the cause of access anomaly of one or more embodiments of the present application. Those skilled in the art will understand that these detection devices can be configured using commercially available hardware components through the steps taught in this solution.

[0164]

[0170] FIG14 is a schematic structural diagram of a device for detecting the cause of an access anomaly according to an embodiment of the present application. As shown in FIG14, the device may include the following modules.

[0165]

[0171] Receiving module 11, used to receive access requests from the database.

[0166]

[0172] The request set determination module 12 is used to determine the target requests with the same attribute information in the access requests as a target request set.

[0167]

[0173] The abnormality determination module 13 is used to determine whether the performance indicator value of the server deployed with the database is abnormal when responding to the target request set.

[0168]

[0174] Optionally, the abnormality determination module 13 is used to determine whether the performance indicator value of the server when responding to the target request set is abnormal based on the response indicator value of the target request set.

[0169]

[0175] Optionally, the abnormality determination module 13 is configured to determine that the performance indicator value of the server is abnormal when responding to the target request set if the response indicator value of the target request set is greater than a preset indicator threshold.

[0170]

[0176] Optionally, the device further includes: a storage module 14, configured to store the attribute information of the access request and the response indicator value of the access request as records in a base table of a log storage system.

[0171]

[0177] The first aggregation module 15 is used to aggregate the response index values ​​of the target requests in the base table to use the aggregation results as the response index values ​​of the target request set.

[0172]

[0178] Optionally, the device further includes: a materialized view module 16, configured to generate, based on the base table, a first materialized view including attribute information of the access request and a response indicator value of the request set.

[0173]

[0179] The second aggregation module 17 is used to aggregate the response indicator values ​​of the historical request set in the first materialized view, wherein the historical requests in the historical request set have the same attribute information as the target request, and the generation time of the historical requests is within a preset historical period.

[0174]

[0180] A threshold determination module 18 is used to determine the preset indicator threshold based on the aggregation result.

[0175]

[0181] Optionally, the aggregation result includes the quantile and maximum value of the response index value of the historical request set.

[0176]

[0182] The threshold determination module 18 is configured to determine an alternative threshold value based on the quantile, the maximum value, and a preset coefficient; and determine a preset indicator threshold value corresponding to the target request set from the alternative threshold values ​​based on the quantile.

[0183] Optionally, the aggregation result includes the quantile and the maximum value of the response indicator value of the historical request set.

[0177]

[0184] The threshold determination module 18 is used to determine the preset indicator threshold based on the quantile and quantile interval in the aggregation result.

[0178]

[0185] Optionally, the request set determination module 12 is used to obtain access requests generated within the detection period contained in the detection request in response to receipt of the detection request; in the base table, at least one record corresponding to the target request with the same attribute information generated within the detection period is determined as the target request set.

[0179]

[0186] Optionally, the device further includes: a comparison module 19, which is used to compare the quantiles of the response index value of the target request set and the response index value of the historical request set if the performance index value of the server is abnormal when responding to the target request.

[0180]

[0187] The abnormality degree determination module 20 is used to determine the abnormality degree corresponding to the target request set based on the comparison result, and the abnormality degree reflects the correlation between the server's response to the target request set and the abnormal performance indicator value of the server.

[0181]

[0188] Optionally, the abnormality determination module 13 is also used to determine whether the performance indicator value of the server is abnormal when responding to the target request set based on the number of target requests in the target request set.

[0182]

[0189] Optionally, the attribute information of the access request includes a format identifier of the access request.

[0183]

[0190] The device further includes: a subset division module 21, configured to divide the target request set into at least one subset according to a time window and a generation time of the target request.

[0184]

[0191] The abnormality determination module 13 is used to determine the detection result corresponding to any subset based on the number of target requests in any subset, and the detection result reflects whether the performance indicator value of the server is abnormal when responding to the target request set; based on the detection results corresponding to different subsets, determine whether the performance indicator value of the server is abnormal when responding to the target request set.

[0185]

[0192] In which, the number of the time windows is multiple groups, the lengths of the multiple time windows in each group of time windows are the same and the start times of the time windows at the same position in different groups of time windows are different.

[0186]

[0193] Optionally, the attribute information also includes: a preset time period to which the generation time of the access request belongs.

[0187]

[0194] The storage module 14 is also used to store the attribute information of the access request and the response index value of the access request as records in the base table of the log storage system.

[0188]

[0195] The materialized view module 16 is further configured to generate, based on the base table, a second materialized view containing attribute information of the access request and the number of the access requests.

[0189]

[0196] The device also includes: a quantity acquisition module 22, configured to acquire the quantity of target requests in any subset from the second materialized view.

[0190]

[0197] Optionally, the abnormality determination module 13 is further configured to determine that the performance indicator value of the server is abnormal when responding to the target request in any subset if the response indicator value of any subset is greater than a preset quantity threshold.

[0191]

[0198] Optionally, the second aggregation module 17 is further configured to aggregate the number of historical requests in the historical request set in the second materialized view, wherein the historical requests in the historical request set have the same attribute information as the target request, and the generation time of the historical requests is within a preset historical time period.

[0192]

[0199] The threshold determination module 18 is configured to determine the preset number threshold based on the aggregation result.

[0200] Optionally, the apparatus further includes: a format identifier determination module 23, configured to perform syntax analysis on the access request to obtain a syntax tree; perform format unification processing on original character strings in the access request that serve as nodes in the syntax tree; concatenate the processed original character strings to obtain a concatenated character string; and determine a hash calculation result of the concatenated character string as the format identifier of the access request.

[0193]

[0201] Optionally, the attribute information of the access request also includes at least one of the first identifier of the user who generates the access request, the second identifier of the server, the type of operation of the access request on the data in the database, and the preset time period to which the generation time of the access request belongs.

[0194]

[0202] The device shown in FIG14 can execute the method of the embodiments shown in FIG1 to FIG12. For parts not described in detail in this embodiment, reference can be made to the relevant description of the embodiments shown in FIG1 to FIG12. The execution process and technical effects of this technical solution are described in the embodiments shown in FIG1 to FIG12, and will not be repeated here.

[0195]

[0203] In a possible design, the access abnormality cause detection method provided in the above embodiments can be applied to an electronic device. As shown in FIG15, the electronic device may include: a processor 31 and a memory 32. The memory 32 is used to store a program that supports the electronic device to execute the access abnormality cause detection method provided in the embodiments shown in FIG1 to FIG12, and the processor 31 is configured to execute the program stored in the memory 32.

[0196]

[0204] The program includes one or more computer instructions, wherein the one or more computer instructions, when executed by the first processor 31, can implement the following steps: receiving a database access request; determining target requests having the same attribute information in the access request as a target request set; and determining whether a performance indicator value of a server deployed with the database is abnormal when responding to the target request set.

[0197]

[0205] Optionally, the processor 31 is also used to execute all or part of the steps in the embodiments shown in Figures 1 to 12 above.

[0198]

[0206] The structure of the electronic device may further include a communication interface 33 for the electronic device to communicate with other devices or communication systems.

[0199]

[0207] In addition, an embodiment of the present application provides a computer storage medium for storing computer software instructions used by the above-mentioned electronic device, which includes a program involved in executing the method for detecting the cause of access anomaly shown in Figures 1 to 12 above.

[0200]

[0208] In addition, an embodiment of the present application provides a computer program product. The computer program product includes a computer program or instructions. When the computer program or instructions are executed by a processor, the processor is enabled to implement the steps or functions of the method for detecting the cause of an access anomaly shown in Figures 1 to 12 above.

[0201]

[0209] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

Claims 1. A method for detecting the cause of access exception, wherein, Including: Receiving an access request to a database; Determining target requests with the same attribute information in the access request as a target request set; Determining whether the performance metric value is abnormal when the server deploying the database responds to the target request set.

2. The method according to claim 1, wherein, The determining whether the performance metric value is abnormal when the server deploying the database responds to the target request set includes: If the response metric value of the target request set is greater than a preset metric threshold, determining that the performance metric value is abnormal when the server responds to the target request set.

3. The method according to claim 2, wherein The method further includes: Storing the attribute information of the access request and the response metric value of the access request as a record in the base table of the log storage system; Aggregating the response metric values of the target requests in the base table to use the aggregation result as the response metric value of the target request set.

4. The method according to claim 3, wherein The method further includes: Generating a first materialized view including the attribute information of the access request and the response metric value of the request set according to the base table; Aggregating the response metric values of the historical request sets in the first materialized view, where the historical requests in the historical request sets have the same attribute information as the target requests, and the generation time of the historical requests is within a preset historical period; Determining the preset metric threshold according to the aggregation result.

5. The method according to claim 4, wherein The aggregation result includes the quantiles and extreme values of the response metric values of the historical request sets; The determining the preset metric threshold according to the aggregation result includes: Determining alternative thresholds according to the quantiles, the extreme values, and a preset coefficient; Determining the preset metric threshold corresponding to the target request set from the alternative thresholds according to the quantiles.

6. The method according to claim 3, wherein The determining target requests with the same attribute information in the access request as a target request set includes: In response to receiving a detection request, obtaining the access requests generated during the detection period included in the detection request; In the base table, determining at least one record corresponding to the target requests with the same attribute information generated during the detection period as the target request set.

7. The method according to claim 4, wherein The method further includes: If the performance metric value is abnormal when the server responds to the target request, comparing the response metric value of the target request set with the quantiles of the response metric values of the historical request sets; Determining the abnormal degree corresponding to the target request set according to the comparison result, where the abnormal degree reflects the correlation between the server's response to the target request set and the abnormal performance metric value of the server.

8. The method according to claim 1, wherein The determining whether the performance metric value is abnormal when the server deploying the database responds to the target request set includes: Determining whether the performance metric value is abnormal when the server responds to the target request set according to the number of target requests in the target request set.

9. The method according to claim 8, wherein The attribute information of the access request includes the format identifier of the access request; The method further includes: Dividing the target request set into at least one according to a time window and the generation time of the target requests Subset; determining whether the performance metric value is abnormal when the server responds to the target request set according to the number of target requests in the target request set includes: if the number of target requests in any subset is greater than a preset number threshold, determining the detection result corresponding to the any subset, where the detection result reflects that the performance metric value is abnormal when the server responds to the target requests in the any subset; determining whether the performance metric value is abnormal when the server responds to the target request set according to the detection results corresponding to different subsets respectively.

10. The method according to claim 9, wherein, The number of the time windows is multiple groups, and the lengths of the multiple time windows in each group of time windows are the same and the start times of the time windows at the same position in different groups of time windows are different.

11. The method according to claim 9, wherein, The attribute information further includes: a preset time period to which the generation time of the access request belongs; the method further includes: storing the attribute information of the access request and the response metric value of the access request as a record in a base table of a log storage system; generating a second materialized view including the attribute information of the access request and the number of the access requests according to the base table; obtaining the number of target requests in the any subset from the second materialized view.

12. The method according to claim 11, wherein, The method further includes: aggregating the number of historical requests in a historical request set in the second materialized view, where the historical requests in the historical request set have the same attribute information as the target requests, and the generation time of the historical requests is within a preset historical time period; determining the preset number threshold according to the aggregation result.

13. A detection system, wherein, Including: A database, a log storage system, and a monitoring platform; The database is configured to receive access requests; The monitoring platform is configured to send detection requests; Displaying a detection result reflecting whether the performance metric value is abnormal when the server deploying the database responds to a target request set; the log storage system is configured to receive the access requests; In response to receiving the detection request, determining target requests with the same attribute information in the access requests as the target request set; Determining the detection result.

14. An electronic device, wherein, Including: A memory and a processor; wherein, an executable code is stored on the memory, and when the executable code is executed by the processor, the processor is caused to execute the detection method for the cause of access exception according to any one of claims 1 to 12.

15. A non-transitory machine-readable storage medium, wherein, An executable code is stored on the non-transitory machine-readable storage medium, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the detection method for the cause of access exception according to any one of claims 1 to 12.

16. A computer program product, wherein Including a computer program or instruction, and when the computer program or instruction is executed by a processor, the processor is caused to be able to implement the steps in the detection method for the cause of access exception according to any one of claims 1 to 12.

Images