A data processing method, device, equipment and computer readable storage medium

Abstract

The present disclosure provides a data processing method, apparatus, device and computer-readable storage medium. The method includes: obtaining a monitoring value group of a first monitoring instance; when the quantile parameter N of the first monitoring instance is less than or equal to a first preset value, calculating the N quantile value of the monitoring value group with a first calculation strategy; otherwise, calculating the N quantile value of the monitoring value group with a second calculation strategy; searching for the first monitoring result corresponding to the first monitoring instance in the first mapping relationship managed by the first management tool corresponding to the first monitoring item; calling the first management tool to store the N quantile value or the address of the N quantile value in the first monitoring result. Compared with the prior art, the embodiments of the present disclosure can improve the monitoring effect of the system operation status, help users better understand the system operation status, and can better assist in locating online problems.

Description

Technical Field

[0001] The present disclosure relates to the field of Internet technology, and in particular to a data processing method, device, equipment and computer-readable storage medium. Background Art

[0002] In the field of Internet technology, the application of monitoring and statistical technology is becoming more and more common. Monitoring and statistical technology refers to counting and counting some content in the program to observe and monitor the operation of the system.

[0003] At present, when using monitoring statistics technology, generally only overall monitoring statistics at the monitoring item level are performed. In this way, the monitoring granularity is very rough, resulting in poor monitoring effect on system operation status in the existing technology. Summary of the invention

[0004] The embodiments of the present disclosure provide a data processing method, apparatus, device and computer-readable storage medium to solve the problem of poor monitoring effect on system operation status in the prior art.

[0005] In order to solve the above technical problems, the present disclosure is implemented as follows:

[0006] In a first aspect, an embodiment of the present disclosure provides a data processing method, including:

[0007] Obtaining a monitoring value group of a first monitoring instance;

[0008] When the quantile parameter N of the first monitoring instance is less than or equal to a first preset value, the N quantile value of the monitoring value group is calculated using a first calculation strategy; otherwise, the N quantile value of the monitoring value group is calculated using a second calculation strategy;

[0009] In a first mapping relationship managed by a first management tool corresponding to a first monitoring item, searching for a first monitoring result corresponding to the first monitoring instance; wherein the first monitoring item is a monitoring item to which the first monitoring instance belongs, and the first mapping relationship is a mapping relationship between each monitoring instance under the first monitoring item and a corresponding monitoring result;

[0010] Call the first management tool to store the N quantile value or the address of the N quantile value in the first monitoring result; wherein the address of the N quantile value is the address where the N quantile value is stored after the N quantile value is calculated.

[0011] In a second aspect, an embodiment of the present disclosure provides a data processing device, including:

[0012] An acquisition module, used to obtain a monitoring value group of a first monitoring instance;

[0013] a first processing module, configured to calculate the Nth quantile of the monitoring value group using a first calculation strategy when the quantile parameter N of the first monitoring instance is less than or equal to a first preset value; otherwise, calculate the Nth quantile of the monitoring value group using a second calculation strategy;

[0014] A search module, used to search for a first monitoring result corresponding to the first monitoring instance in a first mapping relationship managed by a first management tool corresponding to the first monitoring item; wherein the first monitoring item is a monitoring item to which the first monitoring instance belongs, and the first mapping relationship is a mapping relationship between each monitoring instance under the first monitoring item and a corresponding monitoring result;

[0015] A storage module is used to call the first management tool to store the N quantile value or the address of the N quantile value in the first monitoring result; wherein, the address of the N quantile value is the address where the N quantile value is stored after the N quantile value is calculated.

[0016] In a third aspect, an embodiment of the present disclosure provides a data processing device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the above-mentioned data processing method when executed by the processor.

[0017] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned data processing method are implemented.

[0018] In a fifth aspect, the present disclosure provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned data processing method when executed by a processor.

[0019] In the embodiment of the present disclosure, when the monitoring value group of the first monitoring instance is obtained, the N quantile value of the monitoring value group can be calculated according to the comparison result of the quantile value parameter N of the first monitoring instance and the first preset value with the corresponding calculation strategy. Next, the first monitoring result corresponding to the first monitoring instance can be found through the search operation, and the N quantile value or the address of the N quantile value can be stored in the first monitoring result through the management tool call operation. Afterwards, the N quantile value can be found very conveniently according to the information stored in the first monitoring result. It can be seen that in the embodiment of the present disclosure, for data of numerical type, monitoring statistics at the monitoring instance level can be performed. Compared with the prior art, the monitoring granularity in the embodiment of the present disclosure is finer. In addition, when performing monitoring statistics at the monitoring instance level, the calculation strategy used when calculating the N quantile value can be different according to performance requirements. Then, the calculation accuracy of the N quantile value can be guaranteed. Therefore, compared with the prior art, the embodiment of the present disclosure can improve the monitoring effect of the system operation, help users better understand the system operation, and can better assist in locating online problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the description of the embodiments of the present disclosure will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flowchart of a data processing method provided by an embodiment of the present disclosure;

[0022] Figure 2 A structural block diagram of a data processing device provided in an embodiment of the present disclosure;

[0023] Figure 3 A schematic diagram of the structure of a data processing device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0025] The following first describes the data processing method provided by the embodiment of the present disclosure.

[0026] It should be noted that the data processing method provided in the embodiments of the present disclosure is applied to a data processing device. Here, the data processing device may be an electronic device with data processing capabilities, such as a server. Of course, the type of data processing device is not limited thereto and may be determined based on actual conditions. The embodiments of the present disclosure do not impose any limitation on this.

[0027] See also Figure 1 , which shows a flow chart of a data processing method provided by an embodiment of the present disclosure. Figure 1 As shown, the method comprises the following steps:

[0028] Step 101: Obtain a monitoring value group of a first monitoring instance.

[0029] It should be noted that when using monitoring statistics technology, there may be multiple monitoring items, each monitoring item may have multiple dimensions, and based on the multiple dimensions, multiple monitoring instances under the monitoring item may be formed; wherein, the instance may also be referred to as instance. Here, the multiple monitoring items may include monitoring items corresponding to the rate type, such as the query rate per second (Query Per Second, qps), and monitoring items corresponding to the numerical type, such as request delay, number of users, etc.

[0030] Specifically, for the monitoring item qps, it can have two dimensions: user and request type. Users can include user1 and user2, and request types can include read and write. Then, through permutations and combinations, four monitoring instances under qps can be obtained, namely: user1's read request, user2's write request, user2's read request, and user2's write request. Since qps corresponds to the rate type, it can be accumulated within a monitoring statistical cycle. At the end of the cycle, a result of the accumulated value / cycle seconds is output, and then the accumulated value is cleared to zero, and the next monitoring statistical cycle is entered to restart the statistics. For example, if the statistical cycle is 1 minute and the request is called 120 times, then qps is 120 / 60 seconds = 2.

[0031] Similarly, for the monitoring item of request delay, it can also have two dimensions: user and request type. Users can include user3, and request types can include read and write. Then, through permutations and combinations, two monitoring instances under the monitoring item of request delay can be obtained, namely: user3's read request and user3's write request. Since request delay corresponds to a numerical type, several items including sum (accumulated value), min (minimum value), max (maximum value), avg (average value), and last_value (last value) can be retained for request delay. Among them, except last_value, the others will be cleared as the monitoring statistical cycle changes. What can be seen in the end is the maximum value, minimum value, average value, etc. of the request delay within one minute; since the calculation of the average value requires sum / number of times (count), the actual data structure may not store avg, but count, and avg can be calculated at the end of the monitoring statistical cycle.

[0032] It should be noted that the first monitoring instance can be any monitoring instance under a monitoring item corresponding to the value type. For example, the first monitoring instance can be a read request of user3, or a write request of user3. Assuming that user3 initiates 1000 read requests in a monitoring statistical cycle, the monitoring value group of user3's read request is: 1000 delays corresponding to 1000 read requests; assuming that user3 initiates 500 write requests in a monitoring statistical cycle, the monitoring value group of user3's write request is: 500 delays corresponding to 500 write requests.

[0033] Step 102, when the percentile parameter N of the first monitoring instance is less than or equal to the first preset value, calculate the N percentile value of the monitoring value group using the first calculation strategy; otherwise, calculate the N percentile value of the monitoring value group using the second calculation strategy.

[0034] It should be noted that the quantile value involved in the embodiments of the present disclosure may be a percentile value (which may also be referred to as percentile), and both N and the first preset value need to be greater than 0 and less than 1. Specifically, N may be 50%, 80%, 99%, 99.9%, 99.99%, 99.999%, or 99.9999%, and the first preset value may be 99.9%. Of course, the values ​​of N and the first preset value are not limited thereto, and may be determined specifically according to actual conditions, and are not listed one by one here.

[0035] Here, the first calculation strategy can be a pre-set quantile calculation strategy that matches low performance requirements; the second calculation strategy can be a pre-set quantile calculation strategy that matches high performance requirements. In step 102, N can be compared with a first preset value (assuming it is 99.9%); when N is greater than 99.9%, for example, when N is 99.9999%, since a system with high performance requirements will require a delay of 99.9999%, then the quantile calculation can be performed with the second calculation strategy that matches high performance requirements; otherwise, the quantile calculation can be performed with the first calculation strategy that matches low performance requirements.

[0036] Step 103, searching for the first monitoring result corresponding to the first monitoring instance in the first mapping relationship managed by the first management tool corresponding to the first monitoring item; wherein the first monitoring item is the monitoring item to which the first monitoring instance belongs, and the first mapping relationship is the mapping relationship between each monitoring instance under the first monitoring item and the corresponding monitoring result.

[0037] It should be noted that a globally unique MetricManager can be set in the data processing device. As the only entry for global calls, MetricManager can save the mapping relationship between monitoring items and management tools (which can also be called Metric) (for the sake of distinction, it will be referred to as the second mapping relationship later). The second mapping relationship can specifically be the mapping relationship between the monitoring item name and the management tool. Here, the management tool corresponding to any monitoring item can be used to manage a mapping relationship, which is the mapping relationship between each monitoring instance under the monitoring item and the corresponding monitoring result; wherein, each monitoring result can be represented by a counter, that is, each monitoring instance can uniquely correspond to a counter, and the counter can be used as a basic counter.

[0038] In step 103, the first monitoring item to which the first monitoring instance belongs can be determined first, and then the first management tool corresponding to the monitoring item name of the first monitoring item can be found in the second mapping relationship. Next, the counter corresponding to the first monitoring instance (assuming it is the first counter) can be found in the first mapping relationship.

[0039] Step 104, calling the first management tool to store the N quantile value or the address of the N quantile value in the first monitoring result; wherein the address of the N quantile value is the address where the N quantile value is stored after the N quantile value is calculated.

[0040] In step 104, the first management tool can be directly called to store the N percentile value calculated in step 102 in the first counter; or, the N percentile value calculated in step 102 can be first stored in a specific location, the stored address can be recorded, and then the first management tool can be called to store the recorded address in the first counter.

[0041] In the embodiment of the present disclosure, when the monitoring value group of the first monitoring instance is obtained, the N quantile value of the monitoring value group can be calculated according to the comparison result of the quantile value parameter N of the first monitoring instance and the first preset value with the corresponding calculation strategy. Next, the first monitoring result corresponding to the first monitoring instance can be found through the search operation, and the N quantile value or the address of the N quantile value can be stored in the first monitoring result through the management tool call operation. Afterwards, the N quantile value can be found very conveniently according to the information stored in the first monitoring result. It can be seen that in the embodiment of the present disclosure, for data of numerical type, monitoring statistics at the monitoring instance level can be performed. Compared with the prior art, the monitoring granularity in the embodiment of the present disclosure is finer. In addition, when performing monitoring statistics at the monitoring instance level, the calculation strategy used when calculating the N quantile value can be different according to performance requirements. Then, the calculation accuracy of the N quantile value can be guaranteed. Therefore, compared with the prior art, the embodiment of the present disclosure can improve the monitoring effect of the system operation, help users better understand the system operation, and can better assist in locating online problems.

[0042] Optionally, using a first calculation strategy, calculating the Nth quantile of the monitoring value group includes:

[0043] Determine a corresponding container for each monitoring value in the monitoring value group from a plurality of containers arranged in sequence; wherein each container includes a plurality of storage locations arranged in sequence;

[0044] For each monitoring value in the monitoring value group, add it to the corresponding storage position in the corresponding container according to whether there is an idle storage position in the corresponding container;

[0045] According to N and the storage information of the monitoring value group, calculate the Nth percentile value of the monitoring value group.

[0046] Here, the container may be a barrel, the number of barrels may be 64, and the storage location in the container may be a slot. Of course, the type and number of the container and the type of the storage location are not limited thereto and may be determined according to actual conditions, and are not listed here one by one. For ease of understanding, the subsequent embodiments are described by taking the case where the container is a barrel, the number of barrels is 64, and the storage location is a slot as an example.

[0047] Here, the storage information of the monitoring value group may at least represent the total number of monitoring values ​​added to each bucket.

[0048] In this embodiment, for a system with low performance requirements, the monitoring values ​​in the monitoring value group can be stored in a bucketing manner, and the N quantile values ​​can be conveniently calculated based on the bucketing results.

[0049] Optionally, each container is provided with a corresponding monitoring value range. Here, there may be a one-to-one correspondence between the container and the monitoring value range.

[0050] From the plurality of containers arranged in sequence, a corresponding container is determined for each monitoring value in the monitoring value group, including:

[0051] For the first monitoring value C, logD(C) is calculated; wherein C is any monitoring value in the monitoring value group, and D is a second preset value;

[0052] Determine the container corresponding to the monitoring value range to which logD(C) belongs;

[0053] The determined container is used as the container determined for C.

[0054] Here, D can be 2. Of course, D can also be 3, 4 or other values, which are not listed here. For ease of understanding, this embodiment is described by taking the case where D is 2 as an example.

[0055] Here, 64 sequentially arranged buckets may be pre-set, and the numbers of the 64 buckets may be 0, 1, 2, ..., 63, respectively; wherein the monitoring value range corresponding to the bucket numbered 0 (i.e., the first bucket) may be [0, 2 1 ), the monitoring value range corresponding to the bucket numbered 1 (i.e. the second bucket) can be [2 1 , 2 2 ), the monitoring value range corresponding to the bucket numbered 2 (i.e. the third bucket) can be [2 2 , 2 3 ), and so on, so that the maximum supported is (2 64 -1).

[0056] After calculating logD(C) for the first monitoring value C, logD(C) can be compared with all monitoring value ranges to determine the container corresponding to the monitoring value range to which logD(C) belongs. Specifically, when logD(C) is 0 or 1, the container corresponding to the monitoring value range to which logD(C) belongs is the first bucket; when logD(C) is 2 or 3, the container corresponding to the monitoring value range to which logD(C) belongs is the second bucket; when logD(C) is 4, 5, 6 or 7, the container corresponding to the monitoring value range to which logD(C) belongs is the third bucket.

[0057] After determining the container corresponding to the monitoring value range to which logD(C) belongs, the determined container can be directly used as the container determined for C. Then, when the determined container is the first bucket, C can be added to the first bucket. When the determined container is the third bucket, C can be added to the third bucket.

[0058] It can be seen that by adopting the above implementation, it is possible to very conveniently determine a corresponding container for each monitored value.

[0059] Optionally, for each monitoring value in the monitoring value group, according to whether there is a storage location in a free state in the corresponding container, adding it to a corresponding storage location in the corresponding container includes:

[0060] For the first monitoring value, determining whether there is a storage location in an idle state in the first container corresponding to the first monitoring value; wherein the first monitoring value is any monitoring value in the monitoring value group;

[0061] When there is an idle storage location in the first container, the first monitoring value is filled into the idle storage location; otherwise, the first monitoring value is overwritten into the occupied storage location.

[0062] Here, the idle state and the occupied state are two relative states of the storage location. When the storage location is in the idle state, no data is stored in the storage location. When the storage location is in the occupied state, data is stored in the storage location.

[0063] After determining the first container for the first monitoring value, it can be determined whether there are slots in the first container that are in an idle state. If the judgment result is yes, it means that the slots in the first container are not filled, and then the slots can be directly filled; if the judgment result is no, it means that the slots in the first container are already filled, and then they can be covered from the beginning.

[0064] Specifically, each bucket may include 256 slots arranged in sequence. After determining the corresponding container (assuming it is the third bucket) for the first monitoring value C, it may be determined whether there is an idle slot in the third bucket.

[0065] If the result is yes, and only the first 150 slots in the third bucket are occupied, then C can be filled into the 151st slot in the third bucket. Afterwards, if the container corresponding to the next monitoring value is still the third bucket, the next monitoring value can be filled into the 152nd slot in the third bucket, and so on.

[0066] If the result of the determination is no, then C can be overwritten to the first slot in the third bucket. Afterwards, if the container corresponding to the next monitoring value is still the third bucket, the next monitoring value can be overwritten to the second slot in the third bucket, and so on. No further details are given here.

[0067] It can be seen that by adopting the above implementation mode, each monitoring value can be added to the corresponding storage position in the corresponding container very conveniently, and in the entire monitoring value storage process, the data is not sorted but randomly overwritten. That is to say, although some monitoring values ​​are discarded, some data in each value range are retained, and the overall distribution of the monitoring values ​​is retained, which can ensure the accuracy of the subsequently calculated N percentile values.

[0068] Optionally, calculating the Nth quantile of the monitoring value group according to N and the storage information of the monitoring value group includes:

[0069] Calculate the product F of the total number E of monitoring values ​​in the monitoring value group and N;

[0070] Determine the total number of monitoring values ​​added to each container;

[0071] Determine a second container; wherein the first total number is less than F, the sum of the first total number and the second total number is greater than or equal to F, the first total number is the total number of monitoring values ​​added to the containers sorted before the second container, and the second total number is the total number of monitoring values ​​added to the second container;

[0072] The Nth percentile value of the monitoring value group is calculated based on the difference between F and the first total number, the second total number, and the total number of storage locations in the second container that are in an occupied state.

[0073] It should be noted that during the entire monitoring value storage process, for each container, the monitoring values ​​added thereto can be counted. Specifically, whenever a new monitoring value is added to a container, 1 can be added to the current count value corresponding to the container. In this way, the total number of monitoring values ​​added to each container can be obtained very conveniently based on the final count value corresponding to each container.

[0074] Here, calculating the Nth percentile value of the monitoring value group according to the difference between F and the first total number, the second total number, and the total number of storage locations in the second container that are in an occupied state may include:

[0075] G is calculated using the formula G=S / I*J, where S is the difference between F and the first total number, I is the second total number, and J is the total number of occupied storage locations in the second container;

[0076] When G is an integer, the monitoring value at the Gth storage position in the second container is taken as the Nth quantile of the monitoring value group; otherwise, linear interpolation is performed on the monitoring values ​​at the G1th storage position and the G2th storage position in the second container to obtain the Nth quantile of the monitoring value group; wherein G1 is the result of rounding down G, and G2 is the result of rounding up G.

[0077] To facilitate understanding, the specific implementation process of this embodiment is described below with a specific example.

[0078] Assuming N is 99.9%, the total number of monitored values ​​E in the monitored value group is 1000, then the product F of E and N is 999. In addition, assuming that according to the final count values ​​corresponding to each bucket, the total number of monitored values ​​added to the first bucket is 300, and the total number of monitored values ​​added to the second bucket is 700. Since 300 is less than 1000, and the sum of 300 and 700 is equal to 1000, the second bucket is used as the second container above, and the difference between F and 300 is 699, that is, S = 699. Next, S = 699, I = 700, and J = 256 can be substituted into the formula G = S / I*J, and G = 255.63 can be obtained.

[0079] Since G is not an integer, we can determine the downward rounding result G1 of G and the upward rounding result G2 of G. It is easy to see that G1 is 255 and G2 is 256. Then, we can perform linear interpolation on the two monitoring values ​​currently located in the 255th slot and the 256th slot of the second bucket to obtain the Nth percentile value of the monitoring value group.

[0080] It should be noted that if G is an integer, such as 255, the monitoring value currently located at the 255th slot of the second bucket can be directly used as the Nth percentile value of the monitoring value group.

[0081] It can be seen that in this embodiment, by using the bucketing method and combining it with the proportional scaling operation, the Nth percentile value under low performance requirements can be calculated very conveniently.

[0082] Optionally, using a second calculation strategy, calculating the Nth quantile of the monitoring value group includes:

[0083] Calculate the total number E of monitoring values ​​in the monitoring value group and the ratio K of the preset total number corresponding to N;

[0084] When K is less than or equal to 1, the Nth quantile of the monitoring value group is calculated using the first sub-strategy; otherwise, the Nth quantile of the monitoring value group is calculated using the second sub-strategy.

[0085] Here, the preset total number corresponding to N is 100000 or 1000000. Of course, the value of the preset total number is not limited to this, and is not listed here one by one. For ease of understanding, the following embodiments are all described by taking the case where the preset total number is 1000000 as an example.

[0086] Optionally, using the first sub-strategy, calculating the Nth quantile of the monitoring value group includes:

[0087] Calculate the product F of E and N;

[0088] When F is an integer, the monitoring value with the Fth largest value in the monitoring value group is taken as the Nth percentile value of the monitoring value group; otherwise, linear interpolation is performed on the monitoring values ​​with the F1th largest value and the F2th largest value in the monitoring value group to obtain the Nth percentile value of the monitoring value group; wherein F1 is the result of rounding down F, and F2 is the result of rounding up F.

[0089] Here, when E is 1000000 (i.e. 1 million), if the high-performance demand system requires an index of 6 nines (i.e. N is 99.9999%), and the product of E and N F is 999999, then when the 1000000 monitoring values ​​are arranged in order from small to large, the N quantile value is located at the 999999th positive number, that is, the N quantile value is the 999999th largest monitoring value, that is, the N quantile value is located at the second to last place; if the high-performance demand system requires an index of 5 nines (i.e. N is 99.999%), and the product of E and N F is 999990, then the 1000000 monitoring values ​​are arranged in order from small to large, and the N quantile value is located at the 999999th positive number, that is, the N quantile value is the 999999th largest monitoring value, that is, the N quantile value is located at the second to last place; When the 1000000 monitoring values ​​are arranged in order from small to large, the N quantile value is located at the 999990th positive number, that is, the N quantile value is the 999990th largest monitoring value, that is, the N quantile value is located at the 11th from the bottom; if the high-performance demand system requires an indicator of 4 9s (that is, N is 99.99%), the product F of E and N is 999900, then when the 1000000 monitoring values ​​are arranged in order from small to large, the N quantile value is located at the 999900th positive number, that is, the N quantile value is the 999990th largest monitoring value, that is, the N quantile value is located at the 101st from the bottom. Then, for the case where E is 1000000 and N is 99.9999%, 99.999% or 99.99%, only 101 monitoring values ​​need to be saved at most.

[0090] Here, when E is 900000 (900,000), if the high-performance demand system requires an indicator of 6 9s (that is, N is 99.9999%), the product F of E and N is 899999.1. Since 899999.1 is rounded down to 899999 and rounded up to 900000, the 899999th and 900000th largest monitoring values ​​among the 900,000 monitoring values ​​can be linearly interpolated, that is, the two smallest monitoring values ​​among the 900,000 monitoring values ​​can be linearly interpolated to obtain the N percentile value.

[0091] It can be seen that this implementation method can very conveniently calculate the Nth percentile value under high performance requirements.

[0092] Optionally, using a second sub-strategy, calculating the Nth quantile of the monitoring value group includes:

[0093] The monitoring value group is divided into R groups in order of value size; wherein, in each of the first R-1 groups, the total number of monitoring values ​​is the preset total number;

[0094] Calculate the product M of the preset total number and N, and for each of the first R-1 groups, take the Mth largest monitoring value as the Nth percentile value of this group;

[0095] For the Rth group, calculate the product P of the total number of monitoring values ​​included in it and N; when P is an integer, take the monitoring value with the Pth largest value in this group as the Nth quantile value of this group; otherwise, perform linear interpolation on the monitoring values ​​with the P1th largest value and the P2th largest value in this group to obtain the Nth quantile value of this group; where P1 is the result of rounding down P, and P2 is the result of rounding up P;

[0096] The average of the R N quantile values ​​of the R groups is taken as the N quantile value of the monitoring value group.

[0097] To facilitate understanding, the specific implementation process of this embodiment is described below with a specific example.

[0098] Assuming that N is 99.9999%, the preset total number corresponding to N is 1000000, and E is 2010000, the monitored values ​​can be divided into 3 groups according to the order of numerical value; among them, the total number of monitored values ​​in the first group is 1000000, the total number of monitored values ​​in the second group is 1000000, and the total number of monitored values ​​in the third group is 10000.

[0099] Next, the product M of 1000000 and 99.9999% can be calculated, that is, M is 999999. Then, for the first or second group, the 999999th largest (i.e. the second to last) monitoring value in this group can be used as the Nth percentile value of this group.

[0100] In addition, the product P of 10000 and 99.9999% can also be calculated, that is, P is 9999.99. Since P is not an integer, the rounding result P1 and the rounding result P2 of P can be determined first. Obviously, P1 is 9999 and P is 10000. Then the 9999th and 10000th largest monitoring values ​​in the third group can be linearly interpolated to obtain the Nth percentile value of the third group. It should be pointed out that if P is an integer, such as 9999, the 9999th largest monitoring value in the third group can be directly used as the Nth percentile value of the third group.

[0101] Finally, the Nth quantile value of the first group, the Nth quantile value of the second group, and the Nth quantile value of the third group can be calculated to obtain the Nth quantile value of the entire monitoring value group.

[0102] It can be seen that this implementation method can very conveniently calculate the Nth percentile value under high performance requirements.

[0103] It is easy to see that regardless of whether K is greater than 1, the N quantile value can be easily calculated based on the first sub-strategy or the second sub-strategy.

[0104] Optionally, each group of mapping relationships in the first mapping relationship serves as a node in the first hop list;

[0105] The method further includes:

[0106] When the proportion of nodes satisfying the preset deletion condition in the first jump table is greater than the preset proportion, a new second jump table is created, and the nodes in the first jump table that do not satisfy the preset deletion condition are copied to the second jump table;

[0107] After the copying is completed, the first jump table is locked, and the second jump table is updated accordingly according to the update of the first jump table during the copying process;

[0108] After the update is completed, the lock on the first jump table is released, the first jump table is controlled to exit the service state, and the second jump table is controlled to enter the service state.

[0109] Here, the skip list can also be called SkipList. The SkipList retains the pointer and is lock-free, that is, SkipList can be lock-free for ordinary read and write queries, and SkipList will not have a significant impact on performance. In addition, SkipList does not support deletion operations. The preset ratio can be 30%, 40%, 50% or other values, which are not listed here one by one.

[0110] It should be noted that the user can initiate a request to the data processing device for updating the first jump table according to actual needs; wherein the request for updating the first jump table can be an addcounter request (i.e., a request to add a node to the first jump table), a deletecounter request (i.e., a request to delete a node from the first jump table), or an updatecounter request (i.e., a request to update an existing node in the first jump table). Assuming that user1 exits the system, the user can initiate a deletecounter request related to user1, and at this time, the data processing device can determine that the node related to user1 meets the preset deletion condition.

[0111] Here, the proportion of nodes in the first jump table that meet the preset deletion condition (i.e., the ratio of the number of nodes that meet the preset deletion condition to the total number of nodes in the first jump table) can be determined according to the set time interval, and the proportion can be compared with the preset proportion. In the case where the proportion is greater than the preset proportion, it can be considered that there are many nodes that need to be deleted in the first jump table, then a new second jump table can be created, and the first jump table can be traversed and searched to determine the nodes in the first jump table that do not meet the preset deletion condition, and the determined nodes are all copied (which can be considered as one copy) to the second jump table.

[0112] It should be noted that during a copy process, the first jump table is still in a service state, that is, the user can call a thread to access the first jump table to update the first jump table, so the content of the first jump table that has been copied to the second jump table will change. In view of this, after a copy is completed, the first jump table can be locked and traversed again to determine the update of the first jump table during the copy process, and according to the update, the second jump table is updated in the same way. During the update of the second jump table, since the first jump table is in a locked state, the first jump table cannot change.

[0113] After the second jump table is updated, the lock on the first jump table can be released, the first jump table can be controlled to exit the service state, and the second jump table can be controlled to enter the service state. At this time, the second jump table takes over the first jump table to provide services to the outside.

[0114] It should be noted that each set of mapping relationships in the second mapping relationship can be used as a node in the third jump table. In specific implementation, the add, set or subtract method of MetricManager can be called to pass metricname (name of the metric), labels (which are used to characterize the monitoring instance) and specific value into the third jump table.

[0115] It should be noted that the design of Metric does not support deletion, that is, the monitoring items themselves do not support deletion, and the demand for monitoring items will not increase or decrease in the code as the system operates. Therefore, if you want to count qps, you need to plan it from the beginning, for example, write the mapping relationship between qps and the corresponding Metric in the second mapping relationship from the beginning. However, the monitoring instance needs to support dynamic creation and deletion. For example, for qps monitoring, if user1 exits the system, the counter of the instance related to user1 needs to be deleted. Therefore, the second mapping relationship can directly use SkipList, but for the first mapping relationship, the mapping relationship needs to be deleted, so it is necessary to design an efficient deletion method.

[0116] In view of this, in this embodiment, when the SkipList is used in the first mapping relationship, if the proportion of nodes to be deleted in the SkipList is greater than the preset proportion, a new SkipList can be created, and the old SkipList can be traversed to copy the nodes in the old SkipList that do not meet the preset deletion conditions to the new SkipList. Since new writes may be concurrently written to the old SkipList during the copying process, after the copying is completed, the old SkipList can be locked and traversed twice to complete the synchronization of the new and old SkipLists. After the synchronization is completed, the lock on the old SkipList can be released, the old SkipList is no longer accessible, and the new SkipList officially provides services, thus realizing the deletion of the mapping relationship.

[0117] It should be pointed out that the above embodiment only traverses the old SkipList twice, and the number of traversals may also be three or more. For example, only traversal is performed during the first two traversals, and locking is performed synchronously during the third traversal, which is also feasible.

[0118] It should be pointed out that not all monitoring items corresponding to numerical types need to calculate N percentile values. For example, the monitoring item of user quantity does not need to count N percentile values, but only needs to focus on last_value, that is, the change curve of user quantity. In this way, for monitoring instances under monitoring items corresponding to numerical types, the possible items in the corresponding monitoring results are: sum, min, max, count, last_value, percentile; for monitoring instances under monitoring items corresponding to rate types, the possible items in the corresponding monitoring results are: last_value, count.

[0119] In summary, this embodiment supports the implementation of dimensions, and the granularity of monitoring data is relatively fine, which is conducive to assisting in locating online problems and helping users understand the use of the system. In addition, in this embodiment, for high-performance requirements (for example, indicators that require 4 9s, 5 9s, or 6 9s), only the largest monitoring values ​​can be retained, and the calculation of N quantile values ​​can be conveniently implemented based on the retained monitoring values; for low-performance requirements (for example, indicators that require 2 9s), the calculation of N quantile values ​​can be conveniently and accurately implemented based on the bucketing method. In addition, when using SkipList, this embodiment can also conveniently implement dynamic creation and deletion operations of nodes. Therefore, this embodiment can achieve high-performance, multi-dimensional, thread-safe, and easy-to-use monitoring statistics at a relatively low cost.

[0120] The data processing device provided by the embodiment of the present disclosure is described below.

[0121] See also Figure 2 , which shows a structural block diagram of a data processing device 200 provided by an embodiment of the present disclosure. Figure 2 As shown, the data processing device 200 includes:

[0122] An acquisition module 201 is used to obtain a monitoring value group of a first monitoring instance;

[0123] The first processing module 202 is used to calculate the Nth quantile of the monitoring value group using the first calculation strategy when the quantile parameter N of the first monitoring instance is less than or equal to the first preset value; otherwise, calculate the Nth quantile of the monitoring value group using the second calculation strategy;

[0124] A search module 203 is used to search for a first monitoring result corresponding to a first monitoring instance in a first mapping relationship managed by a first management tool corresponding to a first monitoring item; wherein the first monitoring item is a monitoring item to which the first monitoring instance belongs, and the first mapping relationship is a mapping relationship between each monitoring instance under the first monitoring item and a corresponding monitoring result;

[0125] The storage module 204 is used to call the first management tool to store the N quantile value or the address of the N quantile value in the first monitoring result; wherein the address of the N quantile value is the address where the N quantile value is stored after the N quantile value is calculated.

[0126] Optionally, the first processing module 202 includes:

[0127] A determination submodule, used to determine a corresponding container for each monitoring value in the monitoring value group from a plurality of containers arranged in sequence; wherein each container includes a plurality of storage locations arranged in sequence;

[0128] An adding submodule, for adding each monitoring value in the monitoring value group to a corresponding storage position in the corresponding container according to whether there is a storage position in an idle state in the corresponding container;

[0129] The first calculation submodule is used to calculate the Nth percentile value of the monitoring value group according to N and the storage information of the monitoring value group.

[0130] Optionally, each container is provided with a corresponding monitoring value range;

[0131] Identify submodules, including:

[0132] A first calculation unit, configured to calculate logD(C) for a first monitoring value C, wherein C is any monitoring value in the monitoring value group, and D is a second preset value;

[0133] A first determining unit, used to determine a container corresponding to a monitoring value range to which logD(C) belongs;

[0134] The second determining unit is used to use the determined container as the container determined for C.

[0135] Optionally, add submodules, including:

[0136] A third determination unit is used to determine, for the first monitoring value, whether there is a storage location in an idle state in the first container corresponding to the first monitoring value; wherein the first monitoring value is any monitoring value in the monitoring value group;

[0137] The adding unit is used to fill the first monitoring value into the storage position in the idle state when there is a storage position in the first container; otherwise, the first monitoring value is overwritten into the storage position in the occupied state.

[0138] Optionally, the first computing submodule includes:

[0139] A second calculation unit, used for calculating the product F of the total number E of monitoring values ​​in the monitoring value group and N;

[0140] a fourth determining unit, determining a total number of monitoring values ​​added to each container;

[0141] a fifth determining unit, configured to determine a second container; wherein the first total number is less than F, the sum of the first total number and the second total number is greater than or equal to F, the first total number is the total number of monitoring values ​​added to the containers sorted before the second container, and the second total number is the total number of monitoring values ​​added to the second container;

[0142] The third calculation unit is used to calculate the Nth percentile value of the monitoring value group according to the difference between F and the first total number, the second total number, and the total number of storage locations in the second container that are in an occupied state.

[0143] Optionally, the third computing unit includes:

[0144] A calculation subunit, configured to calculate G using the formula G=S / I*J, wherein S is the difference between F and the first total number, I is the second total number, and J is the total number of storage locations in the second container that are in an occupied state;

[0145] The processing subunit is used to, when G is an integer, use the monitoring value located at the Gth storage position in the second container as the Nth quantile value of the monitoring value group; otherwise, linearly interpolate the monitoring values ​​located at the G1th storage position and the G2th storage position in the second container to obtain the Nth quantile value of the monitoring value group; wherein G1 is the result of rounding down G, and G2 is the result of rounding up G.

[0146] Optionally, the first processing module includes:

[0147] A second calculation submodule is used to calculate the total number E of monitoring values ​​in the monitoring value group and the ratio K of the preset total number corresponding to N;

[0148] The third calculation submodule is used to calculate the Nth quantile of the monitoring value group using the first sub-strategy when K is less than or equal to 1; otherwise, calculate the Nth quantile of the monitoring value group using the second sub-strategy.

[0149] Optionally, the second computing submodule includes:

[0150] A fourth calculation unit, used for calculating the product F of E and N;

[0151] The sixth determination unit is used to, when F is an integer, take the monitoring value with the Fth largest value in the monitoring value group as the Nth percentile value of the monitoring value group; otherwise, perform linear interpolation on the monitoring values ​​with the F1th largest value and the F2th largest value in the monitoring value group to obtain the Nth percentile value of the monitoring value group; wherein F1 is the result of rounding down F, and F2 is the result of rounding up F.

[0152] Optionally, the third computing submodule includes:

[0153] A division unit is used to divide the monitoring value group into R groups according to the order of value size; wherein, in each of the first R-1 groups, the total number of monitoring values ​​is a preset total number;

[0154] A fifth calculation unit is used to calculate the product M of the preset total number and N, and for each of the first R-1 groups, the monitoring value ranked at the Mth position is used as the Nth quantile value of the group;

[0155] The processing unit is used to calculate, for the Rth group, the product P of the total number of monitoring values ​​included therein and N; when P is an integer, the monitoring value with the Pth largest value in the group is used as the Nth quantile value of the group; otherwise, linear interpolation is performed on the monitoring values ​​with the P1th largest value and the P2th largest value in the group to obtain the Nth quantile value of the group; wherein P1 is the result of rounding down P, and P2 is the result of rounding up P;

[0156] The seventh determining unit is used to use the average value of the R N quantile values ​​of the R groups as the N quantile value of the monitoring value group.

[0157] Optionally, each group of mapping relationships in the first mapping relationship serves as a node in the first hop list;

[0158] The data processing device 200 further includes:

[0159] A second processing module is used to create a new second jump table when the proportion of nodes that meet the preset deletion condition in the first jump table is greater than the preset proportion, and copy the nodes that do not meet the preset deletion condition in the first jump table to the second jump table;

[0160] A third processing module is used to lock the first jump table after the copying is completed, and update the second jump table accordingly according to the update of the first jump table during the copying process;

[0161] The fourth processing module is used to release the lock on the first jump table after the update is completed, control the first jump table to exit the service state, and control the second jump table to enter the service state.

[0162] It can be seen that in the embodiments of the present disclosure, monitoring statistics at the monitoring instance level can be performed for data of numerical types. Compared with the prior art, the monitoring granularity in the embodiments of the present disclosure is finer. In addition, when performing monitoring statistics at the monitoring instance level, the calculation strategy used when calculating the N quantile value may differ according to performance requirements. Then, the calculation accuracy of the N quantile value can be guaranteed. Therefore, compared with the prior art, the embodiments of the present disclosure can improve the monitoring effect of the system operation status, help users better understand the system operation status, and can better assist in locating online problems.

[0163] The data processing device provided by the embodiment of the present disclosure is described below.

[0164] See also Figure 3 , which shows a schematic diagram of the structure of a data processing device 300 provided by an embodiment of the present disclosure. Figure 3 As shown, the data processing device 300 includes: a processor 301, a memory 303, a user interface 304 and a bus interface.

[0165] The processor 301 is used to read the program in the memory 303 and execute the following process:

[0166] Obtaining a monitoring value group of a first monitoring instance;

[0167] When the quantile parameter N of the first monitoring instance is less than or equal to the first preset value, the N quantile value of the monitoring value group is calculated using the first calculation strategy; otherwise, the N quantile value of the monitoring value group is calculated using the second calculation strategy;

[0168] In a first mapping relationship managed by a first management tool corresponding to the first monitoring item, searching for a first monitoring result corresponding to the first monitoring instance; wherein the first monitoring item is a monitoring item to which the first monitoring instance belongs, and the first mapping relationship is a mapping relationship between each monitoring instance under the first monitoring item and the corresponding monitoring result;

[0169] The first management tool is called to store the N quantile value or the address of the N quantile value in the first monitoring result; wherein the address of the N quantile value is the address where the N quantile value is stored after the N quantile value is calculated.

[0170] exist Figure 3 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 301 and various circuits of memory represented by memory 303 are linked together. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. For different user devices, the user interface 304 may also be an interface that can be connected to external or internal devices, and the connected devices include but are not limited to keypads, displays, speakers, microphones, joysticks, etc.

[0171] The processor 301 is responsible for managing the bus architecture and general processing, and the memory 303 can store data used by the processor 301 when performing operations.

[0172] Optionally, the processor 301 is specifically configured to:

[0173] Determine a corresponding container for each monitoring value in the monitoring value group from a plurality of containers arranged in sequence; wherein each container includes a plurality of storage locations arranged in sequence;

[0174] For each monitoring value in the monitoring value group, add it to the corresponding storage position in the corresponding container according to whether there is an idle storage position in the corresponding container;

[0175] According to N and the storage information of the monitoring value group, calculate the Nth percentile value of the monitoring value group.

[0176] Optionally, each container is provided with a corresponding monitoring value range;

[0177] The processor 301 is specifically configured to:

[0178] For the first monitoring value C, logD(C) is calculated; wherein C is any monitoring value in the monitoring value group, and D is a second preset value;

[0179] Determine the container corresponding to the monitoring value range to which logD(C) belongs;

[0180] The determined container is used as the container determined for C.

[0181] Optionally, the processor 301 is specifically configured to:

[0182] For the first monitoring value, determining whether there is a storage location in an idle state in the first container corresponding to the first monitoring value; wherein the first monitoring value is any monitoring value in the monitoring value group;

[0183] When there is an idle storage location in the first container, the first monitoring value is filled into the idle storage location; otherwise, the first monitoring value is overwritten into the occupied storage location.

[0184] Optionally, the processor 301 is specifically configured to:

[0185] Calculate the product F of the total number E of monitoring values ​​in the monitoring value group and N;

[0186] Determine the total number of monitoring values ​​added to each container;

[0187] Determine a second container; wherein the first total number is less than F, the sum of the first total number and the second total number is greater than or equal to F, the first total number is the total number of monitoring values ​​added to the containers sorted before the second container, and the second total number is the total number of monitoring values ​​added to the second container;

[0188] The Nth percentile value of the monitoring value group is calculated based on the difference between F and the first total number, the second total number, and the total number of storage locations in the second container that are in an occupied state.

[0189] Optionally, the processor 301 is specifically configured to:

[0190] G is calculated using the formula G=S / I*J, where S is the difference between F and the first total number, I is the second total number, and J is the total number of occupied storage locations in the second container;

[0191] When G is an integer, the monitoring value at the Gth storage position in the second container is taken as the Nth quantile of the monitoring value group; otherwise, linear interpolation is performed on the monitoring values ​​at the G1th storage position and the G2th storage position in the second container to obtain the Nth quantile of the monitoring value group; wherein G1 is the result of rounding down G, and G2 is the result of rounding up G.

[0192] Optionally, the processor 301 is specifically configured to:

[0193] Calculate the total number E of monitoring values ​​in the monitoring value group and the ratio K of the preset total number corresponding to N;

[0194] When K is less than or equal to 1, the Nth quantile of the monitoring value group is calculated using the first sub-strategy; otherwise, the Nth quantile of the monitoring value group is calculated using the second sub-strategy.

[0195] Optionally, the processor 301 is specifically configured to:

[0196] Calculate the product F of E and N;

[0197] When F is an integer, the monitoring value with the Fth largest value in the monitoring value group is taken as the Nth percentile value of the monitoring value group; otherwise, linear interpolation is performed on the monitoring values ​​with the F1th largest value and the F2th largest value in the monitoring value group to obtain the Nth percentile value of the monitoring value group; wherein F1 is the result of rounding down F, and F2 is the result of rounding up F.

[0198] Optionally, the processor 301 is specifically configured to:

[0199] The monitoring value group is divided into R groups in order of value size; wherein, in each of the first R-1 groups, the total number of monitoring values ​​is the preset total number;

[0200] Calculate the product M of the preset total number and N, and for each of the first R-1 groups, take the Mth largest monitoring value as the Nth percentile value of this group;

[0201] For the Rth group, calculate the product P of the total number of monitoring values ​​included in it and N; when P is an integer, take the monitoring value with the Pth largest value in this group as the Nth quantile value of this group; otherwise, perform linear interpolation on the monitoring values ​​with the P1th largest value and the P2th largest value in this group to obtain the Nth quantile value of this group; where P1 is the result of rounding down P, and P2 is the result of rounding up P;

[0202] The average of the R N quantile values ​​of the R groups is taken as the N quantile value of the monitoring value group.

[0203] Optionally, each group of mapping relationships in the first mapping relationship serves as a node in the first hop list;

[0204] The processor 301 is further configured to:

[0205] When the proportion of nodes satisfying the preset deletion condition in the first jump table is greater than the preset proportion, a new second jump table is created, and the nodes in the first jump table that do not satisfy the preset deletion condition are copied to the second jump table;

[0206] After the copying is completed, the first jump table is locked, and the second jump table is updated accordingly according to the update of the first jump table during the copying process;

[0207] After the update is completed, the lock on the first jump table is released, the first jump table is controlled to exit the service state, and the second jump table is controlled to enter the service state.

[0208] It can be seen that in the embodiments of the present disclosure, monitoring statistics at the monitoring instance level can be performed for data of numerical types. Compared with the prior art, the monitoring granularity in the embodiments of the present disclosure is finer. In addition, when performing monitoring statistics at the monitoring instance level, the calculation strategy used when calculating the N quantile value may differ according to performance requirements. Then, the calculation accuracy of the N quantile value can be guaranteed. Therefore, compared with the prior art, the embodiments of the present disclosure can improve the monitoring effect of the system operation status, help users better understand the system operation status, and can better assist in locating online problems.

[0209] Preferably, the embodiment of the present disclosure also provides a data processing device, including a processor 301, a memory 303, and a computer program stored in the memory 303 and executable on the processor 301. When the computer program is executed by the processor 301, each process of the above-mentioned data processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0210] The embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned data processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0211] The embodiments of the present disclosure also provide a computer program product, which is stored in a computer-readable storage medium. The computer program product is executed by at least one processor to implement the various processes of the above-mentioned data processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0212] The embodiments of the present disclosure are described above in conjunction with the accompanying drawings, but the present disclosure is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present disclosure, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present disclosure and the claims, all of which are within the protection of the present disclosure.

Claims

1. A data processing method, comprising: Obtaining a monitoring value group of a first monitoring instance; When the quantile parameter N of the first monitoring instance is less than or equal to a first preset value, calculating the N quantile value of the monitoring value group using a first calculation strategy; Otherwise, using a second calculation strategy, calculating the Nth quantile of the monitoring value group; In a first mapping relationship managed by a first management tool corresponding to a first monitoring item, searching for a first monitoring result corresponding to the first monitoring instance; wherein the first monitoring item is a monitoring item to which the first monitoring instance belongs, and the first mapping relationship is a mapping relationship between each monitoring instance under the first monitoring item and a corresponding monitoring result; Calling the first management tool to store the N quantile value or the address of the N quantile value in the first monitoring result; wherein the address of the N quantile value is the address where the N quantile value is stored after the N quantile value is calculated; The calculating of the Nth quantile of the monitoring value group by the first calculation strategy includes: Determining a corresponding container for each monitoring value in the monitoring value group from a plurality of containers arranged in sequence; wherein each container includes a plurality of storage locations arranged in sequence; For each monitoring value in the monitoring value group, according to whether there is a storage position in a free state in the corresponding container, add it to the corresponding storage position in the corresponding container; Calculate the Nth percentile of the monitoring value group according to N and the storage information of the monitoring value group; The calculating of the Nth quantile of the monitoring value group by the second calculation strategy includes: Calculate the total number E of monitoring values ​​in the monitoring value group and the ratio K of the preset total number corresponding to N; When K is less than or equal to 1, the Nth quantile of the monitoring value group is calculated using the first sub-strategy; otherwise, the Nth quantile of the monitoring value group is calculated using the second sub-strategy; The calculating the Nth quantile of the monitoring value group by the first sub-strategy includes: Calculate the product F of E and N; When F is an integer, the monitoring value with the Fth largest value in the monitoring value group is taken as the Nth percentile value of the monitoring value group; otherwise, linear interpolation is performed on the monitoring values ​​with the F1th largest value and the F2th largest value in the monitoring value group to obtain the Nth percentile value of the monitoring value group; wherein F1 is the result of rounding down F, and F2 is the result of rounding up F; The calculating the Nth quantile of the monitoring value group by the second sub-strategy includes: Divide the monitoring value group into R groups in order of value size; wherein, in each of the first R-1 groups, the total number of monitoring values ​​is the preset total number; Calculate the product M of the preset total number and N, and for each of the first R-1 groups, take the monitoring value with the Mth largest value as the Nth quantile value of the group; For the Rth group, calculate the product P of the total number of monitoring values ​​included in it and N; when P is an integer, take the monitoring value with the Pth largest value in this group as the Nth quantile value of this group; otherwise, perform linear interpolation on the monitoring values ​​with the P1th largest value and the P2th largest value in this group to obtain the Nth quantile value of this group; where P1 is the result of rounding down P, and P2 is the result of rounding up P; The average of the R N quantile values ​​of the R groups is used as the N quantile value of the monitoring value group.

2. The method according to claim 1, wherein: Each container is set with a corresponding monitoring value range; Determining a corresponding container for each monitoring value in the monitoring value group from a plurality of containers arranged in sequence includes: For a first monitoring value C, logD(C) is calculated; wherein C is any monitoring value in the monitoring value group, and D is a second preset value; Determine the container corresponding to the monitoring value range to which logD (C) belongs; The determined container is used as the container determined for C.

3. The method according to claim 1, wherein: The step of adding each monitoring value in the monitoring value group to a corresponding storage position in the corresponding container according to whether there is an idle storage position in the corresponding container includes: For a first monitoring value, determining whether there is a storage location in an idle state in the first container corresponding to the first monitoring value; wherein the first monitoring value is any monitoring value in the monitoring value group; When there is an idle storage location in the first container, the first monitoring value is filled into the idle storage location; otherwise, the first monitoring value is overwritten into the occupied storage location.

4. The method according to claim 1, wherein: The calculating the Nth quantile of the monitoring value group according to N and the storage information of the monitoring value group includes: Calculate the product F of the total number E of monitoring values ​​in the monitoring value group and N; Determine the total number of monitoring values ​​added to each container; Determine a second container; wherein the first total number is less than F, the sum of the first total number and the second total number is greater than or equal to F, the first total number is the total number of monitoring values ​​added to the containers sorted before the second container, and the second total number is the total number of monitoring values ​​added to the second container; The Nth quantile of the monitoring value group is calculated based on the difference between F and the first total number, the second total number, and the total number of occupied storage locations in the second container.

5. The method according to claim 4, wherein: The calculating the Nth percentile value of the monitoring value group according to the difference between F and the first total number, the second total number, and the total number of storage locations in the second container that are in an occupied state includes: G is calculated using the formula G=S / I*J, wherein S is the difference between F and the first total number, I is the second total number, and J is the total number of occupied storage locations in the second container; When G is an integer, the monitoring value at the Gth storage position in the second container is used as the Nth quantile of the monitoring value group; otherwise, linear interpolation is performed on the monitoring values ​​at the G1th storage position and the G2th storage position in the second container to obtain the Nth quantile of the monitoring value group; wherein G1 is the result of rounding down G, and G2 is the result of rounding up G.

6. The method according to claim 1, wherein: Each group of mapping relationships in the first mapping relationships is used as a node in the first hop list; The method further comprises: If the proportion of nodes satisfying the preset deletion condition in the first jump table is greater than the preset proportion, creating a second jump table, and copying the nodes in the first jump table that do not satisfy the preset deletion condition to the second jump table; After the copying is completed, the first jump table is locked, and according to the update of the first jump table during the copying process, the second jump table is updated accordingly; After the update is completed, the lock on the first jump table is released, the first jump table is controlled to exit the service state, and the second jump table is controlled to enter the service state.

7. A data processing device, comprising: An acquisition module, used to obtain a monitoring value group of a first monitoring instance; A first processing module, configured to calculate the Nth quantile of the monitoring value group using a first calculation strategy when the quantile parameter N of the first monitoring instance is less than or equal to a first preset value; Otherwise, using a second calculation strategy, calculating the Nth quantile of the monitoring value group; A search module, used to search for a first monitoring result corresponding to the first monitoring instance in a first mapping relationship managed by a first management tool corresponding to the first monitoring item; wherein the first monitoring item is a monitoring item to which the first monitoring instance belongs, and the first mapping relationship is a mapping relationship between each monitoring instance under the first monitoring item and a corresponding monitoring result; A storage module, used for calling the first management tool to store the N quantile value or the address of the N quantile value in the first monitoring result; wherein the address of the N quantile value is the address where the N quantile value is stored after the N quantile value is calculated; The first processing module comprises: A determination submodule, used to determine a corresponding container for each monitoring value in the monitoring value group from a plurality of containers arranged in sequence; wherein each container includes a plurality of storage locations arranged in sequence; An adding submodule, for adding each monitoring value in the monitoring value group to a corresponding storage position in the corresponding container according to whether there is a storage position in an idle state in the corresponding container; A calculation submodule, used for calculating the Nth percentile value of the monitoring value group according to N and the storage information of the monitoring value group; The first processing module comprises: A second calculation submodule is used to calculate the total number E of monitoring values ​​in the monitoring value group and the ratio K of the preset total number corresponding to N; The third calculation submodule is used to calculate the Nth quantile of the monitoring value group using the first sub-strategy when K is less than or equal to 1; otherwise, calculate the Nth quantile of the monitoring value group using the second sub-strategy; The second computing submodule includes: A fourth calculation unit, used for calculating the product F of E and N; a sixth determining unit, for, when F is an integer, taking the monitoring value with the Fth largest value in the monitoring value group as the Nth quantile of the monitoring value group; otherwise, performing linear interpolation on the monitoring values ​​with the F1th largest value and the F2th largest value in the monitoring value group to obtain the Nth quantile of the monitoring value group; wherein F1 is the result of rounding down F, and F2 is the result of rounding up F; The third computing submodule comprises: A division unit is used to divide the monitoring value group into R groups according to the order of value size; wherein, in each of the first R-1 groups, the total number of monitoring values ​​is a preset total number; A fifth calculation unit is used to calculate the product M of the preset total number and N, and for each of the first R-1 groups, the monitoring value ranked at the Mth position is used as the Nth quantile value of the group; The processing unit is used to calculate, for the Rth group, the product P of the total number of monitoring values ​​included therein and N; when P is an integer, the monitoring value with the Pth largest value in the group is used as the Nth quantile value of the group; otherwise, linear interpolation is performed on the monitoring values ​​with the P1th largest value and the P2th largest value in the group to obtain the Nth quantile value of the group; wherein P1 is the result of rounding down P, and P2 is the result of rounding up P; The seventh determining unit is used to use the average value of the R N quantile values ​​of the R groups as the N quantile value of the monitoring value group.

8. The device according to claim 7, wherein: Each container is set with a corresponding monitoring value range; The determination submodule comprises: A first calculation unit, configured to calculate logD(C) for a first monitoring value C, wherein C is any monitoring value in the monitoring value group, and D is a second preset value; A first determining unit, used to determine a container corresponding to a monitoring value range to which logD (C) belongs; The second determining unit is used to use the determined container as the container determined for C.

9. The device according to claim 7, wherein: The adding submodule includes: A third determining unit is used to determine, for a first monitoring value, whether there is a storage location in an idle state in the first container corresponding to the first monitoring value; wherein the first monitoring value is any monitoring value in the monitoring value group; An adding unit is used to fill the first monitoring value into the storage position in the idle state when there is a storage position in the first container; otherwise, the first monitoring value is overwritten into the storage position in the occupied state.

10. The device according to claim 7, wherein: The computing submodule comprises: A second calculation unit, used for calculating the product F of the total number E of monitoring values ​​in the monitoring value group and N; a fourth determining unit, determining a total number of monitoring values ​​added to each container; a fifth determining unit, configured to determine a second container; wherein the first total number is less than F, the sum of the first total number and the second total number is greater than or equal to F, the first total number is the total number of monitoring values ​​added to containers sorted before the second container, and the second total number is the total number of monitoring values ​​added to the second container; The third calculation unit is used to calculate the Nth percentile value of the monitoring value group according to the difference between F and the first total number, the second total number, and the total number of storage locations in the second container that are in an occupied state.

11. The device according to claim 7, wherein: Each group of mapping relationships in the first mapping relationships is used as a node in the first hop list; The device also includes: A second processing module, configured to create a new second jump table when the proportion of nodes satisfying the preset deletion condition in the first jump table is greater than a preset proportion, and copy the nodes in the first jump table that do not satisfy the preset deletion condition to the second jump table; A third processing module, configured to lock the first jump table after the copying is completed, and update the second jump table accordingly according to the update of the first jump table during the copying process; The fourth processing module is used to release the lock on the first jump table after the update is completed, control the first jump table to exit the service state, and control the second jump table to enter the service state.

12. A data processing device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the data processing method according to any one of claims 1 to 6 when executed by the processor.

13. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the data processing method according to any one of claims 1 to 6.

14. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images