Event statistical method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention discloses an event statistical method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: in response to an event statistics request, obtaining a current event value of a to-be-counted current event; for a first sliding window matched with a previous event of the current event, obtaining an intermediate statistical value obtained after a previous event value of the previous event is aggregated to the first sliding window; for a second sliding window matched with the current event, obtaining an event statistical value of the current event in the second sliding window according to the intermediate statistical value and the current event value; wherein for the first time granularity with granularity statistical values in the first sliding window, the intermediate statistical values comprise granularity statistical values corresponding to all the first time granularity, and / or window statistical values obtained by performing statistics on all the granularity statistical values. According to the technical scheme, the statistical performance of the current event can be improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular, to an event statistics method, device, electronic device, storage medium, and program product. Background Art

[0002] In e-commerce scenarios, there is often a need for event statistics. For example, taking the interface access event as an example, in order to protect the platform or merchants from malicious traffic attacks, it is necessary to control the total number of accesses to the platform or merchant's backend interface, such as counting the number of times each device accesses a certain backend interface in the past X seconds. If it is too fast, it is likely to be a crawler, which can be intercepted and human-machine verification can be performed.

[0003] It should be noted that the industry currently usually uses the Flink streaming computing framework to calculate aggregate statistical values ​​within a sliding window to meet event statistics needs.

[0004] In the process of realizing the present invention, the inventors found that the prior art has the following technical problems: the event statistics performance is poor and needs to be solved urgently. Summary of the invention

[0005] The embodiments of the present invention provide an event statistics method, device, electronic device, storage medium and program product, which solve the problem of poor event statistics performance.

[0006] According to one aspect of the present invention, there is provided an event statistics method, which may include:

[0007] In response to the event statistics request, obtaining the current event value of the current event to be counted;

[0008] For a first sliding window that matches a previous event of a current event, obtaining an intermediate statistical value obtained after aggregating a previous event value of the previous event into the first sliding window;

[0009] For the second sliding window matching the current event, obtain the event statistical value of the current event in the second sliding window according to the intermediate statistical value and the current event value;

[0010] Among them, for the first time granularity with granular statistical value in the first sliding window, the intermediate statistical value includes the granular statistical values ​​corresponding to all the first time granularities respectively, and / or the window statistical value obtained by statistics of all the granular statistical values; the granular statistical value is the statistical value of the granular event value of each granular event aggregated under the corresponding first time granularity.

[0011] According to another aspect of the present invention, an event statistics device is provided, which may include:

[0012] A current event value acquisition module, used to obtain the current event value of the current event to be counted in response to an event statistics request;

[0013] An intermediate statistical value acquisition module, for acquiring, for a first sliding window matching a previous event of a current event, an intermediate statistical value obtained after aggregating a previous event value of a previous event into the first sliding window;

[0014] An event statistics obtaining module is used to obtain the event statistics of the current event in the second sliding window according to the intermediate statistics and the current event value for the second sliding window matching the current event;

[0015] Among them, for the first time granularity with granular statistical value in the first sliding window, the intermediate statistical value includes the granular statistical values ​​corresponding to all the first time granularities respectively, and / or the window statistical value obtained by statistics of all the granular statistical values; the granular statistical value is the statistical value of the granular event value of each granular event aggregated under the corresponding first time granularity.

[0016] According to another aspect of the present invention, there is provided an electronic device, which may include:

[0017] at least one processor; and

[0018] a memory communicatively connected to at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor implements the event statistics method provided by any embodiment of the present invention when executing the computer program.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, on which computer instructions are stored. The computer instructions are used to enable a processor to implement the event statistics method provided by any embodiment of the present invention when executed.

[0021] According to another aspect of the present invention, a computer program product is provided, including a computer program, which, when executed by a processor, implements the event statistics method provided by any embodiment of the present invention.

[0022] The technical solution of the embodiment of the present invention obtains the current event value of the current event to be counted in response to an event statistics request, and the current event value can be used to obtain the event statistics value; for the first sliding window matching the previous event of the current event, obtain the intermediate statistical value obtained after aggregating the previous event value of the previous event into the first sliding window, the intermediate statistical value may include the granular statistical values ​​corresponding to each first time granularity with a granular statistical value in the first sliding window, and / or the window statistical value obtained by statistics of all granular statistical values, the granular statistical value is the statistical value of the granular event value of each granular event aggregated into the corresponding first time granularity, that is, what is obtained here is the statistical value, not the event value (that is, data details); for the second sliding window matching the current event, obtain the event statistical value of the current event under the second sliding window according to the intermediate statistical value and the current event value, thereby realizing the event statistics process. Compared with related solutions that use event values ​​for event statistics, the above technical solution obtains statistical values ​​by performing aggregate statistics of event values ​​in a granular manner within a sliding window. This makes it possible to directly use a small number of statistical values ​​for calculations during the event statistics process without using a large number of event values ​​for calculations, thereby reducing the amount of calculations during the event statistics process and improving event statistics performance.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 This is a schematic diagram of a flink streaming computing framework provided by a related solution for calculating aggregated statistical values ​​within a sliding window;

[0026] Figure 2 is a schematic diagram of an example of event statistics request routing provided according to an embodiment of the present invention;

[0027] Figure 3 is a flow chart of an event statistics method provided according to an embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of sliding a sliding window in an event statistics method provided by an embodiment of the present invention;

[0029] Figure 5 is a flowchart of another event statistics method provided according to an embodiment of the present invention;

[0030] Figure 6 is a schematic diagram of aggregating granular event values ​​according to time granularity in another event statistics method provided by an embodiment of the present invention;

[0031] Figure 7 is a flowchart of another event statistics method provided according to an embodiment of the present invention;

[0032] Figure 8 is a schematic diagram of a data structure in another event statistics method provided according to an embodiment of the present invention;

[0033] Fig. 9 is a flowchart of another event statistics method provided according to an embodiment of the present invention;

[0034] Fig.10 is a flowchart of another event statistics method provided according to an embodiment of the present invention;

[0035] Fig.11 is a schematic diagram of another data structure in another event statistics method provided according to an embodiment of the present invention;

[0036] Fig.12 is a structural block diagram of an event statistics device provided according to an embodiment of the present invention;

[0037] Fig.13 It is a schematic diagram of the structure of an electronic device for implementing the event statistics method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0039] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The situations of "target", "original", etc. are similar and will not be repeated here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0040] Before introducing the embodiments of the present invention, an implementation process of a related event statistics scheme and the reasons why the event statistics performance is poor are first exemplified to better understand why the event statistics scheme proposed in the embodiments of the present invention can improve the event statistics performance.

[0041] For example, see Figure 1 , the scheme that uses the Flink streaming computing framework to calculate aggregated statistical values ​​in sliding windows can abstract data (i.e., the event value of the event described above) into a continuous data stream. When the sliding window slides according to the specified time, each piece of data received can be assigned to a sliding window that meets the conditions according to the object configuration, and then the aggregate calculation function corresponding to the sliding window is triggered to perform statistical calculations on all the data in the sliding window. However, the object of statistics in this scheme is data (or data details). When the amount of data is large, it is impossible to perform fast statistics due to the large amount of calculation, resulting in poor statistical performance, which needs to be solved urgently.

[0042] In addition, it should be noted that the event statistics solution proposed in the embodiment of the present invention can be applied in a distributed scenario, and this application process is first exemplarily described here.

[0043] Exemplarily, the event statistics solution can provide externally the ability to calculate the aggregated statistical value in the sliding window through an online service, which can be applied to the statistical calculation machine. Specifically, a software development kit (SDK) for implementing the routing algorithm can be deployed on the business machine, and the business machine can route the event statistics request corresponding to the computing task through the SDK to route the event statistics request to the statistical calculation machine, and the statistical calculation machine can perform event statistics in response to the event statistics request. Among them, the above-mentioned routing algorithm can, for example, specify the number of machines N of the statistical calculation machine, N can be a power of 2, N can be less than or equal to 2^14 (the cluster is large enough), the data can be logically divided into 16384 (2^14=16384) slots, and the slots can be allocated according to the statistical calculation machine. Specifically, each statistical calculation machine can be designated to be responsible for 2^14 / N slots. For example, when N=128, each statistical calculation machine can be responsible for 128 slots, the first statistical calculation machine can be responsible for slots [0-127], the second statistical calculation machine can be responsible for slots [128-255], and the i-th statistical calculation machine can be responsible for slots [(i-1)*X, i*X-1]. The value of xxhash(group_by)&0X3FFF can be determined, and the statistical calculation machine corresponding to the slot interval where the value falls can be determined, and the event statistical request corresponding to the group_by can be routed to the statistical calculation machine. It should be noted that the above xxhash is a hash algorithm; group_by is the statistical dimension of the current event corresponding to the event statistical request, and the statistical dimension can represent the dimension of the statistical value of the statistical event. For example, if the statistical dimension is a commodity dimension, the event statistical value can be the price of the commodity, and the statistical dimension can be "sku1" for example; 0X3FFF represents 127.

[0044] It is understandable that the above-mentioned statistical calculation machine may be one or more, that is, the event statistics scheme may be applied to each statistical calculation machine. Figure 2 There are business clusters and computing clusters. The business cluster may include one or more business machines, and the computing cluster includes one or more statistical computing machines. For each business machine, the business machine can route the event statistics request to the corresponding statistical computing machine in the computing cluster, so that the event statistics requests in high-concurrency scenarios are dispersed to different statistical computing machines in the computing cluster, and ensure that the event statistics requests of the same computing dimension can be routed to the same statistical computing machine to ensure the integrity of the data received by the statistical computing machine.

[0045] It should be noted that each statistical computing machine can provide statistical capabilities without distinction. Considering whether it can support high-concurrency and high-performance event statistics, the key lies in whether the implementation of event statistics on each statistical computing machine is efficient. Therefore, the embodiments of the present invention mainly describe the event statistics process with higher statistical performance on a single statistical computing machine.

[0046] Figure 3 It is a flow chart of an event statistics method provided in an embodiment of the present invention. This embodiment is applicable to the case of performing event statistics in a sliding window, and is particularly applicable to the case of performing non-deduplication event statistics in a sliding window. The method can be executed by the event statistics device provided in an embodiment of the present invention, and the device can be implemented by software and / or hardware. The device can be integrated on an electronic device (i.e., the statistical calculation machine described above), and the electronic device can be various user terminals or servers.

[0047] See also Figure 3 The method of the embodiment of the present invention specifically includes the following steps:

[0048] S110 . In response to an event statistics request, obtain a current event value of a current event to be counted.

[0049] The event statistics request may be understood as a request for instructing to collect statistics on the current event.

[0050] The current event can be understood as the event to be counted currently. Combined with the application scenarios that may be involved in the embodiments of the present invention, the event can optionally be used to describe an action, specifically, it can be a description of an action issued by an object in the business system, for example, it can be an order event generated by an object placing an order.

[0051] The current event value (value) can be understood as the value of the specific demand statistics of the current event, that is, the object of demand statistics, for example, it can be the price of the product in the ordering scenario.

[0052] In the embodiment of the present invention, the current event value can be obtained from the event statistics request by responding to the event statistics request. On this basis, optionally, information such as the occurrence time of the current event can also be obtained from the event statistics request, which is related to the actual situation and is not specifically limited here.

[0053] S120, for a first sliding window matching a previous event of a current event, obtaining an intermediate statistical value obtained after aggregating a previous event value of the previous event into the first sliding window;

[0054] Wherein, for the first time granularity having the granularity statistical value in the first sliding window, the intermediate statistical value includes the granularity statistical values ​​corresponding to all the first time granularities respectively, and / or the window statistical value obtained by counting all the granularity statistical values;

[0055] The granularity statistics are statistics of granularity event values ​​of granularity events aggregated to the corresponding first time granularity.

[0056] The previous event can be understood as an event counted in response to the previous statistics request, and the previous statistics request can be understood as a request responded to before the event statistics request and responded adjacent to the event statistics request. The previous event value can be understood as the event value of the previous event.

[0057] The first sliding window can be understood as a sliding window that matches the previous event, that is, a sliding window used in the process of counting the previous event. The window size of the first sliding window can be a preset duration. The intermediate statistical value can be understood as a statistical value obtained after aggregating the previous event value into the first sliding window. The number of intermediate statistical values ​​can be one or more, which is related to the actual situation and is not specifically limited here.

[0058] There are multiple time granularities in the first sliding window, wherein, for each time granularity, the time granularity can be understood as the time obtained by dividing the aggregation occurrence time of the aggregation events aggregated to the time granularity by the granularity, and the granularity can be understood as the sliding step of the first sliding window (that is, the minimum time unit when the first sliding window slides). Exemplarily, the window size of the first sliding window is 10s, and the minimum time unit is 1s, then every 1s in the first sliding window can be used as a time granularity. As another example, assuming that the aggregation occurrence time of a certain aggregation event is 2023-08-29 18:01:01:100, when the minimum time unit is 1 second, the corresponding time granularity is 2023-08-29 18:01:01.

[0059] It should be noted that not every time granularity in the first sliding window can reach a corresponding event. For the sake of distinction, the time granularity with a granularity statistical value in all time granularities is called the first time granularity, and then the event occurring at the first time granularity is taken as the granular event aggregated at the first time granularity, and the event value of the granular event is called the granular event value.

[0060] The granularity statistics of the first time granularity can be understood as the statistics of the granularity event values ​​of the granularity events aggregated under the first time granularity; there may be one or more granularity statistics for each first time granularity, and the granularity statistics may be, for example, at least one of a count (COUNT) statistic, a sum (SUM) statistic, an average (AVG) statistic, and a standard deviation (STD) statistic, so that one or more types of event statistics can be determined when counting events. For the first sliding window, all granularity statistics under the first sliding window are counted to obtain a window statistic, which is the event statistic of the previous event under the first sliding window. In combination with the application scenarios that may be involved in the embodiments of the present invention, the window statistic may be optionally referred to as a cache.

[0061] In the embodiment of the present invention, for the first sliding window, an intermediate statistical value obtained after aggregating and counting the previous event value into the first sliding window may be obtained. In the embodiment of the present invention, the manner of obtaining the intermediate statistical value is not specifically limited.

[0062] S130. For a second sliding window matching the current event, obtain an event statistical value of the current event in the second sliding window according to the intermediate statistical value and the current event value.

[0063] The second sliding window can be understood as a sliding window that matches the current event, that is, a sliding window used in the process of counting the current event. The second sliding window may be the same or different from the first sliding window described above, which depends on the actual situation and is not specifically limited here.

[0064] The event statistics value can be understood as the statistical value of the current event under the second sliding window. The event statistics value can be, for example, at least one of the number statistics value, sum statistics value, average statistics value, standard deviation statistics value, etc. This is related to the actual situation and is not specifically limited here.

[0065] In the embodiment of the present invention, for the second sliding window, the event statistical value can be obtained according to the intermediate statistical value and the current event value by performing at least one statistical operation of summing, counting, averaging, and standard deviation. In the embodiment of the present invention, the method of obtaining the event statistical value according to the intermediate statistical value and the current event value is not specifically limited.

[0066] In an embodiment of the present invention, after obtaining the event statistical value of the current event under the second sliding window, the current event can be updated to the previous event, the current event value can be updated to the previous event value, the second sliding window can be updated to the first sliding window, and the event statistical value can be updated to the intermediate statistical value, so as to provide a statistical basis for subsequent event statistics.

[0067] It can be understood that the technical solution of the embodiment of the present invention has at least the following advantages: because the aggregated results are stored in a granular manner, the amount of calculation for subsequent event statistics is reduced, especially for some businesses that are not sensitive to granularity or for scenarios with a large number of requests for the same granularity, the amount of calculation can be saved by hundreds or even thousands of times, thereby solving the problem of poor event statistics performance; and, unlike the flink streaming computing framework, it can be synchronized with the business system to support synchronous computing scenarios. The two work together to meet the low-latency business needs of synchronously counting the event statistics of massive events in tens of millions of high-concurrency scenarios.

[0068] The technical solution of the embodiment of the present invention obtains the current event value of the current event to be counted in response to an event statistics request, and the current event value can be used to obtain the event statistics value; for the first sliding window matching the previous event of the current event, obtain the intermediate statistical value obtained after aggregating the previous event value of the previous event into the first sliding window, the intermediate statistical value may include the granular statistical values ​​corresponding to each first time granularity with a granular statistical value in the first sliding window, and / or the window statistical value obtained by statistics of all granular statistical values, the granular statistical value is the statistical value of the granular event value of each granular event aggregated into the corresponding first time granularity, that is, what is obtained here is the statistical value, not the event value (that is, data details); for the second sliding window matching the current event, obtain the event statistical value of the current event under the second sliding window according to the intermediate statistical value and the current event value, thereby realizing the event statistics process. Compared with related solutions that use event values ​​for event statistics, the above technical solution obtains statistical values ​​by performing aggregate statistics of event values ​​in a granular manner within a sliding window. This makes it possible to directly use a small number of statistical values ​​for calculations during the event statistics process without using a large number of event values ​​for calculations, thereby reducing the amount of calculations during the event statistics process and improving event statistics performance.

[0069] In an optional technical solution, the second sliding window is obtained by:

[0070] Get the occurrence time of the current event;

[0071] When the occurrence time is within the first sliding window, the first sliding window is used as the second sliding window; and / or, when the occurrence time is greater than the cutoff time of the first sliding window, the first sliding window is slid to obtain a second sliding window containing the occurrence time.

[0072] The occurrence time can be understood as the time when the current event occurs, that is, the time when the corresponding action occurs, such as the time when an order is placed in an ordering scenario.

[0073] The deadline time can be understood as the maximum time among all times covered by the first sliding window.

[0074] In an embodiment of the present invention, the occurrence time is obtained. Considering that, for example, the current event may be late, and the occurrence time is less than the deadline, if the occurrence time is used as the deadline of the second sliding window, it may result in the use of 2 times or even more memory to store the granular statistical values ​​corresponding to each late event. Therefore, when the occurrence time is within the first sliding window, the first sliding window can be used as the second sliding window to avoid unnecessary storage of granular statistical values ​​corresponding to each late event, thereby saving memory; and / or, when the occurrence time is greater than the deadline, the first sliding window can be slid to obtain a second sliding window containing the occurrence time, thereby achieving a second sliding window that meets the event statistics requirements.

[0075] On this basis, it should be noted that the sliding window of the current application is always based on the latest event. For example, see Figure 4 , when the sliding window has moved from [t1, t] to [t2, t3], even if the current event occurrence time is t, the current event is still counted based on the sliding window of [t2, t3] instead of the sliding window of [t1, t].

[0076] In the embodiment of the present invention, when the occurrence time is greater than the deadline, the first sliding window may be slid in the time direction corresponding to the occurrence time; the first sliding window may be slid in the direction that satisfies the second sliding window including the occurrence time; the first sliding window may be slid in a larger time direction; the first sliding window may be slid according to a preset sliding step length, etc. In the embodiment of the present invention, the manner of sliding the first sliding window when the occurrence time is greater than the deadline is not specifically limited.

[0077] In an embodiment of the present invention, when the occurrence time is less than the deadline of the first sliding window, the first sliding window can be used as the second sliding window; it should be noted that in this case, if the occurrence time lags behind the first sliding window, the current event value can be directly discarded, and the intermediate statistical value is returned, that is, based on the intermediate statistical value and the current event value, the event statistical value of the current event under the second sliding window is obtained, including: when the occurrence time is less than the deadline of the first sliding window, the intermediate statistical value is used as the event statistical value of the current event under the second sliding window.

[0078] In the embodiment of the present invention, by determining the numerical relationship between the occurrence time and the deadline of the first sliding window, the second sliding window that meets the event statistics requirements is accurately obtained.

[0079] Figure 5 It is a flowchart of another event statistics method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, all granular statistical values ​​and / or window statistical values ​​are stored in the memory, and the intermediate statistical value obtained after the previous event value of the previous event is aggregated into the first sliding window is obtained, including: from the memory, obtaining the intermediate statistical value obtained after the previous event value of the previous event is aggregated into the first sliding window. Among them, the explanation of the terms that are the same or corresponding to the above-mentioned embodiments will not be repeated here.

[0080] See also Figure 5 The method of this embodiment may specifically include the following steps:

[0081] S210: In response to an event statistics request, obtain a current event value of a current event to be counted.

[0082] S220, for a first sliding window matching a previous event of a current event, obtaining from a memory an intermediate statistical value obtained after aggregating a previous event value of the previous event into the first sliding window;

[0083] Wherein, for the first time granularity having the granularity statistical value in the first sliding window, the intermediate statistical value includes the granularity statistical values ​​corresponding to all the first time granularities respectively, and / or the window statistical value obtained by counting all the granularity statistical values;

[0084] The granularity statistics are statistics of granularity event values ​​of granularity events aggregated to the corresponding first time granularity.

[0085] The memory may be understood as a memory storing all granular statistical values ​​and / or window statistical values.

[0086] It should be noted that, for example, the window size of the first sliding window is 3600 seconds, the sliding step is 1s, and the first sliding window corresponds to 3600 time granularities. However, in most businesses, traffic is sparse data, that is, the same device may not place orders frequently. Therefore, according to the device dimension, the order volume is counted, and the data in the first sliding window is very sparse. There may be only one first time granularity among the 3600 time granularities, which may cause a lot of memory waste. Therefore, only all granular statistics and / or window statistics can be stored in memory.

[0087] For example, see Figure 6 , the first sliding window may include each first time granularity ( Figure 6For each first time granularity, the granular event values ​​of each granular event under the first time granularity can be aggregated to obtain the granular statistical value stored in the memory. That is, in a high concurrency scenario, if 100 million current event values ​​are received for the same first time granularity, only one granular statistical value needs to be stored, thereby greatly reducing memory usage.

[0088] In the embodiment of the present invention, since the intermediate statistical value may include all granular statistical values ​​and / or window statistical values, and all granular statistical values ​​and / or window statistical values ​​are stored in the memory, the intermediate statistical value may be obtained from the memory.

[0089] S230. For a second sliding window matching the current event, obtain an event statistical value of the current event in the second sliding window according to the intermediate statistical value and the current event value.

[0090] Compared with related solutions that require storing all data details in the sliding window in the memory, thereby occupying a large amount of memory resources, the technical solution of the embodiment of the present invention stores all granular statistical values ​​and / or window statistical values ​​in the memory, that is, a small amount of statistical values ​​is stored in the memory instead of a large amount of data details, thereby solving the problem of large-scale occupation of memory resources and saving memory resources.

[0091] Figure 7 It is a flow chart of another event statistics method provided in an embodiment of the present invention. This embodiment is optimized based on the above technical solutions. In this embodiment, optionally, the intermediate statistical value includes at least all granularity statistical values, and all granularity statistical values ​​are stored in the memory in the form of an array. Among them, the explanation of the terms that are the same or corresponding to the above embodiments is not repeated here.

[0092] See also Figure 7 The method of this embodiment may specifically include the following steps:

[0093] S310: In response to an event statistics request, obtain a current event value of a current event to be counted.

[0094] S320, for a first sliding window matching a previous event of the current event, obtaining from the memory an intermediate statistical value obtained after aggregating the previous event value of the previous event into the first sliding window;

[0095] Wherein, for the first time granularity having granularity statistics in the first sliding window, the intermediate statistics at least include all granularity statistics stored in the memory in the form of an array;

[0096] The granularity statistics are statistics of granularity event values ​​of granularity events aggregated to the corresponding first time granularity.

[0097] In the embodiment of the present invention, the intermediate statistical value includes at least all the granularity statistical values, which are stored in the memory in the form of an array. The array structure of all the granularity statistical values ​​stored in the memory in the form of an array can be compact, because the empty granularity statistical value is not stored in the array; the array structure can be ordered or unordered. In the embodiment of the present invention, the representation form of the array structure is not limited.

[0098] In an embodiment of the present invention, each granular statistical value can be stored in the memory in the form of an array according to its corresponding granular event attributes, wherein the granular event attributes include at least one of the event type, occurrence time, statistical number, statistical dimension, window size, sliding step size, etc. of each granular event at the first time granularity corresponding to the granular statistical value.

[0099] For example, see Figure 8 Each granular statistical value can be stored in the memory in the form of an array according to its corresponding event type, window size and statistical dimension, that is, the granular statistical values ​​with the same window size and the same event type are stored together as the same type of aggregate data; the granular statistical values ​​with the same statistical dimension in the granular statistical values ​​of the same type of aggregate data are stored together in the form of an array, and the first field of each group of values ​​can store the maximum event time (maxeventTime) of the occurrence time of each granular event corresponding to each granular statistical value stored therein, so as to play a role in window sliding and determining the elimination statistical value.

[0100] In the embodiment of the present invention, not only all granularity statistics values ​​may be stored in the memory in the form of an array, but optionally, the first time granularity corresponding to each granularity statistics value may also be associated with the granularity statistics value and stored in the memory in the form of an array.

[0101] S330: For a second sliding window matching the current event, obtain an event statistical value of the current event in the second sliding window according to the intermediate statistical value and the current event value.

[0102] The technical solution of the embodiment of the present invention can avoid further saving of memory resources by storing all granularity statistics values ​​in the form of an array in the memory.

[0103] An optional technical solution, when no window statistics value is stored in the memory, the event statistics method may further include:

[0104] When the length of the array is equal to or greater than a preset length threshold, a window statistic is calculated based on all granularity statistics in the array, and the window statistic is stored in a memory so that the intermediate statistic obtained from the memory includes the window statistic.

[0105] The length threshold is the minimum value of the length of the array when the window statistics are counted when no window statistics are stored in the memory.

[0106] It can be understood that if the length of the array is less than the length threshold, the window statistics may not be stored in the memory, which means that the current scenario is sparse data. Although the algorithm complexity may be 0(n) at this time, due to the small amount of data, it can be achieved while ensuring statistical performance and low memory usage; when the length of the array is equal to or greater than the preset length threshold, it means that with the increase of time, the amount of data of the intermediate statistical values ​​corresponding to the first sliding window expands greatly, which may cause the statistical speed of event statistics to slow down. Therefore, in this case, the window statistics with a space complexity of o(1) can be counted according to all the granular statistical values ​​in the array, and the window statistics can be stored in the memory, so that when the event is counted, the intermediate statistical values ​​including the window statistical values ​​can be directly obtained from the memory. On the basis of adding a small amount of additional storage window statistical values, the algorithm complexity can be reduced from the traditional 0(n) algorithm to 0(log(n)), which can avoid the problem of slowing down the statistical speed due to additional statistics of a large number of granular statistical values, thereby speeding up the statistical speed and further solving the problem of poor event statistical performance.

[0107] Fig. 9 It is a flowchart of another event statistics method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, the event statistics method also includes: obtaining the occurrence time of the current event, and determining the second time granularity to which the current event belongs under the second sliding window according to the occurrence time; when all the first time granularities include the second time granularity, based on the current event value, the granularity statistics under the first time granularity corresponding to the second time granularity stored in the array are updated; and / or, when all the first time granularities do not include the second time granularity, the second time granularity is used as the new first time granularity, and the granularity statistics under the new first time granularity obtained based on the current event value are stored in the array. Among them, the explanations of the terms that are the same or corresponding to the above-mentioned embodiments are not repeated here.

[0108] See also Fig. 9 The method of this embodiment may specifically include the following steps:

[0109] S410: In response to an event statistics request, obtain a current event value of a current event to be counted.

[0110] S420, for a first sliding window matching a previous event of a current event, obtaining from a memory an intermediate statistical value obtained after aggregating a previous event value of the previous event into the first sliding window;

[0111] Wherein, for the first time granularity having granularity statistics in the first sliding window, the intermediate statistics at least include all granularity statistics stored in the memory in the form of an array;

[0112] The granularity statistics are statistics of granularity event values ​​of granularity events aggregated to the corresponding first time granularity.

[0113] S430: For a second sliding window matching the current event, obtain an event statistical value of the current event in the second sliding window according to the intermediate statistical value and the current event value.

[0114] S440: Obtain the occurrence time of the current event, and determine the second time granularity to which the current event belongs in the second sliding window according to the occurrence time.

[0115] Among them, the second time granularity can be understood as the time granularity to which the current event belongs in each time granularity in the second sliding window. In an embodiment of the present invention, the second time granularity can be determined according to the occurrence time. In combination with the above example, illustratively, the occurrence time can be divided by the minimum time unit to determine the second time granularity. For example, if the occurrence time is 2023-08-2918:01:01:100, when the minimum time unit is 1 second, the second time granularity is 2023-08-29 18:01:01.

[0116] S450: When all first time granularities include the second time granularity, based on the current event value, update the granularity statistics at the first time granularity corresponding to the second time granularity stored in the array.

[0117] In an embodiment of the present invention, when all first time granularities include the second time granularity, it means that the current event value can be counted at the first time granularity corresponding to the second time granularity. Therefore, based on the current event value, the granularity statistics value at the first time granularity corresponding to the second time granularity stored in the array can be updated. For example, the updated granularity statistics value can be obtained by aggregating the current event value at the first time granularity of the second time granularity.

[0118] For example, the current event value and current event attributes of the current event with an occurrence time of 1 second expressed in the form of (statistical dimension, current event value, occurrence time, statistical number, window size, sliding step length) are obtained, specifically (sku1,10,et1,sum,10,1), (sku1,10,et1,sum,10,1), (sku1,9,et1,sum,10,1), (sku1,8,et1,sum,10,1), (sku1,9,et1,sum,10,1) and (sku1,10,et1,sum,10,1). There is no need to store the above 6 data. You only need to store the granular statistical values ​​corresponding to the above current events, that is, you only need to store Store (sku1,et1,sum), where sum is the granularity statistics corresponding to the sum of each current event value, and et1 can be the occurrence time, or it can refer to the first time granularity to which the occurrence time belongs; specifically, after obtaining the current event values ​​and current event attributes of the first 5 current events, the first time granularity corresponding to the occurrence time is stored (sku1,et1,46), and after obtaining the current event value and current event attributes of the 6th current event, based on the current event value of the 6th current event, the granularity statistics at the first time granularity stored in the array are updated, that is, sum=46+10 is calculated, and the corresponding storage at the first time granularity corresponding to the occurrence time is obtained (sku1,et1,56).

[0119] S460: When all first time granularities do not include the second time granularity, use the second time granularity as a new first time granularity, and obtain a granularity statistical value at the new first time granularity based on the current event value, and store it in an array.

[0120] In an embodiment of the present invention, when all first time granularities do not include the second time granularity, it means that the granularity statistics at the first time granularity corresponding to the second time granularity may not be stored in the memory. Therefore, the second time granularity can be used as the new first time granularity, and the granularity statistics at the new first time granularity based on the current event value can be stored in the array.

[0121] The technical solution of the embodiment of the present invention is to update the granularity statistics stored in the array based on the current event value when all the first time granularities include the second time granularity, and / or, when all the first time granularities do not include the second time granularity, obtain the granularity statistics at the new first time granularity based on the current event value and store them in the array, thereby storing the aggregated statistical results of the current event value in the array to provide a statistical basis for subsequent event statistics.

[0122] In an optional technical solution, the array is unordered, and before determining the second time granularity to which the current event belongs in the second sliding window according to the occurrence time, the event statistics method further includes:

[0123] When the length of the array is equal to or greater than a preset length threshold, the storage position of each granularity statistical value in the array is adjusted according to the first time granularity corresponding to all granularity statistical values ​​in the array, so as to convert the unordered array into an ordered array.

[0124] The storage location may be understood as the location where the particle size statistics value is stored in the array.

[0125] In an embodiment of the present invention, when the length of the array is less than a preset length threshold, it means that the current scenario is sparse data. The array can be made unordered, and there is no need to establish relevant additional indexes. When performing event statistics, a traversal method can be used to obtain granular statistical values ​​from the memory or locate the granular statistical values ​​that need to be located. At this time, although the algorithm complexity is 0(n), due to the small amount of data, that is, n is small, high statistical performance can be guaranteed; when the length of the array is equal to or greater than the preset length threshold, storing an array with a large amount of data and no order may cause the statistical speed of event statistics to slow down. Therefore, in this case, according to the first time granularity corresponding to all the granular statistical values ​​in the array, the storage position of each granular statistical value in the array is adjusted to convert the unordered array into an ordered array, so that when performing event statistics, granularity positioning can be quickly performed, for example, the first time granularity corresponding to the second time granularity can be quickly located, thereby speeding up the update speed of the granular statistical value.

[0126] Fig.10 It is a flowchart of another event statistics method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, when the second sliding window is obtained by sliding the first sliding window, and the intermediate statistical value at least includes all granularity statistical values, the event statistical value of the current event under the second sliding window is obtained according to the intermediate statistical value and the current event value, including: when all first time granularities include elimination time granularities that are not located under the second sliding window, the elimination statistical value under the elimination time granularity is determined from all granularity statistical values; according to the intermediate statistical value, the current event value and the elimination statistical value, the event statistical value of the current event under the second sliding window is obtained. Among them, the explanation of the terms that are the same or corresponding to the above-mentioned embodiments will not be repeated here.

[0127] See also Fig.10 The method of this embodiment may specifically include the following steps:

[0128] S510: In response to an event statistics request, obtain a current event value of a current event to be counted.

[0129] S520: for a first sliding window matching a previous event of a current event, obtain an intermediate statistical value obtained after aggregating a previous event value of the previous event into the first sliding window;

[0130] Wherein, for the first time granularity having the granularity statistical value in the first sliding window, the intermediate statistical value includes the granularity statistical values ​​corresponding to all the first time granularities respectively, and / or the window statistical value obtained by counting all the granularity statistical values;

[0131] The granularity statistics are statistics of granularity event values ​​of granularity events aggregated to the corresponding first time granularity.

[0132] S530. For the second sliding window matching the current event, when the second sliding window is obtained by sliding the first sliding window, and the intermediate statistical value includes at least all granularity statistical values, and when all first time granularities include elimination time granularities that are not located under the second sliding window, determine the elimination statistical value under the elimination time granularity from all granularity statistical values.

[0133] The elimination time granularity can be understood as the first time granularity that is not in the second sliding window; the elimination time granularity can also be understood as the first time granularity at which the demand is eliminated due to window sliding. The elimination statistical value can be understood as the statistical value under the elimination time granularity.

[0134] It can be understood that when the second sliding window is obtained by sliding the first sliding window and the intermediate statistical values ​​include at least all granular statistical values, it means that the intermediate statistical values ​​may include granular statistical values ​​that do not belong to the second sliding window. In order to ensure that the event statistical values ​​are statistical values ​​under the second sliding window, when all first time granularities include elimination time granularities that are not under the second sliding window, the granular statistical values ​​under the elimination time granularity can be determined from all granular statistical values ​​as the elimination statistical values, so that the event statistical values ​​under the second sliding window can be obtained subsequently through the elimination statistical values.

[0135] For example, in the first sliding window, which is a sliding window of [0s, 10s], if the occurrence time is 11s and the sliding step is 1s, the first sliding window is slid to obtain the second sliding window [1s, 11s]. The first time granularity [0s, 1s] is not located under the second sliding window, and it can be used as the elimination time granularity, and the granularity statistical value under the elimination time granularity is used as the elimination statistical value.

[0136] S540: Obtain an event statistical value of the current event in the second sliding window according to the intermediate statistical value, the current event value and the eliminated statistical value.

[0137] In the embodiment of the present invention, when it is determined that the elimination statistic is obtained, the event statistic is obtained through the intermediate statistic, the current event value and the elimination statistic; there may be a situation where there is no granular statistic under the elimination time granularity, so when it is not determined that the elimination statistic is obtained, the event statistic can be obtained through the intermediate statistic and the current event value. In the embodiment of the present invention, the method of obtaining the event statistic according to the intermediate statistic, the current event value and the elimination statistic is not specifically limited.

[0138] For example, when the event statistics are number statistics, the granularity statistics may include the number of events of each granularity at the first time granularity: count , according to the gl count , current event value and elimination statistics value, the event statistics value COUNT is obtained through the following formula 当前 :

[0139] COUNT current = gl1 count +gl2 count +…+glN count +1 ( Current Request ) -gl count disuse;

[0140] Where N is the total number of particle size statistics, 1 (当前请求) Indicates the number of current event values ​​(the number of current events), gl count淘汰 Elimination statistics.

[0141] For example, when the event statistics are summed statistics, the granularity statistics may include the sum of the event values ​​of each granularity at the first time granularity. SUM , can be based on each gl SUM , current event value and elimination statistics, the event statistics SUM is obtained through the following formula 当前 :

[0142] SUM current = gl1 sum +gl2 sum +…+glN sum +value ( Current Request ) -gl sum disuse;

[0143] Among them, value (当前请求) Indicates the current event value.

[0144] For example, when the event statistics are average statistics, the granularity statistics may include gl count and gl SUM , according to the gl count And each gl SUM , current event value and elimination statistics, the event statistics value AVG is obtained through the following formula 当前 :

[0145]

[0146] For example, when the event statistics are the standard deviation statistics, the particle size statistics include gl count ,gl SUM And the sum of squares of event values ​​at each granularity at the first time granularity X i The granularity event value of the i-th granularity event at the first time granularity can be calculated based on the gl count , each gl SUM and each Current event value and elimination statistics value, calculated by standard deviation formula The event statistics S is obtained, where n is the number of granular event values.

[0147] The technical solution of the embodiment of the present invention can ensure that the event statistical value is a statistical value under the second sliding window by using the determined elimination statistical value as a reference factor for obtaining the event statistical value, thereby avoiding the influence of the elimination statistical value that does not belong to the second sliding window on the accuracy of the event statistical value.

[0148] An optional technical solution, wherein the intermediate statistical value also includes a window statistical value, and obtaining the event statistical value of the current event under the second sliding window according to the intermediate statistical value, the current event value and the elimination statistical value may include:

[0149] According to the window statistics, the current event value and the elimination statistics, the event statistics of the current event in the second sliding window are obtained.

[0150] In the embodiment of the present invention, the event statistics are obtained based on the window statistics, which can avoid the problem of slowing down the statistics due to additional statistics of a large number of granular statistics, thereby speeding up the statistics. However, since the window statistics are obtained by statistics of all granular statistics, if the event statistics are obtained directly based on the window statistics, the elimination statistics that do not belong to the second sliding window may affect the event statistics. Therefore, the event statistics can be obtained based on the window statistics, the current event value and the elimination statistics. In the embodiment of the present invention, the method of obtaining the event statistics based on the window statistics, the current event value and the elimination statistics is not specifically limited.

[0151] For example, when the event statistics are count statistics, the window statistics may include the number of events of each granularity in the first sliding window. count , according to the cache count , current event value and elimination statistics value, the event statistics value COUNT is obtained through the following formula 当前 :

[0152] COUNT 当前 =cache count +1 (当前请求) -gl count淘汰 .

[0153] Exemplarily, when the event statistics are summed statistics, the granularity statistics include the sum cache of each granularity event value under the first sliding window. SUM , according to the cache SUM , current event value and elimination statistics, the event statistics SUM is calculated by the following formula 当前 :

[0154] SUM 当前 =cache sum +value (当前请求) -gl sum淘汰 .

[0155] For example, when the event statistics are average statistics, the granularity statistics include cache SUM and cache count , according to the cache SUM and cache count , current event value and elimination statistics, the event statistics AVG is obtained by accelerating the calculation through the following formula 当前 :

[0156]

[0157] For example, when the event statistics are used to calculate the standard deviation statistics, the granularity statistics include cache SUM , cache count and the sum of squares of each granularity event value under the first sliding window The standard deviation can be calculated by the formula And the formula n = cache count +1, and then perform deformation derivation to obtain the following formula:

[0158]

[0159]

[0160] Thus, according to cache SUM 、cache count and the current event value and the elimination statistic value ∑ 0<i< j X i and the formula derived above, the event statistic value (n - 1)STD is calculated quickly 当前 2 .

[0161] The technical solution of the embodiment of the present invention can obtain the event statistic value based on the window statistic value and the elimination statistic value, which can not only speed up the statistic speed, but also ensure the accuracy of the event statistic value.

[0162] To better understand the technical solution of the above embodiment of the present invention, an optional example is provided here. Exemplarily Fig.11 The window statistic value, the overall granularity statistic value 1 (gl1), the overall granularity statistic value 2 (gl2), and the overall granularity statistic value 3 (gl3) are shown, as well as the first time granularity 1 (et1), the first time granularity 2 (et2), and the first time granularity 3 (et3) corresponding to each granularity statistic value respectively. They can be stored in the memory in an ordered array form according to the occurrence time of each granularity event corresponding to each first time granularity, and et1 < et2 < et3. Based on this, the following steps are carried out

[0163] In response to an event statistic request, obtain the current event value and the occurrence time et of the current event to be statistically analyzed

[0164] In the case where et is greater than et3, the second time granularity to which the current event belongs can be used as the new first time granularity, and the granularity statistic value determined according to the current event value can be added to the end of the array; in the case where the occurrence time span corresponding to the array is within the first sliding window, the intermediate statistic value can be obtained, and the event statistic value can be obtained according to the intermediate statistic value and the current event value; in the case where the occurrence time span corresponding to the array is equal to the first sliding window, the elimination statistic value can be determined. For example, the granularity statistic value corresponding to the smallest first time granularity stored in the array can be used as the elimination statistic value, and the intermediate statistic value can be obtained. The event statistic value is obtained according to the intermediate statistic value, the current event value, and the elimination statistic value

[0165] In the case where et is less than et3 minus the window size, the current event value is discarded, and the event statistic value is determined according to the intermediate statistic value. For example, the window statistic value can be used as the event statistic value

[0166] When et is less than or equal to et3, and et is greater than or equal to et3 minus the window size, a binary search (time complexity O(logn)) is used in the array according to et to search for each first time granularity to determine whether all first time granularities contain the second time granularity. Further, when all first time granularities include the second time granularity, based on the current event value, the granularity statistics under the first time granularity corresponding to the second time granularity stored in the array are updated; when all first time granularities do not include the second time granularity and the corresponding occurrence time span of the array is within the first sliding window, the second time granularity to which the current event belongs is used as the new first time granularity, and the granularity statistics determined according to the current event value are inserted into the corresponding position in the array found by binary search, and the intermediate statistics are obtained, and the event statistics are obtained according to the intermediate statistics and the current event value; when all first time granularities do not include the second time granularity and the corresponding occurrence time span of the array is equal to the first sliding window, the second time granularity to which the current event belongs is used as the new first time granularity, and the granularity statistics determined according to the current event value are inserted into the corresponding position in the array found by binary search, and the intermediate statistics are obtained, the elimination statistics are determined, and the event statistics are obtained according to the intermediate statistics, the current event value and the elimination statistics.

[0167] Fig.12 This is a structural block diagram of an event statistics device provided in an embodiment of the present invention, and the device is used to execute the event statistics method provided in any of the above embodiments. The device and the event statistics method of the above embodiments belong to the same inventive concept, and the details not described in detail in the embodiment of the event statistics device can refer to the embodiment of the above event statistics method. Fig.12 The device may specifically include: a current event value acquisition module 610, an intermediate statistical value acquisition module 620 and an event statistical value acquisition module 630.

[0168] The current event value acquisition module 610 is used to obtain the current event value of the current event to be counted in response to the event statistics request;

[0169] An intermediate statistical value acquisition module 620, for acquiring, for a first sliding window matching a previous event of a current event, an intermediate statistical value obtained after aggregating a previous event value of a previous event into the first sliding window;

[0170] An event statistics obtaining module 630 is used to obtain, for a second sliding window matching the current event, an event statistics value of the current event in the second sliding window according to the intermediate statistics value and the current event value;

[0171] Among them, for the first time granularity with granular statistical value in the first sliding window, the intermediate statistical value includes the granular statistical values ​​corresponding to all the first time granularities respectively, and / or the window statistical value obtained by statistics of all the granular statistical values; the granular statistical value is the statistical value of the granular event value of each granular event aggregated under the corresponding first time granularity.

[0172] Optionally, all granularity statistics and / or window statistics are stored in a memory, and the intermediate statistics acquisition module 620 includes:

[0173] The intermediate statistical value acquisition submodule is used to obtain, from the memory, the intermediate statistical value obtained after aggregating the previous event value of the previous event into the first sliding window.

[0174] Optionally, based on the above device, the intermediate statistical values ​​at least include all granularity statistical values, and all granularity statistical values ​​are stored in the memory in the form of an array.

[0175] Optionally, based on the above device, the device may further include:

[0176] A second time granularity determination module is used to obtain the occurrence time of the current event, and determine the second time granularity to which the current event belongs in the second sliding window according to the occurrence time;

[0177] A granularity statistics updating module, configured to update the granularity statistics at the first time granularity corresponding to the second time granularity stored in the array based on the current event value when all the first time granularities include the second time granularity; and / or,

[0178] The granularity statistics storage module is used to use the second time granularity as the new first time granularity when all first time granularities do not contain the second time granularity, and store the granularity statistics at the new first time granularity based on the current event value in an array.

[0179] Optionally, based on the above device, the array is unordered, and the device may further include:

[0180] The array conversion module is used to adjust the storage position of each granularity statistical value in the array according to the first time granularity corresponding to all granularity statistical values ​​in the array before determining the second time granularity to which the current event belongs under the second sliding window according to the occurrence time, when the length of the array is equal to or greater than the preset length threshold, so as to convert the unordered array into an ordered array.

[0181] Optionally, based on the above device, when no window statistics are stored in the memory, the device may further include:

[0182] The window statistics storage module is used to calculate the window statistics based on all granular statistics in the array when the length of the array is equal to or greater than a preset length threshold, and store the window statistics in the memory so that the intermediate statistics obtained from the memory include the window statistics.

[0183] Optionally, when the second sliding window is obtained by sliding the first sliding window, and the intermediate statistical value at least includes all granular statistical values, the event statistical value obtaining module 630 includes:

[0184] An elimination statistical value determination submodule, used to determine the elimination statistical value at the elimination time granularity from all granularity statistical values ​​when all first time granularities include the elimination time granularity that is not located under the second sliding window;

[0185] The event statistics obtaining submodule is used to obtain the event statistics of the current event in the second sliding window according to the intermediate statistics, the current event value and the elimination statistics.

[0186] Optionally, based on the above device, the intermediate statistical value also includes a window statistical value, and the event statistical value obtains a submodule including:

[0187] The event statistics obtaining unit is used to obtain the event statistics of the current event in the second sliding window according to the window statistics, the current event value and the elimination statistics.

[0188] Optionally, the device may further include the following modules to obtain the second sliding window:

[0189] The occurrence time acquisition module is used to obtain the occurrence time of the current event;

[0190] The first sliding window is used as a module, and is used to use the first sliding window as a second sliding window when the occurrence time is within the first sliding window; and / or,

[0191] The second sliding window obtaining module is used to slide the first sliding window to obtain a second sliding window containing the occurrence time when the occurrence time is greater than the cutoff time of the first sliding window.

[0192] The event statistics device provided by the embodiment of the present invention obtains the current event value of the current event to be counted in response to the event statistics request through the current event value acquisition module, and the current event value can be used to obtain the event statistics value; obtains the intermediate statistics value obtained after aggregating the previous event value of the previous event into the first sliding window for the first sliding window matching the previous event of the current event through the intermediate statistics value acquisition module, and the intermediate statistics value may include the granularity statistics values ​​corresponding to each first time granularity with granularity statistics value in the first sliding window, and / or the window statistics value obtained by statistics of all granularity statistics values, and the granularity statistics value is the statistics value of the granularity event value of each granularity event aggregated into the corresponding first time granularity, that is, what is obtained here is the statistics value, not the event value (that is, data details); obtains the event statistics value of the current event in the second sliding window according to the intermediate statistics value and the current event value for the second sliding window matching the current event through the event statistics value acquisition module, thereby realizing the event statistics process. Compared with related solutions that use event values ​​for event statistics, the above technical solution obtains statistical values ​​by performing aggregate statistics of event values ​​in a granular manner within a sliding window. This makes it possible to directly use a small number of statistical values ​​for calculations during the event statistics process without using a large number of event values ​​for calculations, thereby reducing the amount of calculations during the event statistics process and improving event statistics performance.

[0193] The event statistics device provided in the embodiment of the present invention can execute the event statistics method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0194] It is worth noting that in the embodiment of the above-mentioned event statistics device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0195] Fig.13 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0196] like Fig.13As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0197] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0198] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the event statistics method.

[0199] In some embodiments, the event statistics method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the event statistics method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the event statistics method in any other appropriate manner (e.g., by means of firmware).

[0200] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0201] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0202] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0203] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0204] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0205] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0206] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0207] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An event statistics method, characterized in that: include: In response to the event statistics request, obtaining the current event value of the current event to be counted; For a first sliding window matching a previous event of the current event, obtaining an intermediate statistical value obtained after aggregating a previous event value of the previous event into the first sliding window; For a second sliding window matching the current event, obtaining an event statistical value of the current event in the second sliding window according to the intermediate statistical value and the current event value; Among them, for the first time granularity with granular statistical value in the first sliding window, the intermediate statistical value includes the granular statistical values ​​corresponding to all the first time granularities respectively, and / or the window statistical value obtained by statistics of all the granular statistical values; the granular statistical value is the statistical value of the granular event value of each granular event aggregated under the corresponding first time granularity.

2. The method according to claim 1, characterized in that All of the granular statistical values ​​and / or the window statistical values ​​are stored in a memory, and the obtaining of an intermediate statistical value obtained after aggregating the previous event value of the previous event into the first sliding window includes: An intermediate statistical value obtained after aggregating the previous event value of the previous event into the first sliding window is obtained from the memory.

3. The method according to claim 2, characterized in that The intermediate statistical values ​​at least include all the particle size statistical values, and all the particle size statistical values ​​are stored in the memory in the form of an array.

4. The method according to claim 3, characterized in that Also includes: Acquire the occurrence time of the current event, and determine, according to the occurrence time, a second time granularity to which the current event belongs in the second sliding window; In the case where all the first time granularities include the second time granularity, based on the current event value, the granularity statistics at the first time granularity corresponding to the second time granularity stored in the array are updated; and / or, In the case that all the first time granularities do not include the second time granularity, the second time granularity is used as the new first time granularity, and the granularity statistics value at the new first time granularity is obtained based on the current event value and stored in the array.

5. The method according to claim 4, characterized in that The array is unordered. Before determining the second time granularity to which the current event belongs in the second sliding window according to the occurrence time, the method further includes: When the length of the array is equal to or greater than a preset length threshold, the storage position of each granularity statistical value in the array is adjusted according to the first time granularity corresponding to each of the granularity statistical values ​​in the array, so as to convert the unordered array into an ordered array.

6. The method according to claim 3, characterized in that: In the case where the window statistics are not stored in the memory, the method further includes: When the length of the array is equal to or greater than a preset length threshold, the window statistics are calculated based on all the granular statistics in the array, and the window statistics are stored in the memory, so that the intermediate statistics obtained from the memory include the window statistics.

7. The method according to claim 1, characterized in that When the second sliding window is obtained by sliding the first sliding window, and the intermediate statistical value includes at least all the granular statistical values, obtaining the event statistical value of the current event in the second sliding window according to the intermediate statistical value and the current event value includes: In a case where all of the first time granularities include an elimination time granularity that is not located under the second sliding window, determining an elimination statistical value under the elimination time granularity from all of the granularity statistical values; The event statistical value of the current event in the second sliding window is obtained according to the intermediate statistical value, the current event value and the eliminated statistical value.

8. The method according to claim 7, characterized in that The intermediate statistical value also includes the window statistical value, and obtaining the event statistical value of the current event under the second sliding window according to the intermediate statistical value, the current event value and the eliminated statistical value includes: The event statistic value of the current event in the second sliding window is obtained according to the window statistic value, the current event value and the elimination statistic value.

9. The method according to claim 1, characterized in that: The second sliding window is obtained by: Obtaining the occurrence time of the current event; When the occurrence time is within the first sliding window, using the first sliding window as the second sliding window; and / or, When the occurrence time is greater than the cutoff time of the first sliding window, the first sliding window is slid to obtain the second sliding window including the occurrence time.

10. An event statistics device, characterized in that: include: A current event value acquisition module, used to obtain the current event value of the current event to be counted in response to an event statistics request; An intermediate statistical value acquisition module, configured to acquire, for a first sliding window matching a previous event of the current event, an intermediate statistical value obtained after aggregating a previous event value of the previous event into the first sliding window; An event statistics obtaining module, configured to obtain, for a second sliding window matching the current event, an event statistics value of the current event in the second sliding window according to the intermediate statistics value and the current event value; Among them, for the first time granularity with granular statistical value in the first sliding window, the intermediate statistical value includes the granular statistical values ​​corresponding to all the first time granularities respectively, and / or the window statistical value obtained by statistics of all the granular statistical values; the granular statistical value is the statistical value of the granular event value of each granular event aggregated under the corresponding first time granularity.

11. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the event statistics method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the event statistics method as described in any one of claims 1 to 9 when executed.

13. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the event statistics method according to any one of claims 1 to 9.

Images