Event processing method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention discloses an event processing method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: in response to an event deduplication request, obtaining a current deduplication feature and current occurrence time of a current event, and obtaining a first sliding window and a first array matched with the current event; determining the repeatability between each matched associated deduplication feature and the current deduplication feature according to the associated deduplication features which are stored in the first array and matched with the first sliding window; under the condition that it is determined that an associated deduplication feature repeated with the current deduplication feature exists according to the repeatability, the associated occurrence time corresponding to the repeated associated deduplication feature and stored in the first array is processed according to the current occurrence time, and the deduplication process of the current event is completed. According to the technical scheme provided by the embodiment of the invention, the problems of low duplicate removal performance and memory waste are solved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular, to an event processing method, device, electronic device, storage medium, and program product. Background Art

[0002] In e-commerce risk control scenarios, it is often necessary to calculate the deduplicated statistical values ​​of events for risk control.

[0003] Here, we take the order event as an example. In order to identify whether a device is an abnormal device, we can count the number of different objects that have placed orders through the device in the past day, that is, we can deduplicate the objects (distinct) and then count the number of objects (count). It can be seen that event deduplication is the first step in the event deduplication statistics process.

[0004] In the process of implementing the present invention, the inventors found that the prior art has the following technical problems: poor deduplication performance and memory waste, which need to be solved urgently. Summary of the invention

[0005] The embodiments of the present invention provide an event processing method, device, electronic device, storage medium and program product, which solve the problems of low deduplication performance and memory waste.

[0006] According to one aspect of the present invention, there is provided an event processing method, which may include:

[0007] In response to an event deduplication request, obtaining a current deduplication feature and a current occurrence time of a current event, and obtaining a first sliding window and a first array matching the current event, wherein the first array stores associated deduplication features and associated occurrence times of each associated event associated with the current event;

[0008] For the associated deduplication features stored in the first array and matched with the first sliding window, determine the repetitiveness between each matched associated deduplication feature and the current deduplication feature;

[0009] When it is determined based on repeatability that there is an associated deduplication feature that repeats the current deduplication feature, the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array is processed according to the current occurrence time to complete the deduplication process of the current event.

[0010] According to another aspect of the present invention, there is provided an event processing device, which may include:

[0011] A first array acquisition module, configured to, in response to an event deduplication request, acquire a current deduplication feature and a current occurrence time of a current event, and acquire a first sliding window and a first array matching the current event, wherein the first array stores associated deduplication features and associated occurrence times of each associated event associated with the current event;

[0012] A repeatability determination module, for determining the repeatability between each associated deduplication feature matched with the current deduplication feature, for the associated deduplication features stored in the first array and matched with the first sliding window;

[0013] The associated occurrence time processing module is used to process the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array according to the current occurrence time when it is determined based on repeatability that there is an associated deduplication feature that is repeated with the current deduplication feature, so as to complete the deduplication process of the current event.

[0014] According to another aspect of the present invention, there is provided an electronic device, which may include:

[0015] at least one processor; and

[0016] a memory communicatively connected to at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor implements the event processing method provided by any embodiment of the present invention when executing the computer program.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, on which computer instructions are stored. The computer instructions are used to enable a processor to implement the event processing method provided by any embodiment of the present invention when executed.

[0019] According to another aspect of the present invention, a computer program product is provided, including a computer program, which implements the event processing method provided by any embodiment of the present invention when executed by a processor.

[0020] The technical solution of the embodiment of the present invention can obtain the current deduplication features and current occurrence time of the current event, and obtain the first sliding window and the first array matching the current event by responding to the event deduplication request, wherein the first array stores the associated deduplication features and the associated occurrence time of each associated event associated with the current event; for the associated deduplication features stored in the first array that match the first sliding window, the repetitiveness between each matched associated deduplication feature and the current deduplication feature is determined to determine whether there are associated deduplication features that are repeated with the current deduplication features in the first array; in the case where it is determined that there are repeated associated deduplication features based on the repetitiveness, the associated occurrence time corresponding to the repeated associated deduplication features stored in the first array is processed according to the current occurrence time to complete the deduplication process of the current event. Compared to storing associated deduplication features and associated occurrence times in the form of HashMap for event deduplication, the above technical solution saves memory during the event deduplication process because there is no need to store additional pointers by storing them in the form of an array. In addition, the associated deduplication features cached in the memory can be directly used for repeatability judgment without involving multiple pointer query processes, thereby improving the deduplication performance.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flow chart of an event processing method provided according to an embodiment of the present invention;

[0024] Figure 2 is a flowchart of another event processing method provided according to an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of hash conflict processing in another event processing method provided according to an embodiment of the present invention;

[0026] Figure 4 is a flowchart of another event processing method provided according to an embodiment of the present invention;

[0027] Figure 5 is a flowchart of another event processing method provided according to an embodiment of the present invention;

[0028] Figure 6 is a flowchart of another event processing method provided according to an embodiment of the present invention;

[0029] Figure 7 is a schematic diagram of a target tree in another event processing method provided according to an embodiment of the present invention;

[0030] Figure 8 is a schematic diagram of performing deduplication statistics by using a target tree in another event processing method provided according to an embodiment of the present invention;

[0031] Fig. 9 is a schematic diagram of time granularity elimination in another event processing method provided according to an embodiment of the present invention;

[0032] Fig.10 is a structural block diagram of an event processing device provided according to an embodiment of the present invention;

[0033] Fig.11 It is a structural schematic diagram of an electronic device for implementing the event processing method of an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The situations of "target", "original", etc. are similar and will not be repeated here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] Before introducing the embodiment of the present invention, the implementation process of the relevant event processing scheme and the reasons why the problem of poor deduplication performance and memory waste occurs are explained by way of example, so as to better understand why the event processing scheme proposed in the embodiment of the present invention can solve this problem.

[0037] For example, in the event deduplication scenario, the relevant solution is implemented based on HashMap, specifically through HashMap <Value,<Time,Increment> >Store the deduplication feature (Value), occurrence time (Time) and event value (Increment) of an event, where Increment can also be called the growth value. However, due to the characteristics of the HashMap data structure itself, in addition to storing Value, Time and Increment, it also stores many pointers, such as pointers to Value, pointers to Time and Increment, and pointers to Entry nodes (Entry nodes are unique nodes in HashMap), and these pointers stored in memory will cause a lot of memory waste. In addition, when using HashMap <Value,<Time,Increment> >When determining whether a Value is repeated, it is necessary to query pointers multiple times to locate multiple memory addresses, which makes it impossible to properly utilize the memory cache, resulting in poor deduplication performance.

[0038] In view of this, the embodiment of the present invention proposes an implementation scheme for storing Value, Time and Increment in the form of an array. The array data structure does not need to involve pointers, which saves memory and can directly perform duplicate judgment using the Value cached in the memory, thereby improving deduplication performance.

[0039] Next, this will be elaborated in detail.

[0040] Figure 1 It is a flowchart of an event processing method provided in an embodiment of the present invention. This embodiment can be applied to the case of event processing, and is particularly applicable to the case of event deduplication. The method can be executed by the event processing device provided in an embodiment of the present invention, and the device can be implemented by software and / or hardware, and the device can be integrated on an electronic device, and the electronic device can be various user terminals or servers.

[0041] See also Figure 1 The method of the embodiment of the present invention specifically includes the following steps:

[0042] S110. In response to an event deduplication request, obtain the current deduplication features and current occurrence time of the current event, and obtain a first sliding window and a first array matching the current event, wherein the first array stores associated deduplication features and associated occurrence times of each associated event associated with the current event.

[0043] The event deduplication request may be understood as a request for instructing to deduplicate the current event.

[0044] The current event can be understood as the event to be deduplicated. Combined with the application scenarios that may be involved in the embodiments of the present invention, the event can optionally be used to describe an action, specifically, a description of an action issued by an object in a business system, for example, an order event generated by an object placing an order.

[0045] The current deduplication feature can be understood as a feature used to determine whether to perform deduplication for the current event. For example, in the application scenario of counting the number of orders placed by different login objects on the same device, this feature can be the login object's identification code (pin). The current occurrence time can be understood as the time when the current event occurs, that is, the time when the corresponding action occurs, such as the time when the order is placed in the ordering scenario.

[0046] The first sliding window can be understood as a sliding window that matches the current event, that is, a sliding window that is applied in the process of deduplication statistics of the current event. It should be noted that the sliding window currently applied is always based on the latest event. For example, when the sliding window has moved from [t1, t] to [t2, t3], even if the current occurrence time of the current event is t, the current event is still counted based on the sliding window [t2, t3] instead of the sliding window [t1, t].

[0047] An associated event can be understood as an event that is associated with the current event. In combination with the application scenarios that may be involved in the embodiments of the present invention, for example, in the application scenario of counting the number of orders placed by different login objects of the same device, the associated event and the current event are both order events generated by the device.

[0048] The first array can be understood as an array storing the associated deduplication features and associated occurrence time of each associated event, that is, each associated deduplication feature and each associated occurrence time are stored in a flat form in the memory. In the embodiment of the present invention, the first array can be optionally called LongExpireMap.

[0049] The meaning of the associated deduplication feature is similar to that of the current deduplication feature, and will not be repeated here. The same is true for the association occurrence time, and will not be repeated here either.

[0050] In the embodiment of the present invention, by responding to an event deduplication request, a current deduplication feature, a current occurrence time, a first sliding window and a first array are obtained.

[0051] S120. For the associated deduplication features stored in the first array and matching the first sliding window, determine the repetitiveness between each matched associated deduplication feature and the current deduplication feature.

[0052] Among them, because deduplication statistics are performed under the first sliding window, the associated deduplication features whose associated occurrence time is within the first sliding window among all associated deduplication features can be used as matched associated deduplication features. Further, by determining the repetitiveness between each matched associated deduplication feature and the current deduplication feature, it is determined whether there is an associated deduplication feature among these associated deduplication features that is repeated with the current deduplication feature.

[0053] S130. When it is determined based on repeatability that there is an associated deduplication feature that is repeated with the current deduplication feature, the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array is processed according to the current occurrence time to complete the deduplication process of the current event.

[0054] Among them, when there are repeated associated deduplication features, this indicates that deduplication processing is required for the current event. At this time, the associated occurrence time corresponding to the repeated associated deduplication features stored in the first array can be processed according to the current occurrence time.

[0055] For example, when the current occurrence time is greater than the associated occurrence time, the associated occurrence time can be replaced by the current occurrence time, and the associated occurrence event corresponding to the associated occurrence time is used as the event to be deduplicated; otherwise, the current occurrence time can be discarded, and the current occurrence event is used as the event to be deduplicated. In the above two cases, the deduplication process related to the current event is actually completed.

[0056] The technical solution of the embodiment of the present invention can obtain the current deduplication features and current occurrence time of the current event, and obtain the first sliding window and the first array matching the current event by responding to the event deduplication request, wherein the first array stores the associated deduplication features and the associated occurrence time of each associated event associated with the current event; for the associated deduplication features stored in the first array that match the first sliding window, the repetitiveness between each matched associated deduplication feature and the current deduplication feature is determined to determine whether there are associated deduplication features that are repeated with the current deduplication features in the first array; in the case where it is determined that there are repeated associated deduplication features based on the repetitiveness, the associated occurrence time corresponding to the repeated associated deduplication features stored in the first array is processed according to the current occurrence time to complete the deduplication process of the current event. Compared to storing associated deduplication features and associated occurrence times in the form of HashMap for event deduplication, the above technical solution saves memory during the event deduplication process because there is no need to store additional pointers by storing them in the form of an array. In addition, the associated deduplication features cached in the memory can be directly used for repeatability judgment without involving multiple pointer query processes, thereby improving the deduplication performance.

[0057] An optional technical solution, the above event processing method further includes:

[0058] Obtaining a preset failure number threshold and the failure number of associated deduplication features stored in the first array that are not matched with the first sliding window;

[0059] When the number of failures is greater than or equal to the failure number threshold, the associated deduplication features and associated occurrence times of the failures in the first array are progressively cleaned up.

[0060] Among them, the invalid number threshold (validNum) can be understood as the minimum value of the invalid number when the invalid associated deduplication features and associated occurrence times are gradually cleaned up; the invalid number can be pre-set or determined based on at least one of the response frequency of event deduplication requests, the window size of the first sliding window, the sliding step size, and the size of the first array.

[0061] The expiration number (expireNum) can be understood as the number of associated deduplication features stored in the first array that are not matched with the first sliding window. Specifically, the number of associated deduplication features corresponding to the association occurrence time that is less than the start time (i.e., the expiration time, expireTime) can be taken as the expiration number.

[0062] In the embodiment of the present invention, the cleaning is not performed as soon as the failed associated deduplication features and associated occurrence times are detected. This is because if there are many failed associated deduplication features and associated occurrence times, delay spike problems are likely to occur. Therefore, in the embodiment of the present invention, when the number of failures is greater than or equal to the failure number threshold, these failed associated deduplication features and associated occurrence times are progressively cleaned to avoid delay spike problems and achieve a more efficient overall cleaning process.

[0063] Figure 2 It is a flow chart of another event processing method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, after obtaining the current deduplication feature of the current event, the event processing method also includes: hashing the current deduplication feature, and updating the current deduplication feature according to the obtained hash value. Among them, the explanations of the terms that are the same or corresponding to the above-mentioned embodiments are not repeated here.

[0064] See also Figure 2 The method of this embodiment may specifically include the following steps:

[0065] S210. In response to an event deduplication request, obtain the current deduplication features and current occurrence time of the current event, and obtain a first sliding window and a first array matching the current event, wherein the first array stores associated deduplication features and associated occurrence times of each associated event associated with the current event.

[0066] S220. Hash the current deduplication feature, and update the current deduplication feature according to the obtained hash value.

[0067] The hash value (hashValue) can be understood as the value obtained by hashing the current deduplication feature.

[0068] It is understandable that the first array is only used to determine whether the features are repeated, and it has no meaning of storing actual data. Therefore, the current deduplication features are hashed to save memory usage. Moreover, the xxHash algorithm used here has a collision rate of only one in a billion, which can fully meet the deduplication scenario.

[0069] S230. For the associated deduplication features stored in the first array and matching the first sliding window, determine the repetitiveness between each matched associated deduplication feature and the current deduplication feature.

[0070] S240. When it is determined based on repeatability that there is an associated deduplication feature that is repeated with the current deduplication feature, the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array is processed according to the current occurrence time to complete the deduplication process of the current event.

[0071] Compared with directly storing the current deduplication features in the memory, the technical solution of the embodiment of the present invention can convert the current deduplication features with a longer length into hash values ​​with a shorter length and store them in the memory by hashing the current deduplication features. This can save memory and reduce the amount of repetitive calculations, thereby further solving the problems of poor deduplication performance and memory waste.

[0072] An optional technical solution, after updating the current deduplication feature according to the obtained hash value, the event processing method further includes:

[0073] When it is determined according to the repeatability that there is no repeated associated deduplication feature in the first array, the current deduplication feature is processed to obtain a target slot in the first array;

[0074] If the target slot is occupied, determine a step slot in the first array that is a preset step length away from the target slot;

[0075] According to the step slot, the current deduplication feature and the current occurrence time are stored in the first array.

[0076] Among them, when there are no repeated associated deduplication features, this means that deduplication processing is not required, and the current deduplication features and current occurrence time can be directly stored in the first array to be used as associated deduplication features and associated occurrence time in subsequent event deduplication.

[0077] The target slot can be understood as the slot obtained after processing the current deduplication feature, and the current deduplication feature and the current occurrence time need to be stored according to the slot. Exemplarily, after hashing the current deduplication feature, the hash value can be taken as a remainder or equal to the value of the preset value, and the slot corresponding to the obtained processing result in the first array is used as the target slot.

[0078] In the embodiment of the present invention, when the target slot is not occupied, the current deduplication feature and the current occurrence time may be directly stored in the target slot.

[0079] In the embodiment of the present invention, when the target slot is occupied, it indicates that a hash conflict has occurred, and a step slot can be reselected with a preset step length to store the current deduplication feature and the current occurrence time. That is, the step slot can be understood as a slot with a preset step length from the target slot.

[0080] For example, see Figure 3, the current deduplication feature can be hashed to obtain hash value 2 (hash2), and thus {hash2, time2} can be obtained, wherein occurrence time 2 (time2) represents the current occurrence time; when there are no repeated associated deduplication features in the first array, hash2 can be processed to obtain the target slot; when the target slot is occupied by hash value 1 and occurrence time 1 (ie, {hash1, time1}), that is, when there is a hash conflict, determine the stride slot in the first array that is apart from the target slot by a preset step length (stride), and then store {hash2, time2} in the stride slot.

[0081] The above technical solution can realize the effective storage of the current deduplication features and the current occurrence time in the case of hash conflicts.

[0082] Based on the above solution, another optional technical solution is to store the current deduplication feature and the current occurrence time in the first array according to the step slot, including:

[0083] When the step slot is occupied, obtaining the expiration time corresponding to the first sliding window;

[0084] When the associated occurrence time of the occupied step slot is less than or equal to the expiration time, the associated deduplication feature stored in the step slot is replaced based on the current deduplication feature, and the associated occurrence time stored in the step slot is replaced based on the current occurrence time.

[0085] The expiration time can be understood as the maximum time when the association corresponding to the first sliding window fails; the expiration time can be the start time of the first sliding window, or a time adjacent to the start time and less than the start time, and so on.

[0086] In the case where the step slot is occupied, the expiration time may be obtained to determine whether the associated deduplication feature and the associated occurrence time stored in the step slot can be replaced.

[0087] In an embodiment of the present invention, when the associated occurrence time occupying the step slot is less than or equal to the expiration time, that is, it means that the associated deduplication feature and the associated occurrence time have expired and can be replaced, then the associated deduplication feature stored in the step slot can be replaced based on the current deduplication feature, and the associated occurrence time stored in the step slot can be replaced based on the current occurrence time.

[0088] In an embodiment of the present invention, when the associated occurrence time of the occupied step slot is greater than the expiration time, the step slot can be updated to the target slot, and the step of determining the step slot in the first array that is apart from the target slot by a preset step length is repeated; or, the first array is expanded to store the current deduplication features and the current occurrence time in a third array obtained by expanding the first array; and so on.

[0089] The above technical solution can avoid frequent expansion of the first array and save computing resources by replacing the invalid data occupying the step slot.

[0090] Figure 4 It is a flowchart of another event processing method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, in the process of progressively copying each associated deduplication feature and each associated occurrence time stored in the first array to the second array, the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array is processed according to the current occurrence time, which may include: when the repeated associated deduplication feature has not been copied to the second array, the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array is processed according to the current occurrence time; and / or, the above-mentioned event processing method also includes: when the repeated associated deduplication feature has been copied to the second array, the associated occurrence time corresponding to the repeated associated deduplication feature stored in the second array is processed according to the current occurrence time. Among them, the explanations of the terms that are the same as or corresponding to the above-mentioned embodiments are not repeated here.

[0091] See also Figure 4 The method of this embodiment may specifically include the following steps:

[0092] S310. In response to an event deduplication request, obtain the current deduplication features and current occurrence time of the current event, and obtain a first sliding window and a first array matching the current event, wherein the first array stores associated deduplication features and associated occurrence times of each associated event associated with the current event.

[0093] S320. For the associated deduplication features stored in the first array and matching the first sliding window, determine the repetitiveness between each matched associated deduplication feature and the current deduplication feature.

[0094] S330. In the process of progressively copying each associated deduplication feature and each associated occurrence time stored in the first array to the second array, if it is determined based on repeatability that there is an associated deduplication feature that is repeated with the current deduplication feature, it is determined whether the repeated associated deduplication feature is copied to the second array.

[0095] The second array may be understood as an array to which each associated deduplication feature and each associated occurrence time are required to be asymptotically copied.

[0096] It can be understood that, in the process of progressive copying, a rehash table (RehashTable) data structure can be created to copy the data stored in the first array to the second array in segments. Therefore, in an embodiment of the present invention, in the process of progressively copying each associated deduplication feature and each associated occurrence time stored in the first array to the second array, it can be determined whether the repeated associated deduplication feature is copied to the second array to determine whether to process the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array, or to process the associated occurrence time stored in the second array.

[0097] In an embodiment of the present invention, optionally, when it is determined based on repeatability that there are no repeated associated deduplication features in the first array, the repeatability between each window deduplication feature stored in the second array and matching the first sliding window and the current deduplication feature can be determined, so as to process the second array based on the determination result using the current occurrence time, or the current deduplication feature and the current occurrence time. This processing process is similar to that of processing the first array and will not be repeated here.

[0098] S340. When the repeated associated deduplication feature is not copied to the second array, the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array is processed according to the current occurrence time to complete the deduplication process of the current event.

[0099] In an embodiment of the present invention, when the repeated associated deduplication features are not copied to the second array, that is, it means that the second array may not have repeated associated deduplication features, the associated occurrence time corresponding to the repeated associated deduplication features stored in the first array can be processed according to the current occurrence time to complete the deduplication process of the current event.

[0100] S350. When the repeated associated deduplication feature has been copied to the second array, the associated occurrence time corresponding to the repeated associated deduplication feature stored in the second array is processed according to the current occurrence time to complete the deduplication process of the current event.

[0101] In an embodiment of the present invention, when the repeated associated deduplication features have been copied to the second array, it means that the repeated associated deduplication features already exist in the second array, and the corresponding associated occurrence time stored in the second array can be processed according to the current occurrence time to complete the deduplication process of the current event.

[0102] The technical solution of the embodiment of the present invention can avoid performance problems caused by one-time copying when the number of copies is large by progressively copying each associated deduplication feature and each associated occurrence time stored in the first array to the second array, and on this basis, different processing methods are implemented for repeated associated deduplication features that have been copied and those that have not been copied to the second array, which can avoid deduplication errors or deduplication failures.

[0103] An optional technical solution, the above event processing method further includes:

[0104] Obtaining a storage slot of the repeated associated deduplication feature in the first array and obtaining a copy slot, wherein the copy slot is used to indicate the slot in the first array where the associated deduplication feature that is most recently copied from the first array to the second array is located;

[0105] According to the positional relationship between the storage slot and the copy slot, it is determined whether the repeated associated deduplication feature has been copied to the second array.

[0106] The storage slot may be understood as a slot in the first array that stores repeated associated deduplication features.

[0107] The copy slot (resizeOffset) can be understood as being used to indicate the slot in the first array where the associated deduplication feature most recently copied from the first array to the second array is located, that is, the slot in the first array where the associated deduplication feature most recently copied from the current time is located.

[0108] In an embodiment of the present invention, it can be determined whether the repeated associated deduplication features have been copied to the second array based on the positional relationship. For example, when the positional relationship represents that the number of the storage slot in the first array is smaller than the number of the copy slot in the first array, it means that the associated deduplication features have been copied to the second array. Otherwise, it means that the associated deduplication features have not been copied to the second array.

[0109] In an embodiment of the present invention, by determining whether the repeated associated deduplication features have been copied to the second array based on the positional relationship between the storage slot and the copy slot, it is possible to accurately determine whether the repeated associated deduplication features have been copied to the second array.

[0110] Another optional technical solution, the above event processing method further includes:

[0111] For a memory used to store the second array, the memory is divided into a plurality of memory blocks;

[0112] For each uninitialized memory block among the multiple memory blocks, the memory block is initialized when the memory block is required to store the associated deduplication features and the associated occurrence time copied from the first array to the second array.

[0113] The memory block can be understood as a continuous storage space in the memory, which can be used to store data.

[0114] It is understandable that when the amount of data stored in the first array is too much, it is easy to form time-consuming delay spikes in the online scenario when applying for memory each time when expanding, shrinking or copying, and the performance is mainly consumed in the process of initializing the large memory to 0. Therefore, in order to avoid initializing the large memory to 0 at one time, the memory can be divided into multiple memory blocks. In an embodiment of the present invention, for each uninitialized memory block, the memory block can be initialized when the memory block needs to be used to store the associated deduplication features and the associated occurrence time copied from the first array to the second array.

[0115] Exemplarily, the memory can be divided into multiple memory blocks (Segments), and a Boolean array is used to store a mark to indicate whether each memory block has been initialized. In conjunction with progressive copying of data, for each memory block that has not been initialized, if the memory block needs to be used to store associated deduplication features and associated occurrence times copied from the first array to the second array, the memory block is initialized, and the initialized memory card is marked as true, indicating that it has been initialized, thereby avoiding repeated initialization.

[0116] In the embodiment of the present invention, for each uninitialized memory block, the memory block is initialized when it is needed to store data, thereby avoiding the formation of time-consuming delay spikes.

[0117] In order to better understand the technical solution of the above-mentioned embodiment of the present invention, an optional example is provided here. Exemplarily, in the process of progressively copying the data stored in the first array to the second array, it is checked whether there are repeated associated deduplication features in the first array; if there are no repeated associated deduplication features in the first array, it is checked whether there are repeated associated deduplication features in the second array; if there are repeated associated deduplication features in the second array, the associated occurrence time corresponding to the repeated associated deduplication features stored in the second array is processed according to the current occurrence time; if there are no repeated associated deduplication features in the second array, the current deduplication features and the current occurrence time are stored in the second array; if there are repeated associated deduplication features in the first array, it can be determined whether the repeated associated deduplication features have been copied to the second array according to the storage slot and position relationship. If they have been copied to the second array, the step of searching whether there are repeated associated deduplication features in the second array is performed; if they have not been copied to the second array, the associated occurrence time corresponding to the repeated associated deduplication features stored in the first array is processed according to the current occurrence time.

[0118] Figure 5 It is a flowchart of another event processing method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, the event processing method also includes: obtaining window statistics for a second sliding window that matches the previous event of the current event, wherein the window statistics are deduplicated statistics of the previous event under the second sliding window; obtaining the current event value of the current event and the associated event value of the associated event corresponding to the repeated associated deduplication feature, and obtaining the deduplicated statistics of the current event under the first sliding window based on the window statistics, the associated event value and the current event value. The explanations of the terms that are the same or corresponding to the above-mentioned embodiments are not repeated here.

[0119] See also Figure 5 The method of this embodiment may specifically include the following steps:

[0120] S410. In response to an event deduplication request, obtain the current deduplication features and current occurrence time of the current event, and obtain a first sliding window and a first array matching the current event, wherein the first array stores associated deduplication features and associated occurrence times of each associated event associated with the current event.

[0121] S420. For the associated deduplication features stored in the first array and matching the first sliding window, determine the repetitiveness between each matched associated deduplication feature and the current deduplication feature.

[0122] S430. When it is determined based on repeatability that there is an associated deduplication feature that is repeated with the current deduplication feature, the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array is processed according to the current occurrence time to complete the deduplication process of the current event.

[0123] S440. For a second sliding window that matches a previous event of the current event, obtain a window statistic value obtained after aggregating the previous event value of the previous event into the second sliding window.

[0124] The previous event value may be understood as the event value of the previous event.

[0125] The window statistics (Cache) can be understood as the deduplication statistics of the previous event under the second sliding window. The window statistics can also be understood as the statistical value of the window event values ​​of each window event aggregated under the second sliding window; the window statistics can be stored in a third array, storing the window statistics, and using the window statistics as one of the factors for determining the deduplication statistics, which can speed up the deduplication statistics.

[0126] S450. Obtain the current event value of the current event and the associated event value of the associated event corresponding to the repeated associated deduplication feature, and obtain the deduplication statistical value of the current event under the first sliding window based on the window statistical value, the associated event value and the current event value.

[0127] Among them, the current event value (Value) can be understood as the numerical value of the current event that specifically requires deduplication statistics. In combination with the application scenarios that may be involved in the embodiments of the present invention, optionally, in the case of requiring deduplication statistics increment value, such as counting the number of orders placed by different login objects of the same device, the current event value can be, for example, an increment value (increment), that is, the number is increased by 1 each time the device places an order.

[0128] In the embodiment of the present invention, the current event value and the associated event value may be obtained, and then the deduplication statistical value may be obtained according to the window statistical value, the associated event value and the current event value.

[0129] In the embodiment of the present invention, the window statistical value and the current event value may be aggregated, and the influence of the associated event value may be removed from the aggregation result to obtain a deduplicated statistical value.

[0130] In the embodiment of the present invention, the window size of the first sliding window can be fixed, and this situation can be regarded as a single window scenario, that is, the deduplication statistics are within the sliding window of the fixed window size. In combination with the application scenarios that may be involved in the embodiment of the present invention, optionally, the number of different login objects of the same device within 1 day can be deduplicated. The solution of the embodiment of the present invention is particularly suitable for single window scenarios. If the window size of the first sliding window needs to be changed, a new data structure of this type can be recreated for statistics.

[0131] The technical solution of the embodiment of the present invention obtains deduplication statistics through window statistics, associated event values ​​and current event values. Firstly, by associating event values, it avoids the statistical value of the current event under the first sliding window from being affected by duplicate data, thereby improving the accuracy of deduplication statistics; secondly, by utilizing window statistics, the efficiency of deduplication statistics is improved.

[0132] In an optional technical solution, when the first sliding window is obtained by sliding the second sliding window, the event processing method further includes:

[0133] When there is an elimination time granularity that is not in the first sliding window among all the time granularities of the second sliding window, obtaining an elimination statistical value at the elimination time granularity, wherein the elimination statistical value is obtained by counting the granular event values ​​of each granularity event aggregated to the elimination time granularity;

[0134] According to the window statistics, the associated event value and the current event value, the deduplication statistics of the current event in the first sliding window are obtained, including:

[0135] According to the window statistics, the elimination statistics, the associated event values ​​and the current event value, the deduplication statistics of the current event in the first sliding window are obtained.

[0136] Among them, there are multiple time granularities in the second sliding window, wherein, for each time granularity, the time granularity can be understood as the time obtained by dividing the aggregation occurrence time of the aggregation events aggregated to the time granularity by the granularity, and the granularity can be understood as the sliding step of the second sliding window (that is, the minimum time unit when the second sliding window slides). Exemplarily, the window size of the second sliding window is 10s, and the minimum time unit is 1s, then every 1s in the second sliding window can be used as a time granularity. As another example, assuming that the aggregation occurrence time of a certain aggregation event is 2023-08-29 18:01:01:100, when the minimum time unit is 1 second, the corresponding time granularity is 2023-08-29 18:01:01.

[0137] It should be noted that a time granularity may have a granularity statistic value, which may be understood as a statistic value of granularity event values ​​aggregated at each granularity event at its corresponding time granularity.

[0138] In an embodiment of the present invention, all granular statistical values ​​can be stored in a third array (Array) sorted by granularity such as time seconds or minutes, and the third array can also store the time granularity corresponding to each granular statistical value and / or the number of granular events aggregated under the time granularity corresponding to each granular statistical value, so as to obtain the above data stored in the third array for event statistics. It should be noted that in the online computing statistics scenario, the data stored in the third array can be ordered, which can ensure the rapid positioning of the eliminated statistical values ​​and the efficient insertion of data into the third array.

[0139] The elimination time granularity can be understood as the time granularity that is not located under the first sliding window.

[0140] The elimination statistics value can be understood as the granularity statistics value at the elimination time granularity; the elimination statistics value is obtained by statistically analyzing the granularity event values ​​of each granularity event aggregated at the elimination time granularity.

[0141] It can be understood that the first sliding window is obtained by sliding the second sliding window, which means that the aggregated statistics in the window statistics may have granularity statistics that do not belong to the first sliding window. In order to ensure that the deduplication statistics are statistics under the first sliding window, the elimination statistics can be obtained when there are elimination time granularities that are not located in the first sliding window in all time granularities. For example, the granularity statistics under the elimination time granularity can be determined from all granularity statistics as the elimination statistics, so that the deduplication statistics under the first sliding window can be obtained later through the elimination statistics.

[0142] In the embodiment of the present invention, when the elimination statistics are obtained, the deduplication statistics can be obtained through the window statistics, the elimination statistics, the associated event values ​​and the current event values.

[0143] The technical solution of the embodiment of the present invention can ensure that the deduplication statistical value is a statistical value under the first sliding window by using the determined elimination statistical value as a reference factor for obtaining the deduplication statistical value, thereby avoiding the influence of the elimination statistical value that does not belong to the first sliding window on the accuracy of the deduplication statistical value.

[0144] Another optional technical solution is an event processing method, wherein the event processing method further includes:

[0145] Obtaining the associated time granularity corresponding to the associated event value and the current time granularity corresponding to the current event value in all time granularities of the first sliding window;

[0146] The granularity statistics value at the associated time granularity is subtracted from the associated event value, and the granularity statistics value at the current time granularity is added to the current event value.

[0147] The associated time granularity is the time granularity to which the associated event corresponding to the associated event value belongs in all time granularities. The current time granularity is the time granularity to which the current event belongs in all time granularities. The associated time granularity and the current time granularity can be obtained.

[0148] Subtract the associated event value from the granularity statistics at the associated time granularity, and update the first result obtained to the granularity statistics at the associated time granularity; and add the current event value to the granularity statistics at the current time granularity, and update the second result obtained to the granularity statistics at the current time granularity.

[0149] Exemplarily, in a scenario where the deduplication statistics are counts (COUNT) or sums, when the current occurrence time is less than or equal to the associated occurrence time, the current event is discarded, and the granular statistical values ​​at the associated time granularity and the current time granularity are not processed; when the current occurrence time is greater than the associated occurrence time, in a counting scenario, the granular statistical value at the associated time granularity can be subtracted by 1, and the granular statistical value at the current time granularity can be added by 1; in a counting scenario, the granular statistical value at the associated time granularity can be subtracted from the associated event value, and the granular statistical value at the current time granularity can be added to the current event value.

[0150] In the embodiment of the present invention, it can be ensured that all time granularities are not affected by repeated events, and the accuracy of deduplication statistics is guaranteed when deduplication statistics of events are performed based on statistics of each granularity.

[0151] Figure 6 It is a flowchart of another event processing method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, the event processing method also includes: obtaining a target tree that matches the current event and has the same structure as the B+ tree, wherein the leaf nodes of the target tree store granularity statistics at the time granularity represented by the leaf nodes; for the first node corresponding to the repeated associated deduplication feature in all leaf nodes, obtaining the associated event value of the associated event corresponding to the repeated associated deduplication feature, and updating the granularity statistics stored in the first node based on the associated event value; for the second node corresponding to the current deduplication feature in all leaf nodes, based on the current event value of the current event, updating the granularity statistics stored in the second node. Among them, the explanations of the terms that are the same or corresponding to the above-mentioned embodiments are not repeated here.

[0152] See also Figure 6 The method of this embodiment may specifically include the following steps:

[0153] S510. In response to an event deduplication request, obtain the current deduplication features and current occurrence time of the current event, and obtain a first sliding window and a first array matching the current event, wherein the first array stores associated deduplication features and associated occurrence times of each associated event associated with the current event.

[0154] S520. For the associated deduplication features stored in the first array and matching the first sliding window, determine the repetitiveness between each matched associated deduplication feature and the current deduplication feature.

[0155] S530. When it is determined based on repeatability that there is an associated deduplication feature that is repeated with the current deduplication feature, the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array is processed according to the current occurrence time to complete the deduplication process of the current event.

[0156] S540. Obtain a target tree that matches the current event and has the same structure as the B+ tree, wherein the leaf nodes of the target tree store granularity statistics at the time granularity represented by the leaf nodes.

[0157] Among them, B+ tree (B Plus Tree) is a multi-way balanced search tree.

[0158] The target tree is a tree that matches the current event and has the same structure as the B+ tree; the target tree can be, for example, a tree that has stored granularity statistics at the current time granularity that matches the current event, or a tree that has the same structure as the B+ tree and can store granularity statistics at the current time granularity that matches the current event.

[0159] Leaf nodes can be understood as nodes without child nodes in the target tree; leaf nodes can be connected without linked lists, thereby reducing the overhead of inserting data and pointer overhead. Leaf nodes can store granularity statistics at the time granularity represented by the leaf node, that is, each leaf node can represent a time granularity, and each leaf node can store granularity statistics at the time granularity it represents.

[0160] S550. For the first node corresponding to the repeated associated deduplication feature in all leaf nodes, obtain the associated event value of the associated event corresponding to the repeated associated deduplication feature, and update the granularity statistics value stored in the first node based on the associated event value.

[0161] Among them, the first node can be understood as a leaf node corresponding to repeated associated deduplication features among all leaf nodes, and can also be understood as a leaf node representing the associated time granularity among all leaf nodes.

[0162] In an embodiment of the present invention, for the first node, the associated event value can be obtained, and the granular statistical value stored in the first node can be updated based on the associated event value. For example, the statistics of the associated event value in the granular statistical value stored in the first node can be removed, so that the updated granular statistical value is the statistical value of each granular event value except the associated event value at the time granularity represented by the first node; for another example, the associated occurrence time and the current occurrence time can be compared, and when the current occurrence time is greater than the associated occurrence time, the statistics of the associated event value in the granular statistical value stored in the first node are removed.

[0163] In an embodiment of the present invention, the association occurrence time and the current occurrence time can also be compared. When the current occurrence time is greater than the association occurrence time, the association event value of the association event corresponding to the repeated association deduplication feature can be obtained, and based on the association event value, the granular statistical value stored in the first node can be updated by removing the statistics of the association event value in the granular statistical value stored in the first node.

[0164] The leaf nodes in the target tree may be sorted according to the time granularity of the representation, and thus the first node (index n) may be found in the target tree in reverse order by the associated occurrence time corresponding to the associated event value.

[0165] S560: For the second node corresponding to the current deduplication feature in all leaf nodes, based on the current event value of the current event, update the granularity statistics stored in the second node.

[0166] Among them, the second node can be understood as the leaf node corresponding to the current deduplication feature among all leaf nodes, and can also be understood as the leaf node representing the current time granularity among all leaf nodes.

[0167] For the second node, the current event value can be obtained, and the granular statistics stored in the second node can be updated based on the current event value (chilren[n].statis). For example, the statistics of the associated event value in the granular statistics stored in the first node can be added, so that the updated granular statistics are the statistics of each granular event value including the current event value at the time granularity represented by the second node; for another example, the associated occurrence time and the current occurrence time can be compared, and when the current occurrence time is greater than the associated occurrence time, the statistics of the associated event value in the granular statistics stored in the first node are added.

[0168] In the embodiment of the present invention, the correlation occurrence time and the current occurrence time may be compared. When the current occurrence time is greater than the correlation occurrence time, the granularity statistics value stored in the second node may be updated based on the current event value.

[0169] There may also be a case where there is no second node corresponding to the current deduplication feature in all leaf nodes. In this case, a second node (chilren[n]) corresponding to the current deduplication feature may be inserted into the target tree.

[0170] The technical solution of the embodiment of the present invention, by acquiring the target tree, and then updating the granularity statistics stored in the first node based on the associated event value, and updating the granularity statistics stored in the second node based on the current event value, can make the granularity statistics stored in the target tree all granularity statistics that are not affected by repeated events, thereby ensuring the accuracy of the stored granularity statistics.

[0171] An optional technical solution, the directory node of the target tree stores the statistical results of the granularity statistical values ​​stored in each child node of the directory node, and the directory node is a node other than a leaf node in the target tree; the above event processing method also includes:

[0172] Based on the associated event value, the statistical results respectively stored in the ancestor nodes of the first node are updated, and based on the current event value, the statistical results respectively stored in the ancestor nodes of the second node are updated.

[0173] Among them, the directory node can be understood as a node in the target tree other than a leaf node, that is, a node with child nodes in the target tree. For each directory node, the node directly pointed to by the directory node is the child node of the directory node. For each directory node, the statistical results stored in the directory node may include the results obtained by statistically analyzing the granularity statistical values ​​stored in each of its child nodes, and at least one of the granularity statistical values ​​corresponding to the event granularity represented by each of its corresponding leaf nodes; the statistical results may also include the number of granular events aggregated by the time granularity represented by each of its corresponding leaf nodes.

[0174] For each node of the target tree, an ancestor node can be understood as a node on the path from the root node to the parent node of the node.

[0175] In the embodiment of the present invention, the statistical results respectively stored in each ancestor node of the first node may be updated based on the associated event value. For example, the statistics of the associated event value in the statistical results respectively stored in the first node may be removed; for another example, the associated occurrence time and the current occurrence time may be compared, and when the current occurrence time is greater than the associated occurrence time, the statistics of the associated event value in the statistical results respectively stored in the first node may be removed.

[0176] In an embodiment of the present invention, the statistical results respectively stored in each ancestor node of the second node may be updated based on the current event value. For example, the statistics of the current event value may be added to the statistical results respectively stored in the second node; for another example, the associated occurrence time and the current occurrence time may be compared, and when the current occurrence time is greater than the associated occurrence time, the statistics of the associated event value may be added to the statistical results respectively stored in the first node.

[0177] In an embodiment of the present invention, the associated occurrence time and the current occurrence time can be compared. When the current occurrence time is greater than the associated occurrence time, the statistical results stored in each ancestor node of the first node are updated, and based on the current event value, the statistical results stored in each ancestor node of the second node are updated.

[0178] For example, see Figure 7 , the current occurrence time corresponds to time granularity 2 (time2), and the current event value used to update the granularity statistics stored in the second node can be represented as {time2, increment}. According to {time2, increment}, the granularity statistics stored in the second node representing time granularity 2 (time2) and the statistical results (statis) stored in the directory nodes of the first and second layers representing time granularity 1 (time1) in the structure of the target tree can be updated. For example, increment can be added to statis.

[0179] The technical solution of the embodiment of the present invention, by updating the statistical results respectively stored in each ancestor node of the first node based on the associated event value, and updating the statistical results respectively stored in each ancestor node of the second node based on the current event value, can make the statistical results stored in the target tree all statistical results that are not affected by repeated events, thereby ensuring the accuracy of the stored statistical results.

[0180] Based on the above solution, in another optional technical solution, the time granularity represented by the directory node is the smallest time granularity among the time granularities represented by each child node of the directory node, and the deadline of the first sliding window is the largest time granularity corresponding to the target tree; the above method also includes:

[0181] Obtain the start time of the first sliding window, and perform path search on the target tree based on the start time to obtain the path node;

[0182] For each path node, when the path node is a directory node, obtain the statistical results stored in the directory node that is structurally adjacent to the path node, and when the path node is a leaf node, obtain the granularity statistics stored in the leaf node that represents a time granularity greater than the path node and corresponds to the same directory node as the path node;

[0183] Based on all obtained granular statistical values ​​and all statistical results, a deduplication statistical value of the current event in the first sliding window is obtained.

[0184] The deadline can be understood as the maximum time granularity corresponding to the target tree.

[0185] The start time of the first sliding window is the minimum time of the first sliding window; the start time can be determined according to the setting of the object, and can also be determined according to the window size of the first sliding window set by the object. In the embodiment of the present invention, the window range of the first sliding window can be optionally changed arbitrarily, but the deduplication event statistics are based on the event with the largest occurrence time, that is, the deadline cannot be changed arbitrarily, the start time and the window size can be arbitrarily not fixed, and the solution of the embodiment of the present invention supports path node search at the start time of any first sliding window.

[0186] The path node can be understood as a node obtained by performing a path search on the target tree based on the start time.

[0187] The start time may be obtained, a path search may be performed on the target tree based on the start time, and the node obtained by the search may be used as a path node. In the embodiment of the present invention, the method of obtaining the start time and performing a path search on the target tree based on the start time to obtain the path node is not specifically limited.

[0188] In an embodiment of the present invention, for example, the starting leaf node to which the time granularity corresponding to the start time belongs on the target tree can be determined, and the adjacent leaf nodes whose represented time granularity is smaller than the time granularity corresponding to the start time and are structurally adjacent to the starting leaf node can be determined. The adjacent leaf nodes and the ancestor nodes of the adjacent leaf nodes can be used as path nodes.

[0189] For example, see Figure 8 , based on time granularity 3 (time3) as the start time, path search can be started from the root node, and two directory nodes representing time granularity 1 (time1) and a leaf node representing time granularity 2 (time2) can be found as path nodes. Figure 8 The time granularity 2 in is the time granularity corresponding to the open interval, so the start time is time3.

[0190] It can be understood that, since granularity statistics can be stored in the target tree, and the target tree has the same structure as the B+ tree, the target tree can be used to perform deduplication statistics on the current event, and in particular, can be used to perform deduplication statistics on the current event under the first sliding window of different window sizes. Therefore, the deduplication statistics can be determined through the target tree. Specifically, for each path node, when the path node is a directory node, the statistical results stored in the directory node that is structurally adjacent to the path node can be obtained, and when the path node is a leaf node, the granularity statistics stored in the leaf node that represents a time granularity greater than the path node and structurally corresponds to the same directory node as the path node can be obtained. Based on all the obtained granularity statistics and all the statistical results, the deduplication statistics are obtained, for example, all the granularity statistics and all the statistical results are counted to obtain the deduplication statistics.

[0191] For example, see Figure 8 , according to each path node, for each path node, when the path node is a directory node representing time granularity 1 (time1), the statistical results stored in the directory nodes representing time granularity 5 (time5) and time granularity 9 (time9) respectively adjacent to the path node in structure are obtained, and when the path node is a leaf node representing time granularity 2 (time2), the granularity statistical values ​​stored in the leaf nodes representing time granularity 3 (time3) and time granularity 4 (time4) respectively, which represent a time granularity greater than the path node and structurally correspond to the same directory node as the path node, are obtained. Based on all the obtained granularity statistical values ​​and all the statistical results, the deduplication statistical value under (time2, time16] is obtained, that is, the statistical results of the directory nodes representing time granularity 9 (time9) and time granularity 5 (time5) respectively in the left half of the structure of the target tree, and the granularity statistical values ​​of the leaf nodes representing time granularity 3 (time3) and time granularity 4 (time4) are counted, and the obtained statistical results are used as the deduplication statistical value.

[0192] In an embodiment of the present invention, the window size of the first sliding window may vary with the start time, and this situation can be regarded as a multi-window scenario, that is, the deduplication statistics are in multiple sliding windows of different window sizes. It should be noted that the time consumption of adopting the solution of the embodiment of the present invention in a multi-window scenario may be higher than the time consumption of adopting the solution of the embodiment of the present invention in a single-window scenario, and the memory consumption of adopting the solution of the embodiment of the present invention in a multi-window scenario may be greater, but in the case of the need for statistics of multiple window events, the solution of the embodiment of the present invention will be much lower than the total time consumption of query statistics of the solution of the embodiment of the present invention through multiple windows in a single-window scenario, and the memory storage will also be lower because only one copy of the detailed data needs to be stored.

[0193] An optional example can determine the method of deduplication statistics based on a predetermined deduplication statistics scenario. For example, in a single-window scenario, a step can be performed for the second sliding window to obtain the window statistic value obtained after aggregating the previous event value of the previous event into the second sliding window, so as to perform deduplication statistics in the single-window scenario; in a multi-window scenario, a step can be performed to obtain the start time, so as to perform deduplication statistics in the multi-window scenario.

[0194] In an embodiment of the present invention, through the path node, and when the path node is a directory node, the statistical results stored in the directory node that is structurally adjacent to the path node are obtained, and when the path node is a leaf node, the granularity statistical value stored in the leaf node that represents a time granularity greater than the path node and corresponds to the same directory node as the path node is obtained, based on all the obtained granularity statistical values ​​and all the statistical results, the deduplication statistical value of the current event under the first sliding window is obtained, and the deduplication statistics of the current event can be achieved when the start time is not fixed.

[0195] In another optional technical solution, the directory node in the target tree is a node other than a leaf node in the target tree, and the above event processing method further includes:

[0196] For a target node whose child nodes are leaf nodes representing the smallest time granularity among all directory nodes, if the time granularity represented by each child node of the target node does not fall within the first sliding window, all child nodes of the target node are deleted.

[0197] The target node may be understood as a directory node whose child node is a leaf node representing the smallest time granularity among all directory nodes.

[0198] It is understandable that in the subsequent event processing process, the first sliding window may slide as the second sliding window. As the window slides, it is not necessary to trigger the deletion of nodes every time the window slides. When the time granularity represented by each child node of the target node does not fall within the maximum sliding window, all child nodes of the target node can be deleted. For example, see Fig. 9 , when the time granularity represented by the lower left leaf node in the structure of the target tree is less than the maximum start time of the maximum sliding window, that is, the maximum start time is greater than the maximum time granularity represented by each child node of the target node, the entire lower left leaf node can be deleted. After deleting the entire lower left leaf node, the pointer lowerLeftNode of the lower left leaf node can also be updated to quickly delete the lower left leaf node next time. It should be noted that the window size of the first sliding window can be unfixed, but in order to delete nodes, a fixed maximum sliding window (MaxWindow) can be set to determine the expiration and deletion of nodes.

[0199] In an embodiment of the present invention, for a target node, when the time granularities respectively represented by the child nodes of the target node do not fall within the first sliding window, that is, the time granularities respectively represented by the child nodes of the target node are all elimination time granularities, all the child nodes of the target node can be deleted to achieve elimination of the nodes corresponding to the elimination time granularities in the target tree.

[0200] In an embodiment of the present invention, a directory node may store statistical results, and the statistical results may include the number of granular events aggregated at the time granularity represented by its corresponding leaf nodes. Therefore, the number of deleted granular events aggregated at the time granularity represented by the deleted leaf nodes can be calculated through the statistical results. Specifically, after deleting all child nodes of the target node, the call of the addExpireNum function can be triggered, and the deleted data that needs to be deleted in the first array is determined based on the numerical relationship between the calculated deletion number and the invalid number threshold, and the first array is cleaned by deleting the data.

[0201] In an embodiment of the present invention, after deleting all child nodes of the target node, the statistical results stored in the ancestor nodes of the target node can be updated according to all deleted child nodes to remove the influence of the data stored in all deleted child nodes on the statistical results stored in the ancestor nodes. The algorithm complexity of calculating the number of deletions and updating the statistical results can be O(logN).

[0202] In an embodiment of the present invention, by deleting all child nodes of the target node when the time granularity represented by each child node of the target node does not fall within the first sliding window, it is possible to avoid waste of resources caused by frequent deletion of leaf nodes and ensure the balance of the target tree.

[0203] In order to better understand the technical solution of the above-mentioned embodiment of the present invention, an optional example is provided herein. Exemplarily, the difference between the target tree and the B+ tree may be that: the directory node of the target tree indexes the minimum time in the time granularity represented by each child node under it, and ensures that the time granularity represented by the root node (times[0]) is the minimum time granularity among its leaf nodes; the directory node of the target tree may store the statistical results (statis) of the granularity statistics stored in each child node of the directory node, so as to speed up the event statistics; the leaf nodes (childrens) of the target tree store the granularity statistics under the time granularity (times) represented by the leaf nodes, and may also store the time granularity represented by the leaf nodes. The opinion leaf node represents at least one of the number of granular events aggregated at the time granularity. The target tree can store a lower left leaf pointer (lowerLeftNode) to speed up the query of whether the node should be deleted. The target tree is relative to the fixed-length node in the B+ tree. Since the occurrence time or time granularity passed in by the target tree is ordered as a whole, the left node of the target tree can be shrunk by half after the split, and since the occurrence time or time granularity is ordered as a whole, the left node of the target tree generally will not insert data to ensure that the data stored in the directory node or leaf node can fill the entire array, and the stored data is in a flat filling state, which is an efficient use of memory.

[0204] Fig.10 This is a structural block diagram of an event processing device provided in an embodiment of the present invention, and the device is used to execute the event processing method provided in any of the above embodiments. The device and the event processing method of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the event processing device, refer to the embodiment of the above event processing method. Fig.10 The device may specifically include: a first array acquisition module 610, a repeatability determination module 620 and an association occurrence time processing module 630.

[0205] The first array acquisition module 610 is used to obtain the current deduplication feature and the current occurrence time of the current event in response to the event deduplication request, and obtain the first sliding window and the first array matching the current event, wherein the first array stores the associated deduplication feature and the associated occurrence time of each associated event associated with the current event;

[0206] The repeatability determination module 620 is used to determine the repeatability between each associated deduplication feature matched with the current deduplication feature, for the associated deduplication features stored in the first array and matched with the first sliding window;

[0207] The associated occurrence time processing module 630 is used to process the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array according to the current occurrence time when it is determined based on repeatability that there is an associated deduplication feature that is repeated with the current deduplication feature, so as to complete the deduplication process of the current event.

[0208] Optionally, the device may further include:

[0209] The current deduplication feature update module is used to hash the current deduplication feature after obtaining the current deduplication feature of the current event, and update the current deduplication feature according to the obtained hash value.

[0210] Optionally, based on the above device, the device may further include:

[0211] A target slot obtaining module is used for, after updating the current deduplication feature according to the obtained hash value, processing the current deduplication feature to obtain the target slot in the first array when it is determined according to the repeatability that there is no repeated associated deduplication feature in the first array;

[0212] A step slot determination module, used to determine a step slot in the first array that is a preset step distance from the target slot when the target slot is already occupied;

[0213] The current occurrence time storage module is used to store the current deduplication feature and the current occurrence time in the first array according to the step slot.

[0214] Optionally, based on the above device, the current occurrence time storage module may include:

[0215] The expiration time acquisition submodule is used to acquire the expiration time corresponding to the first sliding window when the step slot is occupied;

[0216] The associated occurrence time replacement submodule is used to replace the associated deduplication features stored in the step slot based on the current deduplication features when the associated occurrence time of the occupied step slot is less than or equal to the expiration time, and to replace the associated occurrence time stored in the step slot based on the current occurrence time.

[0217] Optionally, the device may further include:

[0218] A failure number acquisition module, used to acquire a preset failure number threshold and the failure number of the associated deduplication features stored in the first array that are not matched with the first sliding window;

[0219] The progressive cleaning module is used to progressively clean up the failed associated deduplication features and associated occurrence times in the first array when the failure number is greater than or equal to the failure number threshold.

[0220] Optionally, in the process of progressively copying each associated deduplication feature and each associated occurrence time stored in the first array to the second array,

[0221] The association occurrence time processing module 630 may include:

[0222] A first association occurrence time processing submodule is used to process the association occurrence time corresponding to the repeated association deduplication feature stored in the first array according to the current occurrence time when the repeated association deduplication feature has not been copied to the second array; and / or,

[0223] The device may also include:

[0224] The second association occurrence time processing submodule is used to process the association occurrence time corresponding to the repeated association deduplication feature stored in the second array according to the current occurrence time when the repeated association deduplication feature has been copied to the second array.

[0225] Optionally, based on the above device, the device may further include:

[0226] A copy slot acquisition module, used to acquire the storage slot of the repeated associated deduplication feature in the first array and to acquire the copy slot, wherein the copy slot is used to indicate the slot in the first array where the associated deduplication feature most recently copied from the first array to the second array is located;

[0227] The copy determination module is used to determine whether the repeated associated deduplication feature has been copied to the second array according to the positional relationship between the storage slot and the copy slot.

[0228] Optionally, based on the above device, the device may further include:

[0229] A memory partitioning module, used for partitioning the memory used for storing the second array into a plurality of memory blocks;

[0230] The memory block initialization module is used to initialize each memory block that has not been initialized among the multiple memory blocks when the memory block needs to be used to store the associated deduplication features and associated occurrence time copied from the first array to the second array.

[0231] Optionally, the device may further include:

[0232] A window statistics acquisition module, used to acquire a window statistics value for a second sliding window matching a previous event of a current event, wherein the window statistics value is a deduplicated statistics value of the previous event under the second sliding window;

[0233] The deduplication statistics obtaining module is used to obtain the current event value of the current event and the associated event value of the associated event corresponding to the repeated associated deduplication features, and obtain the deduplication statistics of the current event under the first sliding window based on the window statistics, the associated event value and the current event value.

[0234] Optionally, based on the above device, when the first sliding window is obtained by sliding the second sliding window, the device may further include:

[0235] An elimination statistics value acquisition module is used to obtain the elimination statistics value at the elimination time granularity when there is an elimination time granularity that is not located in the first sliding window among all the time granularities of the second sliding window, wherein the elimination statistics value is obtained by counting the granular event values ​​of each granularity event aggregated to the elimination time granularity;

[0236] The modules for obtaining deduplication statistics may include:

[0237] The deduplication statistics obtaining submodule is used to obtain the deduplication statistics of the current event under the first sliding window according to the window statistics, the elimination statistics, the associated event values ​​and the current event value.

[0238] Optionally, based on the above device, the device may further include:

[0239] A current time granularity acquisition module, used to acquire the associated time granularity corresponding to the associated event value and the current time granularity corresponding to the current event value from all time granularities of the first sliding window;

[0240] The current event value addition and subtraction module is used to subtract the associated event value from the granularity statistics value at the associated time granularity, and to add the current event value to the granularity statistics value at the current time granularity.

[0241] Optionally, the device may further include:

[0242] A target tree acquisition module is used to acquire a target tree that matches the current event and has the same structure as the B+ tree, wherein the leaf nodes of the target tree store granularity statistics at the time granularity represented by the leaf nodes;

[0243] A first granularity statistics value updating module is used to obtain, for a first node corresponding to a repeated association deduplication feature in all leaf nodes, an associated event value of an associated event corresponding to the repeated association deduplication feature, and update the granularity statistics value stored in the first node based on the associated event value;

[0244] The second granularity statistics updating module is used to update the granularity statistics stored in the second node corresponding to the current deduplication feature in all leaf nodes based on the current event value of the current event.

[0245] Optionally, based on the above device, the directory node of the target tree stores the statistical results of the granularity statistical values ​​respectively stored in each child node of the directory node, and the directory node is a node other than a leaf node in the target tree;

[0246] The device may also include:

[0247] The statistical result updating module is used to update the statistical results respectively stored in each ancestor node of the first node based on the associated event value, and to update the statistical results respectively stored in each ancestor node of the second node based on the current event value.

[0248] Optionally, based on the above device, the time granularity represented by the directory node is the smallest time granularity among the time granularities represented by each child node of the directory node, and the deadline of the first sliding window is the largest time granularity corresponding to the target tree;

[0249] The device may also include:

[0250] A path node obtaining module is used to obtain the start time of the first sliding window, and perform path search on the target tree based on the start time to obtain a path node;

[0251] A granularity statistics obtaining module is used for obtaining, for each path node, the statistical results stored in the directory node structurally adjacent to the path node when the path node is a directory node, and for obtaining, for each path node, the granularity statistics stored in the leaf node whose time granularity is greater than the path node and corresponds to the same directory node as the path node when the path node is a leaf node;

[0252] The deduplication statistics obtaining module is used to obtain the deduplication statistics of the current event in the first sliding window based on all the obtained granularity statistics and all the statistical results.

[0253] Optionally, based on the above device, the directory node in the target tree is a node other than a leaf node in the target tree, and the device may further include:

[0254] The child node deletion module is used to delete all child nodes of the target node whose child nodes are leaf nodes representing the smallest time granularity among all directory nodes, if the time granularity represented by each child node of the target node does not fall within the first sliding window.

[0255] The event processing device provided by the embodiment of the present invention can obtain the current deduplication features and current occurrence time of the current event and obtain the first sliding window and the first array matching the current event in response to the event deduplication request through the first array acquisition module, wherein the first array stores the associated deduplication features and associated occurrence times of each associated event associated with the current event; through the repeatability determination module, for the associated deduplication features stored in the first array that match the first sliding window, determine the repeatability between each matched associated deduplication feature and the current deduplication feature, so as to determine whether there is an associated deduplication feature that repeats the current deduplication feature in the first array; through the associated occurrence time processing module, when it is determined that there is a repeated associated deduplication feature according to the repeatability, the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array is processed according to the current occurrence time to complete the deduplication process of the current event. Compared to storing associated deduplication features and associated occurrence times in the form of a HashMap for event deduplication, the above-mentioned device stores them in the form of an array, thereby saving memory during the event deduplication process because there is no need to store additional pointers, and because the associated deduplication features cached in the memory can be directly used for repeatability judgment, there is no need to involve multiple pointer query processes, thereby improving the deduplication performance.

[0256] The event processing device provided in the embodiment of the present invention can execute the event processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0257] It is worth noting that in the embodiment of the above-mentioned event processing device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0258] Fig.11 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0259] like Fig.11 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0260] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0261] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as an event processing method.

[0262] In some embodiments, the event handling method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the event handling method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the event handling method in any other appropriate manner (e.g., by means of firmware).

[0263] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0264] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0265] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0266] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0267] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0268] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0269] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0270] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An event processing method, characterized in that: include: In response to an event deduplication request, obtaining a current deduplication feature and a current occurrence time of a current event, and obtaining a first sliding window and a first array matching the current event, wherein the first array stores associated deduplication features and associated occurrence times of each associated event associated with the current event; For the associated deduplication features stored in the first array and matched with the first sliding window, determine the repetitiveness between each of the matched associated deduplication features and the current deduplication feature; When it is determined based on the repeatability that there is an associated deduplication feature that is repeated with the current deduplication feature, the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array is processed according to the current occurrence time to complete the deduplication process of the current event.

2. The method according to claim 1, characterized in that After obtaining the current deduplication feature of the current event, the method further includes: The current deduplication feature is hashed, and the current deduplication feature is updated according to the obtained hash value.

3. The method according to claim 2, characterized in that After the current deduplication feature is updated according to the obtained hash value, the method further includes: In a case where it is determined according to the repeatability that there is no repeated associated deduplication feature in the first array, processing the current deduplication feature to obtain a target slot in the first array; In the case where the target slot is occupied, determining a step slot in the first array that is away from the target slot by a preset step length; According to the step slot, the current deduplication feature and the current occurrence time are stored in the first array.

4. The method according to claim 3, characterized in that According to the step slot, storing the current deduplication feature and the current occurrence time in the first array includes: When the step slot is occupied, obtaining an expiration time corresponding to the first sliding window; When the associated occurrence time occupying the step slot is less than or equal to the expiration time, the associated deduplication feature stored in the step slot is replaced based on the current deduplication feature, and the associated occurrence time stored in the step slot is replaced based on the current occurrence time.

5. The method according to claim 1, characterized in that Also includes: Acquire a preset failure number threshold and the failure number of the associated deduplication features stored in the first array that are not matched with the first sliding window; When the failure number is greater than or equal to the failure number threshold, the associated deduplication features and the associated occurrence times of the failures in the first array are progressively cleaned up.

6. The method according to claim 1, characterized in that In the process of progressively copying each of the association deduplication features and each of the association occurrence times stored in the first array to the second array, The processing of the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array according to the current occurrence time includes: In the case where the repeated associated deduplication feature is not copied to the second array, the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array is processed according to the current occurrence time; and / or, The method further comprises: In the case where the repeated associated deduplication feature has been copied to the second array, the associated occurrence time corresponding to the repeated associated deduplication feature stored in the second array is processed according to the current occurrence time.

7. The method according to claim 6, characterized in that Also includes: Obtaining a storage slot of the repeated associated deduplication feature in the first array and obtaining a copy slot, wherein the copy slot is used to indicate the slot in the first array where the associated deduplication feature that is most recently copied from the first array to the second array is located; According to the positional relationship between the storage slot and the copy slot, it is determined whether the repeated associated deduplication feature has been copied to the second array.

8. The method according to claim 6, characterized in that Also includes: For a memory used to store the second array, dividing the memory into a plurality of memory blocks; For each of the multiple memory blocks that have not been initialized, the memory block is initialized when the memory block is needed to store the associated deduplication features and the associated occurrence time copied from the first array to the second array.

9. The method according to claim 1, characterized in that: Also includes: For a second sliding window matching a previous event of the current event, obtaining a window statistic value, wherein the window statistic value is a deduplicated statistic value of the previous event under the second sliding window; Obtain the current event value of the current event and the associated event value of the associated event corresponding to the repeated associated deduplication feature, and obtain the deduplication statistical value of the current event under the first sliding window based on the window statistical value, the associated event value and the current event value.

10. The method according to claim 9, characterized in that In a case where the first sliding window is obtained by sliding the second sliding window, the method further includes: When there is an elimination time granularity that is not located in the first sliding window among all the time granularities of the second sliding window, obtaining an elimination statistical value at the elimination time granularity, wherein the elimination statistical value is obtained by counting the granularity event values ​​of each granularity event aggregated to the elimination time granularity; The obtaining, according to the window statistical value, the associated event value and the current event value, a deduplication statistical value of the current event under the first sliding window includes: According to the window statistical value, the elimination statistical value, the associated event value and the current event value, a deduplication statistical value of the current event in the first sliding window is obtained.

11. The method according to claim 9, characterized in that Also includes: Acquire an associated time granularity corresponding to the associated event value and a current time granularity corresponding to the current event value from all time granularities of the first sliding window; The associated event value is subtracted from the granularity statistics value at the associated time granularity, and the current event value is added to the granularity statistics value at the current time granularity.

12. The method according to claim 1, characterized in that Also includes: Acquire a target tree that matches the current event and has the same structure as the B+ tree, wherein a leaf node of the target tree stores a granularity statistical value at a time granularity represented by the leaf node; For the first node corresponding to the repeated associated deduplication feature in all the leaf nodes, obtaining the associated event value of the associated event corresponding to the repeated associated deduplication feature, and updating the granularity statistics value stored in the first node based on the associated event value; For the second nodes corresponding to the current deduplication feature in all the leaf nodes, the granularity statistics stored in the second nodes are updated based on the current event value of the current event.

13. The method according to claim 12, characterized in that The directory node of the target tree stores the statistical results of the granularity statistical values ​​respectively stored in each child node of the directory node, and the directory node is a node in the target tree except the leaf node; The method further comprises: Based on the associated event value, the statistical results respectively stored in the ancestor nodes of the first node are updated, and based on the current event value, the statistical results respectively stored in the ancestor nodes of the second node are updated.

14. The method according to claim 13, characterized in that The time granularity represented by the directory node is the smallest time granularity among the time granularities represented by each child node of the directory node, and the deadline of the first sliding window is the largest time granularity corresponding to the target tree; The method further comprises: Obtaining the start time of the first sliding window, and performing path search on the target tree based on the start time to obtain a path node; For each of the path nodes, when the path node is the directory node, obtain the statistical results stored in the directory node that is structurally adjacent to the path node, and when the path node is the leaf node, obtain the granularity statistical value stored in the leaf node that represents a time granularity greater than that of the path node and corresponds to the same directory node as the path node; Based on all obtained granularity statistics and all obtained statistical results, a deduplicated statistical value of the current event in the first sliding window is obtained.

15. The method according to claim 12, characterized in that The directory nodes in the target tree are nodes in the target tree other than the leaf nodes, and the method further includes: For a target node whose child nodes among all the directory nodes are leaf nodes representing the smallest time granularity, if the time granularity represented by each child node of the target node does not fall within the first sliding window, all child nodes of the target node are deleted.

16. An event processing device, characterized in that: include: A first array acquisition module, configured to, in response to an event deduplication request, acquire a current deduplication feature and a current occurrence time of a current event, and acquire a first sliding window and a first array matching the current event, wherein the first array stores associated deduplication features and associated occurrence times of each associated event associated with the current event; a repeatability determination module, configured to determine the repeatability between each of the associated deduplication features matched with the current deduplication feature and the associated deduplication features stored in the first array and matched with the first sliding window; An associated occurrence time processing module is used to process the associated occurrence time corresponding to the repeated associated deduplication feature stored in the first array according to the current occurrence time when it is determined according to the repeatability that there is an associated deduplication feature that is repeated with the current deduplication feature, so as to complete the deduplication process of the current event.

17. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor performs the event processing method according to any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the event processing method as described in any one of claims 1 to 15 when executed.

19. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the event processing method according to any one of claims 1 to 15.

Images