An event backlog processing method, apparatus, equipment, and medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JINAN INSPUR DATA TECH CO LTD
- Filing Date
- 2022-10-21
- Publication Date
- 2026-06-30
Smart Images
Figure CN115599521B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of cloud computing data center technology, and in particular to an event backlog processing method, apparatus, equipment, and medium. Background Technology
[0002] Currently, an increasing number of companies are developing their own cloud platform projects, capable of managing multiple virtualization or container clusters. These cloud platforms and managed clusters each have their own event systems, generating events when certain conditions occur. These events largely characterize the operational status of the cloud platform or managed cluster. The cloud platform also has a unified event alerting module (Alerter) to handle events, generating alerts and sending notifications. Each business module sends its event information to the Alerter. However, for buffering and decoupling purposes, business modules typically send events to an event queue, which the Alerter then retrieves and processes. Under normal circumstances, once an event is sent to the queue by a business module, the Alerter immediately retrieves it for processing. Normally, the event queue contains very few messages, indicating a relatively healthy event system. However, the timing and quantity of events generated by business modules are often unpredictable. For example, system anomalies (such as node failures or network problems) can generate a large number of events in a short period, and some of these events are highly urgent, repeatedly sending them to the queue for alerts. In such cases, Alerter often cannot fully process the events in the queue, resulting in a large backlog of unprocessed events. Furthermore, some environments, due to resource limitations, cannot run multiple Alerter modules concurrently, as processing events across different Alerters often disrupts the order of events. Additionally, when Alerter modules process backlogged events, they often handle delayed content, causing the events to lose their timeliness. For example, the queue might contain multiple instances of "a" (where "a" represents a type, such as "a" indicating high CPU (Central Processing Unit) utilization, e.g., a1 - 91% CPU utilization, a2 - 92% CPU utilization, a3 - 93% CPU utilization). If Alerter continues processing, it will still process the events in the order of a1, a2, a3. However, at this point, a3 is clearly the latest information, and a1 and a2 are already outdated even if they are processed. Processing such outdated events is meaningless.
[0003] As can be seen from the above, in the process of handling event backlog, how to improve the efficiency of event backlog handling, ensure the real-time nature and integrity of events, reduce the impact on cloud platform performance, and enhance the stability of cloud platform are problems that need to be solved in this field. Summary of the Invention
[0004] In view of this, the purpose of this invention is to provide an event backlog processing method, apparatus, device, and medium that can improve the efficiency of event backlog processing, ensure the real-time performance and integrity of events, reduce the impact on cloud platform performance, and enhance the stability of the cloud platform. The specific solution is as follows:
[0005] Firstly, this application discloses a method for handling event backlogs, including:
[0006] Monitor the number of events in the local event queue;
[0007] If the number of events exceeds a preset backlog value, then events to be processed are determined from all events in the event queue.
[0008] The event type and event generation time of the event to be processed are determined, and the event to be processed is deduplicated based on the event type and the event generation time to obtain the latest event to be processed;
[0009] The latest pending event is sent to a pre-established temporary event queue so that the client can process the latest pending event in the temporary event queue.
[0010] Optionally, monitoring the number of events in the local event queue includes:
[0011] Get the number of events in the current local event queue;
[0012] Determine whether the number of events is greater than a preset event backlog value.
[0013] Optionally, determining the event type and event generation time of the event to be processed, and performing deduplication on the event to be processed based on the event type and the event generation time to obtain the latest event to be processed, includes:
[0014] Determine the event type and event generation time of the event to be processed;
[0015] The event to be processed of the same event type is deduplicated, and the time difference between the event generation time and the current time is calculated.
[0016] The minimum time difference is determined, and the event to be processed corresponding to the minimum time difference is taken as the latest event to be processed.
[0017] Optionally, determining the events to be processed from all events in the event queue includes:
[0018] When the number of events exceeds the preset event backlog value, a connection is established between the local event processing module and the local event queue, and the current time is recorded. Then, the current time is used as the event backlog time, and it is determined whether the event backlog time is greater than the preset event processing cache time.
[0019] If the time backlog is greater than the preset event processing cache time, then the event to be processed is determined from all events in the event queue.
[0020] Optionally, sending the latest pending event to a pre-established temporary event queue includes:
[0021] A temporary event queue is created locally according to the preset queue creation method;
[0022] The connection is disconnected, and a connection is established between the local event processing module and the temporary event queue. Then, the latest event to be processed is sent to the temporary event queue using the connection.
[0023] Optionally, the client processes the latest pending event in the temporary event queue, including:
[0024] The order of the latest pending events in the event queue is determined so that the client processes the latest pending events in the temporary event queue according to the order of events.
[0025] Optionally, after the client processes the latest pending event in the temporary event queue, it further includes:
[0026] Determine whether the number of events in the local event queue is still greater than the preset event backlog value;
[0027] If the number of events in the event queue is not greater than the preset event backlog value, then the connection between the local event processing module and the temporary event queue is disconnected.
[0028] Secondly, this application discloses an event backlog processing apparatus, comprising:
[0029] The monitoring module is used to monitor the number of events in the local event queue;
[0030] The pending event determination module is used to determine pending events from all events in the event queue if the number of events is greater than a preset event backlog value.
[0031] The latest pending event determination module is used to determine the event type and event generation time of the pending event, and to perform deduplication processing on the pending event based on the event type and the event generation time to obtain the latest pending event;
[0032] The event processing module is used to send the latest pending event to a pre-established temporary event queue so that the client can process the latest pending event in the temporary event queue.
[0033] Thirdly, this application discloses an electronic device, including:
[0034] Memory, used to store computer programs;
[0035] A processor is used to execute the computer program to implement the aforementioned event backlog processing method.
[0036] Fourthly, this application discloses a computer storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed event backlog processing method.
[0037] As can be seen, this application provides an event backlog handling method, including monitoring the number of events in a local event queue; if the number of events exceeds a preset event backlog value, identifying events to be processed from all events in the event queue; determining the event type and event generation time of the events to be processed, and performing deduplication on the events to be processed based on the event type and event generation time to obtain the latest events to be processed; and sending the latest events to be processed to a pre-established temporary event queue so that the client can process the latest events to be processed in the temporary event queue. This application, by monitoring the event queue and intervening in event processing when the event queue is backlogged, allows the Eventer module to process the data in the event queue in batches, removing low-timeliness events and retaining high-timeliness events. This reduces the number of events and resolves the event backlog while ensuring the timeliness of events. Furthermore, after the event backlog is resolved, the Eventer module automatically exits, maintaining the original processing module, ensuring data integrity, reducing the impact on cloud platform performance, and ensuring the stability of the cloud platform. This invention monitors the event queue and intervenes to process events when the queue becomes backlogged. The Eventer module can batch process data in the event queue, removing low-timeliness events and retaining high-timeliness events. This reduces the number of events and resolves the backlog while ensuring timely events. Furthermore, after the backlog is processed, the Eventer module automatically exits, maintaining the original processing module and ensuring data integrity. Compared to starting multiple Alerter modules for event processing, this reduces the impact on cloud platform performance and ensures cloud platform stability. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0039] Figure 1 This application discloses a flowchart of an event backlog processing method.
[0040] Figure 2 This application discloses a flowchart of an event backlog processing method.
[0041] Figure 3 This application discloses a specific flowchart of an event backlog processing method.
[0042] Figure 4 This is a schematic diagram of the structure of an event backlog processing device disclosed in this application;
[0043] Figure 5 This application provides a structural diagram of an electronic device. Detailed Implementation
[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0045] Currently, an increasing number of companies are developing their own cloud platform projects, capable of managing multiple virtualization or container clusters. These cloud platforms and managed clusters each have their own event systems, generating events when certain conditions occur. These events largely characterize the operational status of the cloud platform or managed cluster. The cloud platform also has a unified event alerting module (Alerter) to handle events, generating alerts and sending notifications. Each business module sends its event information to the Alerter. However, for buffering and decoupling purposes, business modules typically send events to an event queue, which the Alerter then retrieves and processes. Under normal circumstances, once an event is sent to the queue by a business module, the Alerter immediately retrieves it for processing. Normally, the event queue contains very few messages, indicating a relatively healthy event system. However, the timing and quantity of events generated by business modules are often unpredictable. For example, system anomalies (such as node failures or network problems) can generate a large number of events in a short period, and some of these events are highly urgent, repeatedly sending them to the queue for alerts. In such cases, Alerter often cannot fully process the events in the queue, resulting in a large backlog of unprocessed events. Furthermore, some environments, due to resource limitations, cannot run multiple Alerter modules concurrently, as processing events across different Alerters often disrupts the order of events. Additionally, when Alerter modules process backlogged events, they often handle delayed content, causing the events to lose their timeliness. For example, the queue might contain multiple instances of "a" (where "a" represents a type, such as "a" indicating high CPU (Central Processing Unit) utilization, e.g., a1 - 91% CPU utilization, a2 - 92% CPU utilization, a3 - 93% CPU utilization). If Alerter continues processing, it will still process events in the order of a1, a2, a3. However, at this point, a3 is clearly the most up-to-date, while a1 and a2, even if processed, are already outdated. Processing such stale events is meaningless. As can be seen, improving the efficiency of event backlog processing, ensuring the real-time nature and integrity of events, reducing the impact on cloud platform performance, and enhancing cloud platform stability are problems that need to be solved in this field. The key point of this invention is to monitor the event queue and intervene in event processing when the queue is backlogged. The Eventer module can batch process the data in the event queue, removing low-timeliness events and retaining high-timeliness events. This reduces the number of events and resolves the backlog while ensuring the timeliness of events.Meanwhile, once the backlog of events has been processed, the event module will automatically exit, maintaining the original processing module and ensuring data integrity. Compared to starting multiple Alerter modules for event processing, this reduces the impact on cloud platform performance and ensures cloud platform stability.
[0046] This application proposes an event processing module called Eventer. When events accumulate, the Eventer module can intervene to process them in batches, using a queue window as the unit. This batch processing primarily involves event filtering and deduplication. Simultaneously, when there is no backlog, the module can exit promptly, maintaining the original event processing module and ensuring data integrity. Then, within the Eventer module, data from a queue window is read in batches and processed. For each event type, only the most recent data is written to the queue. This process removes expired data and focuses on processing new data, ensuring the real-time nature of events.
[0047] See Figure 1 As shown, this embodiment of the invention discloses an event backlog processing method, which may specifically include:
[0048] Step S11: Monitor the number of events in the local event queue.
[0049] In this embodiment, the specific steps for monitoring the number of events in the local event queue are as follows: obtain the current number of events in the local event queue, and then determine whether the number of events is greater than a preset event backlog value.
[0050] Specifically, the execution parameters need to be set in the cloud platform: The first setting is the size N of the event queue window, which determines the number of events to read from the queue at once when a backlog occurs. The default value for N is 1000. The second setting is the queue backlog listening time T, which listens to the event queue. When the number of events in the queue exceeds N, the timer starts. When the duration exceeds T, event preprocessing begins. The default value for T is 10 seconds. Finally, after setting these parameters, the listening task can monitor the event queue according to the set parameters and determine when to intervene in the event processing module Eventer.
[0051] Step S12: If the number of events is greater than the preset event backlog value, then determine the events to be processed from all events in the event queue.
[0052] Step S13: Determine the event type and event generation time of the event to be processed, and perform deduplication processing on the event to be processed based on the event type and the event generation time to obtain the latest event to be processed.
[0053] In this embodiment, the event type and event generation time of the event to be processed are determined. Then, the events to be processed of the same event type are deduplicated. At the same time, the time difference between the event generation time and the current time corresponding to the event to be processed of the same event type is calculated, the minimum time difference is determined, and the event to be processed corresponding to the minimum time difference is taken as the latest event to be processed.
[0054] Step S14: Send the latest pending event to a pre-established temporary event queue so that the client can process the latest pending event in the temporary event queue.
[0055] The key point of this invention is to monitor the event queue and intervene in event processing when the queue becomes backlogged. The Eventer module can batch process the data in the event queue, removing low-timeliness events and retaining high-timeliness events. This reduces the number of events and resolves the backlog while ensuring the timeliness of events. Furthermore, after the backlog is processed, the Eventer module automatically exits, maintaining the original processing module and ensuring data integrity. Compared to starting multiple Alerter modules for event processing, this reduces the impact on cloud platform performance and ensures cloud platform stability.
[0056] In this embodiment, the number of events in the local event queue is monitored. If the number of events exceeds a preset backlog value, events to be processed are identified from all events in the event queue. The event type and event generation time of the events to be processed are determined, and deduplication is performed on the events to be processed based on the event type and event generation time to obtain the latest events to be processed. The latest events to be processed are sent to a pre-established temporary event queue so that the client can process the latest events to be processed in the temporary event queue. This application monitors the event queue and intervenes in event processing when the event queue is backlogged. The Eventer module can process the data in the event queue in batches, removing low-timeliness events and retaining high-timeliness events. This reduces the number of events and solves the event backlog while ensuring the timeliness of events. At the same time, after the event backlog is processed, the Eventer module automatically exits, maintaining the original processing module, ensuring data integrity, reducing the impact on cloud platform performance, and ensuring the stability of the cloud platform.
[0057] See Figure 2 As shown, this embodiment of the invention discloses an event backlog processing method, which may specifically include:
[0058] Step S21: Monitor the number of events in the local event queue.
[0059] Step S22: If the number of events is greater than the preset event backlog value, then establish a connection between the local event processing module and the local event queue, record the current time, and then use the current time as the event backlog time. Determine whether the backlog time is greater than the preset event processing cache time. If the backlog time is greater than the preset event processing cache time, then determine the events to be processed from all events in the event queue.
[0060] Step S23: Determine the event type and event generation time of the event to be processed, and perform deduplication processing on the event to be processed based on the event type and the event generation time to obtain the latest event to be processed.
[0061] Step S24: Send the latest pending event to a pre-established temporary event queue so that the client can process the latest pending event in the temporary event queue.
[0062] In this embodiment, a temporary event queue is established locally according to a preset queue establishment method. Then, the connection relationship is disconnected, and a connection relationship is established between the local event processing module and the temporary event queue. The latest pending event is then sent to the temporary event queue using the connection relationship. The event order of the latest pending event in the event queue is determined so that the client processes the latest pending event in the temporary event queue according to the event order. Then, it is determined whether the number of events in the local event queue is still greater than a preset event backlog value. If the number of events in the event queue is not greater than the preset event backlog value, the connection relationship between the local event processing module and the temporary event queue is disconnected.
[0063] This application proposes a cloud platform event backlog handling method based on a queue window. Assuming the event queue window size N is 1000 (i.e., the preset event backlog value is 1000), and the queue backlog listening time T is 10s (i.e., the preset event processing buffer time is 10s), after setting, the listening task can monitor the event queue according to the set parameters and determine when to intervene in the event processing module (Eventer) and under what circumstances to exit the event processing module. Specifically, a listening task is added to monitor the event queue. Once a backlog is detected and meets a certain time requirement (i.e., when the number of events in the event queue is greater than 1000, the current time is recorded, and the number of events is monitored to see if it is less than 1000 within the next 10 seconds), if it is still greater than 1000 within the next 10 seconds, an event processing (Eventer) task will be started. Figure 3Taking event types a, b, and c in the event queue as an example, if the number of events from c1 to a3 is 1000, when a new event a3 needs to be processed, the Eventer module needs to intervene. At this time, a3 has already been generated, and a2 and a1 are no longer real-time data for this type. If the processing is still done in this order, notifications for a2 and a1 will still be received. However, the actual situation is that a3 is the latest, and a2 and a1 are no longer real-time data, so the notifications are meaningless. Similarly, b and c are in the same situation. Therefore, it is necessary to batch process the events in the event queue, mainly including deduplication and old data removal, to reduce the number of events and ensure the real-time nature of the events. For example, after the final processing, c2, b2, and a3 are obtained, and then c2, b2, and a3 are sent to a pre-established temporary event queue tmp_queue. The client reads data from tmp_queue through the Alerter module for processing. After each batch processing by Eventer, the above conditions will be checked again.
[0064] In this application, a monitoring task for the queue is established to determine whether data backlog has occurred in the event queue and whether the event processing module should intervene. Specifically, if the number of events in the event queue is greater than N during the duration T, the Eventer will re-examine the above conditions after each batch processing. If the conditions are not met, the Eventer will exit and will not perform the next judgment.
[0065] In this embodiment, the number of events in the local event queue is monitored. If the number of events exceeds a preset backlog value, events to be processed are identified from all events in the event queue. The event type and event generation time of the events to be processed are determined, and deduplication is performed on the events to be processed based on the event type and event generation time to obtain the latest events to be processed. The latest events to be processed are sent to a pre-established temporary event queue so that the client can process the latest events to be processed in the temporary event queue. This application monitors the event queue and intervenes in event processing when the event queue is backlogged. The Eventer module can process the data in the event queue in batches, removing low-timeliness events and retaining high-timeliness events. This reduces the number of events and solves the event backlog while ensuring the timeliness of events. At the same time, after the event backlog is processed, the Eventer module automatically exits, maintaining the original processing module, ensuring data integrity, reducing the impact on cloud platform performance, and ensuring the stability of the cloud platform. The key point of this invention is to monitor the event queue and intervene in event processing when the queue becomes backlogged. The Eventer module can batch process the data in the event queue, removing low-timeliness events and retaining high-timeliness events. This reduces the number of events and resolves the backlog while ensuring the timeliness of events. Furthermore, after the backlog is processed, the Eventer module automatically exits, maintaining the original processing module and ensuring data integrity. Compared to starting multiple Alerter modules for event processing, this reduces the impact on cloud platform performance and ensures cloud platform stability.
[0066] See Figure 4 As shown, an embodiment of the present invention discloses an event backlog processing device, which may specifically include:
[0067] Monitoring module 11 is used to monitor the number of events in the local event queue;
[0068] The pending event determination module 12 is used to determine pending events from all events in the event queue if the number of events is greater than a preset event backlog value.
[0069] The latest pending event determination module 13 is used to determine the event type and event generation time of the pending event, and to perform deduplication processing on the pending event based on the event type and the event generation time to obtain the latest pending event;
[0070] The event processing module 14 is used to send the latest pending event to a pre-established temporary event queue so that the client can process the latest pending event in the temporary event queue.
[0071] In this embodiment, the number of events in the local event queue is monitored. If the number of events exceeds a preset backlog value, events to be processed are identified from all events in the event queue. The event type and event generation time of the events to be processed are determined, and deduplication is performed on the events to be processed based on the event type and event generation time to obtain the latest events to be processed. The latest events to be processed are sent to a pre-established temporary event queue so that the client can process the latest events to be processed in the temporary event queue. This application monitors the event queue and intervenes in event processing when the event queue is backlogged. The Eventer module can process the data in the event queue in batches, removing low-timeliness events and retaining high-timeliness events. This reduces the number of events and solves the event backlog while ensuring the timeliness of events. At the same time, after the event backlog is processed, the Eventer module automatically exits, maintaining the original processing module, ensuring data integrity, reducing the impact on cloud platform performance, and ensuring the stability of the cloud platform.
[0072] The key point of this invention is to monitor the event queue and intervene in event processing when the queue becomes backlogged. The Eventer module can batch process the data in the event queue, removing low-timeliness events and retaining high-timeliness events. This reduces the number of events and resolves the backlog while ensuring the timeliness of events. Furthermore, after the backlog is processed, the Eventer module automatically exits, maintaining the original processing module and ensuring data integrity. Compared to starting multiple Alerter modules for event processing, this reduces the impact on cloud platform performance and ensures cloud platform stability.
[0073] In some specific embodiments, the monitoring module 11 may specifically include:
[0074] The event count acquisition module is used to acquire the number of events in the current local event queue;
[0075] The judgment module is used to determine whether the number of events is greater than a preset event backlog value.
[0076] In some specific embodiments, the latest pending event determination module 13 may specifically include:
[0077] The event type determination module is used to determine the event type and event generation time of the event to be processed;
[0078] The deduplication module is used to deduplicatize the events to be processed of the same event type, and at the same time calculate the time difference between the event generation time and the current time corresponding to the events to be processed of the same event type.
[0079] The latest pending event determination module is used to determine the minimum time difference and take the pending event corresponding to the minimum time difference as the latest pending event.
[0080] In some specific embodiments, the event-to-be-processed determination module 12 may specifically include:
[0081] The connection establishment module is used to establish a connection between the local event processing module and the local event queue when the number of events is greater than a preset event backlog value, record the current time, and then use the current time as the event backlog time to determine whether the event backlog time is greater than a preset event processing cache time.
[0082] The pending event determination module is used to determine pending events from all events in the event queue if the time backlog time is greater than a preset event processing cache time.
[0083] In some specific embodiments, the event handling module 14 may specifically include:
[0084] The temporary event queue creation module is used to create temporary event queues locally according to a preset queue creation method.
[0085] The latest pending event sending module is used to disconnect the connection relationship, establish a connection relationship between the local event processing module and the temporary event queue, and then use the connection relationship to send the latest pending event to the temporary event queue.
[0086] In some specific embodiments, the event handling module 14 may specifically include:
[0087] The event order determination module is used to determine the event order of the latest pending events in the event queue, so that the client processes the latest pending events in the temporary event queue according to the event order.
[0088] In some specific embodiments, the event handling module 14 may specifically include:
[0089] The judgment module is used to determine whether the number of events in the local event queue is still greater than the preset event backlog value;
[0090] The connection disconnection module is used to disconnect the connection between the local event processing module and the temporary event queue if the number of events in the event queue is not greater than a preset event backlog value.
[0091] Figure 5This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the event backlog processing method performed by the electronic device disclosed in any of the foregoing embodiments.
[0092] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0093] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.
[0094] The operating system 221 manages and controls the various hardware devices on the electronic device 20 and the computer program 222 to enable the processor 21 to perform operations and processing on the data 223 in the memory 22. The operating system 221 can be Windows, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the event backlog processing method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the event backlog processing device from external devices, as well as data collected by its own input / output interface 25.
[0095] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0096] Furthermore, embodiments of this application also disclose a computer-readable storage medium storing a computer program, which, when loaded and executed by a processor, implements the event backlog processing method steps disclosed in any of the foregoing embodiments.
[0097] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0098] The above provides a detailed description of the event backlog processing method, apparatus, device, and storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for handling event backlog, characterized in that, include: Monitor the number of events in the local event queue; If the number of events exceeds the preset event backlog value, a connection is established between the local event processing module and the local event queue, and the events to be processed are determined from all events in the event queue. The event type and event generation time of the event to be processed are determined, and the event to be processed is deduplicated based on the event type and the event generation time to obtain the latest event to be processed; The latest pending event is sent to a pre-established temporary event queue so that the client can process the latest pending event in the temporary event queue; The process involves determining the event type and event generation time of the events to be processed, and performing deduplication on the events to be processed based on the event type and event generation time to obtain the latest events to be processed. This includes: determining the event type and event generation time of the events to be processed; performing deduplication on events to be processed of the same event type, and simultaneously calculating the time difference between the event generation time of the events to be processed of the same event type and the current time; determining the minimum time difference, and taking the event to be processed corresponding to the minimum time difference as the latest event to be processed. Sending the latest pending event to a pre-established temporary event queue includes: establishing a temporary event queue locally according to a preset queue establishment method; disconnecting the connection between the local event processing module and the local event queue, and establishing a connection between the local event processing module and the temporary event queue; and then using the connection between the local event processing module and the temporary event queue to send the latest pending event to the temporary event queue.
2. The event backlog processing method according to claim 1, characterized in that, The monitoring of the number of events in the local event queue includes: Get the number of events in the current local event queue; Determine whether the number of events is greater than a preset event backlog value.
3. The event backlog processing method according to any one of claims 1 to 2, characterized in that, The step of determining the events to be processed from all events in the event queue includes: When the number of events exceeds the preset event backlog value, a connection is established between the local event processing module and the local event queue, and the current time is recorded. Then, the current time is used as the event backlog time, and it is determined whether the event backlog time is greater than the preset event processing cache time. If the time backlog is greater than the preset event processing cache time, then the event to be processed is determined from all events in the event queue.
4. The event backlog processing method according to claim 1, characterized in that, The client processes the latest pending event in the temporary event queue, including: The order of the latest pending events in the event queue is determined so that the client processes the latest pending events in the temporary event queue according to the order of events.
5. The event backlog processing method according to claim 1, characterized in that, After the client processes the latest pending event in the temporary event queue, the process further includes: Determine whether the number of events in the local event queue is still greater than the preset event backlog value; If the number of events in the event queue is not greater than the preset event backlog value, then the connection between the local event processing module and the temporary event queue is disconnected.
6. An event backlog processing device, characterized in that, include: The monitoring module is used to monitor the number of events in the local event queue; The pending event determination module is used to determine pending events from all events in the event queue if the number of events is greater than a preset event backlog value. The latest pending event determination module is used to determine the event type and event generation time of the pending event, and to perform deduplication processing on the pending event based on the event type and the event generation time to obtain the latest pending event; The event processing module is used to send the latest pending event to a pre-established temporary event queue so that the client can process the latest pending event in the temporary event queue. The process involves determining the event type and event generation time of the events to be processed, and performing deduplication on the events to be processed based on the event type and event generation time to obtain the latest events to be processed. This includes: determining the event type and event generation time of the events to be processed; performing deduplication on events to be processed of the same event type, and simultaneously calculating the time difference between the event generation time of the events to be processed of the same event type and the current time; determining the minimum time difference, and taking the event to be processed corresponding to the minimum time difference as the latest event to be processed. Sending the latest pending event to a pre-established temporary event queue includes: establishing a temporary event queue locally according to a preset queue establishment method; disconnecting the connection between the local event processing module and the local event queue, and establishing a connection between the local event processing module and the temporary event queue; and then using the connection between the local event processing module and the temporary event queue to send the latest pending event to the temporary event queue.
7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the event backlog processing method as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the event backlog processing method as described in any one of claims 1 to 5.