Event streaming management method, system, equipment and medium

Abstract

The invention discloses an event streaming management method, system and device and a medium, belongs to the technical field of event management, and aims to solve the technical problems of large load, untimely processing, high delay and poor real-time performance of a traditional event management system when a large number of events are generated, and the technical scheme adopted by the invention is as follows: event rule configuration: defining an event management rule; wherein the event management rule comprises an event source needing to be monitored, an event type, an event monitoring time period, an event pushing role, an event processing rule and an event linkage action; gathering events: collecting the generated events and temporarily storing the events in a message queue in a centralized manner; event analysis: after subscribing the received event, carrying out storage analysis according to an event rule; and event processing: processing the event.

Description

Technical Field

[0001] The present invention relates to the technical field of event management, and in particular to an event streaming management method, system, device and medium. Background Art

[0002] Event management is a systematic approach to effectively manage and control various events that occur within and outside an organization. In the IT field, event management specifically refers to a series of processes that link event status change information with personnel responses to achieve specific business needs. It involves multiple links such as the collection, analysis, classification, priority determination, response, processing and follow-up of event information.

[0003] In the process of data interaction between multiple systems or services, event management systems are often used to decouple business operations and achieve system flexibility. However, most event management systems process events in batches, which can lead to problems such as heavy load, untimely processing, high latency, and poor real-time performance when a large number of events are generated. Summary of the invention

[0004] The technical task of the present invention is to provide an event streaming management method, system, device and medium to solve the problems of heavy load, untimely processing, high delay and poor real-time performance in traditional event management systems when a large number of events are generated.

[0005] The technical task of the present invention is achieved in the following manner: a method for event streaming management, the method is as follows:

[0006] Event rule configuration: define event management rules; event management rules include event sources to be monitored, event types, event monitoring time periods, event push roles, event processing rules, and event linkage actions;

[0007] Event aggregation: collect the generated events and temporarily store them in the message queue;

[0008] Event analysis: After subscribing to received events, storage and analysis are performed according to event rules;

[0009] Event processing: Processing events.

[0010] Preferably, after the event rule configuration is completed, the event rule extension message is sent to the processing center based on the eKuiper middleware, and the event rule is used as the flow rule of the corresponding event flow; then the event rule configuration is used to clarify the collection, transmission, processing and linkage rules of the event flow, providing parameter dependencies for subsequent functions.

[0011] As a preferred option, during the event aggregation process, the data of the RabbitMQ open source component is connected to the data stream as the data source according to the eKuiper source plug-in, and one event stream can be reused for events of the same rule;

[0012] Among them, RabbitMQ open source components are used as message queues. According to the event flow rules configured by event rules, the message queues are configured with switch storage system topic events, business topic events, and intelligent analysis topic events of various topic types;

[0013] After the events are aggregated, they need to be diverted according to the event rule configuration. Diversion depends on the eKuiper open source component, and suitable data source plug-ins need to be developed according to business needs;

[0014] or,

[0015] Develop event stream channel multiplexing function to improve data processing efficiency.

[0016] As a preferred option, during the event analysis process, the event stream subscribes to relevant topics according to business rules or event terminals, parses the original event data, cleans the data, and binds the business event processing extended information with the original information to form all the event information, which is then imported into the event stream through the eKuiper sink plug-in and finally sent back to the final event topic of the RabbitMQ open source component.

[0017] As a preferred embodiment, after configuring the event source, query the flow rules configured for the corresponding event source during the event rule configuration process and apply them;

[0018] The stream concept of event analysis is bound to a specific business process ID or a specified event monitoring device ID, and supports access to data sources of multiple protocols to achieve higher scalability and flexibility; data sources include MySQL, RabbitMQ, MQTT, Redis, and HTTP;

[0019] When subscribing to the data in the RabbitMQ open source component, explain the technology and business process of event analysis. After the event stream data is connected, clean the data by writing the sink plug-in of the eKuiper component, analyze the data according to the custom rule information in the business system, keep the data of interest, and cut and assemble it. The cutting and assembling are as follows: bind the triggering action and linkage information of the corresponding event, and finally form event data containing business extension information;

[0020] For event data containing original event information and corresponding processing rule extended information, event analysis sends the event data back to switches of different topics of RabbitMQ open source components for event processing subscription.

[0021] Preferably, during the event processing, the event data after event analysis specification is subscribed, and after parsing, the specific event source and the corresponding processing rules are obtained. Then, according to the event rules, the processing request is further sent to the specific business processing system, such as the recording system for event recording, the screenshot system for event screenshots, and the notification system for email and SMS notifications. In this way, a complete event flow from event occurrence to event processing feedback is continuously carried out until the event flow is terminated;

[0022] The analysis and evaluation of abnormal events provide positive feedback to equipment management personnel, reduce the probability of events occurring, and complete the logical closed loop of the event management system.

[0023] An event streaming management system, the system comprising:

[0024] The event rule configuration module is used to define event management rules; among them, event management rules include event sources to be monitored, event types, event monitoring time periods, event push roles, event processing rules, and event linkage actions;

[0025] The event aggregation module is used to connect the data of the RabbitMQ open source component to the data stream as the data source based on the self-developed eKuiper source plug-in, and supports multiplexing an event stream for events of the same rule;

[0026] The event analysis module is used to combine the original event information of the event source with business rules to aggregate the complete event stream data and send it back to the RabbitMQ open source component;

[0027] The event processing module is used to subscribe to the final event data, parse the format and send it to the relevant business system for event resolution, analyze and judge the event data and feed it back to the management personnel to complete the closed loop of the event management system.

[0028] Preferably, after completing the event rule configuration, the event rule configuration module sends the event rule extension message to the processing center based on the eKuiper middleware, and uses the event rule as the flow rule of the corresponding event flow; and then clarifies the collection, transmission, processing and linkage rules of the event flow through the event rule configuration, providing parameter dependencies for subsequent functions;

[0029] The event aggregation module uses RabbitMQ open source components as message queues. According to the event flow rules configured by the event rules, the message queues are configured with switch storage system topic events, business topic events, and intelligent analysis topic events of various topic types. After the events are aggregated, they need to be diverted according to the event rule configuration. The diversion depends on the eKuiper open source component, and it is necessary to develop a suitable data source plug-in according to business needs.

[0030] The flow concept of the event analysis module is bound to a specific business process ID or a specified event monitoring device ID, and supports access to data sources of multiple protocols to obtain higher scalability and flexibility; among which, data sources include MySQL, RabbitMQ, MQTT, Redis and HTTP;

[0031] When subscribing to the data in the RabbitMQ open source component, explain the technology and business process of event analysis. After the event stream data is accessed, clean the data by writing the sink plug-in of the eKuiper component, analyze the data according to the custom rule information in the business system, keep the data of interest and cut and assemble it. The cutting and assembling are as follows: bind the triggering action and linkage information of the corresponding event to finally form event data containing business extension information; for event data containing the original event information and the corresponding processing rule extension information, event analysis sends the event data back to the switches of different topics of the RabbitMQ open source component for event processing subscription use;

[0032] The event processing module subscribes to the event data after event analysis specification, obtains the specific event source and the corresponding processing rules after analysis, and further sends the processing request to the specific business processing system according to the event rules, such as the video recording system for event recording, the screenshot system for event screenshots, and the notification system for email and SMS notifications. In this way, a complete event flow from event occurrence to event processing feedback is carried out continuously until the termination of this event flow; the analysis and judgment of abnormal events form positive feedback to the equipment management personnel, reduce the probability of event occurrence, and complete the logical closed loop of the event management system.

[0033] An electronic device comprising: a memory and at least one processor;

[0034] Wherein, the memory stores computer-executable instructions;

[0035] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the event streaming management method as described above.

[0036] A computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the event streaming management method as described above is implemented.

[0037] The event streaming management method, system, device and medium of the present invention have the following advantages:

[0038] (I) The present invention is based on eKuiper as the hub of stream processing, including an event rule configuration module, an event aggregation module, an event analysis module and an event processing module, wherein the event aggregation module uses the events published to the RabbitMQ component as the source of the event stream; the event analysis module aggregates the original event information of the event source into complete event stream data in combination with the business rules and sends it back to RabbitMQ; the event processing module subscribes to the final event data, parses the format and sends it to the relevant business system for event resolution, and analyzes and judges the event data and feeds it back to the management personnel, finally completing the closed loop of the event management system;

[0039] (ii) The present invention adopts modular design. Event collection and event processing configuration are often not in the same field. The occurrence of an event does not know how to be executed or what action to trigger to solve the event itself. A medium is needed to connect the two. Therefore, the event management system splits and decouples the event module and the business module to improve the flexibility and scalability of the system.

[0040] (III) The present invention adopts the four-module design of the streaming system, so that the complete life cycle of events from generation to extinction can be stored in a streaming manner, overcoming the problems of soaring system resource usage and large event response delays in traditional event management systems when processing massive event sources, thereby improving the real-time and efficiency of event processing and reducing maintenance difficulty and cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The present invention is further described below in conjunction with the accompanying drawings.

[0042] Attached Figure 1 The flowchart of the event streaming management method is shown in FIG. DETAILED DESCRIPTION

[0043] The event streaming management method, system, device and medium of the present invention are described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Embodiment 1:

[0045] As attached Figure 1 As shown, this embodiment provides an event streaming management method, which is specifically as follows:

[0046] S1. Event rule configuration: define event management rules; event management rules include event sources to be monitored, event types, event monitoring time periods, event push roles, event processing rules, and event linkage actions;

[0047] S2, event aggregation: collect the generated events and temporarily store them in the message queue;

[0048] S3, event analysis: After subscribing to received events, storage analysis is performed according to event rules;

[0049] S4. Event processing: process the event.

[0050] After the event rule configuration in step S1 of this embodiment is completed, the event rule extension message is sent to the processing center based on the eKuiper middleware, and the event rule is used as the flow rule of the corresponding event flow; then the event rule configuration is used to clarify the collection, transmission, processing and linkage rules of the event flow, providing parameter dependencies for subsequent functions.

[0051] In the event aggregation process in step S2 of this embodiment, the data of the RabbitMQ open source component is connected to the data stream as the data source according to the eKuiper source plug-in, and one event stream can be reused for events of the same rule;

[0052] Among them, RabbitMQ open source components are used as message queues. According to the event flow rules configured by event rules, the message queues are configured with switch storage system topic events, business topic events, and intelligent analysis topic events of various topic types;

[0053] After the events are aggregated, they need to be diverted according to the event rule configuration. Diversion depends on the eKuiper open source component, and suitable data source plug-ins need to be developed according to business needs;

[0054] or,

[0055] Develop event stream channel multiplexing function to improve data processing efficiency.

[0056] In the event analysis process in step S3 of this embodiment, the event stream subscribes to relevant topics according to business rules or event terminals, parses the original event data, cleans the data, and binds the business event processing extended information with the original information to form all the event information, which is then imported into the event stream through the eKuiper sink plug-in and finally sent back to the final event topic of the RabbitMQ open source component.

[0057] In this embodiment, after configuring the event source, the flow rules configured for the corresponding event source during the event rule configuration process are queried and applied;

[0058] The stream concept of event analysis is bound to a specific business process ID or a specified event monitoring device ID, and supports access to data sources of multiple protocols to achieve higher scalability and flexibility; data sources include MySQL, RabbitMQ, MQTT, Redis, and HTTP;

[0059] When subscribing to the data in the RabbitMQ open source component, explain the technology and business process of event analysis. After the event stream data is connected, clean the data by writing the sink plug-in of the eKuiper component, analyze the data according to the custom rule information in the business system, keep the data of interest, and cut and assemble it. The cutting and assembling are as follows: bind the triggering action and linkage information of the corresponding event, and finally form event data containing business extension information;

[0060] For event data containing original event information and corresponding processing rule extended information, event analysis sends the event data back to switches of different topics of RabbitMQ open source components for event processing subscription.

[0061] In the event processing process in step S4 of this embodiment, the event data after event analysis specification is subscribed, and the specific event source and the corresponding processing rules are obtained after parsing, and the processing request is further sent to the specific business processing system according to the event rule, such as the recording system for event recording, the screenshot system for event screenshot, and the notification system for email and SMS notification, so that a complete event flow from event occurrence to event processing feedback is continuously carried out until the event flow is terminated;

[0062] The analysis and evaluation of abnormal events provide positive feedback to equipment management personnel, reduce the probability of events occurring, and complete the logical closed loop of the event management system.

[0063] Embodiment 2:

[0064] This embodiment provides an event streaming management system, which includes:

[0065] The event rule configuration module is used to define event management rules; among them, event management rules include event sources to be monitored, event types, event monitoring time periods, event push roles, event processing rules, and event linkage actions;

[0066] The event aggregation module is used to connect the data of the RabbitMQ open source component to the data stream as the data source based on the self-developed eKuiper source plug-in, and supports multiplexing an event stream for events of the same rule;

[0067] The event analysis module is used to combine the original event information of the event source with business rules to aggregate the complete event stream data and send it back to the RabbitMQ open source component;

[0068] The event processing module is used to subscribe to the final event data, parse the format and send it to the relevant business system for event resolution, analyze and judge the event data and feed it back to the management personnel to complete the closed loop of the event management system.

[0069] After the event rule configuration module in this embodiment completes the event rule configuration, it sends the event rule extension message to the processing center based on the eKuiper middleware, and uses the event rule as the flow rule of the corresponding event flow; then, through the event rule configuration, the event flow collection, transmission, processing and linkage rules are clarified to provide parameter dependencies for subsequent functions.

[0070] The event aggregation module in this embodiment uses the RabbitMQ open source component as a message queue. According to the event flow rules configured by the event rules, the message queue is configured with switch storage system topic events, business topic events and intelligent analysis topic events of various topic types; after the events are aggregated, they need to be diverted according to the event rule configuration. The diversion depends on the eKuiper open source component, and suitable data source plug-ins need to be developed according to business needs.

[0071] The flow concept of the event analysis module in this embodiment is bound to a specific business process ID or a specified event monitoring device ID, and supports access to data sources of multiple protocols to obtain higher scalability and flexibility; among them, data sources include MySQL, RabbitMQ, MQTT, Redis and HTTP;

[0072] When subscribing to data in the RabbitMQ open source component, explain the technology and business process of event analysis. After the event stream data is accessed, clean the data by writing the sink plug-in of the eKuiper component, analyze the data according to the custom rule information in the business system, keep the data of interest and cut and assemble it. The cutting and assembly are as follows: bind the triggering action and linkage information of the corresponding event to finally form event data containing business extension information; for event data containing the original event information and the corresponding processing rule extension information, event analysis sends the event data back to the switches of different topics of the RabbitMQ open source component for event processing subscription.

[0073] The event processing module in this embodiment subscribes to the event data after event analysis specification, obtains the specific event source and the corresponding processing rules after analysis, and further sends the processing request to the specific business processing system according to the event rules, such as the recording system for event recording, the screenshot system for event screenshots, and the notification system for email and SMS notifications. In this way, a complete event flow from event occurrence to event processing feedback is carried out continuously until the termination of this event flow; the analysis and judgment of abnormal events form positive feedback to the equipment management personnel, reduce the probability of event occurrence, and complete the logical closed loop of the event management system.

[0074] Embodiment 3:

[0075] This embodiment also provides an electronic device, including: a memory and at least one processor;

[0076] Wherein, the memory stores computer-executable instructions;

[0077] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the event streaming management method described in any one of the present inventions.

[0078] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or any conventional processor, etc.

[0079] The memory can be used to store computer programs and / or modules. The processor realizes various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory can also include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, at least one disk storage period, a flash memory device, or other volatile solid-state storage devices.

[0080] Embodiment 4:

[0081] This embodiment also provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions are loaded by a processor, so that the processor executes the event streaming management method in any embodiment of the present invention. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code that implements the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0082] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0083] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0084] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0085] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An event streaming management method, characterized in that: The method is as follows: Event rule configuration: define event management rules; event management rules include event sources to be monitored, event types, event monitoring time periods, event push roles, event processing rules, and event linkage actions; Event aggregation: collect the generated events and temporarily store them in the message queue; Event analysis: After subscribing to received events, storage and analysis are performed according to event rules; Event processing: Processing events.

2. The event streaming management method according to claim 1, characterized in that: After the event rule configuration is completed, the event rule extension message is sent to the processing center based on the eKuiper middleware, and the event rule is used as the flow rule of the corresponding event flow; then the event rule configuration is used to clarify the collection, transmission, processing and linkage rules of the event flow, providing parameter dependencies for subsequent functions.

3. The event streaming management method according to claim 1, characterized in that: During the event aggregation process, the data of the RabbitMQ open source component is connected to the data stream as the data source according to the eKuiper source plug-in, and one event stream can be reused for events with the same rule; Among them, RabbitMQ open source components are used as message queues. According to the event flow rules configured by event rules, the message queues are configured with switch storage system topic events, business topic events, and intelligent analysis topic events of various topic types; After the events are aggregated, they need to be diverted according to the event rule configuration. Diversion depends on the eKuiper open source component, and suitable data source plug-ins need to be developed according to business needs; or, Develop event stream channel multiplexing function.

4. The event streaming management method according to claim 1, characterized in that: During the event analysis process, the event stream subscribes to relevant topics according to business rules or event terminals, parses the original event data, cleans the data, and binds the business event processing extended information with the original information to form all the event information, which is then imported into the event stream through the eKuiper sink plug-in and finally sent back to the final event topic of the RabbitMQ open source component.

5. The event streaming management method according to claim 1, characterized in that: After configuring the event source, query the flow rules configured for the corresponding event source during the event rule configuration process and apply them; The stream concept of event analysis is bound to a specific business process ID or a specified event monitoring device ID, and supports access to data sources of multiple protocols; data sources include MySQL, RabbitMQ, MQTT, Redis, and HTTP; When subscribing to the data in the RabbitMQ open source component, explain the technology and business process of event analysis. After the event stream data is connected, clean the data by writing the sink plug-in of the eKuiper component, analyze the data according to the custom rule information in the business system, keep the data of interest, and cut and assemble it. The cutting and assembling are as follows: bind the triggering action and linkage information of the corresponding event, and finally form event data containing business extension information; For event data containing original event information and corresponding processing rule extended information, event analysis sends the event data back to switches of different topics of RabbitMQ open source components for event processing subscription.

6. The event streaming management method according to any one of claims 1 to 5, characterized in that: During event processing, subscribe to the event data after event analysis specification, parse to obtain the specific event source and corresponding processing rules, and send the processing request to the specific business processing system according to the event rules; The analysis and evaluation of abnormal events provide positive feedback to equipment management personnel, reduce the probability of events occurring, and complete the logical closed loop of the event management system.

7. An event streaming management system, characterized in that: The system includes: The event rule configuration module is used to define event management rules; among them, event management rules include event sources to be monitored, event types, event monitoring time periods, event push roles, event processing rules, and event linkage actions; The event aggregation module is used to connect the data of the RabbitMQ open source component to the data stream as the data source based on the self-developed eKuiper source plug-in, and supports multiplexing an event stream for events of the same rule; The event analysis module is used to combine the original event information of the event source with business rules to aggregate the complete event stream data and send it back to the RabbitMQ open source component; The event processing module is used to subscribe to the final event data, parse the format and send it to the relevant business system for event resolution, analyze and judge the event data and feed it back to the management personnel to complete the closed loop of the event management system.

8. The event streaming management system according to claim 7, characterized in that: After the event rule configuration module completes the event rule configuration, it sends the event rule extension message to the processing center based on the eKuiper middleware, and uses the event rule as the flow rule of the corresponding event flow; then, through the event rule configuration, it clarifies the collection, transmission, processing and linkage rules of the event flow, and provides parameter dependencies for the subsequent functions; The event aggregation module uses RabbitMQ open source components as message queues. According to the event flow rules configured by the event rules, the message queues are configured with switch storage system topic events, business topic events, and intelligent analysis topic events of various topic types. After the events are aggregated, they need to be diverted according to the event rule configuration. The diversion depends on the eKuiper open source component, and it is necessary to develop a suitable data source plug-in according to business needs. The flow concept of the event analysis module is bound to a specific business process ID or a specified event monitoring device ID, and supports access to data sources of multiple protocols; wherein the data sources include MySQL, RabbitMQ, MQTT, Redis and HTTP; When subscribing to the data in the RabbitMQ open source component, explain the technology and business process of event analysis. After the event stream data is accessed, clean the data by writing the sink plug-in of the eKuiper component, analyze the data according to the custom rule information in the business system, keep the data of interest and cut and assemble it. The cutting and assembling are as follows: bind the triggering action and linkage information of the corresponding event to finally form event data containing business extension information; for event data containing the original event information and the corresponding processing rule extension information, event analysis sends the event data back to the switches of different topics of the RabbitMQ open source component for event processing subscription use; The event processing module subscribes to the event data after event analysis specification, obtains the specific event source and the corresponding processing rules after analysis, and further sends the processing request to the specific business processing system according to the event rules; the analysis and judgment of abnormal events form positive feedback to the equipment management personnel, reduce the probability of event occurrence, and complete the logical closed loop of the event management system.

9. An electronic device, characterized in that: include: memory and at least one processor; Wherein, the memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the event streaming management method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the event streaming management method as described in any one of claims 1 to 6 is implemented.

Images