An asynchronous event-driven MessageEngine framework

Abstract

This invention discloses an asynchronous event-driven MessageEngine framework, consisting of a message loop component, a message loop process management component, and a message pump component. After the internal components of the framework are initialized, a listening thread is entered. When a message request arrives, the framework determines whether it is an event or a task. If it is a task, it calls the standard task processing interface for processing; if it is an event, it follows the relevant event processing logic. The message pump component uses an infinitely looping listening thread to continuously listen for and process events and tasks until the external control framework stops. The MessagePump.Delegate interface is called to handle immediate messages, delayed messages, and idle messages respectively. This invention solves the technical problems of asynchronous event-driven frameworks using a single-threaded asynchronous model, which makes it difficult to fully utilize computer CPU resources and lacks a comprehensive message notification monitoring and collection statistics mechanism, thus hindering the stable operation of the system.

Description

Technical Field

[0001] This invention relates to the field of network communication technology, and in particular to an asynchronous event-driven MessageEngine framework. Background Technology

[0002] Event-driven design is a popular solution for addressing the complex interactions of business logic in large systems, which can lead to strong coupling between system functions and business logic, poor scalability, and slow response speed. It is of great significance for building high-performance, high-concurrency, and highly scalable large systems.

[0003] Existing asynchronous event-driven frameworks developed in Java are based on a single-threaded asynchronous event-driven model. During the use of components, there is only one event listener thread. In modern multi-core computers, this cannot fully utilize the computer's core resources, nor does it provide a sufficiently comprehensive message notification monitoring and collection statistics mechanism, making it difficult to support the stable operation of the system. Summary of the Invention

[0004] This invention provides an asynchronous event-driven MessageEngine framework to address the technical problems of existing asynchronous event-driven frameworks, which use a single-threaded asynchronous model, making it difficult to fully utilize computer CPU resources and lack comprehensive message notification monitoring and collection statistics mechanisms, thus hindering the stable operation of the system.

[0005] In view of this, the present invention provides an asynchronous event-driven MessageEngine framework, including a message loop component MessageLoop, a message loop process management component MessageLoopRunner, and a message pump component MessagePump;

[0006] The MessageLoopRunner is a process management component that, upon receiving an external message, publishes the message to either the EventRunner or the TaskRunner based on the message's attributes. The message attribute is either an event or a task.

[0007] The TaskRunner component is used to determine when a message should be executed. If it should be executed immediately, the message is placed in the immediate execution task pool; otherwise, the message is placed in the temporary execution task pool to await scheduling.

[0008] The EventRunner component is used to process messages and then transmit the processed messages to the MessageLoopDefault component.

[0009] The MessageLoopDefault component is used to determine whether the message is in a runnable state at the current time after receiving a message. If so, the message is pushed to the immediate queue and the message pump component MessagePump is woken up to execute the message content. Otherwise, the message is pushed to the delayed queue or the idle queue.

[0010] The MessagePump component is used to call the doWork, doDelayedWork, and doIdleWork methods of the MessagePump.Delegate interface to handle immediate messages, delayed messages, and idle messages, respectively.

[0011] The MessageLoop component internally contains the event callback handler MessageCallback and the monitoring handler MetricHandler;

[0012] MessageCallback is used to define the execution logic when an event is triggered and callback is called, while MetricHandler is used to define the exception handling logic when an event execution fails.

[0013] Optionally, each group of message loop component MessageLoop, message loop process management component MessageLoopRunner, and message pump component MessagePump is a submodule of the MessageEngine framework. Each submodule communicates with the parent bus of the MessageEngine framework through its internal subbus.

[0014] Optionally, the execution logic when the event is triggered by the MessageCallback definition is as follows:

[0015] When the event is triggered and the callback is called, the processor chain factory FlowProcessFactory is invoked. FlowProcessFactory calls the pre-processing based on the pre-implanted PreAwareProcess to verify whether the source of the MessageCallback material is compliant. If the material is not compliant, it is returned. If the material is compliant, the tree batch processor TreeBatchProcess is invoked.

[0016] The TreeBatchProcess internally provides the SingleSplitProcess, BatchSaveProcess, and BatchExcelProcess processors. The SingleSplitProcess processor splits aggregated data into individual records for separate business processing. After all business logic processing is completed, the TreeBatchProcess calls the BatchSaveProcess processor to save the data in batches. If there is non-compliant data during business processing, the TreeBatchProcess pushes the non-compliant data to the BatchExcelProcess processor, which then exports the business data to Excel in batches. After all business processes are completed, the FlowProcessFactory calls the AfterAwareProcess processor to process the data based on the execution results.

[0017] Optionally, MetricHandler includes three core application interfaces: notify, consumer, and error.

[0018] Optionally, the MessageEngine framework establishes the association between the child bus and the parent bus through the parent function.

[0019] As can be seen from the above technical solutions, the asynchronous event-driven MessageEngine framework provided by this invention has the following advantages:

[0020] The asynchronous event-driven MessageEngine framework provided by this invention consists of a message loop component MessageLoop, a message loop process management component MessageLoopRunner, and a message pump component MessagePump. After the components inside the MessageEngine framework are initialized, a listening thread is entered. When a message request comes in, the MessageEngine framework will determine whether it is an event or a task. If it is a task, the TaskRunner standard interface is called for processing. If it is an event, the relevant logic of EventRunner is followed. MessagePump has an infinitely looping listening thread that continuously listens for and processes events and tasks until the external control framework stops. MessagePump calls the doWork, doDelayedWork, and doIdleWork methods of the MessagePump.Delegate interface to handle immediate messages, delayed messages, and idle messages, respectively. The asynchronous event-driven MessageEngine framework provided by this invention supports multiple threads binding multiple MessageLoop component instances, enabling multi-path loop listening in multiple threads, fully utilizing the CPU resources of modern multi-core computers. It solves the technical problems of existing asynchronous event-driven frameworks that use a single-threaded asynchronous model, making it difficult to fully utilize computer CPU resources and lacking a comprehensive message notification monitoring and collection statistics mechanism, thus making it difficult to support the stable operation of the system.

[0021] Meanwhile, the asynchronous event-driven MessageEngine framework provided by this invention not only provides asynchronous event publishing and subscription, but also supports asynchronous task operations. That is, it can handle both events and tasks as message source types, thus solving the problem of limited functionality in existing frameworks / components.

[0022] The asynchronous event-driven MessageEngine framework provided by this invention has its own event bus for each sub-module. Different sub-modules can use their own event bus to listen for and process business, or they can trigger the notifyParaent method to distribute the event to the parent bus when the event meets the specified conditions. This enables interaction between different sub-modules and realizes more complex business scenarios.

[0023] The asynchronous event-driven MessageEngine framework provided by this invention binds a MessageCallback instance to each MessageLoop as a subscriber. When an event is consumed by the corresponding subscriber, subsequent processing is carried out by a multi-stage, multi-business processor chain based on the industrial pipeline concept, with each performing its own function. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a structural block diagram of an asynchronous event-driven MessageEngine framework provided in this invention;

[0026] Figure 2 This is a simplified timing diagram of the internal components of the MessageEngine framework provided in this invention;

[0027] Figure 3 This is a flowchart of the pre-defined industrial pipeline processing after the MessageCallback consumption event provided in this invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] For easier understanding, please refer to Figure 1 This invention provides an embodiment of an asynchronous event-driven MessageEngine framework, including a message loop component MessageLoop, a message loop process management component MessageLoopRunner, and a message pump component MessagePump;

[0030] The MessageLoopRunner is a process management component that, upon receiving an external message, publishes the message to either the EventRunner or the TaskRunner based on the message's attributes. The message attribute is either an event or a task.

[0031] The TaskRunner component is used to determine when a message should be executed. If it should be executed immediately, the message is placed in the immediate execution task pool; otherwise, the message is placed in the temporary execution task pool to await scheduling.

[0032] The EventRunner component is used to process messages and then transmit the processed messages to the MessageLoopDefault component.

[0033] The MessageLoopDefault component is used to determine whether the message is in a runnable state at the current time after receiving a message. If so, the message is pushed to the immediate queue and the message pump component MessagePump is woken up to execute the message content. Otherwise, the message is pushed to the delayed queue or the idle queue.

[0034] The MessagePump component is used to call the doWork, doDelayedWork, and doIdleWork methods of the MessagePump.Delegate interface to handle immediate messages, delayed messages, and idle messages, respectively.

[0035] The MessageLoop component internally contains the event callback handler MessageCallback and the monitoring handler MetricHandler;

[0036] MessageCallback is used to define the execution logic when an event is triggered and callback is called, while MetricHandler is used to define the exception handling logic when an event execution fails.

[0037] It's important to note that the external messages received by the MessageEngine framework originate from the client, such as a user's click event. Once the MessageEngine framework is established, it first initializes all internal components, then enters the listening thread. When the MessageEngine framework detects an incoming external message, the MessageLoopRunner receives the message and publishes it to the corresponding component based on its attributes. If the message is an event, it is published to the EventRunner for processing; if the message is a task, it is published to the TaskRunner. The EventRunner processes the message data and transmits the processed message to the MessageLoopDefault component. Upon receiving the message, the MessageLoopDefault component determines whether the message is currently in an runnable state. If so, it pushes the message to the immediate queue, waking up the MessagePump component to execute the message content; otherwise, it pushes the message to the delayed queue or idle queue. The listening thread within the MessagePump component implements infinite message loop control, continuously listening for and processing events. MessagePump calls the `doWork`, `doDelayedWork`, and `doIdleWork` methods of the `MessagePump.Delegate` interface to handle immediate, delayed, and idle messages, respectively. When the task processing component `TaskRunner` receives a message, it processes the task through its internal `ThreadPoolTaskRunner`, determining the message's execution timing. If immediate execution is desired, the message is placed in the immediate execution task pool; otherwise, it is placed in the temporary execution task pool awaiting scheduling. `ThreadPoolTaskRunner` is a specific implementation based on the `TaskRunner` interface standard definition. Its internal `postDelayed()` method implements the specific `runable.run()` behavior, first checking if the specified execution time has arrived; if so, execution is performed immediately; otherwise, the message is added to the waiting scheduling queue.

[0038] The MessageLoop component internally contains an event callback handler (MessageCallback) and a monitoring handler (MetricHandler). MessageCallback is used to define the execution when an event triggers a callback.

[0039] The `MetricHandler` class defines the exception handling logic when an event execution fails. It contains three core interfaces (APIs): `notify`, `consumer`, and `error`. The `notify` interface executes event tracking notifications, the `consumer` interface provides automatic detection and statistics of message processing time (accurate to nanoseconds), and the `error` interface provides configuration functionality for exception alarm notifications; if not configured, it defaults to formatted exception log output.

[0040] During the initialization of the MessageEngine framework, the event callback handler MessageCallback and the monitoring handler MetricHandler need to be configured inside the message loop component MessageLoop. MessageCallback predefines the logical behavior for handling various events, that is, listening to a specified set of events, and internally defines the execution logic for each specific business algorithm. MetricHandler needs to implement the three standard interfaces notify, consumer, and error, and configure the default alarm notification platform integration information.

[0041] The Notify interface executes its instrumentation logic when messages are pushed into the queue or when messages in the queue are consumed, allowing you to obtain the current message backlog / remaining quantity in the immediate and delayed queues.

[0042] Whenever a message in the queue is consumed and completed, the consumer interface is executed to obtain the processing time of the current message, accurate to the nanosecond level. Then, according to specific logic, it is reported to the storage medium for centralized management, which facilitates subsequent statistics and displays the statistical results in a visual interface.

[0043] The error interface can capture error information of all unexpected exceptions and push the error information to the alarm notification platform. If this configuration was not defined when the MetricHandler component was passed in, the log will be formatted and output to the console by default.

[0044] In one embodiment, each group of message loop components (MessageLoop, MessageLoopRunner, and MessagePump) is a submodule of the MessageEngine framework. Each submodule communicates with the parent bus of the MessageEngine framework via its internal subbus. Figure 2As shown, Service / Module A, Service / Module B, and Service / Module C are each a sub-module of the MessageEngine framework. The multi-threaded events within each sub-module are associated with the parent bus MessageEngine-Event Bus of the MessageEngine framework via a sub-EventBus using the parent function. This allows each sub-module to communicate with other sub-modules through the parent bus MessageEngine-Event Bus. For example, if Service / Module A, Service / Module B, and Service / Module C want to communicate, they can do so by pushing and subscribing to MessageEngine. Service / Module A's business pushes messages to its own Sub-Event Bus (an implementation provided by MessageEngine), Service / Module B's business can subscribe and consume the messages, and Sub-Event Bus A calls getParent to retrieve the pre-bound parent Event Bus (an implementation provided by MessageEngine) and pushes the messages to the parent bus. Service / Module B and Service / Module C, which subsequently subscribe to the bus, will then receive the messages.

[0045] In one embodiment, the execution logic when the event is triggered by the MessageCallback definition is as follows: Figure 3As shown, when an event is triggered and a callback occurs, the FlowProcessFactory (which provides a template for how to handle event business logic, using a chain of responsibility design pattern. If the detected event is bound to a dataset, it can be processed using this standard process, which has defined specifications) is called. The FlowProcessFactory calls the pre-implanted PreAwareProcess processor for pre-processing, verifying whether the source of the MessageCallback material is compliant. If the material is non-compliant, it is returned by the ReviewSchedulerProcess processor. If the material is compliant, the TreeBatchProcess processor is called. The TreeBatchProcess processor internally provides the SingleSplitProcess processor and the BatchSave processor. The Process processor and the BatchExcelProcess processor are used. The SingleSplitProcess processor splits the aggregated data into individual records for separate business processing. After all business logic processing is completed, the TreeBatchProcess processor calls the BatchSaveProcess processor to save the data in batches. If there is non-compliant data during business processing, the TreeBatchProcess processor pushes the non-compliant data to the BatchExcelProcess processor, which then exports the business data to Excel in batches. After all business processes are completed, the FlowProcessFactory processor calls the AfterAwareProcess processor to process the data based on the execution results.

[0046] The MessageEngine provided by this invention is a lock-free asynchronous event-driven framework developed based on the Java language and adopting an industrial pipeline processing mindset. It adopts an event bus-driven design concept for framework design, provides a very simple and easy-to-use API for multi-threaded asynchronous publishing and subscription of events or tasks, supports configurable and controllable delayed publishing, supports an event bus mechanism for establishing parent-child relationships, and provides a comprehensive and flexible message notification monitoring and abnormal information collection and statistics mechanism.

[0047] The asynchronous event-driven MessageEngine framework provided by this invention consists of a message loop component MessageLoop, a message loop process management component MessageLoopRunner, and a message pump component MessagePump. After the components inside the MessageEngine framework are initialized, a listening thread is entered. When a message request comes in, the MessageEngine framework will determine whether it is an event or a task. If it is a task, the TaskRunner standard interface is called for processing. If it is an event, the relevant logic of EventRunner is followed. MessagePump has an infinitely looping listening thread that continuously listens for and processes events and tasks until the external control framework stops. MessagePump calls the doWork, doDelayedWork, and doIdleWork methods of the MessagePump.Delegate interface to handle immediate messages, delayed messages, and idle messages, respectively. The asynchronous event-driven MessageEngine framework provided by this invention supports multiple threads binding to multiple MessageLoop component instances. That is, the number of MessageLoops instantiated internally can be specified when initializing the MessageEngine framework. Each MessageLoop is bound to a message pump thread. When an external client pushes a message, it can be specified which MessageLoop in the MessageEngine to push to, corresponding to its own queue and message loop. Multi-path loop listening is started in multiple threads, which makes full use of the CPU resources of modern multi-core computers. This solves the technical problems of existing asynchronous event-driven frameworks that use a single-threaded asynchronous model, which makes it difficult to make full use of computer CPU resources and lacks a comprehensive message notification monitoring and collection statistics mechanism, making it difficult to support the stable operation of the system.

[0048] Meanwhile, the asynchronous event-driven MessageEngine framework provided by this invention not only provides asynchronous event publishing and subscription, but also supports asynchronous task operations. That is, it can handle both events and tasks as message source types, thus solving the problem of limited functionality in existing frameworks / components.

[0049] The asynchronous event-driven MessageEngine framework provided by this invention has its own event bus for each sub-module. Different sub-modules can use their own event bus to listen for and process business, or they can trigger the notifyParaent method to distribute the event to the parent bus when the event meets the specified conditions. This enables interaction between different sub-modules and realizes more complex business scenarios.

[0050] The asynchronous event-driven MessageEngine framework provided by this invention binds a MessageCallback instance to each MessageLoop as a subscriber. When an event is consumed by the corresponding subscriber, subsequent processing is carried out by a multi-stage, multi-business processor chain based on the industrial pipeline concept, with each performing its own function.

[0051] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An asynchronous event-driven MessageEngine framework, characterized in that, This includes the message loop component MessageLoop, the message loop process management component MessageLoopRunner, and the message pump component MessagePump; The MessageLoopRunner is a process management component that, upon receiving an external message, publishes the message to either the EventRunner or the TaskRunner based on the message's attributes. The message attribute is either an event or a task. The TaskRunner component is used to determine when a message should be executed. If it should be executed immediately, the message is placed in the immediate execution task pool; otherwise, the message is placed in the temporary execution task pool to await scheduling. The EventRunner component is used to process messages and then transmit the processed messages to the MessageLoopDefault component. The MessageLoopDefault component is used to determine whether the message is in a runnable state at the current time after receiving a message. If so, the message is pushed to the immediate queue and the message pump component MessagePump is woken up to execute the message content. Otherwise, the message is pushed to the delayed queue or the idle queue. The MessagePump component is used to call the doWork, doDelayedWork, and doIdleWork methods of the MessagePump.Delegate interface to handle immediate messages, delayed messages, and idle messages, respectively. The MessageLoop component internally contains an event callback handler (MessageCallback) and a monitoring handler (MetricHandler). MessageCallback is used to define the execution logic when an event is triggered and callback is called, while MetricHandler is used to define the exception handling logic when an event execution fails. The execution logic when the event is triggered by the MessageCallback definition is as follows: When the event is triggered and the callback is called, the processor chain factory FlowProcessFactory is invoked. FlowProcessFactory calls the pre-processing based on the pre-implanted PreAwareProcess to verify whether the source of the MessageCallback material is compliant. If the material is not compliant, it is returned. If the material is compliant, the tree batch processor TreeBatchProcess is invoked. The TreeBatchProcess internally provides the SingleSplitProcess, BatchSaveProcess, and BatchExcelProcess processors. The SingleSplitProcess processor splits aggregated data into individual records for separate business processing. After all business logic processing is completed, the TreeBatchProcess calls the BatchSaveProcess processor to save the data in batches. If there is non-compliant data during business processing, the TreeBatchProcess pushes the non-compliant data to the BatchExcelProcess processor, which then exports the business data to Excel in batches. After all business processes are completed, the FlowProcessFactory calls the AfterAwareProcess processor to process the data based on the execution results.

2. The asynchronous event-driven MessageEngine framework according to claim 1, characterized in that, Each group of message loop components (MessageLoop, MessageLoopRunner, and MessagePump) is a submodule of the MessageEngine framework. Each submodule communicates with the parent bus of the MessageEngine framework through its internal subbus.

3. The asynchronous event-driven MessageEngine framework according to claim 2, characterized in that, MetricHandler includes three core application interfaces: notify, consumer, and error.

4. The asynchronous event-driven MessageEngine framework according to claim 2, characterized in that, The MessageEngine framework establishes the relationship between child buses and parent buses through the parent function.

Images