Event processing method and device based on staged event-driven architecture, equipment and medium

Abstract

The invention relates to the technical field of computer software application, in particular to an event processing method and device based on a staged event-driven architecture, equipment and a medium. The method comprises the steps that a staged event-driven architecture is adopted, a plurality of modules are divided according to application service logic and are in sequence, each stage is provided with an independent thread pool to process a corresponding task, whether the length of an event queue exceeds a limit value or not is monitored in the event processing process of the modules, and after it is found that the length exceeds the limit value, the event queue is started; and immediately triggering a load regulation mechanism to carry out load regulation and control, and if the length of the event queue is continuously increased and exceeds a length threshold value of the event queue corresponding to the target module, discarding events of which the priorities are lower than a preset priority threshold value in the event queue. The SEDA architecture is utilized to ensure the stability of the system under the condition of high concurrency through a multi-level load regulation mechanism, and the event processing efficiency can be improved to a certain extent.

Description

Technical Field

[0001] The present application relates to the technical field of computer software applications, and in particular to an event processing method, apparatus, device and medium based on a phased event-driven architecture. Background Art

[0002] In modern Internet applications, server systems need to handle a large number of concurrent requests at the same time, especially in high-traffic websites, real-time communication platforms, and large-scale online services. In these scenarios, the server not only needs to handle a large number of user requests, but also needs to ensure the response speed and stability of the system. With the rapid growth of the number of Internet users and the increasing complexity of application functions, traditional server architectures have gradually exposed some problems that cannot be ignored.

[0003] Typically, server systems use a thread-driven model to handle concurrent requests. This approach enables the server to handle multiple requests simultaneously by assigning an independent thread to each request. However, as the number of concurrent requests increases, when there are a large number of threads in the system, the operating system needs to frequently switch the execution context of the threads, which not only consumes a large amount of CPU resources, but also reduces the overall processing performance of the system. Moreover, since each thread needs to be allocated independent resources, under high load conditions, system resources are easily exhausted, resulting in a significant drop in server performance or even a crash.

[0004] Staged Event-Driven Architecture (SEDA) is an architectural pattern that divides complex business processes into a series of independent and sequentially executed processing stages. Each stage focuses on completing a specific logical functional unit and communicates with adjacent stages through event queues.

[0005] The use of a phased event-driven model architecture can, to a certain extent, solve the drawbacks caused by the thread-driven model in processing concurrent requests. However, in high-concurrency scenarios, if the event processing speed of the processing stage does not reach the speed of the event arrival, it may still cause the event processing burden of a single stage. Therefore, how to improve the load balancing of the stages and improve the efficiency of processing events has become a technical problem that technicians in this field need to solve urgently. Summary of the invention

[0006] The purpose of this application is to provide an event processing method, device, equipment and medium based on a phased event-driven architecture, which uses the SEDA architecture to ensure the stability of the system under high concurrency conditions through a multi-level load regulation mechanism, and can improve the event processing efficiency to a certain extent.

[0007] In a first aspect, an event processing method based on a phased event-driven architecture is provided, wherein the phased event-driven architecture includes a plurality of sequential modules divided according to application business logic, and each phase has an independent thread pool to process corresponding tasks;

[0008] The event processing method comprises:

[0009] Acquire the length of the event queue of the target module in real time, wherein the target module is any module among the multiple modules;

[0010] Determine whether the length of the event queue exceeds a length threshold of the event queue corresponding to the target module;

[0011] If the length of the event queue exceeds the length threshold of the event queue corresponding to the target module, a load adjustment mechanism is triggered;

[0012] After adjustment using the load adjustment mechanism, if the length of the event queue continues to grow and exceeds the length threshold of the event queue corresponding to the target module, events in the event queue whose priority is lower than the preset priority threshold are discarded.

[0013] In a preferred example, the present application may be further configured as follows: the load adjustment mechanism includes at least one of the following:

[0014] reducing the rate at which new events are received by the target module;

[0015] Increase the number of threads of the target module;

[0016] Based on the priority of the event, high priority events are handled first;

[0017] Increase scheduling frequency.

[0018] In a preferred example, the present application may be further configured as follows: the process of determining the priority of the event includes:

[0019] Determine, from the attribute information of the event field, attribute information corresponding to each of several target fields that meet the priority of the target module;

[0020] Determine the attribute quantization value and weight value corresponding to each attribute information;

[0021] A weighted calculation is performed based on the attribute quantization value and weight value corresponding to each attribute information to obtain the priority of the event.

[0022] In a preferred example, the present application may be further configured as follows: the target module includes a plurality of submodules responsible for different processing tasks, and the method further includes:

[0023] Determine the task type of the current pending event in the event queue of the target module;

[0024] According to the task type of the current event to be processed, a target submodule is determined from multiple submodules, and the current event to be processed is processed through a thread corresponding to the target submodule.

[0025] In a preferred example, the present application can be further configured as follows: the method further includes:

[0026] If the request event sent by the previous module and received by the target module is a complex event, the request event is split to obtain multiple sub-request events; the multiple sub-request events are written into the event queue of the target module in sequence; the multiple sub-request events in the event queue are processed by the thread corresponding to the target module; after completing the processing of the multiple sub-request events, the sub-processing results corresponding to the multiple sub-request events are merged to obtain the processing results, and the processing results are sent to the next module of the target module so that the next module can perform corresponding business processing.

[0027] In a preferred example, the present application may be further configured as follows: before the real-time acquisition of the length of the event queue of the target module and the number of threads currently used by the target module, the method further includes:

[0028] Load each module according to the configuration file or initialization script;

[0029] Assign corresponding event queue and initial thread pool to each module;

[0030] Set system resource allocation rules and scheduling strategies.

[0031] In a preferred example, the present application may be further configured as follows: the method further includes:

[0032] Get the historical load data of each module;

[0033] Based on the historical load data of each module and the current load data of each module, the load forecast result for the future preset period is obtained;

[0034] Determine whether it is a load peak period based on the resource prediction results. If so, set up a reserved resource pool for use in a future preset period of time.

[0035] In a second aspect, an event processing device based on a phased event-driven architecture is provided, wherein the phased event-driven architecture includes a plurality of sequential modules divided according to application business logic, and each phase has an independent thread pool to process corresponding tasks;

[0036] The event processing device comprises:

[0037] An acquisition module, used for acquiring the length of the event queue of a target module in real time, wherein the target module is any module among the multiple modules;

[0038] A determination module, used to determine whether the length of the event queue exceeds a length threshold of the event queue corresponding to the target module;

[0039] A trigger module, configured to trigger a load regulation mechanism if the length of the event queue exceeds a length threshold of an event queue corresponding to a target module;

[0040] The discard module is used to discard events in the event queue whose priority is lower than a preset priority threshold if the length of the event queue continues to grow and exceeds the length threshold of the event queue corresponding to the target module after adjustment using the load adjustment mechanism.

[0041] In a third aspect, an electronic device is provided, including:

[0042] one or more processors;

[0043] Memory;

[0044] One or more applications, wherein one or more applications are stored in a memory and configured to be executed by one or more processors, and the one or more programs are configured to: perform operations corresponding to the event processing method based on the staged event-driven architecture shown in any possible implementation method of the first aspect.

[0045] In a fourth aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded by a processor and executed according to the steps of an event processing method based on a staged event-driven architecture as shown in any possible implementation method in the first aspect.

[0046] In a fifth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements operations corresponding to the event processing method based on the staged event-driven architecture shown in any possible implementation method in the first aspect.

[0047] To sum up, the method provided by the present application has the following beneficial technical effects: the staged event-driven architecture includes multiple modules divided in sequence according to the application business logic, each stage has an independent thread pool to handle the corresponding tasks, and in the process of the module processing events, the length of the event queue is monitored to see if it exceeds the limit. When it is found that the limit is exceeded, the load adjustment mechanism is immediately triggered to perform load regulation. At the same time, if the length of the event queue continues to grow and exceeds the length threshold of the event queue corresponding to the target module, the events in the event queue whose priority is lower than the preset priority threshold are discarded. The SEDA architecture is used to ensure the stability of the system under high concurrency conditions through a multi-level load adjustment mechanism, and the event processing efficiency can be improved to a certain extent.

[0048] At the same time, the present application also provides an event processing device, equipment and medium based on a phased event-driven architecture, all of which have the above-mentioned beneficial technical effects and will not be described in detail. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions of the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 A schematic diagram of an event processing flow provided in an embodiment of the present application;

[0051] Figure 2 A flowchart of an event processing method based on a phased event-driven architecture provided in an embodiment of the present application;

[0052] Figure 3 A hierarchical structure diagram of a module provided in an embodiment of the present application;

[0053] Figure 4 A hierarchical structure diagram of a middle-level architecture of a module provided in an embodiment of the present application;

[0054] Figure 5 A schematic diagram of a bottom-level implementation provided in an embodiment of the present application;

[0055] Figure 6 It is a structural diagram of an event processing device based on a phased event-driven architecture provided in an embodiment of the present application;

[0056] Figure 7 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, a person skilled in the art may make non-creative modifications to the present embodiment as needed, but such modifications are protected by the patent law as long as they are within the scope of the present application.

[0058] It should be noted that in the optional embodiments of the present application, the object information and other related data involved, when the embodiments in the present application are applied to specific products or technologies, need to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if the embodiments of the present application involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information needs to obtain the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained. The embodiments also need to be implemented with the authorization and consent of the object.

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0061] In order to solve the processing performance and stability problems of servers in high-concurrency scenarios, the most commonly used solution in the prior art is the thread-driven model. This technical solution relies on the thread mechanism provided by the operating system and implements parallel processing of multiple requests by assigning independent threads to each concurrent request. The basic idea of ​​the thread-driven model is to use the parallelism of multi-threading to improve the system's responsiveness and throughput.

[0062] The process of processing events in the thread-driven model may include: when the client sends a request, the server receives the request and immediately creates a new thread for the request or allocates an idle thread from the thread pool to process the request; each thread independently performs the assigned task, which may include I / O operations (such as reading request content, database queries), data processing (such as logical operations, data conversion) and generating response content; after completing the task, the thread packages the processing result into response data and sends it back to the client through the network; after the task is completed, the thread can be destroyed or returned to the thread pool for subsequent requests.

[0063] In the thread-driven model, the overall architecture of the server is a structure in which a "main thread" manages multiple "worker threads". The main thread is responsible for listening to the client's connection request. Once a request is received, the main thread assigns it to a worker thread for processing. Each worker thread runs independently, returns the result after processing, and notifies the main thread that the task has been completed.

[0064] The main thread is responsible for managing the creation and scheduling of worker threads, listening to client connections, and distributing requests. Worker threads independently execute received tasks. In order to avoid the overhead of frequent creation and destruction of threads, the system usually maintains a thread pool. After completing the task, the worker thread returns to the thread pool to wait so that the next request can directly reuse the thread.

[0065] The advantage of the thread-driven model is that it is simple and easy to use, and it fully utilizes the parallel computing capabilities of multi-core processors. Through multi-threaded parallel processing, the server can process multiple requests at the same time, thereby improving the throughput and response speed of the system. This method solves the problem of insufficient concurrent processing capabilities of the server to a certain extent, especially when the number of requests is limited, and can significantly improve system performance.

[0066] However, as the number of concurrent requests increases, the thread-driven model begins to exhibit the following technical defects:

[0067] Context switching overhead: The operation of each thread requires the operating system to frequently perform context switching to allocate CPU time slices. Especially in high concurrency situations, the number of threads increases dramatically, causing the CPU to spend a lot of time on context switching, which in turn reduces the execution time of the actual task.

[0068] Resource contention and locking: When multiple threads access shared resources (such as memory and database connections) at the same time, resource contention and locking problems may occur. This kind of contention can lead to unstable system performance, especially in high concurrency situations, which may cause serious performance bottlenecks.

[0069] Resource management complexity: The thread-driven model needs to allocate independent resources (such as memory and CPU time) to each concurrent task. As the number of threads increases, the risk of system resource exhaustion also increases, increasing the possibility of system crashes. Especially during peak system load periods. For example, in the case of a surge in network requests, the thread-driven model may start a large number of threads to handle these requests, but not all threads can effectively utilize CPU and memory resources. This unpredictability of resource allocation will increase the system's memory consumption and may even cause the system to run out of resources. In addition, resource management in the thread-driven model is usually the responsibility of the operating system, which means that it is difficult for developers to finely control the use of resources. The general resource management strategy of the operating system cannot be optimized for specific application scenarios. Therefore, in some cases, resources may be occupied unnecessarily or unevenly allocated, resulting in large fluctuations in system performance.

[0070] Insufficient load regulation capability: The thread-driven model lacks an effective load regulation mechanism and cannot dynamically adjust resource allocation according to the actual system load, making it difficult for the system to maintain stable performance under high load.

[0071] In summary, the thread-driven model is the main solution for solving high-concurrency server processing problems in the existing technology. Although it has achieved certain results in concurrent processing, it has significant technical limitations in resource management, context switching, resource competition, and load regulation. These problems limit the performance improvement of the thread-driven model in dealing with large-scale, high-concurrency scenarios, and also bring potential risks to the stability of the server system.

[0072] Therefore, in this context, how to design a server architecture that can effectively cope with high concurrency requirements and maintain system stability under high load has become an urgent problem to be solved in the current technical field. This not only involves the concurrent processing capabilities of the server, but also the rational use of system resources and dynamic adjustment capabilities.

[0073] It is in this context that this application is proposed, and aims to solve the above problems through a staged event-driven architecture (SEDA), aiming to solve the problems of limited concurrency performance, complex resource management, and insufficient system scalability in the existing server architecture, and to improve the overall performance and stability of the server system in high concurrency scenarios.

[0074] SEDA divides event-driven applications into multiple stages, each of which independently handles specific tasks, and each stage is connected through queues to form a modular system architecture. Each stage is processed using an independent thread, which not only improves the system's concurrency capabilities, but also enables fine-tuning of the load, thereby ensuring the stability of the system under high load conditions.

[0075] The following is a brief description of the SEDA architecture. Figure 1 A flow chart of event processing provided for an embodiment of the present application, wherein the modules of the SEDA architecture include but are not limited to the following categories: Request receiving module: used to receive network requests from clients, parse request contents and convert them into internal events. Event module: processes received events, which may involve operations such as data calculation and database query. Response generation module: generates response data based on the results of event processing and packages it into a network data packet. Response sending module: responsible for sending the generated response data to the client.

[0076] The output of each module will be passed as input to the next module, and the entire server system forms a pipeline processing flow, and the modules are decoupled through queues; the processing logic of each module can also be flexibly adjusted, and load balancing can be achieved at the same time. It can be understood that the above is only an example, and this embodiment is not limited here.

[0077] The embodiment of the present application provides an event processing method based on a phased event-driven architecture, which is applied to Internet applications, especially in high-traffic websites, real-time communication platforms, and large-scale online services.

[0078] Among them, the staged event-driven architecture includes multiple sequential modules divided according to the application business logic, and each stage has an independent thread pool to process corresponding tasks.

[0079] In some embodiments, in order to prevent the performance problem of a certain module from affecting other modules, the SEDA architecture uses thread isolation technology to ensure that the threads of each module can only access the resources belonging to the module, thereby avoiding cross-module resource contention problems.

[0080] In some embodiments, each module in the SEDA architecture runs independently in one or more threads. The management and scheduling of threads are automatically completed by the system, and developers do not need to manage threads directly. Specific thread management strategies include:

[0081] 1. Dynamic thread pool management: The system maintains a dynamic thread pool for each module and dynamically adjusts the number of threads according to the current load of the module. For example, when the length of the event queue of a module exceeds the preset threshold, the system can automatically increase the number of threads allocated to the module to improve processing capabilities. 2. System-level resource scheduling: The system can monitor the resource usage of the entire server (such as CPU, memory, network bandwidth, etc.), and dynamically adjust the resource allocation strategy of each module according to the resource usage to ensure that system resources are optimally utilized. 3. Thread scheduling strategy: The system adopts polling or priority-based thread scheduling strategy to ensure that events of all modules can be processed in a timely manner. At the same time, the thread scheduler can adjust the scheduling frequency according to the importance and load of the module. 4. Load balancing between modules: When the load of a module is too high and the load of other modules is light, the system can dynamically adjust the task allocation strategy, appropriately reduce the task volume of the module, and increase the task volume of other modules to achieve a balanced distribution of the overall load.

[0082] Specifically, Figure 2 As shown, the method provided in the embodiment of the present application can be executed by an electronic device, and the electronic device is a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The embodiment of the present application is not limited here. The method includes:

[0083] S201, obtaining the length of the event queue of a target module in real time, where the target module is any module among multiple modules;

[0084] The embodiment of the present application is described by taking any module among multiple modules as an example, and the event processing processes of all modules refer to the target module.

[0085] The event queue is a data structure used in the target module to temporarily store pending events. It stores events in a certain order (such as first-in, first-out). The length of the event queue is used to indicate the number of pending events in the event queue, which can be used to evaluate the current load of the queue.

[0086] In some embodiments, the electronic device may periodically send query requests to the target module. After receiving the request, the target module returns the current event queue length and number of threads as a response to the electronic device to achieve real-time monitoring of the event queue length.

[0087] Furthermore, before obtaining the length of the event queue of the target module in real time, it also includes: loading each module according to the configuration file or initialization script; allocating a corresponding event queue and an initial thread pool to each module; and setting the allocation rules and scheduling strategies of system resources.

[0088] S202, determining whether the length of the event queue exceeds the length threshold of the event queue corresponding to the target module;

[0089] S203, if the length of the event queue exceeds the length threshold of the event queue corresponding to the target module, a load adjustment mechanism is triggered;

[0090] Among them, a maximum length limit is set for the event queue of the target module, which is used to determine whether the event queue has approached or exceeded the processing capacity of the target module. It should be noted that the length thresholds of the event queues of the modules can be the same or different. The length threshold of the event queue corresponding to the module can be set by the user based on experience or based on the efficiency of the module in processing events.

[0091] Monitor the length of each event queue in real time. When the queue length does not exceed the preset threshold, continue to monitor. When the queue length exceeds the preset threshold, the current processing capacity of the target module can no longer meet the needs of event processing, and the system needs to take measures to avoid overload or improve processing capacity. Through the load adjustment mechanism, it can prevent system overload and crash to a certain extent, ensure the stability and reliability of the system, and improve the response speed and processing capacity of the system.

[0092] Further, in a possible implementation of the embodiment of the present application, the load adjustment mechanism includes at least one of the following:

[0093] Method 1: Reduce the rate at which new events are received by the target module;

[0094] Method 2: Increase the number of threads of the target module;

[0095] Method 3: Based on the priority of the event, the events with high priority are processed first;

[0096] Method 4: Increase the scheduling frequency.

[0097] For method 1, the load of the target module can be reduced by reducing the rate at which the target module receives new tasks or events. Specifically, the previous module of the target module can be controlled to reduce the rate at which new events are sent, or a buffer can be set to write new events into the buffer and control the target module to reduce the rate at which events are captured.

[0098] Regarding the magnitude of the rate reduction, in one achievable manner, a fixed preset rate can be reduced, and the preset rate can be set by the user according to actual needs; in another achievable manner, the magnitude of the reduced rate can be determined based on the percentage of the event length exceeding and the event processing rate, specifically, the first value is determined according to the percentage and the first corresponding relationship, the first corresponding relationship is the corresponding relationship between multiple percentage ranges and multiple values, wherein the larger the percentage, the larger the value; the second value is determined according to the event processing rate and the second corresponding relationship, the second corresponding relationship is the corresponding relationship between multiple event processing rates and multiple values, the smaller the processing rate, the larger the value; the magnitude of the reduced rate is determined according to the sum of the first value and the second value, and the third corresponding relationship, the third corresponding relationship is the corresponding relationship between multiple values ​​and multiple reduced rate magnitudes, the larger the value, the larger the reduced rate magnitude. In another achievable manner, the reduced step frequency a can be set, the receiving rate a is first reduced, and then after a preset time interval, it is determined whether the event length of the event list still exceeds the limit, if so, the receiving rate a is reduced again until it does not exceed the limit, or the number of reductions reaches the preset number threshold.

[0099] For method 2, increase the number of threads of the target module. The processing capability of the target module can be improved by increasing the number of threads that process events in parallel.

[0100] The target module maintains a thread pool. Generally, some threads will be extracted from the thread pool to participate in event processing, and some threads will be reserved. When the event length of the event list exceeds the limit, the remaining threads in the thread pool of the target module can participate in the event processing task to improve the processing capacity of the target module; the number of additional threads can be a fixed value as long as the thread pool can provide it. Of course, it can also be determined based on the current event processing rate of the target module. The higher the rate, the smaller the number of additional threads.

[0101] For method three, based on the priority of the event, high-priority events are processed first. Among them, the events are sorted according to their priorities in the event queue, and high-priority events are processed first to ensure that critical tasks are processed in a timely manner. It can be seen that in the embodiment of the present application, under high load conditions, the processing order of events can be adjusted according to the importance (priority) of the events, so as to give priority to critical events and improve the overall response speed of the system.

[0102] Method 4, increasing the scheduling frequency, means increasing the frequency of system scheduling tasks to check the task queue, allocate resources and execute tasks more frequently. Increasing the scheduling frequency can reduce the task waiting time. It is understandable that the electronic device can allocate processing time for each thread, and each thread will be assigned a fixed time slice. At the end of the time slice, regardless of whether the task is completed, it will switch to the next task, avoiding the situation where some tasks occupy resources for a long time. In an embodiment of the present application, when the load is high, the time slice of each task is appropriately reduced to speed up the frequency of task switching, thereby improving the responsiveness and throughput of the system. Correspondingly, the time slice can also be increased when the load is low.

[0103] The increased frequency may be a fixed frequency set by the user, or the frequency may be gradually increased according to a set step frequency. This embodiment of the present application does not limit this, and the user may set it according to actual needs.

[0104] S204: After the load regulation mechanism is used for regulation, if the length of the event queue continues to grow and exceeds the length threshold of the event queue corresponding to the target module, events in the event queue whose priority is lower than the preset priority threshold are discarded.

[0105] After the load regulation mechanism is used to regulate, the event queue of the target module is continuously monitored. If the length of the event queue continues to grow and exceeds the length threshold of the event queue corresponding to the target module, some low-priority events can be discarded to ensure that the processing of high-priority events is not affected. The preset priority threshold can be a fixed value or can be dynamically determined based on the fixed discard quantity according to the priority of the events in the current event queue.

[0106] It can be seen that in the embodiment of the present application, the staged event-driven architecture includes a plurality of sequential modules divided according to the application business logic, each stage has an independent thread pool to handle the corresponding tasks, and in the process of the module processing events, the length of the event queue is monitored to see if it exceeds the limit. When it is found that the limit is exceeded, the load regulation mechanism is immediately triggered to perform load regulation. At the same time, if the length of the event queue continues to grow and exceeds the length threshold of the event queue corresponding to the target module, the events in the event queue whose priority is lower than the preset priority threshold are discarded. The SEDA architecture is used to ensure the stability of the system under high concurrency conditions through a multi-level load regulation mechanism, and to a certain extent, the event processing efficiency can be improved.

[0107] Furthermore, in a possible implementation of the embodiment of the present application, the process of determining the priority of an event includes:

[0108] Determine, from the attribute information of the event field, attribute information corresponding to each of several target fields that meet the priority of the target module;

[0109] Determine the attribute quantization value and weight value corresponding to each attribute information;

[0110] A weighted calculation is performed based on the attribute quantization value and weight value corresponding to each attribute information to obtain the priority of the event.

[0111] Among them, in one achievable manner, the corresponding target fields of all stages (modules) are the same. For example, if the field is a client field or an internal event field, if it is a client field, it is directly determined as a high priority, and if it is an internal event field, it is directly determined as a low priority. In another achievable manner, different modules are set with different priority judgment mechanisms and strategies, and the reference fields are different, and corresponding attribute quantization values ​​and weight values ​​are maintained for each module. For example, it is stipulated that: the fields used to calculate the priority in module 1 are field a and field c. When field a is A1, its corresponding attribute quantization value is p1, and the weight is p2. When field a is A2, its corresponding attribute quantization value is p3, and the weight is p4; when field c is C1, its corresponding attribute quantization value is Q1, and the weight is Q2. When field c is C2, its corresponding attribute quantization value is Q3, and the weight is Q4. If it is actually field A1+field C2, at this time, the value corresponding to the determined priority = p1* p2+ Q3* Q4.

[0112] It can be seen that in the embodiment of the present application, the attribute information corresponding to several target fields of each event whose priority is determined is used, and then a weighted calculation is performed based on the attribute quantization value and weight value corresponding to each attribute information to obtain the priority of the event, and corresponding priority setting rules are set for each module, which is more flexible.

[0113] Further, in order to more clearly describe the technical solution of the present application, see Figure 3 , Figure 3 A hierarchical structure diagram of a module provided for an embodiment of the present application is specifically the top-level architecture of SEDA. The top-level architecture mainly reflects the overall process of the system and the relationship between the modules. The entire system is divided into several independent modules, and each module is decoupled through queues.

[0114] In some other embodiments, a middle-level architecture may also be included. For details, see Figure 4 , Figure 4 The hierarchical structure diagram of the middle-level architecture of a module provided in the embodiment of the present application shows the internal structure of each module and the calling relationship between each other. Each module can be further subdivided into multiple submodules, each of which is responsible for different processing tasks.

[0115] See also Figure 5 , Figure 5 A schematic diagram of an underlying implementation provided for an embodiment of the present application, including thread management, queue processing logic, and resource scheduling strategy.

[0116] Based on this, in a possible implementation manner of the embodiment of the present application, the target module includes multiple sub-modules responsible for different processing tasks, and the method further includes:

[0117] Determine the task type of the current pending event in the event queue of the target module;

[0118] According to the task type of the current event to be processed, a target submodule is determined from multiple submodules, and the current event to be processed is processed through a thread corresponding to the target submodule.

[0119] It can be seen that in an embodiment of the present application, after identifying the task type of the current event to be processed, a target sub-module corresponding to the task type can be selected from multiple preset sub-modules. Each sub-module can classify threads to process corresponding tasks to achieve targeted processing, which can improve event processing efficiency to a certain extent.

[0120] Furthermore, a possible implementation method of the embodiment of the present application also includes: if the request event sent by the previous module and received by the target module is a complex event, the request event is split to obtain multiple sub-request events; the multiple sub-request events are written into the event queue of the target module in sequence; the multiple sub-request events in the event queue are processed by the thread corresponding to the target module; after completing the processing of the multiple sub-request events, the sub-processing results corresponding to the multiple sub-request events are merged to obtain the processing results, and the processing results are sent to the next module of the target module so that the next module can perform corresponding business processing.

[0121] In the embodiment of the present application, when the target module receives a request event from the previous module, its complexity can be identified through the specific implementation of the request. As long as there is more than one request task, it is considered a complex event and the split operation is automatically performed. The system writes these split sub-request events into the event queue of the target module in an orderly manner, so that the complex request event is disassembled into multiple sub-request events that are easier to manage and process, effectively reducing the difficulty of single processing and improving processing efficiency; then use the multiple concurrent independent threads corresponding to the target module to process multiple sub-request events in the event queue; after all sub-request events are processed, the system can automatically merge the processing results of these sub-request events to obtain the final processing result, and send the result to the next module of the target module.

[0122] It can be seen that in the embodiment of the present application, by splitting complex request events, and then using multi-threaded parallel processing of the split sub-request events, and merging the sub-processing results, the system's ability to handle complex request events can be improved, processing efficiency can be improved, and system stability and reliability can be enhanced.

[0123] Furthermore, a possible implementation method of the embodiment of the present application also includes: if the length of the event queue does not exceed the length threshold of the event queue corresponding to the target module, obtaining the importance value of the target module; updating the scheduling frequency of the target module according to the importance value of the target module and the length of the event queue.

[0124] In the embodiment of the present application, the thread scheduler can adjust the scheduling frequency according to the importance and load of the module (the length of the event queue). The importance value represents a numerical value of the importance of the target module in the current application, which can be set by the user. The event processing efficiency of the block. Specifically, when the system detects that the length of the event queue of a target module does not exceed the length threshold set for the module, it means that the event processing capacity of the target module has not yet reached saturation. At this time, the electronic device will adjust its scheduling frequency based on the importance value of the target module and the length of the event queue. Modules with high importance values ​​may represent higher business priorities or more urgent processing needs. The scheduling frequency of modules with high importance values ​​can be increased to ensure that they can obtain processing resources in time. In general, the higher the importance value and the longer the length of the event queue, the higher the scheduling frequency. This frequency should not affect the global event processing process of the electronic device, that is, without affecting the event processing process of other modules, the event processing capability of the target module can be achieved.

[0125] Furthermore, a possible implementation of the embodiment of the present application also includes:

[0126] Get the historical load data of each module;

[0127] Based on the historical load data of each module and the current load data of each module, the load forecast result for the future preset period is obtained;

[0128] Determine whether it is a load peak period based on the resource prediction results. If so, set up a reserved resource pool for use in a future preset period of time.

[0129] In the embodiment of the present application, for each module, the data change trend is determined based on the historical load data of the module, and the load prediction result and future load data for a future preset period are determined based on the data change trend and the current load data;

[0130] When the future load data exceeds the preset threshold, it indicates a peak load period. Based on the size of the future load data and the reservation information, the reserved resources are determined and stored in the resource pool. The reservation information is the correspondence between multiple future load data and multiple reserved resources. Alternatively, as long as the future load data exceeds the preset threshold, the set resources can be reserved, and the size of the set resources can be customized by the user. During the peak load period, some resources are allocated from the resource pool to the module; when the load decreases, these resources are released back to the resource pool, which can improve resource utilization and flexibility.

[0131] Based on any of the above embodiments, the present application provides a specific event processing method, including:

[0132] Initialization: When the system starts, each module is loaded according to the configuration file or initialization script, and the corresponding event queue and initial thread pool are allocated to each module. The initialization process also includes the allocation of system resources and the setting of scheduling strategies.

[0133] Event reception: The system receives network requests from the client through the request receiving module, parses the request content into internal events, and puts the events into the corresponding event queue.

[0134] Event processing: The module takes events from its event queue for processing, which may include data calculation, database query, etc. After processing, the result is passed to the next module.

[0135] Response generation and sending: When all modules complete their tasks, the system will generate the final response data and send the data back to the client through the response sending module.

[0136] Furthermore, in the processing of each module, it also includes:

[0137] Load monitoring and regulation: During the event processing process, the system monitors the event queue length and resource usage of each module in real time, and starts the load regulation mechanism when necessary to ensure that the system can still run stably under high load.

[0138] Furthermore, it also includes: resource recycling and cleanup, in which, during the operation of the system, no longer used resources (such as temporary objects, expired data, etc.) are regularly recycled and the event queues of each module are cleaned up to maintain efficient operation of the system.

[0139] It can be seen that the SEDA architecture provided by the embodiment of the present application has strong scalability and adaptability, and can be flexibly adjusted according to different application scenarios. In addition, the SEDA architecture can also adapt to systems of different sizes, from stand-alone server applications to large-scale distributed systems, and can achieve efficient event processing and load management through corresponding configuration adjustments.

[0140] Through the above technical solution, this application realizes the optimization of high-concurrency server applications, which is specifically manifested as follows:

[0141] Through a phased event-driven architecture, the system can handle a large number of concurrent requests more efficiently, avoiding the performance bottleneck caused by thread competition in the traditional architecture.

[0142] Through the load regulation mechanism, the system can automatically adjust the processing strategy under high load conditions to avoid system crashes caused by overload. In the SEDA architecture, each module is connected through an event queue, which is not only used to transmit events, but also can be used as a tool for load regulation.

[0143] The introduction of event queues achieves decoupling between modules, which means that the increase in the load of one module will not directly affect the processing capabilities of other modules. Therefore, when the system faces high load, the performance bottleneck of a single module will not cause the overall system to crash.

[0144] Modular design improves code reusability and maintainability: each module handles specific tasks independently, reducing system complexity so that each module only needs to focus on its own logic without having to deal with the complexity of the entire system. This design allows developers to write, debug, and maintain code more easily, and the system can be flexibly expanded according to needs.

[0145] Since each module is relatively independent and decoupled through the event queue, when a module has a problem or needs to be upgraded, the impact on other modules is minimized. This modular design also supports code reuse, because different applications can share the same modules without having to develop every function from scratch.

[0146] In the SEDA architecture, each module runs independently in its thread pool, and thread management and scheduling are automatically completed by the system. Through this thread management method, the system can dynamically adjust the number of threads according to the load of each module, thereby optimizing resource allocation. In addition, thread isolation allows the system to avoid the common resource contention problem in traditional architecture when processing high-concurrency tasks. The direct technical effect of this thread isolation is to improve the concurrent processing capability of the system while ensuring the stable operation of each module. Even under high load, the response time and performance of the system can be kept within a reasonable range.

[0147] In summary, the present application effectively solves many problems in the prior art through a phased event-driven architecture, and provides a technical solution with superior performance, high stability and easy expansion for high-concurrency server applications. In short, through modular design and decoupling of event queues, the SEDA architecture can still maintain the stability of the system under high load conditions. Even if a module has problems or is overloaded, the operation of other modules will not be directly affected, and the system can continue to operate effectively through the load regulation mechanism; and the modularization and dynamic thread management of the SEDA architecture enable the system to easily cope with tasks of different scales, from small stand-alone applications to large distributed systems. This scalability is not only reflected in the concurrent processing capabilities of the system, but also in the convenience of development and maintenance. The reusability and independence of the modules enable the system to quickly adapt to new needs and environments.

[0148] A device provided in an embodiment of the present application is introduced below. The device described below and the method described above can be referred to each other. The device of this embodiment is set in an electronic device. The staged event-driven architecture includes multiple modules that are sequentially divided according to the application business logic, and each stage has an independent thread pool to process corresponding tasks.

[0149] refer to Figure 6 , Figure 6 6 is a structural block diagram of an apparatus 600 according to one embodiment of the present application, including:

[0150] An acquisition module 610 is used to acquire the length of the event queue of a target module in real time, where the target module is any module among the multiple modules;

[0151] A determination module 620 is used to determine whether the length of the event queue exceeds a length threshold of the event queue corresponding to the target module;

[0152] A trigger module 630, configured to trigger a load adjustment mechanism if the length of the event queue exceeds a length threshold of the event queue corresponding to the target module;

[0153] The discard module 640 is used to discard events in the event queue whose priority is lower than a preset priority threshold if the length of the event queue continues to grow and exceeds the length threshold of the event queue corresponding to the target module after adjustment using the load adjustment mechanism.

[0154] In one achievable manner, the load regulation mechanism includes at least one of the following:

[0155] Reduce the rate at which new events are received by the target module;

[0156] Increase the number of threads of the target module;

[0157] Based on the priority of the event, high priority events are handled first;

[0158] Increase scheduling frequency.

[0159] In one achievable manner, the method further includes: a priority determination module, configured to:

[0160] Determine, from the attribute information of the event field, attribute information corresponding to each of several target fields that meet the priority of the target module;

[0161] Determine the attribute quantization value and weight value corresponding to each attribute information;

[0162] A weighted calculation is performed based on the attribute quantization value and weight value corresponding to each attribute information to obtain the priority of the event.

[0163] In an achievable manner, the target module further includes a plurality of submodules responsible for different processing tasks, and the device further includes:

[0164] A task type determination module, used to determine the task type of the current event to be processed in the event queue of the target module;

[0165] The first event processing module is used to determine a target submodule from multiple submodules according to the task type of the current event to be processed, and process the current event to be processed through a thread corresponding to the target submodule.

[0166] In one achievable manner, the method further includes: a second event processing module, configured to:

[0167] If the request event sent by the previous module and received by the target module is a complex event, the request event is split to obtain multiple sub-request events; the multiple sub-request events are written into the event queue of the target module in sequence; the multiple sub-request events in the event queue are processed by the thread corresponding to the target module; after completing the processing of multiple sub-request events, the sub-processing results corresponding to the multiple sub-request events are merged to obtain the processing result, and the processing result is sent to the next module of the target module so that the next module can perform corresponding business processing.

[0168] In one achievable manner, the present invention further includes:

[0169] The configuration module is used to: load each module according to the configuration file or initialization script; allocate the corresponding event queue and initial thread pool to each module; and set the allocation rules and scheduling strategies of system resources.

[0170] In one achievable manner, the present invention further includes:

[0171] Resource reservation module, used to:

[0172] Obtain the historical load data of each module; obtain the load forecast results for the future preset time period based on the historical load data of each module and the current load data of each module; determine whether it is the load peak period based on the resource forecast results, and if so, establish a reserved resource pool for use in the future preset time period.

[0173] An electronic device is provided in an embodiment of the present application, such as Figure 7 As shown, Figure 7 The electronic device 700 shown includes: a processor 701 and a memory 703. The processor 701 and the memory 703 are connected, such as through a bus 702. Optionally, the electronic device 700 may also include a transceiver 704. It should be noted that in actual applications, the transceiver 704 is not limited to one, and the structure of the electronic device 700 does not constitute a limitation on the embodiments of the present application.

[0174] Processor 701 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 701 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0175] The bus 702 may include a path to transmit information between the above components. The bus 702 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 702 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0176] The memory 703 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0177] The memory 703 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 701. The processor 701 is used to execute the application code stored in the memory 703 to implement the contents shown in the above method embodiment.

[0178] Figure 7 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0179] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding content in the aforementioned method embodiment.

[0180] An embodiment of the present application provides a computer program product, including a computer program, which implements the corresponding contents of the aforementioned method embodiment when the computer program is executed by a processor.

[0181] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0182] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An event processing method based on a phased event-driven architecture, characterized in that: The phased event-driven architecture includes multiple sequential modules divided according to the application business logic, and each phase has an independent thread pool to process corresponding tasks; The event processing method comprises: Acquire the length of the event queue of the target module in real time, wherein the target module is any module among the multiple modules; Determine whether the length of the event queue exceeds a length threshold of the event queue corresponding to the target module; If the length of the event queue exceeds the length threshold of the event queue corresponding to the target module, a load adjustment mechanism is triggered; After adjustment using the load adjustment mechanism, if the length of the event queue continues to grow and exceeds the length threshold of the event queue corresponding to the target module, events in the event queue whose priority is lower than the preset priority threshold are discarded.

2. The event processing method based on the phased event-driven architecture according to claim 1 is characterized in that: The load regulation mechanism includes at least one of the following: reducing the rate at which new events are received by the target module; Increase the number of threads of the target module; Based on the priority of the event, high priority events are handled first; Increase scheduling frequency.

3. The event processing method based on the phased event-driven architecture according to claim 1 is characterized in that: The process of determining the priority of the event includes: Determine, from the attribute information of the event field, attribute information corresponding to each of several target fields that meet the priority of the target module; Determine the attribute quantization value and weight value corresponding to each attribute information; A weighted calculation is performed based on the attribute quantization value and weight value corresponding to each attribute information to obtain the priority of the event.

4. The event processing method based on the phased event-driven architecture according to claim 1 is characterized in that: The target module includes a plurality of submodules responsible for different processing tasks, and the method further includes: Determine the task type of the current pending event in the event queue of the target module; According to the task type of the current event to be processed, a target submodule is determined from multiple submodules, and the current event to be processed is processed through a thread corresponding to the target submodule.

5. The event processing method based on the phased event-driven architecture according to claim 1 is characterized in that: Also includes: If the request event sent by the previous module and received by the target module is a complex event, the request event is split to obtain multiple sub-request events; Writing multiple sub-request events into the event queue of the target module in sequence; Using the thread corresponding to the target module to process multiple sub-request events in the event queue; After completing the processing of multiple sub-request events, the sub-processing results corresponding to the multiple sub-request events are merged to obtain the processing result, and the processing result is sent to the next module of the target module so that the next module can perform corresponding business processing.

6. The event processing method based on the phased event-driven architecture according to claim 1 is characterized in that: Before the real-time acquisition of the length of the event queue of the target module and the number of threads currently used by the target module, the method further includes: Load each module according to the configuration file or initialization script; Assign corresponding event queue and initial thread pool to each module; Set system resource allocation rules and scheduling strategies.

7. The event processing method based on a phased event-driven architecture according to any one of claims 1 to 6, characterized in that: Also includes: Get the historical load data of each module; Based on the historical load data of each module and the current load data of each module, the load forecast result for the future preset period is obtained; Determine whether it is a load peak period based on the resource prediction results. If so, set up a reserved resource pool for use in a future preset period of time.

8. An event processing device based on a phased event-driven architecture, characterized in that: The phased event-driven architecture includes multiple sequential modules divided according to the application business logic, and each phase has an independent thread pool to process corresponding tasks; The event processing device comprises: An acquisition module, used for acquiring the length of the event queue of a target module in real time, wherein the target module is any module among the multiple modules; A determination module, used to determine whether the length of the event queue exceeds a length threshold of the event queue corresponding to the target module; A trigger module, configured to trigger a load regulation mechanism if the length of the event queue exceeds a length threshold of an event queue corresponding to a target module; The discard module is used to discard events in the event queue whose priority is lower than a preset priority threshold if the length of the event queue continues to grow and exceeds the length threshold of the event queue corresponding to the target module after adjustment using the load adjustment mechanism.

9. An electronic device, characterized in that: include: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the one or more processors, and the one or more applications are configured to: execute the steps of the event processing method based on the staged event-driven architecture according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded by the processor and executes the steps of the event processing method based on the staged event-driven architecture according to any one of claims 1 to 7.

Images