Concurrent task processing method, device and equipment based on event-driven architecture

Abstract

The invention relates to a concurrent task processing method and device based on an event-driven architecture, computer equipment, a storage medium and a program product. The method comprises the following steps: receiving concurrent service events which are fed back by an event generation service component and respectively correspond to different service operations, publishing the plurality of concurrent service events to a message queue, and respectively carrying out event classification processing on the plurality of concurrent service events in the message queue according to a trained event classification model, and obtaining an event type corresponding to each concurrent service event, and determining the event type as a service theme corresponding to the concurrent service event. And in response to subscription requests triggered by different event consumption service components for the business theme, asynchronously feeding back at least one concurrent business event associated with the business theme to the event consumption service components, and asynchronously executing task operations corresponding to the concurrent business events through the different event consumption service components to obtain task execution results. By adopting the method, the concurrent task processing efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a concurrent task processing method, apparatus, computer device, computer-readable storage medium, and computer program product based on an event-driven architecture. Background Art

[0002] With the development of computer technology and the popularization of the Internet and mobile applications, high-concurrency scenarios frequently occur in actual application processes. For example, in high-concurrency scenarios such as shopping festival activity scenarios on e-commerce platforms, hot topic dissemination scenarios on social networks, popular live broadcast scenarios on online video platforms, and high-frequency resource interaction scenarios in financial services, the server needs to process task requests from a large number of users within a short period of time.

[0003] In traditional technologies, task requests are usually processed by running an operating system and scheduling multiple processes and / or threads running on the CPU by the operating system, and users do not need to intervene too much in the switching between processes / threads to make full use of the CPU computing power.

[0004] However, in traditional embedded CPUs, resources such as available memory and CPU processing power are limited. At the same time, there are strict requirements for performance such as task processing latency. Existing operating systems are difficult to handle large-scale concurrent tasks, and are prone to performance bottlenecks or data inconsistency problems, which may lead to system crashes or service unavailability. There is still a problem of low efficiency in processing concurrent tasks. Summary of the Invention

[0005] Based on this, in view of the above technical problems, it is necessary to provide a concurrent task processing method, apparatus, computer device, computer-readable storage medium, and computer program product based on an event-driven architecture that can improve the efficiency of processing concurrent tasks.

[0006] In a first aspect, this application provides a concurrent task processing method based on an event-driven architecture, including: receiving concurrent business events respectively corresponding to different business operations fed back by an event generation service component, and publishing the multiple concurrent business events to a message queue; according to a trained event classification model, respectively performing event classification processing on the multiple concurrent business events in the message queue to obtain an event type corresponding to each concurrent business event, and determining the event type as a business theme corresponding to the concurrent business event; in response to subscription requests triggered by different event consumption service components for the business theme, asynchronously feeding back at least one concurrent business event associated with the business theme to the event consumption service component, so as to asynchronously execute task operations corresponding to the concurrent business event through different event consumption service components to obtain task execution results.

[0007] In a second aspect, the present application further provides a concurrent task processing apparatus based on an event-driven architecture, including: a concurrent service event receiving module, configured to receive the concurrent service events respectively corresponding to different service operations fed back by an event generation service component, and publish the multiple concurrent service events to a message queue; a service topic determination module, configured to perform event classification processing on the multiple concurrent service events in the message queue respectively according to a trained event classification model, obtain an event type corresponding to each concurrent service event, and determine the event type as the service topic corresponding to the concurrent service event; a task operation execution module, configured to, in response to a subscription request triggered by different event consumption service components for the service topic, asynchronously feed back at least one concurrent service event associated with the service topic to the event consumption service components, so as to asynchronously execute, through different event consumption service components, the task operations corresponding to the concurrent service events and obtain task execution results.

[0008] In a third aspect, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented: receiving the concurrent service events respectively corresponding to different service operations fed back by an event generation service component, and publishing the multiple concurrent service events to a message queue; performing event classification processing on the multiple concurrent service events in the message queue respectively according to a trained event classification model, obtaining an event type corresponding to each concurrent service event, and determining the event type as the service topic corresponding to the concurrent service event; in response to a subscription request triggered by different event consumption service components for the service topic, asynchronously feeding back at least one concurrent service event associated with the service topic to the event consumption service components, so as to asynchronously execute, through different event consumption service components, the task operations corresponding to the concurrent service events and obtain task execution results.

[0009] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: receiving the concurrent service events respectively corresponding to different service operations fed back by an event generation service component, and publishing the multiple concurrent service events to a message queue; performing event classification processing on the multiple concurrent service events in the message queue respectively according to a trained event classification model, obtaining an event type corresponding to each concurrent service event, and determining the event type as the service topic corresponding to the concurrent service event; in response to a subscription request triggered by different event consumption service components for the service topic, asynchronously feeding back at least one concurrent service event associated with the service topic to the event consumption service components, so as to asynchronously execute, through different event consumption service components, the task operations corresponding to the concurrent service events and obtain task execution results.

[0010] In a fifth aspect, the present application also provides a computer program product, including a computer program which, when executed by a processor, implements the following steps: receiving concurrent business events corresponding to different business operations respectively fed back by an event generation service component, and publishing the multiple concurrent business events into a message queue; performing event classification processing on the multiple concurrent business events in the message queue respectively according to a trained event classification model, obtaining an event type corresponding to each concurrent business event, and determining the event type as a business theme corresponding to the concurrent business event; in response to a subscription request triggered by different event consumption service components for the business theme, asynchronously feeding back at least one concurrent business event associated with the business theme to the event consumption service component, so as to asynchronously execute a task operation corresponding to the concurrent business event through different event consumption service components and obtain a task execution result.

[0011] In the above concurrent task processing method, apparatus, computer device, computer-readable storage medium and computer program product based on an event-driven architecture, by receiving concurrent business events corresponding to different business operations respectively fed back by an event generation service component and publishing the multiple concurrent business events into a message queue, the decoupling between various services can be realized by introducing the message queue as an event bus, enabling the system to be flexibly extended and maintained. Further, according to the trained event classification model, event classification processing is performed on the multiple concurrent business events in the message queue respectively, an event type corresponding to each concurrent business event is obtained, and the event type is determined as the business theme corresponding to the concurrent business event, so as to achieve clustering and classification of the multiple concurrent business events according to the degree of association. Service components can publish and subscribe to events for communication based on the business theme, avoiding the performance bottleneck caused by synchronous calls. By responding to the subscription requests triggered by different event consumption service components for the business theme, at least one concurrent business event associated with the business theme can be asynchronously fed back to the event consumption service component, and through different event consumption service components, the task operation corresponding to the concurrent business event is asynchronously executed to obtain a task execution result. Thus, the event consumption service component can subscribe to multiple event types as needed, flexibly adjust the subscription relationship, and reduce the timeout or system crash caused by the service waiting time during concurrent task processing through the asynchronous processing mechanism, improving the efficiency of concurrent task processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for describing the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0013] Figure 1 It is an application environment diagram of a concurrent task processing method based on an event-driven architecture in an embodiment;

[0014] Figure 2 It is a schematic flowchart of a concurrent task processing method based on an event-driven architecture in an embodiment;

[0015] Figure 3 It is a schematic flowchart of a process for obtaining a trained event classification model in an embodiment;

[0016] Figure 4 It is a schematic flowchart of a concurrent task processing method based on an event-driven architecture in another embodiment;

[0017] Figure 5 It is a timing diagram of a concurrent task processing method based on an event-driven architecture in an embodiment;

[0018] Figure 6 It is a structural block diagram of a concurrent task processing device based on an event-driven architecture in an embodiment;

[0019] Figure 7 It is a structural block diagram of a concurrent task processing device based on an event-driven architecture in another embodiment;

[0020] Figure 8 It is an internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0021] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0022] The concurrent task processing method based on an event-driven architecture provided by the embodiments of the present application can be applied to, for example Figure 1In the application environment shown. Among them, the event generation service component 102 and the event consumption service component 104 communicate with the system server 106 through the network. The data storage system can store the data that the system server 106 needs to process. The data storage system can be integrated on the system server 106, or can be placed on the cloud or other network servers. Among them, the event generation service component 102 and the event consumption service component 104 can be, but are not limited to, various personal computers, laptops, smartphones, tablets, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The system server 106 can be an independent physical data processing server, or a data processing server cluster composed of multiple physical data processing servers, or a cloud data processing server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Among them, the event generation service component 102 and the event consumption service component 104 can be directly or indirectly connected to the system server 106 through wired or wireless communication methods, and this is not limited in the embodiments of the present application.

[0023] Among them, the event generation service component 102, the event consumption service component 104, and the system server 106 can all be used alone to execute the concurrent task processing method based on the event-driven architecture provided in the embodiments of the present application. The event generation service component 102, the event consumption service component 104, and the system server 106 can also cooperate to execute the concurrent task processing method based on the event-driven architecture provided in the embodiments of the present application. For example, taking the event generation service component 102, the event consumption service component 104, and the system server 106 cooperating to execute the concurrent task processing method based on the event-driven architecture provided in the embodiments of the present application as an example, the system server 106 receives the concurrent business events corresponding to different business operations respectively fed back by the event generation service component 104, publishes multiple concurrent business events to the message queue, and classifies the multiple concurrent business events in the message queue respectively according to the trained event classification model to obtain the event type corresponding to each concurrent business event, and determines the event type as the business theme corresponding to the concurrent business event. Further, the system server 106 asynchronously feeds back at least one concurrent business event associated with the business theme in response to the subscription requests triggered by different event consumption service components 104 for the business theme to the event consumption service component 106, so as to asynchronously execute the task operations corresponding to the concurrent business events through different event consumption service components 106 to obtain the task execution results.

[0024] In an exemplary embodiment, as Figure 2 shown, a concurrent task processing method based on an event-driven architecture is provided. This method can be applied to a server or a terminal. Taking this method applied to Figure 1 the system server 106 therein as an example for illustration, it includes the following steps S202 to S206. Among them:

[0025] Step S202: Receive the concurrent business events corresponding to different business operations respectively fed back by the event generation service component, and publish multiple concurrent business events to the message queue.

[0026] Specifically, the system server receives the concurrent business events corresponding to different business operations respectively fed back by the event generation service component by introducing the message queue as the event bus, that is, specifically publishes multiple concurrent business events to the message queue serving as the event bus. It can be understood that by using the message queue as the event bus to receive the events sent by the event production service component and distributing the events to the event consumption service component in a timely and accurate manner according to the predefined routing rules, that is, the consumption service component can receive the events from the event bus and execute the corresponding business logic according to the content and type of the events.

[0027] Among them, message queues such as Kafka (i.e., a distributed message queue system that can be applied to fields such as log collection, event sourcing, real-time data pipelines, and stream processing), and RabbitMQ (i.e., message-oriented middleware based on the Advanced Message Queuing Protocol, supporting multiple message passing patterns, including point-to-point, publish / subscribe, request / response, etc.), play the role of an event bus in the event-driven architecture, responsible for event delivery and decoupling the dependencies between services. The event production service component is a component or service used to generate business events, usually the trigger points of some business logics. When a certain business operation occurs, corresponding events will be generated, that is, after the state changes or the operation is completed, the event production service component publishes the event to the message queue. Among them, Kafka or RabbitMQ can be specifically selected as the appropriate message queue according to requirements. Kafka is suitable for high-throughput distributed scenarios, while RabbitMQ is suitable for complex routing and queue management. Among them, the event consumption service component is a component or service in the system responsible for processing events, which can receive events from the event bus and execute corresponding business logics according to the content and type of the events.

[0028] In an exemplary embodiment, taking an e-commerce system as an example, when business operations such as a user successfully placing an order, completing payment, and shipping occur, event production service components such as the order service component, payment service component, and logistics service component will generate corresponding order creation events, payment completion events, shipping events, etc. The user service center can update the user's shopping record according to the order creation event, the inventory service component can update the inventory quantity according to the shipping event, and the data analysis service component can perform sales data analysis according to the payment completion event.

[0029] Among them, the event consumption service component can subscribe to multiple event types according to needs and can flexibly adjust the subscription relationship. That is, through the event consumption service component, the system can disperse business logics into each component to achieve modular design and loose coupling.

[0030] Step S204, according to the trained event classification model, perform event classification processing on multiple concurrent business events in the message queue respectively, obtain the event type corresponding to each concurrent business event, and determine the event type as the business theme corresponding to the concurrent business event.

[0031] Specifically, according to the feature clustering layer of the trained event classification model, perform clustering processing on multiple concurrent business events in the message queue respectively to obtain a preset number of clustering categories, and according to the decoder of the trained event classification model, perform decoding and reconstruction processing on the preset number of clustering categories respectively to obtain the operation mode corresponding to each clustering category.

[0032] Among them, the system server can use the feature clustering layer of the trained event classification model to perform clustering processing on multiple concurrent service events in the message queue respectively, to obtain a preset number of clustering categories, that is, each clustering category can include multiple concurrent service events. Among them, the greater the theme correlation consistency rate of the event types after clustering, the more similar the spatial coupling characteristics of each operation mode cluster after clustering, and the better the clustering effect. Further, the system server can use the decoder of the trained event classification model to perform decoding and reconstruction processing on the preset number of clustering categories respectively, so as to obtain the operation modes corresponding to each clustering category.

[0033] In an exemplary embodiment, the specific process of performing decoding and reconstruction processing on a preset number of clustering categories by using the decoder of the trained event classification model includes: (1) Feature extraction of input data: The original data (such as time series, images or other high-dimensional data) is first subjected to feature extraction through the encoder Encoder of the neural network. The original data is mapped to a low-dimensional feature space through the encoder to obtain the feature representation of the data. (2) Feature clustering: In the feature space, clustering algorithms (such as K-Means, GMM, etc.) are used to cluster the features to obtain J clusters. Each cluster corresponds to a potential mode or operation mode, and the cluster center is the representative point of each cluster in the feature space. (3) Decoder reconstruction: The clustering results (that is, the feature representations or cluster centers of each cluster) are input into the decoder Decoder. Through the decoder, the low-dimensional feature representation is mapped back to the original data space to generate reconstructed data with the same dimension as the input data. For each cluster, the decoder will generate a corresponding reconstructed data, and these reconstructed data represent the operation modes of this cluster. (4) Output J types of operation modes. Among them, the result of decoder reconstruction is J types of operation modes. Each type of operation mode corresponds to the reconstructed data of a cluster, and these reconstructed data can be regarded as the typical performance of the original data under a certain operation mode.

[0034] Among them, the definition of the J types of operation modes is: The J types of operation modes refer to J different operation modes obtained through clustering and decoder reconstruction. Its significance lies in: Each type of operation mode represents a potential behavior or state in the data. For example: In the power system, the J types of operation modes may correspond to different load states (such as peak, valley, stable, etc.), and in robot control, the J types of operation modes may correspond to different motion strategies (such as walking, running, jumping, etc.). The characteristics of the J types of operation modes are: Each type of operation mode is reconstructed by the decoder from the clustering results, and it is the typical performance of a certain mode in the data.

[0035] Optionally, the definition of the representative operation mode: It refers to the operation mode obtained by reconstructing the clustering center of each cluster through the decoder. Its significance lies in that the clustering center is the "center point" of each cluster in the feature space, representing the typical features of the cluster. After being reconstructed by the decoder, the clustering center is mapped back to the original data space, generating the typical operation mode of the cluster. Its characteristics: The representative operation mode is the most representative one among the J operation modes and can be used to describe the overall features of the cluster, serving as an important basis for understanding and analyzing data patterns.

[0036] Exemplarily, assume there is load data of a power system, and different load operation modes need to be obtained through clustering and decoder reconstruction. The specific process includes: (1) Feature extraction: Extract features from the original load data through the encoder to obtain a low-dimensional feature representation. (2) Feature clustering: Cluster the low-dimensional features to obtain 3 clusters (J = 3), corresponding to "peak load", "stable load", and "valley load" respectively. (3) Decoder reconstruction: Input the clustering center of each cluster into the decoder to reconstruct 3 load operation modes: peak load mode, stable load mode, and valley load mode. (4) Result analysis: J operation modes: 3 load operation modes (peak, stable, valley). Representative operation mode: The reconstructed data of each mode (such as the typical curve of peak load, the typical curve of stable load, etc.).

[0037] In an exemplary embodiment, after obtaining the operation mode corresponding to each clustering category, the system server determines, for each clustering category, the event type corresponding to the clustering category according to the operation mode, and determines the event type as the event type corresponding to the concurrent business events under the clustering category.

[0038] Specifically, since after being reconstructed by the decoder, the clustering center is mapped back to the original data space, generating the typical operation mode of the cluster, the event type corresponding to the clustering center, that is, the corresponding clustering category, can be determined according to the determined operation mode, and further, the determined event type is used as the event type corresponding to each concurrent business event included in the clustering center, that is, the clustering category.

[0039] Step S206, in response to the subscription requests triggered by different event consumption service components for the business topic, asynchronously feedback at least one concurrent business event associated with the business topic to the event consumption service components, so as to asynchronously execute the task operations corresponding to the concurrent business events through different event consumption service components and obtain the task execution results.

[0040] Specifically, the system server consumes the subscription requests triggered by the service components for business topics in response to different events, obtains the business topics corresponding to the subscription requests triggered by each event consumption service component, and obtains at least one concurrent business event associated with each business topic, so as to, through the asynchronous processing mechanism, for each event consumption service component, asynchronously feedback at least one concurrent business event associated with the business topic to the event consumption service component, thereby indicating different event consumption service components through the asynchronous processing mechanism to asynchronously execute the task operations corresponding to the concurrent business events and obtain the task execution results.

[0041] Among them, the asynchronous processing mechanism means that when the system server processes a task, it does not immediately return the result to the caller, but continues to execute the task in the background. After the task is completed, it notifies the caller or updates the system status. By decomposing the task into multiple independent steps, the asynchronous processing mechanism can improve the concurrent processing ability and response speed of the system.

[0042] Specifically, in an event-driven architecture, the production and consumption of events are usually asynchronous. Through the asynchronous processing mechanism, after sending an event, the event generation service component does not need to wait for the processing result of the event consumption service component, so it can continue to process other tasks, while the event consumption service component asynchronously consumes the event according to its own processing ability and resource situation and executes the corresponding business logic.

[0043] In an exemplary embodiment, the asynchronous processing mechanism specifically uses asynchronous programming techniques (such as CompletableFuture in Java, asyncio in Python, Promise in JavaScript) to implement the event processing logic. Among them, event-driven framework: Use an event-driven framework (such as Node.js, Netty) to manage the reception and processing of events. Thread pool management: Reasonably configure the thread pool to avoid exhaustion of thread resources and ensure that events can be processed in a timely manner.

[0044] Specifically, the architecture design includes: 1) Event generation service component: After completing the business logic, publish the event to the Kafka cluster and persist the event in the database at the same time (supporting event backtracking). B. Kafka cluster: As the event bus, it is responsible for the storage and distribution of events and supports high throughput and distributed deployment. C. Event consumption service component: Obtain and process events by subscribing to the Kafka cluster, and it uses the asynchronous processing mechanism to improve the concurrent ability. D. Event storage: Use the database to store events, support event backtracking and replay, and when the system fails, recover the events from the database and republish them to the Kafka cluster.

[0045] In an exemplary embodiment, in a video platform, after a user uploads a video, a series of processes need to be performed, such as video transcoding, generating thumbnails, storing in cloud storage, publishing to a content delivery network (CDN), etc. In the traditional synchronous processing method, after the user uploads a video, they need to wait for all the processes to complete before getting a response, which will cause the user to wait for too long and the system resources to be occupied for a long time.

[0046] In the embodiment of the present application, after the system server adopts an asynchronous processing mechanism, after the user uploads a video, the system immediately returns a response indicating successful upload, and asynchronously executes the video processing task. Among them, the video processing task is sent to the event bus and subscribed to and consumed by a dedicated video processing service. The video processing service can include multiple instances, which respectively process different tasks, such as transcoding service, thumbnail service, storage service, and CDN distribution service. These services can asynchronously process tasks in the background without the user having to wait. Exemplarily, when the user uploads a video, the upload service stores the video file in temporary storage and generates a video upload completion event. The transcoding service subscribes to this event. When it receives the event, it retrieves the video file from the temporary storage for transcoding and stores the transcoded video in cloud storage. At the same time, the thumbnail service also subscribes to this event, generates a thumbnail of the video, and stores it in cloud storage. The storage service is responsible for migrating the video file from temporary storage to long-term storage. The CDN distribution service is responsible for pushing the video file to the CDN network to improve the access speed of the video.

[0047] It can be understood that through the asynchronous processing mechanism, the video platform can efficiently process a large number of video upload requests, improve the concurrent processing ability and response speed of the system. The user can get a response immediately after uploading the video and can continue to browse other content or perform other operations without waiting for the video processing to complete. At the same time, the video processing tasks can be asynchronously executed according to the priority and resource situation, ensuring the stability and reliability of the system. With the continuous development of technology and the increasing complexity of business requirements, the event-driven architecture will be applied and innovated in more fields. For example, it can be combined with machine learning and artificial intelligence technologies to achieve event-based intelligent decision-making and automated processing, or with the help of cloud computing and containerization technologies to achieve a more flexible, efficient, and elastic event-driven architecture.

[0048] In the above concurrent task processing method based on the event-driven architecture, by receiving events, concurrent business events corresponding to different business operations are generated and fed back by the service components, and the multiple concurrent business events are published to the message queue. Thus, by introducing the message queue as the event bus, decoupling between various services can be achieved, enabling the system to be flexibly extended and maintained. Further, according to the trained event classification model, the multiple concurrent business events in the message queue are respectively subjected to event classification processing to obtain the event type corresponding to each concurrent business event, and the event type is determined as the business theme corresponding to the concurrent business event, so as to achieve clustering and classification of the multiple concurrent business events according to the degree of association. The service components can publish and subscribe to events for communication based on the business theme, avoiding the performance bottleneck caused by synchronous calls. By responding to different events to consume the subscription requests triggered by the service components for the business theme, at least one concurrent business event associated with the business theme can be asynchronously fed back to the event consumption service component, and through different event consumption service components, the task operations corresponding to the concurrent business events are asynchronously executed to obtain the task execution results. Thus, the event consumption service component can subscribe to multiple event types as needed, flexibly adjust the subscription relationship, and through the asynchronous processing mechanism, reduce the timeout or system crash situation caused by the service waiting time during concurrent task processing, improving the efficiency of concurrent task processing.

[0049] In an exemplary embodiment, as Figure 3 shown, the steps of obtaining the trained event classification model include the following steps S302 to S308. Among them:

[0050] Step S302, collect operation mode samples for the initial neural network model, and pre-train the initial neural network model according to the operation mode samples to obtain a pre-trained neural network model.

[0051] Among them, the system server can specifically classify events into different types by introducing the CSAC algorithm (fully named Constrained Soft Actor-Critic, which is a reinforcement learning algorithm based on maximum entropy and can achieve more efficient and stable policy optimization in reinforcement learning tasks and is used to handle reinforcement learning problems with constraints) and the neural network model, and assign a theme to each type of event.

[0052] Specifically, the system server collects operation mode samples for the initial neural network model to pre-train the initial neural network model according to the collected multiple operation mode samples to obtain a pre-trained neural network model.

[0053] Among them, the initial neural network model can specifically be: 1) Natural language processing models, including: BERT (Bidirectional Encoder Representations from Transformers): A bidirectional Transformer encoder, pre-trained to capture context information, suitable for tasks such as question answering, sentiment analysis, and text classification; GPT (Generative Pre-trained Transformer): A pre-trained model based on Transformer, used for text generation and also applicable to tasks such as text classification. 2) Image processing models, including: Convolutional Neural Network (CNN): Extracts image features through convolutional layers, pooling layers, and fully connected layers, widely used in image classification; ResNet (Residual Network): Solves the problem of vanishing gradients in the training of deep neural networks through residual learning, widely used in image classification tasks; 3) General classification task models, including: Multi-Layer Perceptron (MLP): Consists of an input layer, one or more hidden layers, and an output layer, suitable for handling simple classification tasks; Inception: Introduces the Inception module, extracts multi-scale features through convolutional kernels of different sizes, and improves network performance and computational efficiency.

[0054] Exemplarily, a sample of the running mode can be understood as sample data representing a potential behavior or state in the data. For example, in a power system, the J type of running mode may correspond to different load states (such as peak, valley, stable, etc.), while in robot control, the J type of running mode may correspond to different motion strategies (such as walking, running, jumping, etc.).

[0055] Step S304: Obtain each original concurrent service event to be processed, and perform feature encoding on each original concurrent service event through the encoding layer of the pre-trained neural network model to obtain deep features corresponding to the original concurrent service events.

[0056] Specifically, the system server obtains each original concurrent service event to be processed and, through the encoding layer of the pre-trained neural network model, achieves the purpose of obtaining deep features of multi-dimensional relevant variables, that is, by performing feature encoding on each original concurrent service event, deep features corresponding to the original concurrent service events can be obtained.

[0057] Among them, in machine learning and deep learning, the deep feature extraction of multi-dimensional correlated variables is an important part of feature engineering. Specifically, the deep features of multi-dimensional correlated variables can be extracted through the following methods: 1. Feature extraction in deep learning: Through deep learning models such as convolutional neural networks (CNNs) and Transformers, deep features are automatically extracted from multi-dimensional data, that is, through multi-layer non-linear transformations, complex patterns and relationships in the data are captured. Among them, the convolutional neural network extracts features step by step through convolutional layers, pooling layers, and fully connected layers, and is suitable for image and sequence data, while the Transformer, based on the self-attention mechanism, can process long sequence data and learn deep semantic features through pre-training. 2. Multi-level feature extraction strategies, which involve extracting features from the original data in stages, using different techniques and algorithms in each stage to enhance the representation ability of the model, including: 1) Multi-level feature extraction of CNN: Use convolutional layers to extract low-level features, then reduce the spatial dimension of the features through pooling layers, and finally extract more abstract features through fully connected layers.

[0058] Step S306: According to the feature clustering layer of the pre-trained neural network model, cluster each original concurrent business event to obtain a preset number of clustering categories, and each clustering category corresponds to at least one original concurrent business event.

[0059] Specifically, the system server can divide each original concurrent business event into a preset number of clusters through the feature clustering layer of the pre-trained neural network model. For each cluster, obtain the corresponding clustering center, and determine the clustering center as a clustering category, and each clustering category corresponds to at least one original concurrent business event.

[0060] Step S308: Determine the model training loss value corresponding to the pre-trained neural network model according to the deep features and clustering categories, and adjust the parameters of the pre-trained neural network model according to the model training loss value to obtain a trained event classification model.

[0061] Among them, the model training loss value of the pre-trained neural network model specifically includes the feature clustering layer loss value corresponding to the feature clustering layer and the encoding reconstruction loss value corresponding to the encoding layer. When calculating the model training loss value, it is necessary to calculate the feature clustering layer loss value corresponding to the feature clustering layer and the encoding reconstruction loss value corresponding to the encoding layer, and then perform weighted summation based on the feature clustering layer loss value and the encoding reconstruction loss value to obtain the specific model training loss value.

[0062] Specifically, for each deep feature, the system server determines the original concurrent business events corresponding to the deep feature, obtains the original business categories corresponding to the original concurrent business events, and for each original concurrent business event, compares the original business category with the clustering category determined according to the pre-trained neural network model to obtain a comparison result. The comparison result includes that the original business category is the same as the clustering category determined according to the pre-trained neural network model, and that the original business category is different from the clustering category determined according to the pre-trained neural network model. When the original business category is the same as the clustering category, it is represented by "1", while when the original business category is different from the clustering category, it is represented by "0".

[0063] Furthermore, the system server also needs to perform normalization processing on each original concurrent business event to obtain original normalized data, so that the model training loss value can be calculated based on the deep feature, clustering category, comparison result, and original normalized data to obtain the model training loss value corresponding to the pre-trained neural network model. Among them, the system server can determine the feature clustering layer loss value corresponding to the feature clustering layer according to the deep feature, clustering category, and comparison result, and the encoding reconstruction loss value corresponding to the encoding layer can be determined according to the original normalized data and the deep feature.

[0064] In an exemplary embodiment, calculating the model training loss value according to the deep feature, clustering category, comparison result, and original normalized data to obtain the model training loss value corresponding to the pre-trained neural network model includes:

[0065] Obtain the number of features corresponding to the deep feature, and obtain the number of categories corresponding to the clustering category; calculate the model training loss value according to the comparison result, deep feature, number of features, clustering category, and number of categories to obtain the feature clustering layer loss value corresponding to the feature clustering layer; calculate the model training loss value according to the original normalized data and the number of features to obtain the encoding reconstruction loss value corresponding to the encoding layer; obtain the preset loss value weight corresponding to the feature clustering layer loss value, and perform weighted summation according to the preset loss value weight, feature clustering layer loss value, and encoding reconstruction loss value to obtain the model training loss value corresponding to the pre-trained neural network model.

[0066] Specifically, for the feature clustering layer loss value corresponding to the feature clustering layer, the system server calculates the model training loss value by obtaining the number of features corresponding to the deep features and the number of categories corresponding to the clustering categories, and based on the comparison result obtained by comparing the original business categories with the clustering categories determined according to the pre-trained neural network model, the deep features, the number of features corresponding to the deep features, the clustering categories, and the number of categories corresponding to the clustering categories, to obtain the feature clustering layer loss value corresponding to the feature clustering layer.

[0067] Exemplarily, through the following formula (1), the feature clustering layer loss value K corresponding to the feature clustering layer is calculated lose :

[0068] ; Formula (1)

[0069] Where K lose represents the feature clustering layer loss value, defined as the sum of the squares of the Euclidean distances between the samples within the cluster and the cluster center, and Q j is the comparison result, used to indicate whether the original business category is consistent with the clustering category determined according to the pre-trained neural network model. Among them, when the original business category is consistent with the clustering category, Q j is represented by "1", while when the original business category is inconsistent with the clustering category, Q j is represented by "0", P0 represents the deep features of the multi-dimensional correlation variables obtained through the encoding layer of the pre-trained neural network model, and M represents the number of one-dimensional data, that is, the deep features of the multi-dimensional correlation variables extracted from M operation mode samples.

[0070] Similarly, for the encoding reconstruction loss value corresponding to the encoding layer, the system server normalizes each original concurrent business event to obtain the original normalized data, and calculates the model training loss value by obtaining the number of features corresponding to the deep features, based on the original normalized data and the number of features, to obtain the encoding reconstruction loss value corresponding to the encoding layer.

[0071] Exemplarily, through the following formula (2), the encoding reconstruction loss value L corresponding to the encoding layer is calculated lose :

[0072] ; Formula (2)

[0073] Where L lose represents the encoding reconstruction loss value, M represents the number of rows of the operation mode samples after the input data is preprocessed, and P M represents the normalized representation of the input original data, that is, the original normalized data.

[0074] Further, after determining the feature clustering layer loss value and the encoding reconstruction loss value, a preset loss value weight corresponding to the feature clustering layer loss value is obtained, and a weighted sum is performed according to the preset loss value weight, the feature clustering layer loss value, and the encoding reconstruction loss value to obtain a model training loss value corresponding to the pre-trained neural network model.

[0075] Exemplarily, the model training loss value L corresponding to the pre-trained neural network model is calculated by the following formula (3):

[0076] ; Formula (3)

[0077] where L represents the model training loss value, L lose represents the encoding reconstruction loss value, and K lose represents the feature clustering layer loss value. represents the preset loss value weight corresponding to the feature clustering layer loss value, which can be adjusted and set according to the actual situation and application scenario, and is not limited to specific values, and is used to prevent excessive distortion during the training of the clustering layer.

[0078] In this embodiment, by collecting operation mode samples for the initial neural network model, pre-training the initial neural network model according to the operation mode samples to obtain a pre-trained neural network model, and obtaining each original concurrent service event to be processed, the encoding layer of the pre-trained neural network model is used to perform feature encoding on each original concurrent service event to obtain deep features corresponding to the original concurrent service events, and according to the feature clustering layer of the pre-trained neural network model, clustering processing is performed on each original concurrent service event to obtain a preset number of clustering categories. Thus, the model training loss value corresponding to the pre-trained neural network model can be determined according to the deep features and the clustering categories, and the parameters of the pre-trained neural network model are adjusted according to the model training loss value to obtain a trained event classification model, realizing the training and parameter adjustment of different levels of the neural network model until a trained event classification model that meets the actual requirements is obtained. And by using the neural network, the expression ability of the reinforcement learning algorithm is improved, enabling it to process high-dimensional state and action spaces, and at the same time, the exploration efficiency is improved through the maximum entropy framework, further improving the model accuracy to improve the accuracy of the event type obtained by using the event classification model for event classification processing.

[0079] In an exemplary embodiment, as Figure 4 shown, a concurrent task processing method based on an event-driven architecture is provided. Taking the method applied to the Figure 1 system server 106 as an example for illustration, it includes the following steps S402 to step S410. Among them:

[0080] Step S402: Receive the concurrent business events corresponding to different business operations fed back by the event generation service component, and publish the multiple concurrent business events to the message queue.

[0081] Specifically, the system server receives the concurrent business events corresponding to different business operations fed back by the event generation service component by introducing a message queue as an event bus, that is, specifically publishes the multiple concurrent business events to the message queue acting as the event bus. Among them, message queues such as Kafka (i.e., a distributed message queue system, which can be applied to fields such as log collection, event source, real-time data pipeline, and stream processing), and RabbitMQ (i.e., a message-oriented middleware, implemented based on the Advanced Message Queuing Protocol, supporting multiple message passing modes, including point-to-point, publish / subscribe, request / response, etc.) play the role of an event bus in the event-driven architecture, responsible for event transmission and decoupling the dependencies between services.

[0082] Step S404: Determine the data storage method corresponding to the concurrent business events, and obtain the data storage component corresponding to the data storage method.

[0083] Specifically, the system server determines the data storage method corresponding to the concurrent business events, such as specifically relational database storage, or distributed file system storage, or message queue storage, or event storage system storage, etc., and further obtains the data storage component corresponding to the data storage method, such as obtaining the relational database corresponding to relational database storage, or obtaining the distributed file system corresponding to distributed file system storage, or obtaining the message queue corresponding to message queue storage, or obtaining the event storage system corresponding to event storage system storage, as the specific data storage component.

[0084] Among them, the implementation method of event storage: By selecting a relational database (such as MySQL), a NoSQL database (such as Cassandra), or the persistent storage of a message queue (such as the persistent storage of Kafka). Among them, relational databases are suitable for storing structured event data and support complex queries and transaction processing; NoSQL databases are suitable for storing unstructured or semi-structured event data and have high scalability and high write performance.

[0085] Step S406: Store the multiple concurrent business events in the data storage component, and upload the data storage component storing the multiple concurrent business events to the cloud server.

[0086] Specifically, after determining the time storage component corresponding to the concurrent business events, by storing the multiple concurrent business events in the data storage component, and packaging and uploading the multiple concurrent business events and the data storage component to the cloud server, the business events are effectively saved.

[0087] Step S408, if a system failure or data loss is detected, trigger an event backtracking request for the data storage component to obtain, based on the cloud server, target concurrent business events associated with the event backtracking request from the stored data storage component.

[0088] Among them, event backtracking refers to the process of re-accessing and processing historical events. When a system failure or data loss occurs, historical events can be re-processed through event backtracking to restore the system state. For example, when data errors or inconsistencies occur, relevant events can be re-processed through event backtracking to repair the data. And in some business scenarios, historical events need to be re-processed to meet new business requirements or correct past mistakes.

[0089] Specifically, when a system failure or data loss is detected, the system server triggers an event backtracking request for the data storage component to obtain, based on the cloud server, target concurrent business events associated with the event backtracking request from the stored data storage component, so that historical events can be re-processed through event backtracking to restore the system state.

[0090] Step S410, republish the target concurrent business events to the message queue to backtrack and replay the target concurrent business events.

[0091] Specifically, after obtaining the target concurrent business events that need to be backtracked and replayed, by republishing the target concurrent business events to the message queue, the purpose of backtracking and replaying the target concurrent business events can be achieved.

[0092] Exemplarily, in a financial trading system, if it is found that there was a previous calculation error, relevant trading events can be re-processed through event backtracking to adjust the account balance. In a financial trading system, the accuracy and consistency of trading data are crucial. That is, through the event storage and backtracking mechanism, the reliability, consistency, and traceability of the system can be ensured.

[0093] Optionally, when a transaction occurs, the transaction system generates a transaction event and stores it in the event store. The transaction event contains information such as the transaction time, transaction amount, and the accounts of both parties involved in the transaction. At the same time, these events are distributed to relevant services through the event bus, such as the account service component, clearing service component, risk control service component, etc. If the system fails or data is lost, the account balance and transaction records of the system can be reconstructed using the historical transaction events in the event store. For example, the account service can re-consume all the transaction events and update the account balance and transaction details based on the type and amount of the events. In this way, even if the system fails, data loss and inconsistency will not occur. In addition, the transaction events in the event store can be used as evidence to prove the legality and accuracy of the transactions. For example, regulatory authorities can require banks to provide transaction records for a specific period, and these data can be quickly queried and exported through the event store.

[0094] In the above concurrent task processing method based on the event-driven architecture, concurrent business events corresponding to different business operations are generated by receiving events from the service component and published to the message queue. Determine the data storage method corresponding to the concurrent business events and obtain the data storage component corresponding to the data storage method; store multiple concurrent business events in the data storage component, and upload the data storage component storing multiple concurrent business events to the cloud server; if a system failure or data loss is detected, trigger an event backtracking request for the data storage component to obtain, based on the cloud server, the target concurrent business event associated with the event backtracking request from the stored data storage component; republish the target concurrent business event to the message queue to backtrack and replay the target concurrent business event.

[0095] In an exemplary embodiment, as Figure 5 shown, a concurrent task processing method based on the event-driven architecture is provided. Referring to Figure 5 it can be seen that the concurrent task processing process involves components such as the system server, event generation service component, event consumption service component, and cloud server. The method specifically includes the following steps:

[0096] Step S501, the event generation service component feeds back concurrent business events corresponding to different business operations to the system server.

[0097] Step S502, the system server receives multiple concurrent business events and publishes the multiple concurrent business events to the message queue.

[0098] Step S503, the system server collects operation mode samples for the initial neural network model, and pre-trains the initial neural network model according to the operation mode samples to obtain a pre-trained neural network model.

[0099] Step S504: The system server obtains each original concurrent business event to be processed, and through the encoding layer of the pre-trained neural network model, performs feature encoding on each original concurrent business event to obtain deep features corresponding to the original concurrent business events.

[0100] Step S505: The system server performs clustering processing on each original concurrent business event according to the feature clustering layer of the pre-trained neural network model to obtain a preset number of clustering categories.

[0101] Step S506: For each deep feature, the system server determines the original concurrent business event corresponding to the deep feature, obtains the original business category corresponding to the original concurrent business event, and for each original concurrent business event, compares the original business category with the clustering category determined according to the pre-trained neural network model to obtain a comparison result.

[0102] Step S507: The system server obtains the number of features corresponding to the deep feature, obtains the number of categories corresponding to the clustering category, and calculates the model training loss value according to the comparison result, deep feature, number of features, clustering category, and number of categories to obtain the feature clustering layer loss value corresponding to the feature clustering layer.

[0103] Step S508: The system server performs normalization processing on each original concurrent business event to obtain original normalized data, and calculates the model training loss value according to the original normalized data and the number of features to obtain the encoding reconstruction loss value corresponding to the encoding layer.

[0104] Step S509: The system server obtains the preset loss value weight corresponding to the feature clustering layer loss value, performs weighted summation according to the preset loss value weight, feature clustering layer loss value, and encoding reconstruction loss value to obtain the model training loss value corresponding to the pre-trained neural network model.

[0105] Step S510: The system server adjusts the parameters of the pre-trained neural network model according to the model training loss value to obtain a trained event classification model, and performs clustering processing on multiple concurrent business events in the message queue respectively according to the feature clustering layer of the trained event classification model to obtain a preset number of clustering categories.

[0106] Step S511: The system server, according to the decoder of the trained event classification model, performs decoding and reconstruction processing on a preset number of clustering categories respectively to obtain the operation modes corresponding to each clustering category. For each clustering category, determine the event type corresponding to the clustering category according to the operation mode, and determine the event type as the event type corresponding to the concurrent service events under the clustering category, so as to determine the event type as the business theme corresponding to the concurrent service events.

[0107] Step S512: The event consumption service component triggers a subscription request for the business theme and feeds back the subscription request to the system server.

[0108] Step S513: In response to the subscription requests triggered by different event consumption service components for the business theme, the system server obtains the business theme corresponding to each subscription request triggered by the event consumption service component and at least one concurrent service event associated with each business theme, and through an asynchronous processing mechanism, asynchronously feeds back at least one concurrent service event associated with the business theme to the event consumption service component.

[0109] Step S514: The event consumption service component receives at least one concurrent service event associated with the business theme fed back by the system server and asynchronously executes the task operations corresponding to the concurrent service events to obtain the task execution results.

[0110] After executing step S501, execute step S515. The system server determines the data storage method corresponding to the concurrent service events, obtains the data storage component corresponding to the data storage method, stores multiple concurrent service events in the data storage component, and uploads the data storage component storing multiple concurrent service events to the cloud server.

[0111] Step S516: If the system server detects a system failure or data loss, trigger an event backtracking request for the data storage component, so as to obtain the target concurrent service event associated with the event backtracking request from the stored data storage component based on the cloud server, and republish the target concurrent service event to the message queue for backtracking and replay of the target concurrent service event.

[0112] In the above concurrent task processing method based on the event-driven architecture, by receiving events, concurrent business events corresponding to different business operations are generated and fed back by the service component, and the multiple concurrent business events are published to the message queue. Thus, by introducing the message queue as the event bus, decoupling between various services can be achieved, enabling the system to be flexibly extended and maintained. Further, according to the trained event classification model, the multiple concurrent business events in the message queue are respectively subjected to event classification processing to obtain the event type corresponding to each concurrent business event, and the event type is determined as the business theme corresponding to the concurrent business event, so as to achieve clustering and classification of multiple concurrent business events according to the degree of association. The service component can publish and subscribe to events based on the business theme for communication, avoiding the performance bottleneck caused by synchronous calls. By responding to different events to consume the subscription requests triggered by the service component for the business theme, at least one concurrent business event associated with the business theme can be asynchronously fed back to the event consumption service component, and the task operation corresponding to the concurrent business event can be asynchronously executed by different event consumption service components to obtain the task execution result. Thus, the event consumption service component can subscribe to multiple event types as needed, flexibly adjust the subscription relationship, and through the asynchronous processing mechanism, reduce the timeout or system crash caused by the service waiting time during concurrent task processing, improving the efficiency of concurrent task processing.

[0113] In an exemplary embodiment, a concurrent task processing method based on the event-driven architecture is provided, which specifically includes the following steps:

[0114] S1. Introduce a message queue as the event bus: Among them, the message queue such as Kafka and RabbitMQ, etc., plays the role of "event bus" in the event-driven architecture, responsible for event transmission and decoupling the dependency relationship between services. The event production service component is the component or service in the system responsible for generating events, usually the trigger point of some business logics. When a certain business operation occurs, the corresponding event will be generated, that is, after the state changes or the operation is completed, the event production service component publishes the event to the message queue, and Kafka or RabbitMQ can be selected as the appropriate message queue according to the requirements. Among them, Kafka is suitable for high-throughput distributed scenarios, while RabbitMQ is suitable for complex routing and queue management. By using the message queue as the event bus, the event sent by the event production service component is received, and the event can be timely and accurately distributed to the event consumption service component according to the predefined routing rules. Among them, the event consumption service component is the component or service in the system responsible for processing events, which can receive events from the event bus and execute the corresponding business logic according to the content and type of the events.

[0115] S2. Define event topics according to business scenarios, publish events, and subscribe to events: Among them, each microservice component (including event production service components, event consumption service components, etc.) communicates through publishing and subscribing to events, which can avoid the performance bottleneck caused by synchronous calls. Among them, event publishing and subscribing are the core mechanisms of the event-driven architecture. The event production service component notifies other services by publishing events, and other event consumption service components respond by subscribing to events.

[0116] Specifically, event topics need to be defined according to business scenarios (such as order_created, user_registered). The event production service component, as well as through event consumption service components, etc., can publish and subscribe to events by topic. Among them, in order to facilitate event management and consumption, events are usually divided into different types, and a topic is specified for each type of event. For example, in an e-commerce system, order-related events can be divided into topics such as "order creation", "order payment", "order shipment", etc., and user-related information can be divided into topics such as "user registration", "user login", "user information update", etc.

[0117] In an exemplary embodiment, by introducing the CSAC algorithm (i.e., the reinforcement learning algorithm based on maximum entropy) and a neural network model, events are divided into different types, and a topic is specified for each type of event. The steps of the CSAC algorithm include: 1. Initialize the deep feature space, pre-train the neural network model to obtain P_0; 2. Initialize the cluster centers to obtain C_j; 3. Train the CSAC model. According to the deep feature space P_0 and the cluster centers C_j, calculate the loss function L and then perform backpropagation to continuously update the neural network model and the parameters in the neural network model. Specifically, the Rand coefficient can be used to calculate the similarity of the training results. If the Rand coefficient is greater than 0.99, it is considered that the two clustering results are consistent. If the clustering results are consistent for five consecutive times, it is considered that the model converges. At this time, the clustering result is the final clustering result and the training stops; 4. Reconstruct the operation mode samples. The clustering results in the feature clustering layer are decoded and reconstructed through the decoder of the neural network model to obtain J types of patterns of the operation mode, and the cluster centers are representative operation modes; 5. After inputting the event into the trained neural network model, obtain the event classification and use this classification as the topic of this type of event.

[0118] In an exemplary embodiment, after the event production service component completes the business logic, it publishes the event to the message queue, while the event consumption service component obtains and processes the event by subscribing to the topic. Among them, event subscription refers to the process in which the event consumption service component subscribes to the interested topic from the event bus and receives and processes the relevant events. It can be understood that through the event consumption service component group mechanism of the message queue, the load balancing processing of events is achieved.

[0119] Specifically, when implementing event subscription, the following aspects need to be considered: 1) Subscription management: The event consumption service component needs to be able to flexibly subscribe to and unsubscribe from event topics, and the dynamic management of the subscription relationship can be achieved through methods such as configuration files, management interfaces, or programming interfaces. For example, through a management interface, the system administrator can add or delete the subscribed topics for different event consumption service components. 2) Event filtering: In some cases, the event consumption service component may only need to process a part of the events in the topic. In order to improve the efficiency of event processing, an event filtering mechanism can be provided, that is, the event consumption service component can filter events according to the type of event, data content, or other conditions, and only process the parts it is interested in. 3) Event sorting: In some business scenarios, the order of events may be very important. For example, in a financial trading system, the processing order of trading events will affect the final account balance. At this time, it is necessary to ensure that events are consumed in the correct order, and the ordered processing of events can be achieved through the ordered guarantee mechanism of the message queue, such as the single-partition linearity of Kafka, or through methods such as event timestamps. 4) Event processing logic: The event consumption service component needs to execute the corresponding business logic according to the received events. When implementing the event processing logic, the code should be kept clear and maintainable as much as possible, and at the same time, exception handling and fault tolerance mechanisms should be considered.

[0120] In an exemplary embodiment, in the e-commerce system according to the embodiments of the present application, after a user submits an order, a series of business operations need to be completed, such as inventory reduction, payment processing, user points update, order notification, etc. In the traditional synchronous call method, the order service needs to sequentially call the WeChat service, payment service, inventory service, etc., and wait for the response of each service before proceeding to the next operation. That is, in the traditional method, in the case of high concurrency, it is easy to cause the response time of the order service to be too long, and even timeout or system crash problems may occur. Through the event publishing and subscribing mode, after the order service component creates an order, it publishes the order creation event to the event bus. Other relevant service components, such as inventory service components, payment service components, user service components, etc., subscribe to the "order creation" topic respectively. When receiving the order creation event, these service components asynchronously process their respective business logics, such as inventory reduction, payment processing, user points update, etc. At the same time, the order service component does not need to wait for the processing results of other services and can directly return a response to the user, improving the response speed and concurrent processing ability of the system.

[0121] In addition, if a service component fails during event processing, it will not affect the normal operation of other service components. The system server can ensure that all events are finally processed correctly through event retransmission or retry mechanisms, improving the reliability and stability of the system. For example, if the payment service component fails during the processing of the order payment event, the order service component can continue to process the subsequent business logic, and the payment service component can reprocess the event after the failure is recovered, ensuring both the availability of the system and the consistency and integrity of business data.

[0122] S3. Store the business event in a storage component and upload the storage component storing the business event to a cloud server: Among them, event storage is used to support the backtracking and replay of business events to ensure the reliability and consistency of the system. Event storage refers to persistently storing events in a dedicated storage system for querying, analyzing, and backtracking when needed. Among them, the storage component can be a relational database, a distributed file system, a message queue, or a dedicated event storage system. Uploading it to the cloud server can enable the business event to be saved.

[0123] In an exemplary embodiment, when implementing database storage, the following aspects need to be considered: 1) Data table design: Design a reasonable table structure for event data, usually including fields such as event ID, event type, event time, and event data. According to business requirements, operations such as partitioning and sharding the data table can be performed to improve query performance and data management efficiency. 2) Data synchronization: Ensure that event data can be synchronized to the database in a timely manner. The event data can be inserted into the database through the event consumption service component of the message queue, or the consistency of the data can be ensured through the transaction mechanism of the database. 3) Data query and analysis: Provide efficient query and analysis interfaces so that users and data analysts can conveniently access and analyze event data. SQL query language or professional data analysis tools can be used for query and analysis. 4) Event persistence: While the event is published to the message queue, the event is persisted to the storage system. The message queue itself usually has a persistent storage function and can be used as a way of event storage. For example, the message persistence of Kafka is stored on disk and supports the indefinite retention or retention by time period of messages. By configuring the storage policy of the message queue, long-term storage and fast query of events can be achieved. Among them, in order to better meet the requirements of event storage, a dedicated event storage system can be adopted to specifically implement the storage, processing, and analysis of event data, providing high throughput, high scalability, and powerful data processing capabilities. 5) Event backtracking and replay: By reading the events in the storage and republishing them to the message queue, event backtracking and replay are realized. Among them, event backtracking refers to the process of re-accessing and processing historical events. When the system fails or data is lost, historical events can be reprocessed through event backtracking to restore the state of the system. For example, when data errors or inconsistencies occur, relevant events can be reprocessed through event backtracking to repair the data. And in some business scenarios, historical events need to be reprocessed to meet new business requirements or correct past mistakes.

[0124] S4. Use the asynchronous processing mechanism to enhance the concurrent processing ability of the system and reduce the response time of requests: Among them, the asynchronous processing mechanism specifically uses asynchronous programming technology to implement the event processing logic. Among them, the event-driven framework: Use an event-driven framework (such as Node.js, Netty) to manage the reception and processing of events. Thread pool management: Reasonably configure the thread pool to avoid exhaustion of thread resources and ensure that events can be processed in a timely manner. 2) Architecture design, including: A. Event publisher: After completing the business logic, publish the event to the Kafka cluster and persist the event to the database at the same time (support event backtracking). B. Kafka cluster: As the event bus, it is responsible for the storage and distribution of events. Supports high throughput and distributed deployment. C. Event subscriber: Obtain events through subscribing to the Kafka cluster and process them. Use the asynchronous processing mechanism to enhance the concurrency ability. D. Event storage: Use the database to store events, support event backtracking and replay. In case of system failure, recover events from the database and republish them to Kafka.

[0125] In an exemplary embodiment, the specific implementation of the architecture design is as follows:

[0126] 1) Event publishing: public void publishEvent(String topic, Event event) {

[0127] kafkaProducer.send(new ProducerRecord<>(topic, event)); / / Event publishing

[0128] eventRepository.save(event); / / Persist the event}

[0129] 2) Event subscription and processing: @KafkaListener(topics = "order_created") public void handleOrderCreatedEvent(Event event) {

[0130] CompletableFuture.runAsync(() -> { / / Asynchronously process the event

[0131] processOrder(event);

[0132] });}

[0133] 3) Event backtracking and replay: public void replayEvents() {

[0134] List <event>events = eventRepository.findAll(); / / Event search

[0135] events.forEach(event -> kafkaProducer.send(new ProducerRecord<>(event.getTopic(), event)));} / / Event replay

[0136] In the above concurrent task processing method based on the event-driven architecture, by introducing a message queue, event publishing and subscribing, event storage and tracing, and an asynchronous processing mechanism, efficient, reliable, and scalable communication and processing in high-concurrency scenarios can be achieved. Among them, using the message queue as the event bus decouples the dependencies between services, improves the flexibility and maintainability of the system, while event publishing and subscribing avoid the performance bottleneck of synchronous calls, achieving a loosely coupled communication method. At the same time, through event storage and tracing, the reliability and consistency of the system can be ensured, supporting data auditing, recovery, and analysis. Utilizing the asynchronous processing mechanism enhances the concurrent processing ability of the system, reduces the response time of requests, and improves the user experience.

[0137] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0138] Based on the same inventive concept, the embodiments of the present application also provide a concurrent task processing device based on the event-driven architecture for implementing the above-mentioned concurrent task processing method based on the event-driven architecture. The implementation solutions provided by this device to solve problems are similar to the implementation solutions described in the above method. Therefore, the specific limitations in one or more embodiments of the following concurrent task processing devices based on the event-driven architecture can refer to the limitations on the concurrent task processing method based on the event-driven architecture in the above text, and will not be repeated here.

[0139] In an exemplary embodiment, as Figure 6 As shown in the figure, a concurrent task processing device based on an event-driven architecture is provided, including: a concurrent service event receiving module 602, a service topic determination module 604, and a task operation execution module 606, where: the concurrent service event receiving module 602 is configured to receive the concurrent service events corresponding to different service operations respectively fed back by an event generation service component, and publish a plurality of concurrent service events to a message queue; the service topic determination module 604 is configured to perform event classification processing on the plurality of concurrent service events in the message queue respectively according to a trained event classification model, obtain the event type corresponding to each concurrent service event, and determine the event type as the service topic corresponding to the concurrent service event; the task operation execution module 606 is configured to respond to the subscription requests triggered by different event consumption service components for the service topic, and asynchronously feed back at least one concurrent service event associated with the service topic to the event consumption service component, so as to asynchronously execute the task operations corresponding to the concurrent service events through different event consumption service components, and obtain task execution results.

[0140] In the above-mentioned concurrent task processing device based on an event-driven architecture, by receiving the concurrent service events corresponding to different service operations respectively fed back by an event generation service component and publishing a plurality of concurrent service events to a message queue, the decoupling between various services can be realized by introducing the message queue as an event bus, enabling the system to be flexibly extended and maintained. Further, according to the trained event classification model, event classification processing is performed on the plurality of concurrent service events in the message queue respectively, the event type corresponding to each concurrent service event is obtained, and the event type is determined as the service topic corresponding to the concurrent service event, so as to achieve clustering and classification of a plurality of concurrent service events according to the degree of association. Service components can publish and subscribe to events for communication based on the service topic, avoiding the performance bottleneck caused by synchronous calls. By responding to the subscription requests triggered by different event consumption service components for the service topic, at least one concurrent service event associated with the service topic can be asynchronously fed back to the event consumption service component, and through different event consumption service components, the task operations corresponding to the concurrent service events are asynchronously executed to obtain task execution results. Thus, the event consumption service component can subscribe to multiple event types as needed, flexibly adjust the subscription relationship, and through the asynchronous processing mechanism, reduce the timeout or system crash caused by the service waiting time during concurrent task processing, and improve the efficiency of concurrent task processing.

[0141] In an exemplary embodiment, the business theme determination module is further configured to: according to the feature clustering layer of the trained event classification model, perform clustering processing on multiple concurrent business events in the message queue respectively to obtain a preset number of clustering categories; according to the decoder of the trained event classification model, perform decoding and reconstruction processing on the preset number of clustering categories respectively to obtain the operation modes corresponding to each clustering category; for each clustering category, determine the event type corresponding to the clustering category according to the operation mode, and determine the event type as the event type corresponding to the concurrent business events under the clustering category.

[0142] In an exemplary embodiment, the task operation execution module is further configured to: in response to subscription requests triggered by different event consumption service components for business themes, obtain the business themes corresponding to the subscription requests triggered by each event consumption service component, and obtain at least one concurrent business event associated with each business theme; through an asynchronous processing mechanism, for each event consumption service component, asynchronously feedback at least one concurrent business event associated with the business theme to the event consumption service component, so as to instruct different event consumption service components through the asynchronous processing mechanism to asynchronously execute the task operations corresponding to the concurrent business events and obtain task execution results.

[0143] In an exemplary embodiment, there is provided a concurrent task processing device based on an event-driven architecture, further including a concurrent business event backtracking module, configured to: determine the data storage mode corresponding to the concurrent business event, and obtain the data storage component corresponding to the data storage mode; store multiple concurrent business events in the data storage component, and upload the data storage component storing multiple concurrent business events to the cloud server; if a system failure or data loss is detected, trigger an event backtracking request for the data storage component, so as to obtain, based on the cloud server, the target concurrent business event associated with the event backtracking request from the stored data storage component; republish the target concurrent business event to the message queue to perform backtracking and replay on the target concurrent business event.

[0144] In an exemplary embodiment, as Figure 7 shown, there is provided a concurrent task processing device based on an event-driven architecture, including: a pre-training module 702, a feature encoding module 704, a clustering processing module 706, and a model training loss value determination module 708, wherein:

[0145] The pre-training module 702 collects operation mode samples for the initial neural network model, and pre-trains the initial neural network model according to the operation mode samples to obtain a pre-trained neural network model; the feature encoding module 704 obtains each original concurrent service event to be processed, and passes through the encoding layer of the pre-trained neural network model to perform feature encoding on each original concurrent service event to obtain deep features corresponding to the original concurrent service events; the clustering processing module 706 performs clustering processing on each original concurrent service event according to the feature clustering layer of the pre-trained neural network model to obtain a preset number of clustering categories, and each clustering category corresponds to at least one original concurrent service event; the model training loss value determination module 708 is used to determine the model training loss value corresponding to the pre-trained neural network model according to the deep features and the clustering categories, and adjust the parameters of the pre-trained neural network model according to the model training loss value to obtain a trained event classification model.

[0146] In an exemplary embodiment, the model training loss value determination module is further configured to: for each deep feature, determine the original concurrent service event corresponding to the deep feature, and obtain the original service category corresponding to the original concurrent service event; for each original concurrent service event, compare the original service category with the clustering category determined according to the pre-trained neural network model to obtain a comparison result; perform normalization processing on each original concurrent service event to obtain original normalized data; calculate the model training loss value according to the deep features, the clustering categories, the comparison results, and the original normalized data to obtain the model training loss value corresponding to the pre-trained neural network model.

[0147] In an exemplary embodiment, the model training loss value determination module is further configured to: obtain the number of features corresponding to the deep features, and obtain the number of categories corresponding to the clustering categories; calculate the model training loss value according to the comparison results, the deep features, the number of features, the clustering categories, and the number of categories to obtain the feature clustering layer loss value corresponding to the feature clustering layer; calculate the model training loss value according to the original normalized data and the number of features to obtain the encoding reconstruction loss value corresponding to the encoding layer; obtain a preset loss value weight corresponding to the feature clustering layer loss value, and perform weighted summation according to the preset loss value weight, the feature clustering layer loss value, and the encoding reconstruction loss value to obtain the model training loss value corresponding to the pre-trained neural network model.

[0148] Each module in the above concurrent task processing device based on the event-driven architecture can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in hardware form or independent of the processor, or stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0149] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structural diagram may be as shown in Figure 8 the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as business operations, concurrent business events, message queues, event classification models, event types, business themes, subscription requests, task operations, and task execution results. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a concurrent task processing method based on an event-driven architecture.

[0150] Those skilled in the art can understand that Figure 8 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0151] In one embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, it implements the steps in the above method embodiments.

[0152] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, it implements the steps in the above method embodiments.

[0153] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor, it implements the steps in the above method embodiments.

[0154] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0155] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in this application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., and are not limited thereto.

[0156] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as within the scope recorded in this application. The above-described embodiments only represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several deformations and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application should be subject to the appended claims.< / event>

Claims

1. A concurrent task processing method based on an event-driven architecture, characterized in that The method includes: Receiving concurrent business events corresponding to different business operations respectively fed back by an event generation service component, and publishing the multiple concurrent business events into a message queue; According to the trained event classification model, respectively performing event classification processing on the multiple concurrent business events in the message queue to obtain an event type corresponding to each concurrent business event, and determining the event type as the business theme corresponding to the concurrent business event; In response to subscription requests triggered by different event consumption service components for business themes, asynchronously feeding back at least one concurrent business event associated with the business theme to the event consumption service component, so as to asynchronously execute task operations corresponding to the concurrent business events through different event consumption service components and obtain task execution results.

2. The method according to claim 1, wherein The step of, according to the trained event classification model, respectively performing event classification processing on the multiple concurrent business events in the message queue to obtain an event type corresponding to each concurrent business event includes: According to the feature clustering layer of the trained event classification model, respectively performing clustering processing on the multiple concurrent business events in the message queue to obtain a preset number of clustering categories; According to the decoder of the trained event classification model, respectively performing decoding and reconstruction processing on the preset number of clustering categories to obtain an operation mode corresponding to each clustering category; For each clustering category, according to the operation mode, determining an event type corresponding to the clustering category, and determining the event type as the event type corresponding to the concurrent business events under the clustering category.

3. The method according to claim 2, wherein The manner of obtaining the trained event classification model includes: Collecting operation mode samples for an initial neural network model, and pre-training the initial neural network model according to the operation mode samples to obtain a pre-trained neural network model; Obtaining each original concurrent business event to be processed, and performing feature encoding on each original concurrent business event through the encoding layer of the pre-trained neural network model to obtain deep features corresponding to the original concurrent business events; According to the feature clustering layer of the pre-trained neural network model, performing clustering processing on each original concurrent business event to obtain a preset number of clustering categories; each clustering category corresponds to at least one original concurrent business event; Determining a model training loss value corresponding to the pre-trained neural network model according to the deep features and the clustering categories, and adjusting parameters of the pre-trained neural network model according to the model training loss value to obtain a trained event classification model.

4. The method according to claim 3, wherein The step of determining a model training loss value corresponding to the pre-trained neural network model according to the deep features and the clustering categories includes: For each deep feature, determining the original concurrent business event corresponding to the deep feature, and obtaining the original business category corresponding to the original concurrent business event; For each original concurrent business event, comparing the original business category with the clustering category determined according to the pre-trained neural network model to obtain a comparison result; Normalize each of the original concurrent service events to obtain original normalized data; Calculate a model training loss value according to the deep features, the clustering categories, the comparison results, and the original normalized data, to obtain a model training loss value corresponding to the pre-trained neural network model.

5. The method according to claim 4, characterized in that, The calculating a model training loss value according to the deep features, the clustering categories, the comparison results, and the original normalized data, to obtain a model training loss value corresponding to the pre-trained neural network model includes: Obtain the number of features corresponding to the deep features, and obtain the number of categories corresponding to the clustering categories; Calculate a model training loss value according to the comparison results, the deep features, the number of features, the clustering categories, and the categories, to obtain a feature clustering layer loss value corresponding to the feature clustering layer; Calculate a model training loss value according to the original normalized data and the number of features, to obtain an encoding reconstruction loss value corresponding to the encoding layer; Obtain a preset loss value weight corresponding to the feature clustering layer loss value, and perform weighted summation according to the preset loss value weight, the feature clustering layer loss value, and the encoding reconstruction loss value, to obtain a model training loss value corresponding to the pre-trained neural network model.

6. The method according to any one of claims 1 to 5, characterized in that, The responding to subscription requests triggered by different event consumption service components for a service theme, and asynchronously feeding back at least one concurrent service event associated with the service theme to the event consumption service components, so as to asynchronously execute task operations corresponding to the concurrent service events through different event consumption service components to obtain task execution results includes: Respond to subscription requests triggered by different event consumption service components for a service theme, obtain the service theme corresponding to each subscription request triggered by an event consumption service component, and obtain at least one concurrent service event associated with each service theme; Through an asynchronous processing mechanism, for each event consumption service component, asynchronously feed back at least one concurrent service event associated with the service theme to the event consumption service component, so as to instruct different event consumption service components through the asynchronous processing mechanism to asynchronously execute task operations corresponding to the concurrent service events to obtain task execution results.

7. The method according to any one of claims 1 to 5, characterized in that The method further includes: Determine a data storage method corresponding to the concurrent service event, and obtain a data storage component corresponding to the data storage method; Store multiple concurrent service events in the data storage component, and upload the data storage component storing multiple concurrent service events to a cloud server; If a system failure or data loss is detected, trigger an event backtracking request for the data storage component, so as to obtain a target concurrent service event associated with the event backtracking request from the stored data storage component based on the cloud server; Republish the target concurrent service event to the message queue to backtrack and replay the target concurrent service event.

8. A concurrent task processing device based on an event-driven architecture, characterized in that, The device includes: A concurrent service event receiving module, configured to receive the concurrent service events respectively corresponding to different service operations fed back by an event generation service component, and publish the multiple concurrent service events to a message queue; A service topic determination module, configured to perform event classification processing on the multiple concurrent service events in the message queue respectively according to a trained event classification model, obtain an event type corresponding to each concurrent service event, and determine the event type as the service topic corresponding to the concurrent service event; A task operation execution module, configured to, in response to a subscription request triggered by different event consumption service components for a service topic, asynchronously feed back at least one concurrent service event associated with the service topic to the event consumption service component, so as to asynchronously execute a task operation corresponding to the concurrent service event through different event consumption service components and obtain a task execution result.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images