Event push methods, devices, computer equipment and storage media

Abstract

This application relates to an event push method, apparatus, computer device, and storage medium. The method includes: acquiring a data update log from a database; reading a target record from the data update log; generating target event information corresponding to the target record; and pushing the target event information to an event queue corresponding to an event processing component. In this solution, the event information does not require the component that processes data update requests to execute, thereby decoupling the processing of data update requests from the event information push steps. In cloud computing scenarios, the database's logging function is used to avoid delays caused by the event information push steps in the data update request processing process, thus reducing the processing latency of data update services.

Description

Technical Field

[0001] This application relates to the field of cloud computing technology, and in particular to an event push method, apparatus, computer device, and storage medium. Background Technology

[0002] In a cloud computing scenario, the processing of a business operation may involve both data updates in the database and event pushes.

[0003] For example, in related technologies, a complete business processing logic in a cloud computing scenario is completed collaboratively by a request processing component, a database, and an event processing component. For instance, the request processing component processes data update requests sequentially according to their received time. For each data update request, the request processing component instructs the database to update the data. Simultaneously, the request processing component generates event information corresponding to the data update request and pushes the event information to an event queue. To avoid misprocessing or data inconsistencies, the request processing component typically saves the data update result to the database (i.e., a data update commit operation) after confirming that the event information has been successfully sent to the event queue, and then proceeds to process the next data update request.

[0004] However, the solutions shown in the related technologies require the request processing component to perform the data update commit operation only after successfully sending the event information, resulting in excessive delay in the data update request processing flow and affecting the efficiency of business processing. Summary of the Invention

[0005] This application provides an event push method, apparatus, computer device, and storage medium, which can improve business processing efficiency in cloud computing scenarios. The technical solution is as follows:

[0006] On the one hand, an event push method is provided, the method including:

[0007] Obtain the data update log of the database, wherein the data update log contains at least one update record, and the update record is used to indicate an update operation of a piece of data in the database;

[0008] Read the target record from the data update log, where the target record is any one of the at least one update record;

[0009] Generate the target event information corresponding to the target record;

[0010] The target event information is pushed to the event queue corresponding to the event processing component, and the event processing component is used to execute the processing operation corresponding to the event information in the event queue.

[0011] On the other hand, an event push device is provided, the device comprising:

[0012] The log acquisition module is used to acquire the data update log of the database. The data update log contains at least one update record, which is used to indicate an update operation of a piece of data in the database.

[0013] A record reading module is used to read a target record from the data update log, wherein the target record is any one of the at least one update record;

[0014] The event information generation module is used to generate target event information corresponding to the target record;

[0015] The event information push module is used to push the target event information to the event queue corresponding to the event processing component, and the event processing component is used to execute the processing operation corresponding to the event information in the event queue.

[0016] In one possible implementation, the target record includes target data before the update and target data after the update, wherein the target data is the data in the database corresponding to the target record;

[0017] The event information generation module is used to convert the target data before the update and the target data after the update into data in a specified format to obtain the target event information; the specified format is a data format that the event processing component can parse.

[0018] In one possible implementation, the device further includes:

[0019] The identifier acquisition module is used to acquire the identifier of the database before the event information generation module converts the target data before the update and the target data after the update into data in a specified format to obtain the target event information;

[0020] The first format query module is used to query the specified format corresponding to the identifier in the database.

[0021] In one possible implementation, the device further includes:

[0022] The format acquisition module is used to acquire the data format of the target data before the event information generation module converts the target data before the update and the target data after the update into data of a specified format to obtain the target event information;

[0023] The second format query module is used to query the specified format corresponding to the data format of the target data.

[0024] In one possible implementation, the device further includes:

[0025] The data type acquisition module is used to acquire the data type of each field in the target record before the event information generation module converts the target data before the update and the target data after the update into data of a specified format to obtain the target event information;

[0026] The third format query module is used to query the data sub-formats corresponding to the data types of each field.

[0027] The format combination module is used to combine the data sub-formats corresponding to the data types of each field to obtain the specified format.

[0028] In one possible implementation, the record reading module is used to read the at least one update record sequentially as the target record according to the generation time of the at least one update record from earliest to latest.

[0029] In one possible implementation, the event information push module is further configured to, in response to a failure to push the target event information, re-push the target event information to the event queue.

[0030] In one possible implementation, the record reading module is further configured to read a new target record from the data update log in response to a successful push of the target event information.

[0031] In one possible implementation, the update record is a record generated by the database in response to a request processing component processing a data update request; the apparatus further includes:

[0032] The request processing control module is used to control the request processing component to process the next data update request in response to the database successfully executing any data update operation corresponding to the update record before the log acquisition module obtains the data update log of the database.

[0033] In one possible implementation, the request processing control module is configured to send an update completion notification to the request processing component through the database in response to the database successfully executing a data update operation corresponding to any of the target records. The update completion notification is used to instruct the request processing component to process the next data update request.

[0034] In one possible implementation, the data update log is a binary log, the log mode of the binary log is a row-based replication mode, and the mirror parameter of the binary log indicates that the binary log is used to record the before and after mirror of the data.

[0035] On the other hand, an event push system is provided, the system including: a request processing component, a database, and an event push component;

[0036] The request processing component is used to perform a data update operation on the target data in the database when processing the first data update request.

[0037] The database is configured to write the update record corresponding to the data update operation into the data update log after the data update operation is successfully executed, and send an update completion notification to the request processing component. The update completion notification is used to instruct the request processing component to process the second data update request, which is the next request after the first data update request.

[0038] The event push component is used to read the target record in the data update log in order from earliest to latest according to the generation time, generate the target event information corresponding to the target record, and push the target event information to the event queue corresponding to the event processing component. The event processing component is used to execute the processing operation corresponding to the event information in the event queue.

[0039] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the event push method as described above.

[0040] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the event push method as described above.

[0041] The technical solution provided in this application may include the following beneficial effects:

[0042] When any data in the database is updated, an update record is generated, and a corresponding event message is added to the event queue. In other words, the event message does not need to be executed by the component that handles the data update request. This decouples the processing of the data update request from the event message push step, avoiding the delay caused by the event message push step in the data update request processing process, thereby reducing the processing latency of the data update business.

[0043] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0045] Figure 1 This is a system configuration diagram of a business processing system according to various embodiments of this application;

[0046] Figure 2 This is a flowchart illustrating an event push method according to an exemplary embodiment;

[0047] Figure 3 yes Figure 2 The illustrated embodiment relates to an event-driven system framework diagram;

[0048] Figure 4 This is a flowchart illustrating an event push method according to an exemplary embodiment;

[0049] Figure 5 yes Figure 4 The illustrated embodiment is a schematic diagram of the update record.

[0050] Figure 6 yes Figure 4 The illustrated embodiment relates to a framework diagram of an event information push notification.

[0051] Figure 7 This is a structural block diagram of an event push device according to an exemplary embodiment;

[0052] Figure 8 This is a schematic diagram of the structure of a computer device according to an exemplary embodiment. Detailed Implementation

[0053] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0054] This application proposes an event push scheme that enables event push notifications. For ease of understanding, the terms used in this application are explained below.

[0055] 1) Cloud technology is a general term encompassing network technology, information technology, integration technology, management platform technology, and application technology based on cloud computing business models. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0056] 2) Cloud computing

[0057] Cloud computing refers to the delivery and usage model of Internet technology (IT) infrastructure, meaning obtaining necessary resources through a network in an on-demand and easily scalable manner. In a broader sense, cloud computing also refers to the delivery and usage model of services, meaning obtaining necessary services through a network in an on-demand and easily scalable manner. These services can be IT and software, Internet-related, or other services. Cloud computing is a product of the development and integration of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0058] With the development of the internet, real-time data streams, and the diversification of connected devices, as well as the demands for search services, social networks, mobile commerce, and open collaboration, cloud computing has rapidly developed. Unlike previous parallel distributed computing, cloud computing will fundamentally revolutionize the entire internet model and enterprise management model.

[0059] 3) Cloud storage

[0060] Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology and distributed storage file systems to bring together a large number of storage devices of various types in the network (storage devices are also called storage nodes) to work together through application software or application interfaces to provide data storage and business access functions to the outside world.

[0061] Currently, the storage method in storage systems is as follows: Logical volumes are created, and during creation, physical storage space is allocated to each logical volume. This physical storage space may consist of a single storage device or the disks of several storage devices. Clients store data on a logical volume, which means storing the data on the file system. The file system divides the data into many parts, each part being an object. Each object contains not only the data but also additional information such as a data identifier (ID, ID entity). The file system writes each object to the physical storage space of that logical volume and records the storage location information of each object. Therefore, when a client requests access to data, the file system can allow the client to access the data based on the storage location information of each object.

[0062] The process by which a storage system allocates physical storage space to a logical volume is as follows: the physical storage space is pre-divided into strips according to the capacity estimate of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of Redundant Array of Independent Disks (RAID). A logical volume can be understood as a strip, thus allocating physical storage space to the logical volume.

[0063] 4) Database

[0064] A database, simply put, can be viewed as an electronic filing cabinet—a place to store electronic files, where users can perform operations such as adding, querying, updating, and deleting data. A "database" is a collection of data stored together in a certain way, capable of being shared by multiple users, with minimal redundancy, and independent of application programs.

[0065] A Database Management System (DBMS) is a computer software system designed to manage databases, generally possessing basic functions such as storage, retrieval, security, and backup. DBMSs can be classified according to the database model they support, such as relational or XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters or mobile devices; or according to the query language used, such as SQL (Structured Query Language) or XQuery; or according to performance priorities, such as maximum scale or maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, supporting multiple query languages ​​simultaneously.

[0066] Please refer to Figure 1 This illustrates a system configuration diagram of a business processing system according to various embodiments of this application. For example... Figure 1 As shown, the system includes a server 120, a database 140, and several terminals 160.

[0067] Server 120 is a single server, or a combination of several servers, or a virtualization platform, or a cloud computing service center.

[0068] Server 120 can be a server that provides backend support for various services in a cloud computing scenario. Server 120 can consist of one or more functional components.

[0069] Optionally, in one possible implementation, such as Figure 1 As shown, server 120 may include request processing component 120a, event push component 120b, and event processing component 120c.

[0070] The aforementioned request processing component 120a, event push component 120b, and event processing component 120c can be implemented by executing different software programs, such as different applications, through the hardware in the server.

[0071] The request processing component 120a is used to interact with the client program installed in the terminal 160 to obtain data update requests initiated by the terminal 160; and to update the data in the database 140 according to the received data update requests.

[0072] The event push component 120b is used to generate corresponding event information based on the data update results of the database 140, and provide the event information to the event processing component 120c.

[0073] The event handling component 120c is used to perform corresponding event handling operations based on the event information provided by the event push component 120b, such as sending corresponding notifications based on the received event information.

[0074] The aforementioned database 140 can be a Redis database, or it can be another type of database. Database 140 is used to store various types of data, such as user information for each user, business data for various services, and so on.

[0075] Terminal 160 and server 120 are connected via a communication network. Optionally, this communication network can be a wired network or a wireless network.

[0076] Optionally, the system may also include a management device ( Figure 1 (Not shown), the management device is connected to the server 120 via a communication network. Optionally, the communication network can be a wired network or a wireless network.

[0077] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats, including Hypertext Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0078] In application layer logic, a common scenario involves processing a user request, such as placing an order or making a payment. After processing the request, the result needs to be persisted to the database (i.e., writing data to the database). Simultaneously, if the request is successfully processed, the result should be sent as an event to an event queue, as other program logic may also be interested in data changes. This is a classic application scenario in distributed systems: event-driven systems.

[0079] These scenarios are more common after cloud architectures are microservice-based and containerized. This is because microservices break down functional modules into smaller units, with each service (or application) handling a specific function. For example, there are applications that handle user requests, applications that handle monitoring and alarms, applications that handle statistical data reports, applications that handle access control, and so on.

[0080] For example, in one possible scenario, application A (such as the one mentioned above) Figure 1 If the application running the request processing component 120a in the system receives an order request to purchase one item a in quantity of 2, then after processing a series of logic steps, it needs to increment the quantity of sold items a in the order database (e.g., database 140 mentioned above) by 2 and decrement the remaining inventory by 2. Simultaneously, it needs to send the remaining inventory quantity x of item a as an event message to the event queue, which will then be processed by application B (e.g., the application running the request processing component 120a mentioned above). Figure 1 The event handling component 120c in the system shown is running an application. The logic of application B is to receive a message, determine whether the current inventory is lower than a pre-configured threshold for the product, and if it is lower than the threshold, send an email to remind the purchasing staff that a purchase is needed.

[0081] With the continuous development of internet applications, the number of internet users is also growing rapidly. Currently, many internet services have extremely high concurrency, and these services usually have high requirements for low latency, high data availability, high reliability, and data consistency. For example, these internet services are often used in e-commerce and financial payment fields. Therefore, in event-driven systems, how to minimize business processing latency while ensuring high availability, high reliability, and consistency of data is of great significance to internet applications.

[0082] Subsequent embodiments of this application propose an event push scheme that can better balance low latency, high data availability, high reliability, and data consistency in an event push-based system.

[0083] Figure 2This is a flowchart illustrating an event push method according to an exemplary embodiment. This event push method can be used in computer devices, such as those described above. Figure 1 In the server of the system shown. This method is used by... Figure 1 Taking the execution of the event push component 120b in the server shown as an example, as... Figure 2 As shown, the event push method may include the following steps:

[0084] Step 21: Obtain the database update log. The data update log contains at least one update record, which indicates an update operation on a piece of data in the database.

[0085] In this embodiment, the database stores several data entries, and also maintains a data update log. Whenever a data entry in the database changes, i.e., a data update occurs, the database adds an update record to the data update log to record this data update.

[0086] In one possible implementation, data updates in the database are triggered by the request processing component when executing a data update request. For example, after receiving a data processing request, the request processing component initiates data modification to the corresponding data in the server. After the data modification is completed, the database adds the information of this data modification to the data update log as a data record.

[0087] Step 22: Read the target record from the data update log. The target record is any one of the at least one update record.

[0088] In this embodiment of the application, when there are unprocessed update records in the data update log, the event push component in the server reads an update record from the data update log as the target record.

[0089] For example, the event push component monitors change requests in the data update log. When a new update record appears in the data update log, the event push component reads the newly added update record from the data update log as the target record for subsequent processing.

[0090] Step 23: Generate the target event information corresponding to the target record.

[0091] Since the format of data update logs is usually very different from that of event information, event processing components usually cannot directly parse data update logs. Therefore, in this embodiment of the application, the event push component can convert the target record into target event information by format transformation.

[0092] Step 24: Push the target event information to the event queue corresponding to the event handling component. The event handling component is used to execute the processing operation corresponding to the event information in the event queue.

[0093] After each event message is generated, the event push component can push the generated event message to the event queue, and the event handling component can retrieve the event message from the event queue and perform the corresponding processing operations.

[0094] For example, please refer to Figure 3 This diagram illustrates an event-driven system framework based on an embodiment of this application. Figure 3 As shown, when the request processing component processes a data update request, it initiates a data update to the database (S31); the database executes the data update operation (S32) and writes the update record corresponding to the data update operation to the data update log (S33); the event push component reads the target record from the data update log (S34), generates event information based on the target record (S35), and pushes the generated event information to the event queue (S36). From the above process... Figure 3 As can be seen from the steps shown, the above scheme generates event information through data update logs, decoupling the event information generation step from the request processing component's processing of data update requests. In other words, when the request processing component processes a data update request, it does not need to wait for the corresponding event information to be successfully sent to the event queue before it can directly process the next data update request.

[0095] In summary, the solution shown in this application embodiment can generate corresponding event information and add it to the event queue based on the update record generated when any data in the database is updated. In other words, the event information does not need to be executed by the component that processes the data update request, thereby decoupling the processing of the data update request from the event information push step. This avoids the delay caused by the event information push step in the data update request processing process, thereby reducing the processing delay of the data update business.

[0096] Figure 4 This is a flowchart illustrating an event push method according to an exemplary embodiment. This event push method can be used in computer devices, such as those described above. Figure 1 In the server of the system shown. This method is used by... Figure 1 Taking the execution of each component in the server shown as an example, such as... Figure 4 As shown, the event push method may include the following steps:

[0097] Step 401: When processing a data update request, the request processing component initiates a data update on the data in the database.

[0098] In one possible implementation, the request processing component can receive data update requests sent by client programs installed on the terminal and process each received data update request one by one.

[0099] Optionally, the data update request mentioned above is a request that requires or can modify the corresponding data in the database. For example, a data update request can be any business processing request initiated by the user, such as an order placement request, a transfer request, etc.

[0100] Optionally, when processing the first data update request, the request processing component sends a data update instruction to the database, which instructs the database to update the corresponding data. Accordingly, upon receiving the data update instruction, the database performs the data update operation on the data corresponding to that instruction.

[0101] For example, taking a request processing component with transfer processing functionality as an example, a transfer transaction generates two data update requests: one is to subtract the transfer amount from the balance of the sending account, and the other is to increase the transfer amount in the balance of the receiving account. For these two data update requests, the request processing component sends data updates to the database in sequence, so that the database updates the values ​​of the balance of the sending and receiving accounts stored in the database in sequence.

[0102] Step 402: After the database successfully executes the data update operation, it writes the update record corresponding to the data update operation into the data update log.

[0103] In other words, the update record is a record generated by the database in response to the request processing component processing a data update request.

[0104] In this embodiment of the application, after each data update operation is performed, the database writes an update record in the data update log to record the data update status.

[0105] In one possible implementation, the data update log is a binary log, the log mode of the binary log is a row-based replication mode, and the mirror parameter of the binary log indicates that the binary log is used to record the before and after mirror of the data.

[0106] In this embodiment of the application, taking the database as a relational database management system database, such as MySQL, the database enables binary logging, i.e., the Binlog function, and sets the Binlog format to ROW (row), and sets the binlog_row_image parameter to FULL. In this way, the binary log will record the values ​​of a row of data in the database before and after modification.

[0107] For example, please refer to Figure 5 It illustrates a schematic diagram of the update record involved in an embodiment of this application, such as... Figure 5 As shown, in a data update operation, for example, a data 51a in database 51 has a value of 100 before modification and a value of 80 after modification; then the update record 52a in the data update log 52 corresponding to this data update operation is: (100, 80).

[0108] Step 403: In response to the database successfully executing any data update operation corresponding to the updated record, control the request processing component to process the next data update request.

[0109] Optionally, in response to the database successfully executing a data update operation corresponding to any of the target records, the server sends an update completion notification to the request processing component through the database. The update completion notification is used to instruct the request processing component to process the next data update request.

[0110] In one possible implementation, after performing a data update operation, the database can immediately perform a commit operation to persist the updated data to the database. The database can then notify the request processing component to process the next data update request.

[0111] From the perspective of the request processing component, during the sequential processing of data update requests, the request processing component initiates a data update operation to the database based on the current data update request. After receiving a notification from the database that the data update operation has been successfully executed, it can directly initiate the data update operation corresponding to the next data update request to the database without waiting for the event information corresponding to the current data update to be successfully sent to the event queue, thereby shortening the processing latency of the data update request.

[0112] Step 404: The event push component retrieves the data update logs from the database.

[0113] In one possible implementation, the event push component can read the database's data update logs at a preset period.

[0114] In one possible implementation, the database update log can be stored in a memory space shared by the database and the event push component. The database writes new update records to the data update log by accessing this shared memory space, and the event push component reads the newly written update records from the data update log by accessing this shared memory space.

[0115] Step 405: The event push component reads the target record from the data update log. The target record contains the target data before the update and the target data after the update. The target data is the data of the corresponding target record in the database.

[0116] The target record is any one of the above-mentioned at least one update record.

[0117] In one possible implementation, the event push component reads at least one update record as the target record in order of its generation time from earliest to latest.

[0118] In this embodiment of the application, the event push component can periodically query or monitor in real time whether there are any unread update records in the database's data update log. If so, it will start from the earliest unread update record and read them sequentially as target records to execute the subsequent event information push process.

[0119] In one possible implementation, before executing the subsequent step 406, the event push component first checks whether the data corresponding to the target record has enabled the event notification function. In response to the data corresponding to the target record having enabled the event notification function, the subsequent step 406 is executed.

[0120] In this embodiment, some data updates in the database may not require the generation of corresponding event information. For example, if the database contains bank card balances for various users, and the event processing component's function is to provide bank card balance change alerts, user A has enabled the bank card balance alert function (i.e., user A's data has enabled event notification), while user B has not. Therefore, when both user A's and user B's bank card balances change, if corresponding event information is generated for both user A's and user B's data updates, the subsequent event processing component will ignore the event information corresponding to user B's data update. In other words, the step of generating corresponding event information for user B's data update is unnecessary. Therefore, through the solution shown in this embodiment, after reading the update record corresponding to user B's data update, the event push component detects that user B has not enabled the bank card balance alert function and ignores the update record. Similarly, after reading the update record corresponding to user A's data update, the event push component detects that user A has enabled the bank card balance alert function and executes subsequent step 406.

[0121] Step 406: The event push component converts the target data before and after the update into data in a specified format to obtain the target event information.

[0122] The specified format is a data format that the event handling component can parse.

[0123] To improve database utilization and reduce data storage space, databases typically pre-define data formats. For example, in a database data list, each data bit in each row corresponds to its own physical meaning. For instance, suppose a row of data contains 3 data bits, representing product name 'a', sales volume 10, and inventory quantity 100, respectively. This row of data only contains the values ​​of the three fields ('a', 10, 100) without specifying the meaning of these values. Similarly, the update record in the data update log only contains the values ​​of each field before and after the modification of this row of data. For event handling components outside the database, they cannot directly parse the actual meaning of the update record. Therefore, the event push component needs to convert the format of the update record into a data format that the event handling component can parse.

[0124] In one exemplary solution, the event push component converts the target data before and after the update into data in a specified format. Before obtaining the target event information, it can also obtain the database identifier and query the specified format corresponding to the database identifier.

[0125] In one possible implementation of this application embodiment, the format of each piece of data in the database is uniform, and correspondingly, the format of each update record in the data update log of the database is also uniform. The server can pre-configure the target format (i.e. the specified format mentioned above) of the update record corresponding to the database during format conversion in the first configuration information. The first configuration information can include the correspondence between the database identifier and the specified format. For different databases, since the update record format is different, the corresponding specified format is also different. That is to say, in the first configuration information mentioned above, the identifier of different databases corresponds to different specified formats.

[0126] Correspondingly, before obtaining the event information, the event push component converts the format of each update record in the data update log of the database and obtains the event information, it can query the corresponding specified format in the first configuration information through the database identifier.

[0127] In one possible implementation, the event push component described above can extract the database identifier from the target record. That is, each update record in the database's data update log can carry the database identifier.

[0128] For example, suppose that the format of each row of data stored in database DB1 is uniform. Each row of data in database DB1 contains 3 data bits, which represent the product name 'a', the sales volume of 10, and the inventory quantity of 100.

[0129] In one possible implementation, after a data update, the values ​​of these three data bits are modified to product name 'a', sales volume '12', and inventory quantity '98'. The update record corresponding to this data update would then be:

[0130] (DB1, "a", 10, 100), (DB1, "a", 12, 98);

[0131] In the data before and after the modification, "DB1" is the identifier of the database.

[0132] In another possible implementation, to save storage space, the database identifier in the update record is independent of the data before and after the update. For example, the update record corresponding to this data update is:

[0133] (DB1), ("a", 10, 100), ("a", 12, 98).

[0134] In one exemplary solution, the event push component converts the target data before and after the update into data in a specified format. Before obtaining the target event information, it also obtains the data format of the target data and queries the specified format corresponding to the data format of the target data.

[0135] In one possible implementation of this application, the format of each data entry in the database is not uniform. For example, the database contains data in two or more different formats. For instance, data of different formats are processed according to different data lists; the format of each row in a single list is uniform, while the format of data in different lists is different.

[0136] Correspondingly, the format of each update record in the database's data update log is also inconsistent. The server can pre-configure the target format (i.e. the specified format mentioned above) for the update records corresponding to different data formats in the database during format conversion in the second configuration information. The second configuration information can contain the correspondence between data formats and specified formats. For data corresponding to different data formats, since the update record format is different, the corresponding specified format is also different. In other words, in the second configuration information mentioned above, different data formats correspond to different specified formats.

[0137] Correspondingly, before the event push component converts the format of each update record in the data update log of the database to obtain the event information, it can query the specified format corresponding to the data format of the current update record in the second configuration information through the database identifier.

[0138] In one possible implementation, the event push component described above can extract the data format from the target record. That is, in the database's data update log, each update record can carry the corresponding data format.

[0139] In one possible implementation, when the format of each row of data in a list in a database is uniform, the data format can be identified by the identifier of the list in which the data is located (such as the list number), or it can be represented by both the identifier of the database in which the data is located and the identifier of the list in which it is located.

[0140] For example, suppose the data in each row stored in database DB1 is not in a uniform format. In particular, a row of data in list C1 of database DB1 contains 3 data bits, which represent the product name a, the sales quantity 10, and the inventory quantity 100.

[0141] In one possible implementation, after a data update, the values ​​of these three data bits are modified to product name 'a', sales volume '12', and inventory quantity '98'. The update record corresponding to this data update would then be:

[0142] (DB1, C1, "a", 10, 100), (DB1, C1, "a", 12, 98);

[0143] In the data before and after the modification, "DB1" is the identifier of the database, and "C1" is the identifier of the list.

[0144] In another possible implementation, to save storage space, the database identifier and list identifier in the update record are independent of the data before and after the update. For example, the update record corresponding to this data update is:

[0145] (DB1, C1), ("a", 10, 100), ("a", 12, 98).

[0146] In one exemplary solution, the event push component converts the target data before and after the update into data in a specified format. Before obtaining the target event information, it also obtains the data type of each field in the target record; queries the data sub-format corresponding to the data type of each field; and combines the data sub-formats corresponding to the data type of each field to obtain the specified format.

[0147] In one possible implementation of this application, since the data format depends on the combination of various fields in the data, a limited number of data types of fields can be combined to create more types of data formats. For example, taking a field with a total of 3 data types as an example, theoretically, 7 data formats can be combined. Therefore, when the number of formats of each piece of data in the database is large, but the data types of the fields are limited, the server directly maintains the data sub-formats corresponding to fields of various data types. When a target record is read, the event push component can query the data sub-formats corresponding to each field according to the data type of each field in the target record, and then combine the queried data sub-formats to obtain the corresponding specified format.

[0148] In one possible implementation, the event push component described above can extract the data types of each field in the target record. In other words, each update record in the database's data update log can carry the data types of the corresponding fields.

[0149] For example, suppose a row of data in the database contains 3 data bits, which represent the product name 'a', the sales volume of 10, and the inventory quantity of 100, respectively, and the corresponding data types are represented by T1, T2, and T3.

[0150] In one possible implementation, after a data update, the values ​​of these three data bits are modified to product name 'a', sales volume '12', and inventory quantity '98'. The corresponding update record for this data update would then be:

[0151] (T1: "a", T2: 10, T3: 100), (T1: "a", T2: 12, T3: 98).

[0152] Step 407: The event push component pushes the target event information to the event queue corresponding to the event handling component.

[0153] If the event push component fails to push the target event information, the event handling component will push the target event information back to the event queue.

[0154] When the event push component successfully pushes the target event information, the event handling component reads the new target record from the data update log.

[0155] In one exemplary scheme, after each event message is generated, the event push component pushes the event message to the event queue. If the push is successful, the next update record is read to generate a new event message. If the push fails, the event message is re-pushed.

[0156] Step 408: The event handling component executes the processing operation corresponding to the event information in the event queue.

[0157] In this embodiment of the application, the event processing component can monitor the event queue. When event information appears in the event queue, it can extract the event information from the event queue and perform the operation corresponding to the event information, such as sending a data update reminder to the user.

[0158] In one possible implementation, the event queue is a first-in-first-out queue, which allows the event handling component to process each event in the order of its generation time, thereby ensuring the processing order of the event information.

[0159] For example, please refer to Figure 6 The diagram illustrates a framework for event information push notifications according to an embodiment of this application. Taking a product inventory database as an example, its structure is as follows:

[0160] CREATE TABLE `order` (

[0161] `id` int(11) DEFAULT NULL AUTO_INCREMENT,

[0162] `goods_name` char(32) DEFAULT NULL DEFAULT 0,

[0163] `goods_number` char(11) NOT NULL DEFAULT 0,

[0164] `goods_left` int(11) DEFAULT NULL DEFAULT 0,

[0165] PRIMARY KEY(`id`),

[0166] KEY `gn` (`goods_name`)

[0167] ) ENGINE=InnoDB DEFAULT CHARSET=utf8;

[0168] The `goods_name` field represents the product name, the `goods_number` field represents the total number of units of the product sold, and `goods_left` represents the remaining inventory of this product. Assume that initially, the database contains only one record: (1, 'a', 10, 100), meaning that 10 units of product 'a' have been sold, and 100 units remain in stock. Figure 6 As shown, the process steps for pushing event information based on this database are as follows:

[0169] Step S61: Application 1 executes order request processing.

[0170] After processing the order request (a sold 2 units), Application 1 (i.e., the request processing component) writes the latest data (1, 'a', 12, 98) to the database (S61a); then, the database immediately returns the response indicating that the order request has been processed (S61b); Application 1 continues to process new order requests and continues to execute step S61 in a loop.

[0171] Step S62: The event push component generates and pushes event information.

[0172] Event push components (such as the Binlog Processor component) listen for changes in the database's binary log (S62a) and obtain the latest changes, which are equivalent to the update records of [(1, 'a', 10, 100), (1, 'a', 12, 98)], which include the data before and after the modification.

[0173] The event push component determines the specified format for format conversion based on the received update records (S62b).

[0174] The event push component parses the raw Binlog binary log according to the database binary log protocol format, performs format conversion (S62c), and transforms it into data that upper-layer applications (such as event handling components) can understand. Specifically, it converts it into the specified format mentioned above. Taking JSON format as an example, the converted event information can be as follows:

[0175] {

[0176] "table": "order",

[0177] "before":{

[0178] "id":1,

[0179] "goods_name":"1",

[0180] "goods_number":10,

[0181] "goods_left:100

[0182] },

[0183] "after":{

[0184] "id":1,

[0185] "goods_name":"1",

[0186] "goods_number":12,

[0187] "goods_left:98

[0188] }

[0189] }

[0190] The event push component sends the parsed data to the event queue (S63d). If the execution is successful, it jumps to step 2 to continuously listen for and process new data changes.

[0191] If a single event sending fails, the event push component continues to execute S63d retry.

[0192] Step S63: Application 2 / 3 (i.e., the event handling component) performs event handling.

[0193] Application 2 / 3 pulls data from the event queue to obtain the latest event information (S63a).

[0194] Application 2 / 3 processes the acquired event information (S63b), and after processing, jumps to S63a to retrieve the latest data.

[0195] above Figure 6 The illustrated process constructs a complete event notification and message processing solution in a microservices environment. Combined with... Figure 6 It can be seen that in the three major steps, steps S61, S62, and S63 are sequentially related and executed strictly in that order, representing the direction of data flow. Furthermore, each sub-step within the three major steps is an infinite loop, processing one data item before moving on to the next. While the smaller steps are executed strictly sequentially, the major steps run asynchronously and independently in several different components.

[0196] Combination Figure 6 The solution shown in this application can achieve the following effects:

[0197] A) Low latency:

[0198] Since application 1 does not need to perform additional message sending (or even retrying on failure) operations after writing data, it reduces one or more network requests (in the case of retrying on failure).

[0199] B) High availability and high reliability:

[0200] Figure 6In this process, the separate representations of data persistence and message sending are re-integrated into a single persistent data representation, ensuring that data will not be lost as long as it is successfully written to the database. During the sending of event information in step S62, the event information is asynchronously retried until it is successfully sent. Furthermore, if the application or database suddenly crashes, the written messages will not be lost because they have already been recorded in the binary log.

[0201] C) Ensure the sequential nature of event information:

[0202] exist Figure 6 In this context, although both the application layer and the database can support tens of thousands or even millions of concurrent operations, the messages recorded in the database binary log are always in chronological order and are accurate. This is guaranteed by the principle and disaster recovery characteristics of the database binary log.

[0203] D) The rationality of the architecture:

[0204] exist Figure 6 In this approach, data writing and message sending are decoupled, ensuring the simplicity of application logic. Furthermore, updates, upgrades, or downtime of one application / component do not affect the availability of another. From a data flow perspective, data flows in a single, linear manner, resulting in a simpler architectural dependency.

[0205] In summary, the solution shown in this application embodiment can generate corresponding event information and add it to the event queue based on the update record generated when any data in the database is updated. In other words, the event information does not need to be executed by the component that processes the data update request, thereby decoupling the processing of the data update request from the event information push step. This avoids the delay caused by the event information push step in the data update request processing process, thereby reducing the processing delay of the data update business.

[0206] Furthermore, the solution presented in this application, by reintegrating data persistence and message notification—two processing logics for the same data—into a single data processing flow, decouples data persistence and message notification. This not only improves the response speed of the application layer but also leverages the capabilities of the database itself to address business processing logic that is essentially a distributed transaction, ensuring strict sequentiality, high availability, and high reliability of data and message processing. This is particularly relevant for fields such as e-commerce and financial payments, where requirements for processing latency, availability, and reliability are extremely high.

[0207] Figure 7 This is a structural block diagram illustrating an event push device according to an exemplary embodiment. The event push device can perform... Figure 2 or Figure 4 All or part of the steps in the illustrated embodiment. The event push device may include:

[0208] The log acquisition module 701 is used to acquire the data update log of the database. The data update log contains at least one update record, which is used to indicate an update operation of a piece of data in the database.

[0209] The record reading module 702 is used to read the target record in the data update log, wherein the target record is any one of the at least one update record;

[0210] Event information generation module 703 is used to generate target event information corresponding to the target record;

[0211] The event information push module 704 is used to push the target event information to the event queue corresponding to the event processing component, and the event processing component is used to execute the processing operation corresponding to the event information in the event queue.

[0212] In one possible implementation, the target record includes target data before the update and target data after the update, wherein the target data is the data in the database corresponding to the target record;

[0213] The event information generation module 703 is used to convert the target data before the update and the target data after the update into data in a specified format to obtain the target event information; the specified format is a data format that the event processing component can parse.

[0214] In one possible implementation, the device further includes:

[0215] The identifier acquisition module is used to acquire the identifier of the database before the event information generation module 703 converts the target data before the update and the target data after the update into data in a specified format to obtain the target event information;

[0216] The first format query module is used to query the specified format corresponding to the identifier in the database.

[0217] In one possible implementation, the device further includes:

[0218] The format acquisition module is used to acquire the data format of the target data before the event information generation module 703 converts the target data before the update and the target data after the update into data of a specified format to obtain the target event information;

[0219] The second format query module is used to query the specified format corresponding to the data format of the target data.

[0220] In one possible implementation, the device further includes:

[0221] The data type acquisition module is used to acquire the data type of each field in the target record before the event information generation module 703 converts the target data before the update and the target data after the update into data of a specified format to obtain the target event information;

[0222] The third format query module is used to query the data sub-formats corresponding to the data types of each field.

[0223] The format combination module is used to combine the data sub-formats corresponding to the data types of each field to obtain the specified format.

[0224] In one possible implementation, the record reading module 702 is used to read the at least one update record sequentially as the target record according to the generation time of the at least one update record from earliest to latest.

[0225] In one possible implementation, the event information push module 704 is further configured to, in response to a failure to push the target event information, re-push the target event information to the event queue.

[0226] In one possible implementation, the record reading module 702 is further configured to read a new target record from the data update log in response to the successful push of the target event information.

[0227] In one possible implementation, the update record is a record generated by the database in response to a request processing component processing a data update request; the apparatus further includes:

[0228] The request processing control module is used to control the request processing component to process the next data update request in response to the database successfully executing any data update operation corresponding to the update record before the log acquisition module 701 acquires the data update log of the database.

[0229] In one possible implementation, the request processing control module is configured to send an update completion notification to the request processing component through the database in response to the database successfully executing a data update operation corresponding to any of the target records. The update completion notification is used to instruct the request processing component to process the next data update request.

[0230] In one possible implementation, the data update log is a binary log, the log mode of the binary log is a row-based replication mode, and the mirror parameter of the binary log indicates that the binary log is used to record the before and after mirror of the data.

[0231] In summary, the solution shown in this application embodiment can generate corresponding event information and add it to the event queue based on the update record generated when any data in the database is updated. In other words, the event information does not need to be executed by the component that processes the data update request, thereby decoupling the processing of the data update request from the event information push step. This avoids the delay caused by the event information push step in the data update request processing process, thereby reducing the processing delay of the data update business.

[0232] Furthermore, the solution presented in this application decouples data persistence and message notification—two different ways of processing data—by integrating them into a single data processing flow. This not only improves the response speed of the application layer but also leverages the capabilities of the database itself to address business processing logic that is essentially a distributed transaction, ensuring the strict ordering, high availability, and high reliability of data and message processing.

[0233] An exemplary embodiment of this application also provides an event push system, the system including: a request processing component, a database, and an event push component. In one possible implementation, the request processing component and the event push component are implemented by running corresponding applications through hardware / virtual hardware in a cloud server.

[0234] The request processing component is used to perform a data update operation on the target data in the database when processing the first data update request.

[0235] The database is used to write the update record corresponding to the data update operation to the data update log after the data update operation is successfully executed, and to send an update completion notification to the request processing component. The update completion notification is used to instruct the request processing component to process the second data update request, which is the next request after the first data update request.

[0236] The event push component is used to read the target records in the data update log in order from earliest to latest according to the generation time, generate the target event information corresponding to the target record, and push the target event information to the event queue corresponding to the event handling component. The event handling component is used to execute the processing operation corresponding to the event information in the event queue.

[0237] Figure 8This is a schematic diagram illustrating the structure of a computer device according to an exemplary embodiment. The computer device can be implemented as a server on the network side. The server can be... Figure 1 The server 120 is shown. The computer device 800 includes a central processing unit 801, a system memory 804 including random access memory (RAM) 802 and read-only memory (ROM) 803, and a system bus 805 connecting the system memory 804 and the central processing unit 801. The computer device 800 also includes a basic input / output system 806 to facilitate the transfer of information between various devices within the computer, and a mass storage device 807 for storing the operating system 813, application programs 814, and other program modules 815.

[0238] The mass storage device 807 is connected to the central processing unit 801 via a mass storage controller (not shown) connected to the system bus 805. The mass storage device 807 and its associated computer-readable media provide non-volatile storage for the computer device 800. That is, the mass storage device 807 may include computer-readable media (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0239] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, flash memory or other solid-state storage technologies, CD-ROM, or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage media are not limited to the above-mentioned types. The system memory 804 and mass storage device 807 described above can be collectively referred to as memory.

[0240] Computer device 800 can be connected to the Internet or other network devices via network interface unit 811 connected to the system bus 805.

[0241] The memory also includes one or more programs, which are stored in the memory, and the central processing unit 801 implements these programs. Figure 2 or Figure 4 All or part of the steps of the method shown.

[0242] In exemplary embodiments, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including a computer program (instructions) that can be executed by a processor of a computer device to perform all or part of the steps of the methods shown in the various embodiments of this application. For example, the non-transitory computer-readable storage medium may be a read-only memory, a random access memory, an optical disc read-only memory, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0243] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0244] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An event push method, characterized in that, The method includes: Obtain the data update log of the database, wherein the data update log contains at least one update record, and the update record is used to indicate an update operation of a piece of data in the database; the database contains data in two or more different formats; Read the target record from the data update log, where the target record is any one of the at least one update record; the target record contains the target data before the update and the target data after the update, and the target data is the data in the database corresponding to the target record. The data format for extracting the target data from the target record; the data format is represented by the identifier of the database where the corresponding data is located and the identifier of the list where it is located; Query a specified format corresponding to the data format of the target data; the specified format is a data format that the event handling component can parse; the correspondence between the data format and the specified format is contained in the second configuration information; The target data before and after the update are converted into data in the specified format to obtain target event information; The target event information is pushed to the event queue corresponding to the event processing component, and the event processing component is used to execute the processing operation corresponding to the event information in the event queue.

2. The method according to claim 1, characterized in that, The specified format corresponding to the data format of the target data in the query includes: Obtain the data type of each field in the target record; Query the data sub-formats corresponding to the data types of each of the fields; The specified format is obtained by combining the data sub-formats corresponding to the data types of each field.

3. The method according to claim 1, characterized in that, The step of reading the target record in the data update log includes: The at least one update record is read sequentially as the target record according to the generation time of the at least one update record from earliest to latest.

4. The method according to claim 1, characterized in that, The method further includes: If the push of the target event information fails, the target event information is pushed back to the event queue.

5. The method according to claim 1, characterized in that, The method further includes: In response to the successful push of the target event information, a new target record is read from the data update log.

6. The method according to claim 1, characterized in that, The update record is a record generated by the database in response to the request processing component processing a data update request; Before obtaining the database update log, the process also includes: In response to the database successfully executing the data update operation corresponding to any of the updated records, the request processing component is controlled to process the next data update request.

7. The method according to claim 6, characterized in that, The step of responding to the successful execution of a data update operation corresponding to any of the updated records in the database, and controlling the request processing component to process the next data update request, includes: In response to the database successfully executing a data update operation corresponding to any of the target records, an update completion notification is sent to the request processing component through the database. The update completion notification is used to instruct the request processing component to process the next data update request.

8. The method according to any one of claims 1 to 7, characterized in that, The data update log is a binary log, the log mode of the binary log is a row-based replication mode, and the mirror parameter of the binary log indicates that the binary log is used to record the before and after mirror of the data.

9. An event push system, characterized in that, The system includes: a request processing component, a database, and an event push component; the database contains data in two or more different formats. The request processing component is used to perform a data update operation on the target data in the database when processing the first data update request. The database is configured to write the update record corresponding to the data update operation into the data update log after the data update operation is successfully executed, and send an update completion notification to the request processing component. The update completion notification is used to instruct the request processing component to process the second data update request, which is the next request after the first data update request. The event push component is used to read target records in the data update log in chronological order of generation time, where each target record contains target data before and after the update. The target data is the data corresponding to the target record in the database. The component extracts the data format of the target data from the target record. The data format is represented by the identifier of the database where the data is located and the identifier of the list where it is located. The component queries a specified format corresponding to the data format of the target data. The specified format is a data format that the event processing component can parse. The correspondence between the data format and the specified format is contained in the second configuration information. The component converts the target data before and after the update into the specified format to obtain target event information. The component then pushes the target event information to the event queue corresponding to the event processing component. The event processing component is used to execute the processing operation corresponding to the event information in the event queue.

10. An event push device, characterized in that, The device includes: The log acquisition module is used to acquire the data update log of the database. The data update log contains at least one update record, which indicates an update operation of a piece of data in the database. The database contains data in two or more different formats. The record reading module is used to read the target record in the data update log, wherein the target record is any one of the at least one update record; the target record contains target data before the update and target data after the update, and the target data is the data in the database corresponding to the target record; The format acquisition module is used to extract the data format of the target data from the target record; the data format is represented by the identifier of the database where the corresponding data is located and the identifier of the list where it is located; The second format query module is used to query a specified format corresponding to the data format of the target data; the specified format is a data format that the event handling component can parse; the correspondence between the data format and the specified format is contained in the second configuration information; The event information generation module is used to convert the target data before the update and the target data after the update into data in the specified format to obtain target event information; The event information push module is used to push the target event information to the event queue corresponding to the event processing component, and the event processing component is used to execute the processing operation corresponding to the event information in the event queue.

11. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, a code set, or an instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by the processor to implement the event push method as described in any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or instruction set is loaded and executed by a processor to implement the event push method as described in any one of claims 1 to 8.

Images