A message notification method, device, medium and equipment

Abstract

The application discloses a message notification method and device, a medium and equipment, relates to the field of data communication, and comprises the following steps: obtaining a task message of a notification task, and sending the task message to a distributed message queue; the distributed message queue comprises a plurality of message queues corresponding to a plurality of message topics; determining a subscribed message topic, and obtaining a target task message from a message queue corresponding to the subscribed message topic; obtaining target registration information corresponding to first identification data in the target task message from a registration center according to the first identification data; determining a destination interface address of the target task message according to the target registration information; and sending the target task message to a target business service corresponding to the destination interface address. The scheme provided in the application can ensure the timeliness and controllability of message notification, and improve the efficiency of server operation.

Description

Technical Field

[0001] This application relates to the field of data communication, specifically to a message notification method, apparatus, medium, and device. Background Technology

[0002] With the development of cloud services, higher demands are being placed on server operations to ensure the stability and continuity of user business operations, especially in business scenarios related to data computing. In the field of server data operations, the data volume is large and the data application scenarios are diverse, frequently involving the interaction between data scheduling systems and business systems. This mainly involves notifying the business system after a task node in the scheduling system is completed to trigger the execution of a certain business service.

[0003] The existing message notification mechanism between the data scheduling system and the business system has shortcomings. For example, the implementation scheme for notifying external systems of scheduling tasks in the server data computing model is complex and prone to coupling; the external notification mechanism of the scheduling system does not support message failure retries, etc. Therefore, it is necessary to improve the message notification mechanism in the server operation domain. Summary of the Invention

[0004] To improve the efficiency and controllability of message notifications, this application provides a message notification method, apparatus, medium, and device. The technical solution is as follows:

[0005] Firstly, this application provides a message notification method, the method comprising:

[0006] Obtain task messages for at least one notification task and send the task messages to a distributed message queue; the distributed message queue includes multiple message queues corresponding to multiple message topics;

[0007] Determine the message topic to be subscribed to, and retrieve the target task message from the message queue corresponding to the subscribed message topic;

[0008] Based on the first identifier data in the target task message, target registration information corresponding to the first identifier data is obtained from the registration center, and the target registration information is generated based on the registration page of the registration center;

[0009] The destination interface address of the target task message is determined based on the target registration information;

[0010] Send the target task message to the target service corresponding to the target interface address.

[0011] Secondly, this application provides a message notification device, the device comprising:

[0012] The acquisition and sending module is used to acquire task messages for at least one notification task and send the task messages to a distributed message queue; the distributed message queue includes multiple message queues corresponding to multiple message topics.

[0013] The second acquisition module is used to determine the subscribed message topic and acquire the target task message from the message queue corresponding to the subscribed message topic;

[0014] The third acquisition module is used to acquire target registration information corresponding to the first identifier data from the registration center based on the first identifier data in the target task message. The target registration information is generated based on the registration page of the registration center.

[0015] The address determination module is used to determine the destination interface address of the target task message based on the target registration information.

[0016] The message sending module is used to send the target task message to the target business service corresponding to the destination interface address.

[0017] Thirdly, this application provides a computer-readable storage medium storing at least one instruction or at least one program, which is loaded and executed by a processor to implement a message notification method as described in the first aspect.

[0018] Fourthly, this application provides a computer device including a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or at least one program being loaded and executed by the processor to implement a message notification method as described in the first aspect.

[0019] Fifthly, the present invention provides a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a message notification method as described in the first aspect.

[0020] The message notification method, apparatus, device, and storage medium provided in this application have the following technical advantages:

[0021] The solution provided in this application introduces a distributed message queue, where publishers store messages separately according to message topics. Subscribers only need to subscribe to the message topic to consume data without needing to know the physical address of the data, which can solve the problem of functional coupling between task scheduling and message notification, and between task scheduling and business services. The solution provided in this application can improve the message sending success rate and enhance the reliability of message notifications through message retries and alarm measures. The solution provided in this application improves the processing efficiency of server operation by setting up a registry center to provide visualized dynamic configuration and management.

[0022] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0023] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram illustrating the implementation environment of a message notification method provided in an embodiment of this application;

[0025] Figure 2 This is a flowchart illustrating a message notification method provided in an embodiment of this application;

[0026] Figures 3(1) and 3(2) are schematic diagrams of the task view in the data scheduling system provided in the embodiments of this application;

[0027] Figure 4 This is a schematic diagram of a partition log for a specific topic provided in an embodiment of this application;

[0028] Figure 5 This is a schematic diagram of the structure of a blockchain system provided in an embodiment of this application;

[0029] Figure 6 This is a schematic diagram of a process for obtaining target registration information corresponding to the first identifier data, provided in an embodiment of this application.

[0030] Figure 7 This is a schematic diagram of a process for retrying or issuing an alarm based on the type of reason for the failure to send, provided in an embodiment of this application.

[0031] Figure 8 This is a schematic diagram of a process for retrying a message that failed to be sent, provided in an embodiment of this application.

[0032] Figure 9 This is a schematic diagram of a message notification method based on a scheduled task provided in an embodiment of this application;

[0033] Figure 10 This is a flowchart illustrating a message notification method based on a Kafka message queue provided in an embodiment of this application;

[0034] Figure 11 This is a schematic diagram of a message notification device provided in an embodiment of this application.

[0035] Figure 12 This is a schematic diagram of the hardware structure of a device for implementing a method provided in an embodiment of this application. Detailed Implementation

[0036] This application provides a message notification method, apparatus, medium, and device. The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0037] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0038] To facilitate understanding of the technical solutions and their effects described in the embodiments of this application, the relevant technical terms are explained in the embodiments of this application:

[0039] Kafka is a distributed, partitioned, multi-replica, multi-subscriber distributed logging system (also used as a message middleware system) coordinated by Zookeeper. It is commonly used for Web / Nginx logging, access logging, message services, etc.

[0040] Messaging System: A messaging system is responsible for delivering data from one application to another. Applications only need to focus on the data itself, without needing to know how it's transferred between applications. Distributed messaging relies on reliable message queues to asynchronously deliver messages between client applications and the messaging system. There are two main messaging patterns: point-to-point and publish-subscribe. Most messaging systems use the publish-subscribe pattern. Kafka is an example of a publish-subscribe system. In a publish-subscribe messaging system, messages are persisted to a Topic. Unlike point-to-point messaging systems, consumers can subscribe to one or more Topics. Consumers can consume all the data in a Topic, and the same piece of data can be consumed by multiple consumers. Data is not immediately deleted after consumption. In a publish-subscribe messaging system, the message producer is called the publisher, and the consumer is called the subscriber.

[0041] Broker: A Kafka cluster consists of one or more servers, and these server nodes are called brokers. Brokers store data for topics.

[0042] Topic: Each message published to the Kafka cluster has a category called a Topic. Physically, messages from different Topics are stored separately. Logically, although messages from a single Topic are stored on one or more brokers, users only need to specify the Topic to produce or consume data without worrying about where the data is stored. Logically, a Topic can be considered a queue; each consumption must specify its Topic, which can be simply understood as specifying which queue to put the message into.

[0043] Partition: Data within a Topic is divided into one or more partitions. Each Topic must have at least one partition. Data within each partition is stored using multiple segment files. Segment files consist of a data file ending in "log" and an index file ending in "index". Data within a partition is ordered, but data order is lost between different partitions. If a topic has multiple partitions, the order of data consumption cannot be guaranteed. In scenarios where a strict guarantee of message consumption order is required, the number of partitions should be set to 1.

[0044] Producer: The producer is the data publisher, responsible for publishing messages to Kafka topics. When sending a message, the producer must specify the topic to which it will be sent. Afterward, all consumers subscribed to that topic will receive the message. Once the broker receives the message from the producer, it appends it to the segment file used for appending data. Messages sent by the producer are stored in a partition; the producer can also specify the partition for data storage.

[0045] Consumer: Consumers can read data from brokers and can consume data from multiple topics.

[0046] Coupling refers to the phenomenon where two or more systems or two forms of motion influence each other and even unite through interaction.

[0047] Redis: Remote Dictionary Server, is an open-source, network-enabled, in-memory or persistent, log-structured, key-value database written in ANSI C, and provides application programming interfaces (APIs) in multiple languages.

[0048] Offline computing, also known as "batch processing," refers to static data processing procedures that are performed offline in batches with relatively high latency. Offline computing is suitable for scenarios with low real-time requirements, such as offline reporting and data analysis. Latencies are typically in the minutes or hours range. Most scenarios involve periodically executing a job task, with task cycles ranging from minutes (e.g., statistical analysis every five minutes) to months or years (e.g., executing a task monthly). MapReduce is a familiar offline computing framework, and Spark SQL is also commonly used for offline computing tasks.

[0049] Please see Figure 1 This is a schematic diagram illustrating the implementation environment of a message notification method provided in this application embodiment, such as... Figure 1 As shown, the implementation environment may include at least client 01, server 02, and server 03.

[0050] Specifically, client 01 may include devices such as smartphones, desktop computers, tablets, laptops, digital assistants, smart wearable devices, monitoring devices, and voice interaction devices. It may also include software running on the device, such as web pages provided to users by service providers, or applications provided by those service providers. Specifically, client 01 can be used to configure non-notification and notification tasks in the data scheduling system hosted on server 02, and can be used to configure business service logic in the business system hosted on server 03. Furthermore, client 01 can also be used to add, delete, modify, or query registration information in the registration center of the data scheduling system or message notification system.

[0051] Specifically, servers 02 and 03 can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Servers 02 and 03 may include network communication units, processors, and memory, etc. Terminals and servers can be directly or indirectly connected via wired or wireless communication, which is not limited herein. Specifically, server 02 can be used to run a data scheduling system and generate task messages for notification tasks, while server 03 can run a business system, receive task messages for notification tasks, and execute corresponding subsequent business services. Servers 02 and 03 can communicate with each other via message components.

[0052] This application embodiment can also be implemented using cloud technology. Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. It can also be understood as a general term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies based on cloud computing business models. Cloud technology requires cloud computing as its support. Cloud computing is a computing model that distributes computing tasks across a resource pool composed of a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." Specifically, server 02, server 03, and the database are located in the cloud. Server 02 or server 03 can be a physical machine or a virtualized machine.

[0053] The following describes a message notification method provided in this application. Figure 2This is a flowchart illustrating a message notification method provided in an embodiment of this application. This application provides the operational steps of the method described in the embodiments or flowchart, but based on conventional or non-inventive methods, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server product execution, the method can be executed sequentially according to the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Please refer to... Figure 2 The execution subject of the message notification method provided in this application embodiment can be an inter-system message notification component, device, or message system, etc., and the method may include the following steps:

[0054] S210: Obtain task messages for at least one notification task and send the task messages to a distributed message queue; the distributed message queue includes multiple message queues corresponding to multiple message topics.

[0055] In the field of server data operations, the data volume is large and the data application scenarios are diverse, frequently involving the interaction between data scheduling systems and business systems. This primarily involves the need for the business system to be notified upon completion of a task node in the data scheduling system, triggering the execution of a specific business service. In a data scheduling system, a task is a program or command running in a specific environment, also known as a job. Tasks can be divided into notification-based tasks and non-notification-based tasks. For example, in a data warehouse scheduling system, a task can be a dataset extraction program or a report generation program, etc. Non-notification tasks can include data ingestion tasks, offline computing tasks, and data output tasks. Data ingestion tasks connect various data sources (such as MySQL) to the data warehouse; offline computing tasks are scripts written using Hive (a data warehouse tool based on Hadoop) or Spark SQL (a module of Spark for processing structured data) to process data according to a data computing model; data output tasks move data from the data warehouse to the storage area of ​​the business system, such as MySQL (a relational database management system) or Clickhouse (a columnar database management system for online analytical processing), so that the business system can read the output data; notification tasks are mainly for message notification between the data warehouse and external business systems, so that the external business system knows that the current data computing task has been completed, and the business system can perform corresponding processing after receiving the message. It is understood that in this embodiment, the notification task mainly completes the message notification between systems, and the interaction between the server's internal data scheduling system and external business systems is one application scenario. Furthermore, it can also be applied to message notifications between the data scheduling system and multiple task execution systems within server operations. Multiple tasks in a task flow are executed by multiple task execution systems, and these tasks have dependencies on each other. The data scheduling system needs to know that the current task has been completed before it can publish subsequent tasks to other task execution systems. In this case, the task execution system sends a task execution status message to the data scheduling system to notify it that the corresponding task has been completed.

[0056] Optionally, a visual task configuration method can be used to facilitate the configuration of tasks and task dependencies based on the task view, or to view the execution status of tasks. Figures 3(1) and 3(2) show a task view composed of multiple offline computing tasks, which can contain multiple task flows. The blank-filled graphical nodes are offline computing tasks, and the grid-filled graphical nodes are notification tasks. In a task view, a notification task may be a non-leaf node or a leaf node. That is, a notification task can be an intermediate node or a termination node of a task flow. A task view can contain one or more message notification tasks. The execution of a notification task can be regarded as a producer publishing data.

[0057] Optionally, when configuring the task view, the data format of the task messages can be set. For example, the task message data uses JSON format (JavaScript Object Notation, a lightweight data interchange format), such as:

[0058] {"id":"1",

[0059] "source":"View Notification Task Name"

[0060] };

[0061] Among them, the key-value pair "id-1" can be regarded as the first identifier data of the notification task, and as the index basis for the notification task-related information.

[0062] Optionally, to improve the throughput of the message queue, task messages from multiple notification tasks can be cached, and these task messages can be published to the distributed message queue in batches or micro-batch. Optionally, the distributed message queue includes, but is not limited to, ZeroMQ, Kafka, and JAFKA. The distributed message queue includes multiple message queues corresponding to multiple message topics, and each message queue corresponds to a server entity in the distributed messaging system cluster.

[0063] In one embodiment of this application, the distributed message queue is a Kafka message queue, and sending the task message to the distributed message queue may include the following steps:

[0064] S211: Determine the message subject of the task message.

[0065] Understandably, a message topic represents the category of a message, and producers must specify which message topic (hereinafter referred to as the topic) to send a message to. Furthermore, producers can also specify the partition where the message is stored.

[0066] S213: Send the task message to the message queue corresponding to the message topic of the task message.

[0067] Understandably, a topic can be logically considered as a message queue, with messages for different topics stored in different message queues. Physically, messages are stored as files, with messages for different topics stored in different partition files on different server nodes.

[0068] Figure 4 This shows the partition logs for a single topic. For example... Figure 4 As shown, the topic is divided into three partitions, each a sorted, immutable sequence of records. Newly published messages are appended to the structured log. Physically, the partition files for these three partitions can be stored on different server node brokers.

[0069] In one embodiment of this application, the task message generated by executing the notification task is stored in a blockchain to facilitate subsequent information retrieval and storage. Figure 5 This is a schematic diagram of the structure of a blockchain system provided in an embodiment of this application. For example... Figure 5 As shown, the server can be a node 200 in the distributed system 100, which can be a blockchain system. The blockchain system can be a distributed system formed by multiple nodes connected through network communication. The nodes can form a peer-to-peer (P2P) network. Any form of computer device, such as a server, client 300, or other electronic device, can become a node in the blockchain system by joining the peer-to-peer network. The blockchain includes a series of blocks that are sequentially generated according to their time sequence. Once a new block is added to the blockchain, it will not be removed. The blocks record the data submitted by the nodes in the blockchain system.

[0070] Blockchain is an emerging application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, it's a decentralized database, a chain of data blocks linked using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer. The blockchain underlying platform can include processing modules such as user management, basic services, smart contracts, and operation monitoring. The user management module is responsible for managing the identity information of all blockchain participants, including maintaining public and private key generation (account management), key management, and maintaining the correspondence between user real identities and blockchain addresses (access management). Under authorization, it also monitors and audits transactions of certain real identities and provides risk control rule configuration (risk control audit). The basic service module is deployed on all blockchain node devices to verify the validity of business requests. After consensus is reached on valid requests, they are recorded in storage. For a new business request, the basic service first performs interface adaptation parsing and authentication (interface adaptation), and then encrypts the business information through a consensus algorithm (consensus management). The blockchain process involves several layers of data transmission and processing. The first layer, the blockchain service layer, provides basic capabilities and implementation frameworks for typical applications. Developers can define contract logic using a programming language and publish it to the blockchain (contract registration). The second layer provides blockchain-based application services for business stakeholders. The third layer provides application services based on blockchain solutions for use by business participants.

[0071] S230: Determine the subscribed message topic and retrieve the target task message from the message queue corresponding to the subscribed message topic.

[0072] In this embodiment, message delivery is implemented based on a distributed message queue. The message delivery modes primarily include point-to-point delivery and publish-subscribe delivery. Kafka, for example, is a high-throughput distributed publish-subscribe messaging system. This application preferably uses the publish-subscribe delivery mode to achieve asynchronous message delivery. Those retrieving data from the message queue are generally called consumers. Consumers listen to subscribed message topics and pull data from the corresponding message queues for consumption. The message topics to be subscribed to are pre-determined in the consumer's program or process, and these subscribed message topics are consistent with the message topics set for the task messages used to notify the task during task configuration. Furthermore, based on the message retry mechanism and publish-subscribe message delivery mode provided in this embodiment, consumers can subscribe to message retry topics and / or failure topics in addition to the task message topics for notification tasks.

[0073] Consumers consume messages in two ways: pull and push. Kafka uses the pull method, where the consumer actively pulls messages after establishing a connection with the broker (the message queue server entity). The advantage of this mode is that the consumer can pull and process messages according to its consumption capacity and can control the message consumption progress. Data is pulled by repeatedly calling the `poll()` method, which returns a list of records. Each record contains the message's key-value pair, as well as topic, partition, and offset information.

[0074] In one embodiment of this application, specifically, obtaining the target task message from the message queue corresponding to the subscribed message topic may include the following steps:

[0075] S231: Determine the message topics to subscribe to.

[0076] S233: Get the preset time parameters.

[0077] If the method of actively pulling data is adopted, the server will keep polling when there is no data. Therefore, a time parameter can be set in the consumer's pull method to allow the consumer to block for a period of time during polling and pull the data when it arrives.

[0078] S235: Based on the time parameter, data is retrieved from the message queue corresponding to the subscribed message topic to obtain the target task message.

[0079] Furthermore, data parameters can be set to ensure that data is retrieved only when the incoming data reaches a certain number of bytes.

[0080] In this embodiment, asynchronous message transmission is achieved based on a distributed message queue and a publish-subscribe messaging pattern, which solves the problem of functional coupling between systems and improves processing efficiency and operational effectiveness.

[0081] S250: Based on the first identifier data in the target task message, obtain the target registration information corresponding to the first identifier data from the registration center. The target registration information is generated based on the registration page of the registration center.

[0082] In this embodiment, the consumer retrieves data and obtains a list of returned records. Each record contains message data content (e.g., in key-value pair format) as well as topic, partition, and offset information. Preferably, a set of key-value pairs in the data content is used as first identifier data, and the registration information corresponding to the notification task or the task message is determined based on the first identifier data.

[0083] This application provides a registration center module, which has a visual registration interface. Developers or operations personnel can pre-configure or query registration information for notification tasks through this interface, achieving visualized configuration and control and improving operational efficiency. The registration center also has a program interface for consumer programs to call to query registration information and send task messages. The registration message represents information such as the external notification method and notification status corresponding to the notification task's task message. The registration information includes the first identifier of the notification task, the address of the requested interface, and the sending status of the notification task's task message. Compared to configuration from the server backend, this method has less impact on server operation. The key data tables comprising the registration information are shown in Table 1.

[0084] Table 1

[0085] 1 R1 0 2 R2 1 ... ... ...

[0086] The Id field can be used to uniquely identify a record; Request represents a request, which can be the address of the target interface to be called, such as http: / / {domain} / method_path?p1=v1&p2=v2; Lock represents a switch, which can take the value 0 or 1, where 0 indicates that the request has been sent and 1 indicates that it has not been sent.

[0087] In one embodiment of this application, specifically, as Figure 6 As shown, obtaining the target registration information corresponding to the first identifier data from the registration center may include the following steps:

[0088] S251: Query the cache server cluster to see if there is target registration information corresponding to the first identifier data; the cache server cluster periodically synchronizes data from the registration center.

[0089] S253: If there is target registration information corresponding to the first identifier data in the cache server cluster, then obtain the target registration information from the cache server cluster.

[0090] S255: If there is no target registration information corresponding to the first identifier data in the cache server cluster, the interface of the registration center is called to query and obtain the target registration information corresponding to the first identifier data.

[0091] The aforementioned high-speed cache server cluster can improve service response speed. When processing time-sensitive task messages, registration information is synchronized to the cache, and then consumers read it. For example, the high-speed cache server cluster can use Redis or Memcache (a distributed high-speed caching system). Redis is an excellent caching server, an in-memory key-value database that supports rich data structures, has a comprehensive set of database operation commands, and operates very quickly, meeting the needs of a caching server. Redis provides single-machine sharded clusters or can be deployed in a distributed manner to build a distributed caching service.

[0092] S270: Determine the destination interface address of the target task message based on the target registration information.

[0093] It is understood that the target registration information is determined by the unique identifier of the target task message. The target registration information records the destination interface address to which the target task message needs to be sent, which can be a URL (Uniform Resource Locator) address. The destination interface address corresponds to the external business service.

[0094] S290: Send the target task message to the target service corresponding to the destination interface address.

[0095] In this embodiment, the target task message is communicated to the corresponding business service, instructing the business system to perform the corresponding operation. For example, after the data is exported, the external business system is notified that it can read the exported data from its own database. If the destination interface address is a URL, the data content in the target task message can be sent to the business service corresponding to the URL address by calling the HttpClient service. The business service is responsible for the actual business logic.

[0096] In this embodiment, a distributed message queue and a publish-subscribe messaging pattern are used, allowing for the design of message retry or alarm mechanisms to ensure message delivery. Specifically, when a message in the subscribed main topic fails to be consumed, the consumer will not process the message in a blocking manner. Instead, a copy of the message can be written to a retry topic for other consumers to consume. If a message in the retry topic fails to be sent after a specified number of retries according to a preset strategy, the copy is written to the failed topic. Messages in the failed topic are then manually processed, thus achieving non-blocking retry logic and ensuring the delivery of task messages.

[0097] In one embodiment of this application, specifically, sending the target task message to the target service corresponding to the target interface address may include the following steps:

[0098] S291: If the target task message cannot be successfully sent, determine the type of reason for the failure to send.

[0099] In one embodiment of this application, message processing may fail, some of which are due to persistent issues such as database constraint failures or invalid message formats. Retrying message processing can be an effective solution in such cases. When message consumption in the subscribed main topic fails, it is handled according to the type of consumption failure, which is divided into three types:

[0100] Type 1: Runtime exceptions, database persistence exceptions, network timeout exceptions, etc.;

[0101] Type 2: Business processing failure (recoverable), business request for retry, etc.;

[0102] Type 3: Invalid parameters or data, etc.

[0103] S293: Retry or issue an alarm based on the reason for the failure to send.

[0104] In one embodiment of this application, such as Figure 7As shown, for type 1, an alarm will be generated if the message fails to be sent, triggering the system to automatically terminate consumption and requiring the user to manually restart consumption. For type 2, if the message fails to be sent, the failed task message will be written to the retry topic for re-consumption. For type 3, if the message fails to be sent, an alarm will be generated and the message will be written to the failure topic. The user will then manually handle the messages in the failure topic upon receiving the alarm. The main topic stores the primary task messages, accounting for 99.95% of the message volume; the retry topic stores messages that need to be retried, accounting for less than 0.05% of the message volume; and the failure topic stores messages that failed to be sent, accounting for less than 0.05% of the message volume.

[0105] Specifically, when the target task message needs to be written to the retry topic for re-consumption, such as... Figure 8 As shown, step S293 may include the following steps:

[0106] S2931: Generate a copy of the target task message based on the target task message.

[0107] S2933: Send a copy of the target task message to the message queue corresponding to the preset message retry topic.

[0108] S2935: Based on a preset retry strategy, read the target task message from the message queue corresponding to the message retry topic and resend the target task message.

[0109] Specifically, the preset retry strategy can be expressed as follows: Messages in the retry topic are consumed according to an incremental delay period; if consumption fails, the message is re-added to the retry topic; a maximum of n retries are allowed; if one of the n retries is successful, no further retries are allowed; if more than n retries occur, the message is added to the failure topic. For example... Figure 7 As shown, the incrementing period can be 5 seconds, 10 seconds, or 20 seconds, and n takes the value of 3.

[0110] In one embodiment of this application, after sending the target task message to the target service corresponding to the destination interface address, the method may further include:

[0111] The system notifies the registry center that the message has been sent, so that the registry center can update the target registration information.

[0112] Specifically, after the target task message is successfully sent, the attribute representing the message sending status in the target registration information is modified, such as changing the Lock value from 1 to 0 (0 indicates that it has been sent, and 1 indicates that it has not been sent).

[0113] Figure 9 This is a schematic diagram of a specific message notification method based on scheduled tasks, in such a case... Figure 9 Under the message notification architecture shown, the main steps include:

[0114] 1. Write the business service logic;

[0115] 2. Write and configure non-notification tasks in the task view;

[0116] 3. Configure notification tasks in the task view;

[0117] 4. Configure scheduled tasks;

[0118] 5. Notify the task node to write to the status table. The status table can be as shown in the following table:

[0119] Table 2

[0120] 1 R1 0 2 R2 1 ... ... ...

[0121] Where: Id is unique and identifies a record; R2 is the request to be invoked (the format can be http: / / {domain} / method_path?p1=v1&p2=v2); the flag bit indicates the execution status of the notification task (1 can indicate pending execution, 0 indicates already executed);

[0122] 6. Scheduled task processor processing:

[0123] (1) Periodically poll and query the status table, corresponding to Figure 9 Step 6;

[0124] (2) Retrieve the records with flag 1, corresponding to Figure 9 Step 7;

[0125] (3) Send the corresponding request from the record. Figure 9 Steps 8 and 9;

[0126] (4) Modify the corresponding record flag position in the status table to 0, corresponding to Figure 9 Step 10: The modified status table can be shown in Table 3 below:

[0127] Table 3

[0128] 1 R1 0 2 R2 0 ... ... ...

[0129] 7. Repeat Figure 9 Steps (1) to (10) in the process.

[0130] As can be seen, message notification methods based on scheduled tasks are prone to functional coupling issues, do not support message failure retries, and cannot provide visual configuration and control.

[0131] Figure 10This is a flowchart illustrating a message notification method based on a Kafka message queue provided in this application. Based on the method embodiments provided in this application, a new message notification mechanism is designed by introducing a Kafka message queue on top of an offline computing data scheduling platform, resolving the problem of functional coupling; it also supports retry and alarm mechanisms, improving reliability; and it provides dynamic configuration and visualization capabilities, enhancing the efficiency of data operation and processing in the server operation field. This improves the service quality of server operators and provides support for stable business operation, meeting customer needs.

[0132] like Figure 10 As shown, the message queue used is Kafka, primarily used to store message data for notification tasks, which can be in JSON format. The Kafka Consumer is the consumer program, whose main function is to consume messages by listening to a specified Kafka topic. An embodiment of this application also provides an example of the consumer program, the content of which can be as follows:

[0133]

[0134]

[0135]

[0136] As can be seen, the solution provided in this application introduces a distributed message queue, where publishers' messages are stored separately according to message topics. Subscribers only need to subscribe to the message topic to consume data without needing to care about the physical address of the data, which can solve the problem of functional coupling between task scheduling and message notification, and between task scheduling and business services. The solution provided in this application can improve the message sending success rate and enhance the reliability of message notifications through message retries and alarm measures. The solution provided in this application improves the processing efficiency of server operation by setting up a registry center to provide visualized dynamic configuration and management, facilitates intelligent operation, and reduces operating costs.

[0137] This application embodiment also provides a message notification device 1100, such as... Figure 11 As shown, the device may include:

[0138] The acquisition and sending module 1110 is used to acquire task messages for at least one notification task and send the task messages to a distributed message queue; the distributed message queue includes multiple message queues corresponding to multiple message topics.

[0139] The second acquisition module 1120 is used to determine the subscribed message topic and acquire the target task message from the message queue corresponding to the subscribed message topic;

[0140] The third acquisition module 1130 is used to acquire target registration information corresponding to the first identifier data from the registration center based on the first identifier data in the target task message, wherein the target registration information is generated based on the registration page of the registration center;

[0141] Address determination module 1140 is used to determine the destination interface address of the target task message based on the target registration information;

[0142] The message sending module 1150 is used to send the target task message to the target business service corresponding to the destination interface address.

[0143] In one embodiment of this application, the acquisition and transmission module 1110 may include:

[0144] The first topic determination unit is used to determine the message topic of the task message;

[0145] The sending unit is used to send the task message to the message queue corresponding to the message topic of the task message.

[0146] In one embodiment of this application, the second acquisition module 1120 may include:

[0147] The second topic determination unit is used to determine the subscribed message topics;

[0148] The time parameter acquisition unit is used to acquire preset time parameters;

[0149] The data retrieval unit is used to retrieve data from the message queue corresponding to the subscribed message topic based on the time parameter to obtain the target task message.

[0150] In one embodiment of this application, the third acquisition module 1130 may include:

[0151] The cache acquisition unit is used to acquire the target registration information from the cache server cluster if the target registration information corresponding to the first identifier data exists in the cache server cluster; the cache server cluster periodically synchronizes data from the registration center.

[0152] The registration center acquisition unit is used to query the registration center interface to obtain the target registration information corresponding to the first identifier data if there is no target registration information corresponding to the first identifier data in the cache server cluster.

[0153] In one embodiment of this application, the message sending module 1150 may include:

[0154] The cause determination unit is used to determine the type of cause for failure to send the target task message if it fails to be sent successfully.

[0155] The retry alarm unit is used to retry or issue an alarm based on the type of reason for the failure to send the message successfully.

[0156] In another embodiment of this application, the retry alarm unit may further include:

[0157] The copy generation subunit is used to generate a copy of the target task message based on the target task message;

[0158] The re-storage subunit is used to send a copy of the target task message to the message queue corresponding to the preset message retry topic.

[0159] The retry subunit is used to read the target task message from the message queue corresponding to the message retry topic and resend the target task message based on a preset retry strategy.

[0160] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0161] This application provides a computer device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement a message notification method as provided in the above method embodiments.

[0162] Figure 12 A schematic diagram of the hardware structure of a device for implementing a message notification method provided in an embodiment of this application is shown. This device may constitute or include the apparatus or system provided in the embodiment of this application. Figure 12As shown, device 12 may include one or more processors 1202 (shown as 1202a, 1202b, ..., 1202n in the figure) 1202 (processor 1202 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 1204 for storing data, and a transmission device 1206 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 12 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, device 12 may also include a... Figure 12 The more or fewer components shown, or having the same Figure 12 The different configurations shown.

[0163] It should be noted that the aforementioned one or more processors 1202 and / or other data processing circuitry are generally referred to herein as "data processing circuitry". This data processing circuitry may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be wholly or partially integrated into any other element within device 12 (or mobile device). As involved in the embodiments of this application, this data processing circuitry serves as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0164] The memory 1204 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method described in the embodiments of this application. The processor 1202 executes various functional applications and data processing by running the software programs and modules stored in the memory 1204, thereby implementing the aforementioned message notification method. The memory 1204 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1204 may further include memory remotely located relative to the processor 1202, and these remote memories can be connected to the device 12 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0165] The transmission device 1206 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of device 12. In one example, the transmission device 1206 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 1206 may be a radio frequency (RF) module for wireless communication with the Internet.

[0166] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows a user to interact with the user interface of device 12 (or a mobile device).

[0167] This application also provides a computer-readable storage medium, which can be disposed in a server to store at least one instruction or at least one program related to implementing a message notification method in the method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the message notification method provided in the above method embodiment.

[0168] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0169] This invention also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the message notification methods provided in the various optional embodiments described above.

[0170] As can be seen from the embodiments of the message notification method, apparatus, medium, and device provided in this application above:

[0171] The solution provided in this application introduces a distributed message queue, where publishers store messages separately according to message topics. Subscribers only need to subscribe to the message topic to consume data without needing to know the physical address of the data, which can solve the problem of functional coupling between task scheduling and message notification, and between task scheduling and business services. The solution provided in this application can improve the message sending success rate and enhance the reliability of message notifications through message retries and alarm measures. The solution provided in this application improves the processing efficiency of server operation by setting up a registry center to provide visualized dynamic configuration and management.

[0172] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0173] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device, equipment, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0174] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0175] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A message notification method, characterized in that, The method includes: Obtain task messages for at least one notification task and send the task messages to a distributed message queue; the distributed message queue includes multiple message queues corresponding to multiple message topics; Determine the message topic to be subscribed to, and retrieve the target task message from the message queue corresponding to the subscribed message topic; Based on the first identifier data in the target task message, target registration information corresponding to the first identifier data is obtained from the registration center, and the target registration information is generated based on the registration page of the registration center; The destination interface address of the target task message is determined based on the target registration information; Send the target task message to the target service corresponding to the target interface address; The system notifies the registration center that the message has been sent, so that the registration center can update the target registration information. The target registration information generated by the registration center also records the message sending status of the target task message.

2. The method according to claim 1, characterized in that, Sending the task message to the distributed message queue includes: Determine the message subject of the task message; The task message is sent to the message queue corresponding to the message topic of the task message.

3. The method according to claim 1, characterized in that, The step of determining the subscribed message topic and retrieving the target task message from the message queue corresponding to the subscribed message topic includes: Specify the message topics to subscribe to; Get the preset time parameters; Based on the time parameter, data is retrieved from the message queue corresponding to the subscribed message topic to obtain the target task message.

4. The method according to claim 1, characterized in that, The step of obtaining target registration information corresponding to the first identifier data from the registration center based on the first identifier data in the target task message includes: If target registration information corresponding to the first identifier data exists in the cache server cluster, the target registration information is obtained from the cache server cluster; the cache server cluster periodically synchronizes data from the registration center. If no target registration information corresponding to the first identifier data exists in the cache server cluster, the interface of the registration center is called to query and obtain the target registration information corresponding to the first identifier data.

5. The method according to claim 1, characterized in that, Sending the target task message to the target service corresponding to the destination interface address includes: If the target task message cannot be sent successfully, determine the type of reason for the failure to send; Retry or issue an alert based on the reason for the failure to send.

6. The method according to claim 5, characterized in that, The retrying based on the cause type includes: Generate a copy of the target task message based on the target task message; Send a copy of the target task message to the message queue corresponding to the preset message retry topic; Based on a preset retry strategy, the target task message is read from the message queue corresponding to the message retry topic and the target task message is resent.

7. A message notification device, characterized in that, The device includes: The acquisition and sending module is used to acquire task messages for at least one notification task and send the task messages to a distributed message queue; the distributed message queue includes multiple message queues corresponding to multiple message topics; The second acquisition module is used to determine the subscribed message topic and acquire the target task message from the message queue corresponding to the subscribed message topic; The third acquisition module is used to query the registration center to obtain the target registration information corresponding to the first identifier data based on the first identifier data in the target task message. The target registration information is generated based on the registration page of the registration center. The address determination module is used to determine the destination interface address of the target task message based on the target registration information. The message sending module is used to send the target task message to the target business service corresponding to the destination interface address; The update notification module is used to notify the registration center that the message has been sent, so that the registration center can update the target registration information. The target registration information generated by the registration center also records the message sending status of the target task message.

8. The apparatus according to claim 7, characterized in that, The acquisition and transmission module includes: The first topic determination unit is used to determine the message topic of the task message; The sending unit is used to send the task message to the message queue corresponding to the message topic of the task message.

9. The apparatus according to claim 7, characterized in that, The second acquisition module includes: The second topic determination unit is used to determine the subscribed message topics; The time parameter acquisition unit is used to acquire preset time parameters; The data retrieval unit is used to retrieve data from the message queue corresponding to the subscribed message topic based on the time parameter to obtain the target task message.

10. The apparatus according to claim 7, characterized in that, The third acquisition module includes: The cache acquisition unit is used to acquire the target registration information from the cache server cluster if the target registration information corresponding to the first identifier data exists in the cache server cluster; the cache server cluster periodically synchronizes data from the registration center. The registration center acquisition unit is used to query the registration center interface to obtain the target registration information corresponding to the first identifier data if there is no target registration information corresponding to the first identifier data in the cache server cluster.

11. The apparatus according to claim 7, characterized in that, The message sending module includes: The cause determination unit is used to determine the type of cause for failure to send the target task message if it fails to be sent successfully. The retry alarm unit is used to retry or issue an alarm based on the type of reason for the failure to send the message successfully.

12. The apparatus according to claim 11, characterized in that, The retry alarm unit also includes: The copy generation subunit is used to generate a copy of the target task message based on the target task message; The re-storage subunit is used to send a copy of the target task message to the message queue corresponding to the preset message retry topic. The retry subunit is used to read the target task message from the message queue corresponding to the message retry topic and resend the target task message based on a preset retry strategy.

13. A computer-readable machine storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to implement a message notification method as described in any one of claims 1 to 6.

14. A computer device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or at least one program being loaded and executed by the processor to implement a message notification method as described in any one of claims 1 to 6.

Images