Message re-throwing method and device, electronic equipment, storage medium and program product

Abstract

The present disclosure provides a message re-throwing method and device, electronic equipment, storage medium and program product, and relates to the technical field of computers. The present disclosure obtains a first key-value message by traversing an embedded database; the value of the first key-value message includes a delivery message, and the key of the first key-value message includes a first delivery timestamp and a first delivery number; and the first key-value message is delivered to a message middleware according to the first delivery timestamp; in response to a delivery failure, the first delivery number is updated to a second delivery number; the second delivery timestamp is determined according to the second delivery number; the second key-value message is generated according to the second delivery timestamp, the second delivery number and the delivery message; the second key-value message is stored in the embedded database as a new first key-value message, and the first key-value message that fails to be delivered is deleted from the embedded database. The present disclosure can quickly re-throw a message, improve the timeliness of a business, and ensure that the performance of database work is not affected by message delivery failure.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to a message re-throwing method and device, electronic equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] RocketMQ (message middleware) is a distributed message queue system for processing message transmission. A delivery device encapsulates file data as a message and sends it to the message middleware for parallel processing of data by a downstream receiving device.

[0003] In the related art, in order to improve the reliability of delivery, when delivery is abnormal, the message is written into a database or other persistent storage, and then a timing task is used to compensate for delivery. However, this scheme can be used when a small amount of message delivery fails when the message middleware does not fail. However, when the message middleware fails and is not available as a whole, a large number of messages will fail to be delivered, and the delivery failure rate is high. When millions of messages are inserted into the database at the same time, it is slow, resulting in a long time interval for re-delivery. The downstream receiving device cannot receive messages for a long time, affecting the timeliness of the business, and the database resources are occupied for a long time during the entire process, causing the overall performance of the database to decline.

[0004] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] The present disclosure provides a message re-throwing method and device, electronic equipment, storage medium and program product to solve the problems of high message delivery failure rate, poor business timeliness and overall decline in database performance.

[0006] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a message re-throw method is provided, comprising: traversing an embedded database to obtain a first key-value message, a value of the first key-value message comprising a delivery message, a key of the first key-value message comprising a first delivery timestamp and a first delivery number, the delivery message being a message that fails to be delivered to a message middleware; delivering the first key-value message to the message middleware according to the first delivery timestamp; in response to a delivery success, deleting the first key-value message from the embedded database; in response to a delivery failure, updating the first delivery number to a second delivery number; determining a second delivery timestamp according to the second delivery number; generating a second key-value message according to the second delivery timestamp, the second delivery number and the delivery message; storing the second key-value message as a new first key-value message in the embedded database, and deleting the first key-value message that fails to be delivered from the embedded database.

[0008] According to another aspect of the present disclosure, a message re-throw device is also provided, comprising: a traversal module configured to traverse an embedded database to obtain a first key-value message; a value of the first key-value message comprising a delivery message, a key of the first key-value message comprising a first delivery timestamp and a first delivery number, the delivery message being a message that fails to be delivered to a message middleware; a delivery module configured to deliver the first key-value message to the message middleware according to the first delivery timestamp; a response module configured to, in response to a delivery success, delete the first key-value message from the embedded database; in response to a delivery failure, update the first delivery number to a second delivery number; determine a second delivery timestamp according to the second delivery number; generate a second key-value message according to the second delivery timestamp, the second delivery number and the delivery message; store the second key-value message as a new first key-value message in the embedded database, and delete the first key-value message that fails to be delivered from the embedded database.

[0009] According to another aspect of the present disclosure, an electronic device is also provided, comprising: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute any of the above-mentioned message re-throw methods via execution of the executable instructions.

[0010] According to another aspect of the present disclosure, a computer readable storage medium is also provided, having a computer program stored thereon, the computer program being executed by a processor to implement any of the above-mentioned message re-throw methods.

[0011] According to another aspect of the present disclosure, a computer program product is also provided, comprising: a computer program or instructions, the computer program or instructions being executed by a processor to implement any of the above-mentioned message re-throw methods.

[0012] In the embodiment of the present disclosure, a first key-value message is obtained by traversing the embedded database; the value of the first key-value message includes a delivery message, and the key of the first key-value message includes a first delivery timestamp and a first delivery number; the delivery message is a message that fails to be delivered to the message middleware; and the first key-value message is delivered to the message middleware according to the first delivery timestamp; in response to successful delivery, the first key-value message is deleted from the embedded database; in response to failed delivery, the first delivery number is updated to a second delivery number; a second delivery timestamp is determined according to the second delivery number; a second key-value message is generated according to the second delivery timestamp, the second delivery number and the delivery message; the second key-value message is stored in the embedded database as a new first key-value message, and the first key-value message that fails to be delivered is deleted from the embedded database. When the message middleware is unavailable due to failure, a large number of delivery messages that fail to be delivered can be quickly and persistently stored in the embedded database. The embedded database can convert random modification of the delivery message into sequential writing, so that millions of messages can be quickly written into the embedded database, and the embedded database can be quickly queried, thereby realizing quick re-delivery of the messages and improving the timeliness of the business. By storing the delivery messages that fail to be delivered in the embedded database, the database resources of the message middleware are not occupied, the performance of the database is not affected by the delivery failure of the messages, and thus the database performance is improved. In addition, re-delivery of the messages according to the delivery timestamp determined according to the delivery number (such as the second delivery timestamp) can solve the problem of invalid delivery and reduce the failure rate of the message re-delivery.

[0013] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0014] The drawings incorporated into the specification and constituting a part of the specification show embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure. It is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained from these drawings without creative labor for those skilled in the art.

[0015] Figure 1 A flowchart showing a process of compensating for delivery by a timing task in the related art is shown.

[0016] Figure 2 A schematic diagram showing the structure of a message re-delivery system in an embodiment of the present disclosure is shown.

[0017] Figure 3 A flowchart showing a message re-delivery method in an embodiment of the present disclosure is shown.

[0018] Figure 4 A flowchart showing a message re-delivery method in another embodiment of the present disclosure is shown.

[0019] Figure 5 A flow chart of a message re-throw method in another embodiment of the present disclosure is shown.

[0020] Figure 6 A flow chart of a message re-throw method in another embodiment of the present disclosure is shown.

[0021] Figure 7 A flow chart of a message re-throw method in another embodiment of the present disclosure is shown.

[0022] Figure 8 A schematic diagram of a message re-throw device in an embodiment of the present disclosure is shown.

[0023] Figure 9 A structural block diagram of an electronic device in an embodiment of the present disclosure is shown.

[0024] Figure 10 A schematic diagram of a computer readable storage medium provided in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0025] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the implementations set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the inventive concept to those skilled in the art. Like reference numerals refer to like elements throughout the description.

[0026] Moreover, the drawings are not necessarily to scale. Like reference numerals designate like parts throughout the specification. Some of the blocks in the diagrams can be functional blocks that can implement software or can be implemented in software, or can be implemented in one or more hardware modules or integrated circuits, or can be implemented in different network and / or processor devices and / or microcontroller devices.

[0027] For ease of understanding, several terms related to the present disclosure are explained as follows:

[0028] Message: a unit of data transmitted between two electronic devices. For example, the electronic device can be a computer.

[0029] Producer: a party that sends a message. The producer is responsible for creating a message and then sending it to the message middleware.

[0030] Consumer: responsible for subscribing to a topic in the message middleware, and pulling messages from the subscribed topic.

[0031] RocketMQ: a distributed message middleware. It has low latency, high reliability, scalability, and ease of use. Producers write messages to queues, and consumers pull messages from queues for business logic.

[0032] RocksDB: RocksDB is an LSM-tree (Log-Structured Merge-Tree) architecture engine developed by Facebook based on LevelDB, which provides key-value storage and read-write functions. The key-value pairs written by the user are first written to the WAL (Write Ahead Log) on the disk, and then written to the SkipList in memory (this part of the structure is also called MemTable). The LSM-tree engine converts the user's random modification (insertion) into sequential writing to the WAL file, so it has higher write throughput than B-tree storage engines.

[0033] Thread: the corresponding English name is "thread", and its function is to execute a specific task in an application program, such as a program, a function, etc.

[0034] Thread suspension: suspend thread execution.

[0035] Thread wake-up: resume thread execution.

[0036] The following embodiments take the collection night batch system as an example for illustration, but the present disclosure is not limited thereto.

[0037] In the collection night batch system, it is necessary to parse the integrated file on the delivery device (such as the integrated file pushed by the upstream billing system) and encapsulate the file data as a message to send to the message middleware for parallel processing of data by other nodes (receiving devices) downstream. For example, in the collection night batch system, it is necessary to parse the contact, customer, and contract integrated files pushed by the upstream billing system, and encapsulate the file data as a message object to send to RocketMQ for parallel processing of data by downstream receiving devices. Since the data volume is in the tens of millions, delivery to the message middleware often reports delivery exceptions.

[0038] In related technologies, in order to solve the problem of delivery reliability, the message can be written to a database or other persistent storage when the delivery is abnormal, and then a timing task is used to compensate for the delivery. As shown in Figure 1 The specific steps of the scheme include the following S101-S105.

[0039] S101, parse the integrated file data.

[0040] S102, the producer creates a message and delivers the message to the message middleware. For example, the producer encapsulates the file data as a message and delivers the message to the RocketMQ Broker service end.

[0041] S103, the message middleware returns whether the message is successfully delivered. If not, S104 is executed. If yes, the process ends.

[0042] S104, write the message that fails to be delivered into a database. For example, write into a database table A.

[0043] S105, start the message re-delivery service. It is to be noted that the message re-delivery service is an independent application (independent process) that reads the data in the database at a fixed time. If data is read, the data is encapsulated as a message and sent to the message middleware.

[0044] The above scheme can be used when a small amount of messages fail to be delivered when RocketMQ is not faulty. However, when RocketMQ is faulty and the whole system is unavailable, a large amount of messages fail to be delivered. The instantaneous writing pressure of the database is very large. In terms of speed, it takes a long time to write into the database. A large amount of message writing into the database may cause the database to be unavailable. After the recovery of RocketMQ, the message that fails to be delivered needs to be quickly re-delivered to the RocketMQ service end, otherwise the timeliness of the downstream consumption is affected.

[0045] The inventors have found that when RocketMQ is faulty and the whole system is unavailable, a large amount of messages fail to be delivered. The simultaneous insertion of millions of messages into the database is slow, which causes the re-delivery time interval to be too long. The downstream consumer cannot consume the message for a long time, which affects the business timeliness. In the whole process, the database resources are occupied for a long time, which causes the overall performance of the database to decline. In addition, the message that fails to be delivered is concentratedly delivered to the message middleware, which easily causes a large amount of message re-delivery to be invalid and the failure rate of message delivery to be high.

[0046] The specific implementation of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0047] Based on this, the message re-throw method and device, electronic equipment, storage medium and program product provided by the embodiments of the present disclosure can be applied to the scenario where the message delivery fails and needs to be re-delivered. For example, the RocksDB is used to deliver the failed message of the integrated file to the message middleware for delayed retry delivery in the collection night batch system. The specific application scenario is not limited by the present disclosure. When the message middleware is unavailable due to failure, a large number of delivery messages that fail to be delivered can be quickly persisted and stored in the embedded database. The embedded database can convert the random modification of the delivery message into sequential writing, so that the embedded database can quickly write the delivery message to the embedded database. In addition, the embedded database can be quickly queried from the embedded database, so as to quickly re-throw the message and improve the timeliness of the business. By storing the failed message in the embedded database, the database resources of the message middleware are not occupied, and the performance of the database work is not affected by the message delivery failure, thereby improving the database performance. In addition, the re-throw of the message according to the delivery timestamp (such as the second delivery timestamp) determined according to the delivery times can solve the problem of invalid delivery and reduce the failure rate of message re-throw.

[0048] It should be noted that the embodiments of the present disclosure and the technical features in the embodiments can be combined with each other without conflict.

[0049] The specific implementation of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0050] Figure 2 A schematic diagram of the structure of a message re-throw system in an embodiment of the present disclosure is shown. The system can apply the message re-throw method or message re-throw device in various embodiments of the present disclosure.

[0051] As shown in Figure 2 The system architecture 200 can include a delivery device 201, a message middleware 202, a receiving device 203, an intermediate device 204 and a storage device 205. The delivery device 201 is used to deliver messages to the message middleware 202, the receiving device 203 is used to obtain messages from the message middleware 202, the intermediate device 204 can store the failed messages in the form of key-value pairs (such as second key-value messages) in the embedded database thereon, and can also re-deliver the messages stored in the embedded database to the message middleware 202, and the storage device 205 is provided with a database for providing the message middleware 202 with a database function. For example, the storage device 205 is used to store the data on the message middleware 202 and manage the structured data.

[0052] In the embodiments of the present disclosure, the embedded database and the database are independently deployed and deployed on different devices (the intermediate device and the storage device), and do not occupy hard disk resources of each other, so that the messages of delivery failure are stored through the embedded database, the database resources of the message middleware are not occupied, and the performance of the database work is ensured not to be affected by the message delivery failure.

[0053] The delivery device 201, the message middleware 202 and the receiving device 203 are connected through a network for communication, the intermediate device 204 and the message middleware 202 can also be connected through a network for communication, and the storage device 205 and the message middleware 202 can also be connected through a network for communication. The network can be a wired network or a wireless network.

[0054] Optionally, the wireless network or the wired network uses standard communication technology and / or protocol. The network is usually the Internet, but can also be any network, including but not limited to a local area network (Local Area Network, LAN), a metropolitan area network (Metropolitan Area Network, MAN), a wide area network (Wide Area Network, WAN), a mobile, wired or wireless network, a private network or any combination of virtual private networks). In some embodiments, technologies and / or formats including Hyper Text Mark-up Language (Hyper Text Mark-up Language, HTML), Extensible Markup Language (Extensible Markup Language, XML) and the like are used to represent the data exchanged through the network. In addition, all or some links can be encrypted using conventional encryption technologies such as Secure Socket Layer (Secure Socket Layer, SSL), Transport Layer Security (Transport Layer Security, TLS), Virtual Private Network (Virtual Private Network, VPN), Internet Protocol Security (Internet Protocol Security, IPsec) and the like. In other embodiments, custom and / or dedicated data communication technologies can be used instead of or in addition to the above data communication technologies.

[0055] The receiving device 203 is responsible for subscribing to the topic in the message middleware 202, and pulling the message from the subscribed topic. The receiving device 203 can be various electronic devices, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, a wearable device, an augmented reality device, a virtual reality device, and the like.

[0056] Optionally, the application clients installed in different receiving devices 203 are the same, or are clients of the same type of application based on different operating systems. Depending on the terminal platform, the specific form of the application client can also be different, for example, the application client can be a mobile phone client, a PC client, etc.

[0057] The delivery device 201 is a party that sends a message. It can be responsible for creating a message and then sending it to the message middleware 202, and the receiving device 203 receives the message from the message middleware 202 and consumes it. It should be noted that the delivery device 201 can send the message that fails to be delivered to the message middleware 202 and the return state of the delivery failure (fault in the following) to the intermediate device 204 after the delivery of the message fails.

[0058] The delivery device 201 can be a server or an electronic device that provides various services. For example, the delivery device 201 can be a background management server that provides support for the operation of the receiving device 203. The background management server can send the message that fails to be delivered to the message middleware 202 to the intermediate device 204, and the intermediate device 204 writes the delivery message in the form of a key-value pair to RocksDB, and re-delivers the message stored in RocksDB to the message middleware 202.

[0059] The intermediate device 204 and the storage device 205 can be servers that provide various services, the intermediate device 204 is installed with an embedded database using an LSM-tree storage engine, and the storage device 205 is installed with a database that provides regular operations, which is not used to store re-delivered data.

[0060] Optionally, the server can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms.

[0061] Those skilled in the art can know that, Figure 1 The number of delivery devices, message middleware, receiving devices, intermediate devices, and storage devices in the above-mentioned system is only illustrative, and any number of delivery devices, message middleware, receiving devices, intermediate devices, and storage devices can be provided according to actual needs. The embodiments of the present disclosure do not limit this.

[0062] The present example embodiment will be described in detail below with reference to the accompanying drawings and examples.

[0063] The message re-throw method provided in the embodiments of the present disclosure can be executed by any electronic device with computing processing capability, and can also be executed by the intermediate device.

[0064] Figure 3 The message re-throw method flowchart in an embodiment of the present disclosure is shown in FIG. 1. Figure 3 As shown, the message re-throw method provided in the embodiments of the present disclosure includes the following S301-S303 when applied to the intermediate device.

[0065] S301, traverse the embedded database to obtain a first key-value message, the value of the first key-value message including a delivery message, the key of the first key-value message including a first delivery timestamp and a first delivery number, and the delivery message being a message that fails to be delivered to a message middleware.

[0066] The embedded database is a Key (key) Value (value) storage engine based on the LSM-tree data structure. The LSM-tree engine has higher write throughput than the B-tree storage engine because it converts the user's random modification into sequential writing of the WAL file, supports high-throughput writing and fast range query, and even if ten million data is written, RocksDB and reading can be completed in a very short time. The first key-value message in the embodiments of the present disclosure is a key-value pair, which is stored in the embedded database.

[0067] In the embodiments of the present disclosure, the first delivery timestamp is used to indicate the time of re-delivering the message. The embodiments of the present disclosure do not limit how the first delivery timestamp is represented. For example, the first delivery timestamp is 1615123200000. The first delivery timestamp represents time in milliseconds.

[0068] In the embodiments of the present disclosure, the first delivery number is used to indicate the total number of times the same message is re-delivered to the message middleware. For example, the delivery device delivers a message to the message middleware, and the message delivery fails. Since the delivery device is the first time to deliver the message to the message middleware, the first delivery number is 0. For another example, the delivery device fails to deliver a message to the message middleware, and the intermediate device re-delivers the message to the message middleware, and then the first delivery number is updated to 1.

[0069] S302, deliver the first key-value message to the message middleware according to the first delivery timestamp.

[0070] In the embodiments of the present disclosure, when the timestamp on the intermediate device is greater than or equal to the first delivery timestamp, the first key-value message is delivered to the message middleware. For example, when the timestamp on the intermediate device is equal to the first delivery timestamp, the delivery message in the first key-value message is delivered to the message middleware.

[0071] S303, in response to the delivery success, deleting the first key-value message from the in- database; in response to the delivery failure, updating the first delivery times to the second delivery times; determining the second delivery timestamp according to the second delivery times; generating the second key-value message according to the second delivery timestamp, the second delivery times and the delivery message; storing the second key-value message as the new first key-value message in the in-database, and deleting the first key-value message which fails to deliver from the in-database.

[0072] In the embodiments of the present disclosure, if the delivery is successful, the first key-value message in the in-database needs to be deleted, so as to avoid message repeated delivery. If the delivery fails, the delivery time (such as the second delivery timestamp) needs to be updated, the delivery message is re-written in the in-database, and the first key-value message originally stored in the in-database is deleted.

[0073] In an embodiment, in order to improve the accuracy of the delivery message delivery, the key of the key-value message can include the message identifier in addition to the delivery timestamp and the delivery times. The message identifier is an identifier for indicating the uniqueness of the message, for example, the message identifier is uuid (universal unique identifier).

[0074] It should be noted that, for the convenience of description, the first key-value message and the second key-value message are collectively referred to as key-value messages. The first delivery timestamp and the second delivery timestamp are collectively referred to as delivery timestamps. The first delivery times and the second delivery times are collectively referred to as delivery times.

[0075] In another embodiment, when the message fails to deliver to the message middleware, the return state is "fault", and the intermediate device can obtain the delivery message after receiving the fault. For example, the intermediate device obtains the delivery message from the delivery device. For another example, when the message successfully delivers to the message middleware, the return state is "correct", and the intermediate device does not need to obtain the delivery message after receiving the correct, which indicates that the delivery is normal.

[0076] It should be noted that as long as the message fails to deliver to the message middleware, the message is written in the in-database in the form of key-value pair, which is not affected by the re-delivery processing. It can also be said that writing the delivery failed message in the in-database in the form of key-value pair and re-delivering the delivery message to the message middleware are two relatively independent threads.

[0077] For example, when the delivery device delivers the message to the message middleware, the message delivery fails, the delivery times of the message is set to 1, the delivery timestamp is determined according to the delivery times 1, the delivery message is written in the in-database as the value of the key-value pair, and the delivery times and the delivery timestamp are written in the in-database as the key of the key-value pair.

[0078] For another example, when the intermediate device delivers a delivery message to the message middleware (the delivery message has a delivery number of 1 in the key-value pair), the delivery message fails to be delivered, the delivery number of the delivery message is updated from 1 to 2, the delivery timestamp is re-determined according to the delivery number 2, and the delivery message, the delivery number and the delivery timestamp are re-written in the form of the key-value pair in the embedded database.

[0079] It should be noted that the present disclosure S301 to S303 is a re-delivery processing thread. When the embedded database has the first key-value message, it indicates that the message delivery fails and the re-delivery processing thread needs to be awakened. The embodiment of the present disclosure can traverse the embedded database at a predetermined time, for example, once every 5 seconds. If the embedded database has the first key-value message and the re-delivery processing is not started, the re-delivery processing is started.

[0080] In another embodiment, the message re-delivery method can further include: in the case that the re-delivery processing thread is started, the re-delivery processing thread is executed cyclically.

[0081] In the embodiment of the present disclosure, the re-delivery processing thread is executed, and even if a new key-value pair is written into the embedded database, the re-delivery processing is not restarted. The embodiment of the present disclosure can maintain single-thread operation (only one re-delivery processing thread is started), and can improve the stability of message re-delivery.

[0082] In another embodiment, the message re-delivery method can further include: when the number of the first key-value messages in the embedded database is 0, the re-delivery processing thread is stopped.

[0083] The embodiment of the present disclosure is the stopping condition of the re-delivery processing thread. When the re-delivery processing thread executes the traversal of the embedded database, the number of the first key-value messages obtained by the traversal is 0, and the re-delivery processing thread is stopped. That is, the re-delivery processing thread is suspended.

[0084] In the embodiment of the present disclosure, when the number of the first key-value messages in the embedded database is 0, it indicates that there is no message to be re-delivered in the embedded database, and the re-delivery processing thread is suspended and waits for data. Suspending the re-delivery processing thread can reduce the computing power, thereby saving the cost and prolonging the service life of the hardware.

[0085] When the message middleware is unavailable due to a failure, a large number of delivery messages that fail to be delivered can be quickly persisted into an embedded database. The embedded database can convert random modification of the delivery messages into sequential writing, so that the embedded database can quickly write and query millions of messages, thereby realizing quick re-delivery of the messages and improving the timeliness of the business. By storing the delivery messages that fail to be delivered in the embedded database, the database resources of the message middleware are not occupied, and the performance of the database is not affected by the delivery failure of the messages, thereby improving the performance of the database. In addition, re-delivery of the messages according to the delivery timestamp determined according to the delivery times can solve the problem of invalid delivery and reduce the failure rate of re-delivery of the messages.

[0086] The following describes how to determine the delivery timestamp of the messages according to the delivery times through several example embodiments.

[0087] In an example embodiment, determining the second delivery timestamp according to the second delivery times can include the following steps A1 and A2.

[0088] Step A1: determining the delay duration according to the second delivery times.

[0089] In the embodiment of the present disclosure, the greater the second delivery times, the greater the corresponding delay duration. As to how to determine the delay duration according to the second delivery times, the embodiment of the present disclosure is not specifically limited. For example, a preset relationship correspondence table is queried to determine the delay duration corresponding to the second delivery times, and the relationship correspondence table is used to indicate the corresponding relationship between the second delivery times and the delay duration. The relationship correspondence table is shown in Table 1.

[0090] Table 1 Relationship Correspondence Table

[0091]

[0092] For another example, a linear graph is used to indicate the corresponding relationship between the second delivery times and the delay duration. For another example, the second delivery times are input into a model (such as a neural network model) to obtain the corresponding delay duration.

[0093] In an embodiment, the first delivery times and the second delivery times are in an exponential relationship with the delay duration.

[0094] In the embodiment of the present disclosure, the exponential relationship indicates that the relationship between the two variables (delivery times and delay duration) is an exponential function relationship. When the delivery times are in an exponential relationship with the delay duration, the delay duration will show exponential growth with the increase of the delivery times, rather than linear or other forms of growth. This indicates that the increase speed of the delay duration is getting faster and faster, rather than increasing at a fixed rate, thereby avoiding multiple invalid re-delivery and improving the success rate of re-delivery.

[0095] In the embodiments of the present disclosure, in the case that the type of the message middleware fault cannot be determined, the message middleware fault recovery time has uncertainty, and the recovery time is several minutes for the fast, and several hours or several days for the slow, so the exponential backoff delay delivery strategy is adopted, which can maximize the success rate of retry delivery and the system resource utilization.

[0096] Step A2, determining the second delivery timestamp according to the delay duration.

[0097] In the embodiments of the present disclosure, the second delivery timestamp can be the sum of the delivery failure time and the delay duration. For example, if the message is delivered on 2024-05-28 16:00, the delay duration (for example, 2 hours) is added to the time 2024-05-28 16:00 to obtain 2024-05-28 18:00, and the time (2024-05-28 18:00) is converted into a timestamp (i.e., the second delivery timestamp).

[0098] It should be noted that the message middleware fault recovery time can be several minutes to several hours. In the case that the recovery time of the message middleware fault is uncertain, the retry delivery strategy of exponential backoff delay is adopted, and once the fault is recovered, the delivery will be completed in a short time, which guarantees the timeliness of the business.

[0099] In the embodiments of the present disclosure, the delay duration is determined by the delivery times, and the second delivery timestamp is determined by the delay duration, so that the success rate of re-delivery and the system resource utilization can be maximized.

[0100] In another exemplary embodiment, determining the delay duration according to the second delivery times can include the following steps B1 to B2.

[0101] Step B1, obtaining the fault type of the message middleware.

[0102] In the embodiments of the present disclosure, the fault types of the message middleware are different, and the recovery times are also different. For example, the faults that can be recovered in a short time include transient network fault, node restart, transient resource depletion, and configuration error, etc. The faults that can be recovered in a long time include hardware fault, serious network fault, system upgrade failure, etc. How to obtain the fault type of the message middleware is not limited in the embodiments of the present disclosure. For example, the fault type is obtained by detecting the fault through a detection device. For another example, the message middleware reports the fault type.

[0103] Step B2, determining the delay duration according to the fault type and the second delivery times. In the case that the second delivery times are the same, the longer the fault recovery time of the fault type is, the longer the determined delay duration is.

[0104] Exemplarily, the fault types can at least include a first fault type and a second fault type, wherein the first fault type corresponds to a recovery time of the message middleware fault being less than a recovery time of the message middleware fault corresponding to the second fault type; in the case of the same second delivery number, the delay duration determined according to the first fault type is less than the delay duration determined according to the second fault type.

[0105] In the embodiments of the present disclosure, according to the fault type of the message middleware, the relationship between the second delivery number and the delay duration is determined, for example, for a short-time fault (such as the first fault type), the delay duration is set to be relatively short. For another example, for a long-time fault (such as the second fault type), the delay duration is set to be relatively long, thereby avoiding the situation that the message delivery fails due to too many messages being written into the message middleware instantaneously, improving the timeliness of the receiving device receiving the messages, and also improving the success rate of message delivery.

[0106] Exemplarily, according to the fault type, the recovery time is determined, before the recovery time, the delay duration corresponding to the second delivery number is relatively long, and after the recovery time, the delay duration corresponding to the second delivery number is relatively short. The success rate of message delivery is improved while meeting the timeliness of delivery.

[0107] For example, the message middleware fault is a security event, and it is necessary to perform security checks, vulnerability patches, data recovery, and system reinforcement on the server of the message middleware, and the actual recovery time thereof is about 5 hours. At this time, the recovery time is set to be 4 hours and 55 minutes, the current time is 12:00, and before 16:55, the relationship between the second delivery number and the delay duration is shown in Table 2.

[0108] Table 2 relationship correspondence table

[0109]

[0110] After 16:55, the relationship between the second delivery number and the delay duration is shown in Table 3.

[0111] Table 3 relationship correspondence table

[0112]

[0113] It should be noted that the embodiments of the present disclosure only show part of the delay duration corresponding to the second delivery number, and more delay duration corresponding to the second delivery number can be set according to the actual application scene and application experience. Based on Table 2 and Table 3, it can be seen that before the message middleware fault is recovered, the delay duration of the message is relatively long, which reduces a large number of invalid deliveries, and when the message middleware fault is about to be recovered, the delay duration of the message is relatively short, which can increase the timeliness of re-delivery.

[0114] The second delivery times and the fault types are used to determine the delay duration in the embodiments of the present disclosure, so that the success rate of re-delivery and the system resource utilization can be maximized.

[0115] In still another example embodiment, as shown in Figure 4 The second delivery timestamp is determined according to the second delivery times, including S401-S403.

[0116] S401, an average load utilization of the message middleware in a sliding time window is obtained.

[0117] In the embodiments of the present disclosure, the average load utilization of the message middleware generally refers to the ratio between the load processed by the message middleware system and the maximum processing capacity of the system in the sliding time window.

[0118] S402, in the case where the average load utilization is less than a preset load utilization threshold, the delay duration is determined according to a preset delay duration initial value, a preset delay duration increment and the second delivery times.

[0119] In the embodiments of the present disclosure, the preset load utilization threshold is not limited in the embodiments of the present disclosure. For example, the preset load utilization threshold is any one of 90%, 85%, 80%, 75% and 70%.

[0120] For example, the delay duration is determined according to the preset delay duration initial value, the preset delay duration increment and the second delivery times, which can include: calculating the second delivery times power of the delay duration increment to obtain a first value; calculating the product of the delay duration initial value and the first value to obtain the delay duration.

[0121] The calculation formula of the delay duration is shown in the following formula 1.

[0122] B1 = I x R c (1)

[0123] Wherein, B1 is the delay duration, I is the delay duration initial value, R is the preset delay duration increment, and c is the second delivery times.

[0124] S403, the second delivery timestamp is determined according to the delay duration.

[0125] In the embodiments of the present disclosure, how to determine the second delivery timestamp according to the delay duration has been described in the above example embodiments, which will not be repeated here.

[0126] It should be noted that when the average load utilization is less than the preset load utilization threshold, it means that the message middleware can process the delivery message at a normal rate.

[0127] The disclosure embodiment can further improve the success rate of delivering messages by dynamically adjusting the delay duration of the delivered messages according to the number of delivery times of the delivered messages and the current message middleware load.

[0128] In yet another exemplary embodiment, after obtaining the average load occupancy of the message middleware in the sliding time window, the method provided by the disclosure embodiment can further include the following steps C1 and C2.

[0129] In step C1, when the average load occupancy is greater than or equal to the preset load occupancy threshold, the total number of successful deliveries in the sliding time window is obtained, and the average processing time is obtained according to the total number of successful deliveries in the sliding time window and the total time of the sliding time window. The delay duration is determined according to the preset delay duration initial value, the preset delay duration increment, the second delivery number, the average processing time, and the average load occupancy.

[0130] In the disclosure embodiment, the preset load occupancy threshold is not limited to a specific value. For example, the preset load occupancy threshold can be any one of 90%, 85%, 80%, 75%, and 70%. For another example, the preset load occupancy threshold can be 80%.

[0131] It should be noted that when the average load occupancy is greater than or equal to the preset load occupancy threshold, it can be confirmed that the message middleware is in a high load state, and the delay duration needs to be increased to reduce the number of message delivery failures.

[0132] In the disclosure embodiment, when the average load occupancy is greater than or equal to the preset load occupancy threshold, the delay duration needs to be adjusted to slow down the delivery speed of the delivered messages, thereby increasing the success rate of delivery.

[0133] In an embodiment, determining the delay duration according to the preset delay duration initial value, the preset delay duration increment, the second delivery number, the average processing time, and the average load occupancy includes steps C11 to C15.

[0134] In step C11, the second delivery number power of the delay duration increment is calculated to obtain a second value.

[0135] In step C12, the product of the delay duration initial value and the second value is calculated to obtain a third value.

[0136] In the disclosure embodiment, the third value can be calculated by formula 1. That is, the third value in the disclosure embodiment is B1 in formula 1.

[0137] In step C13, the ratio of the average processing time to the total time of the sliding time window is calculated to obtain a fourth value.

[0138] In the embodiments of the present disclosure, the average processing time is a ratio of a total time of the sliding time window to a total number of times of successful delivery of the delivered messages.

[0139] In step C14, a ratio of the average load occupancy rate to the preset load occupancy threshold is calculated to obtain a fifth value.

[0140] In the embodiments of the present disclosure, the average load occupancy rate is an average system load in the sliding time window. The embodiments of the present disclosure do not make specific limitations on how to calculate the average load occupancy rate. For example, the total time of the sliding time window is 5 minutes, and the load occupancy rates obtained at 1 minute, 2 minutes, 3 minutes, 4 minutes and 5 minutes are 80%, 90%, 70%, 80% and 80% respectively. The average load occupancy rate is obtained by calculating the average value, and the calculated average load occupancy rate is 80%.

[0141] In step C15, the delay duration is determined based on the third value, the fourth value and the fifth value.

[0142] In the embodiments of the present disclosure, one or more of the third value, the fourth value and the fifth value increases, and the value of the delay duration also increases.

[0143] In an embodiment, the delay duration is determined by the following formula 2.

[0144]

[0145] Wherein, B2 is the delay duration, I is the initial value of the delay duration, R is the preset delay duration increment, c is the second delivery number, T 均 is the average processing time, T 总 is the total time of the sliding time window, F 均 is the average load occupancy rate, F 阈 is the preset load occupancy threshold.

[0146] It should be noted that I x R c is the third value, is the fourth value, is the fifth value.

[0147] In another embodiment, the delay duration is determined by the following formula 3.

[0148]

[0149] Wherein, B2 is the delay duration, I is the initial value of the delay duration, R is the preset delay duration increment, c is the second delivery number, T 均 is the average processing time, T 总 is the total time of the sliding time window, F 均 is the average load occupancy rate, F 阈W is a preset load occupancy threshold value, and W is a regulation coefficient.

[0150] The embodiment of the present disclosure adjusts the delay duration when the average load occupancy rate is greater than or equal to the preset load occupancy threshold value, and the greater the load occupancy rate, the longer the delay time, thereby increasing the success rate of delivery.

[0151] Step C2, determining a second delivery timestamp according to the delay duration.

[0152] In the embodiment of the present disclosure, how to determine the second delivery timestamp according to the delay duration has been described in the above exemplary embodiment, and will not be repeated here.

[0153] The embodiment of the present disclosure can further improve the success rate of delivering messages by dynamically adjusting the delay duration of delivering messages according to the message re-delivery situation (delivery times) and the current message middleware load situation.

[0154] In another exemplary embodiment, determining the delay duration based on the third value, the fourth value and the fifth value can include steps D1 to D4.

[0155] Step D1, obtaining a candidate delay duration based on the third value, the fourth value and the fifth value.

[0156] In the embodiment of the present disclosure, the candidate delay duration can be calculated by formula 2 or formula 3. That is, the candidate delay duration in the embodiment of the present disclosure is B2 in formula 2 or formula 3.

[0157] Step D2, obtaining a preset delay duration threshold.

[0158] In the embodiment of the present disclosure, the delay duration threshold is the maximum delay duration. The maximum delay duration can be set according to the actual application scenario and specific application experience.

[0159] Step D3, in the case where the candidate delay duration is less than or equal to the delay duration threshold, the candidate delay duration is determined as the delay duration.

[0160] Step D4, in the case where the candidate delay duration is greater than the delay duration threshold, the delay duration threshold is determined as the delay duration.

[0161] In order to avoid the delay duration from increasing indefinitely, the embodiment of the present disclosure compares the candidate delay duration with the delay duration threshold, and selects the smaller value as the delay duration, thereby ensuring the timeliness of delivering messages.

[0162] The above describes how to determine the delivery timestamp, and the following describes how to perform message re-delivery and how to handle the re-delivery result.

[0163] In an exemplary embodiment, delivering a first key-value message to a message middleware based on a first delivery timestamp may include: splitting the key of the first key-value message to obtain a first delivery timestamp and a first delivery count; and delivering the message to the message middleware if the current time is greater than or equal to the first delivery timestamp and the first delivery count is less than a preset delivery count.

[0164] In this embodiment of the disclosure, the method for splitting the key in the first key-value message is not limited. For example, the key can be split using the string processing function Split to obtain the first delivery timestamp and the first delivery number of the delivery message.

[0165] For example, the embedded database is traversed to obtain multiple first key-value messages. The key of one first key-value message is extracted from the multiple first key-value messages. The key is [first delivery timestamp, first delivery count]. The key is split by the string processing function Split("|") to obtain the first delivery timestamp and the first delivery count.

[0166] In this embodiment of the disclosure, the current time can be the current timestamp of the intermediate device, or it can be the 24-hour time corresponding to the current timestamp of the intermediate device.

[0167] In this embodiment, there are two conditions for re-delivering a message (i.e., a time condition and a number condition). If the current time is greater than the first delivery timestamp, it means the message delivery time has arrived, i.e., the time condition is met. If the first delivery count is less than a preset delivery count, it means the number condition is met. When both conditions are met, the message is re-delivered to the message middleware. This embodiment does not limit the specific value of the preset delivery count. For example, the preset delivery count can be any value between 5 and 20. For instance, the preset delivery count can be any value among 5, 6, 7, 8, 9, and 10.

[0168] It should be noted that when the first delivery count is greater than or equal to the preset delivery count, no delivery will be made.

[0169] In this embodiment of the disclosure, when the current time is greater than or equal to the first delivery timestamp and the first delivery number is less than the preset delivery number, the message is re-delivered to the message middleware, thereby maximizing the success rate of re-delivery and the utilization rate of system resources.

[0170] In another exemplary embodiment, such as Figure 5 As shown, after delivering the first key-value message to the message middleware according to the first delivery timestamp, the method may further include S501 to S503.

[0171] S501 determines whether the message re-delivery to the message middleware was successful.

[0172] In the embodiments of the present disclosure, how to determine whether the message re-delivery is successful can be determined according to the return state of the message delivery. If the return state is “correct”, the message re-delivery is successful. If the return state is “fault”, the message re-delivery fails. It should be noted that the present disclosure does not limit how to identify the return state by using a certain symbol, as long as the symbol can indicate the success or failure of the delivery.

[0173] S502, in response to the delivery success, deleting the first key-value message corresponding to the message from the embedded database.

[0174] In the embodiments of the present disclosure, in order to avoid the repeated delivery, the first key-value message corresponding to the message of the successful delivery is deleted from the embedded database.

[0175] S503, in response to the delivery failure, updating the first delivery number to the second delivery number; determining the second delivery timestamp according to the second delivery number; generating the second key-value message according to the second delivery timestamp, the second delivery number and the delivery message; storing the second key-value message as a new first key-value message in the embedded database, and deleting the first key-value message of the failed delivery from the embedded database.

[0176] In the embodiments of the present disclosure, with the update of the delivery number, the delivery number gradually increases. For example, after the message delivery of the first key-value message from the intermediate device to the message middleware fails, the value of the delivery number corresponding to the message in the first key-value message is increased by 1, and the second delivery timestamp is determined according to the delivery number after the increase by 1. The second delivery timestamp, the second delivery number and the delivery message are written in the embedded database in the form of key-value pair to obtain the second key-value message, and the first key-value message is deleted from the embedded database, that is, the original first key-value message corresponding to the delivery message in the embedded database is deleted.

[0177] For example, if the current time is greater than the first delivery timestamp and the first delivery number is less than the preset delivery number (such as the maximum delivery number), the delivery message is delivered to the message middleware. If the return state of the message middleware is delivery success, the first key-value message corresponding to the message is deleted from the embedded database, which can be deleted by calling the API delete method. If the return state is failure, the delivery number of the current message is increased by 1, and the second delivery timestamp is recalculated according to the delivery number. The delivery message is copied, a new KEY is used, and the RocksDB is re-written, wherein the format of the KEY is [first delivery timestamp, first delivery number], and finally the original first key-value message is deleted. Then the next first key-value message is taken until all the first key-value messages are re-delivered.

[0178] It should be noted that the number of the first key-value messages traversed from the embedded database can be 0, 1 or multiple. For example, when the number of the first key-value messages traversed from the embedded database for the first time is multiple, the multiple first key-value messages can be regarded as a key-value data set D0. When the key-value data set D0 is processed, the first key-value message in the RocksDB is traversed again to obtain a next batch of key-value data set D1. If the key-value data set D1 is not an empty set, the re-delivery is continued to be performed. If the key-value data set D1 is an empty set, the re-delivery processing thread is suspended, and the data is waited.

[0179] When the re-delivery is unsuccessful, the first delivery number and the first delivery timestamp are updated, and the message whose delivery is unsuccessful is written in the embedded database in the form of a key-value pair, so that the timeliness of delivery of other first key-value messages in the embedded database is ensured, and the delivery of the message is ensured not to be repeated, so that the accuracy of the business service is improved.

[0180] In order to further improve the non-repetition of the message delivered to the message middleware, the disclosure provides the following exemplary embodiments.

[0181] In an exemplary embodiment, a message identifier is added in the key of the first key-value message. The message identifier is an identifier used to indicate the uniqueness of the message.

[0182] For example, the embedded database is traversed to obtain multiple first key-value messages. The KEY (key) of a first key-value message is taken out from the multiple first key-value messages. The KEY is [first delivery timestamp, first delivery number, uuid]. The KEY is split by using a string processing function Split(“|”) to obtain the first delivery timestamp, the first delivery number and the uuid. The uuid is the message identifier.

[0183] When the message re-delivery is performed, in addition to the time condition and the number condition, it is also necessary to judge whether the message identifier of the delivery message is the same as the message identifier of the previously delivered message. If they are different, the delivery message is re-delivered according to the first delivery timestamp. If they are the same, the delivery message is not delivered, and the first key-value message corresponding to the delivery message is deleted from the embedded database.

[0184] By judging whether the message identifier is the same, the disclosure can ensure that the delivered message is not repeated, the delivery message is not repeated, and the receiving device will not consume the message repeatedly, so as to further improve the accuracy of the business service.

[0185] In order to solve the problem that the embedded database occupies a large amount of disk memory due to the failure of the delivery message, as shown in Figure 6 The message re-delivery method can also include S601 to S603.

[0186] S601, traverse the embedded database at a preset cleaning time to obtain a first key-value message.

[0187] In the embodiments of the present disclosure, the preset cleaning time can be the time when the scheduled task polling (also referred to as the cleaning data scheduled task) starts, for example, the embedded database is traversed at 12:00 every night to obtain the first key-value message. For another example, the embedded database is traversed at 12:00 every Monday night to obtain the first key-value message. The present disclosure is not limited to this, and can be set according to the actual application scene and specific application experience.

[0188] It should be noted that the number of the first key-value messages traversed from the embedded database at the preset cleaning time is 0, which indicates that the disk memory does not need to be cleaned.

[0189] S602, obtain a delivery failure time of the first key-value message.

[0190] In the embodiments of the present disclosure, the delivery failure time can be represented by a delivery failure timestamp, and can also be represented by a 24-hour time. The delivery failure time is the time or timestamp when the delivery message fails to be delivered for the last time. For example, the delivery message fails to be delivered at 2024-05-28 16:00, assuming that this is the last delivery and the delivery fails, the timestamp converted from 2024-05-28 16:00 is the delivery failure timestamp.

[0191] S603, if the difference between the current time and the delivery failure time is greater than a preset delivery duration, delete the first key-value message from the embedded database.

[0192] In the embodiments of the present disclosure, the preset delivery duration can be set as needed. For example, the preset delivery duration is 7 days, and for example, the delivery failure time is 2024-05-28 16:00, and a scheduled task polling is set to delete the first key-value message after 7 days from the delivery failure time.

[0193] In the embodiments of the present disclosure, by deleting the delivery message that fails to be delivered for a long time, the problem that the data occupies the disk for a long time can be avoided, so that the work efficiency can be improved and the energy consumption can be reduced.

[0194] In summary, the present disclosure re-delivers the delivery message to the message middleware based on RocksDB, when the message middleware is unavailable due to failure, can quickly persist the large data volume message to the disk, and if the message middleware cannot be recovered due to short-time failure, supports custom delay delivery message. After the message middleware is removed, the failed message can be delivered to the message middleware in a short time, compared with the MySQL (open source relational database management system) storage delivery failed message, the re-delivery time can be greatly reduced, the database resources are not occupied, and the reliable message delivery problem is solved.

[0195] The present disclosure is described below by complete examples.

[0196] As Figure 7 shown, in combination Figure 7 , the specific implementation manner of the example is described according to the flow direction of the data flow, it should be noted that the flow direction of the data flow has been marked in Figure 7 .

[0197] (1) The delivery device parses the integrated file data.

[0198] (2) The delivery device encapsulates the file data into a message.

[0199] (3) The delivery device delivers the message to the message middleware.

[0200] (4) According to the return state of the message middleware, it is judged whether the message is successfully delivered. If the message is successfully delivered, it is ended. If the message fails to be delivered, step (5) is executed.

[0201] (5) In response to the failure of the message to be delivered to the message middleware, the delivery message, the first delivery number and the message identifier are determined, the first delivery timestamp is determined according to the first delivery number, the delivery message is taken as the value of the first key-value message, the first delivery timestamp, the first delivery number and the message identifier are taken as the key of the first key-value message, and the first key-value message is written into the embedded database.

[0202] (6) In the case that the first key-value message exists in the embedded database and the re-delivery processing is not started, the re-delivery processing thread is woken up. Exemplarily, the re-delivery processing thread can be woken up by calling the thread start method, the embedded database is accessed by polling (for example, the disk directory of the embedded database is accessed by polling), if there is no first key-value message, the re-delivery processing thread is suspended by calling the thread Thread. wait method.

[0203] (7) The intermediate device prepares to re-deliver the message to the message middleware. That is, the re-delivery processing is prepared to be executed in a loop, which can include: traversing the embedded database to obtain the first key-value message, the value of the first key-value message including the delivery message, the key of the first key-value message including the first delivery timestamp, the first delivery number and the message identifier; according to the first delivery timestamp, the delivery message is re-delivered to the message middleware. Exemplarily, the first key-value message in the embedded database is obtained by calling the Thread. notify method.

[0204] (8) Traverse the embedded database to obtain a first key-value message, and determine whether the first key-value message exists in the embedded database. If the first key-value message does not exist, the re-throw processing thread is suspended. If the first key-value message exists, step (9) is performed. For example, by calling the API seekToFirst method, the instance data (first key-value message) in the RocksDB is traversed from the beginning.

[0205] (9) Redeliver the message to the message middleware, and determine whether the first key-value message obtained by traversal is delivered. If the re-throw is complete, the embedded database is continuously traversed, and the re-throw processing is performed.

[0206] (10) Determine whether the message is successfully re-thrown. If the re-throw is successful, step (11) is performed, and if the re-throw fails, step (12) is performed.

[0207] (11) Delete the first key-value message corresponding to the message from the embedded database.

[0208] (12) Update the first delivery number and the message identifier, and determine the first delivery timestamp according to the updated first delivery number. The first delivery timestamp, the first delivery number, and the message identifier are written into the embedded database as keys, and the message is written into the embedded database as a value. The original first key-value message is deleted from the embedded database.

[0209] (13) At a preset cleaning time, a cleaning data timing task is started, and the first key-value message cleaning work is performed. For details, refer to S601 to S603.

[0210] In the embodiment of the present disclosure, the total delivery time of the failed message can be reduced, and in the case that the message middleware fails and cannot be recovered for a short time, the message is delivered with an exponential delay, which does not cause more pressure on the message middleware. Compared with the related art which uses disk resources to replace database resources, the system risk is reduced, and the reliable message delivery problem is solved.

[0211] In the embodiment of the present disclosure, when the message middleware fails and is not available as a whole, the RocksDB is equivalent to caching all messages, which indirectly solves the problem of message loss caused by the failure of the message middleware.

[0212] Based on the same inventive concept, the present disclosure also provides a message re-throw device, as described in the following embodiments. Since the principle of solving the problem by the device embodiment is similar to the above-mentioned method embodiment, the implementation of the device embodiment can be referred to the implementation of the above-mentioned method embodiment, and the repeated parts will not be described here.

[0213] Figure 8 A schematic diagram of a message re-throw device in an embodiment of the present disclosure is shown as follows: Figure 8As shown, the apparatus includes a traversal module 801, a delivery module 802, and a response module 803. The traversal module 801 can be configured to traverse an embedded database to obtain a first key-value message; a value of the first key-value message includes a delivery message, and a key of the first key-value message includes a first delivery timestamp and a first delivery count; the delivery message is a message that fails to be delivered to a message middleware. The delivery module 802 can be configured to deliver the first key-value message to the message middleware according to the first delivery timestamp. The response module 803 can be configured to, in response to a delivery success, delete the first key-value message from the embedded database; in response to a delivery failure, update the first delivery count to a second delivery count; determine a second delivery timestamp according to the second delivery count; generate a second key-value message according to the second delivery timestamp, the second delivery count, and the delivery message; and store the second key-value message as a new first key-value message in the embedded database, and delete the first key-value message that fails to be delivered from the embedded database.

[0214] In an embodiment, the response module 803 can be further configured to determine a delay duration according to the second delivery count.

[0215] In an embodiment, the first delivery count and the second delivery count are in an exponential relationship with the delay duration. The response module 803 can be further configured to obtain a failure type of the message middleware; determine the delay duration according to the failure type and the second delivery count; and wherein, in a case where the second delivery count is the same, the longer the failure recovery time of the failure type, the longer the delay duration determined.

[0216] In an embodiment, the response module 803 can be further configured to obtain an average load occupancy rate of the message middleware within a sliding time window; in a case where the average load occupancy rate is less than a preset load occupancy threshold, determine the delay duration according to a preset delay duration initial value, a preset delay duration increment, and the second delivery count; and determine the second delivery timestamp according to the delay duration.

[0217] In an embodiment, the response module 803 can be further configured to calculate the second delivery count power of the delay duration increment to obtain a first value; and calculate a product of the delay duration initial value and the first value to obtain the delay duration.

[0218] In an embodiment, after obtaining the average load occupancy rate of the message middleware within the sliding time window, the response module 803 can be further configured to, in a case where the average load occupancy rate is greater than or equal to the preset load occupancy threshold, obtain a total number of delivery successes within the sliding time window; obtain an average processing time according to the total number of delivery successes within the sliding time window and a total time of the sliding time window; determine the delay duration according to the preset delay duration initial value, the preset delay duration increment, the second delivery count, the average processing time, and the average load occupancy rate; and determine the second delivery timestamp according to the delay duration.

[0219] In an embodiment, the response module 803 can also be configured to calculate a second value by raising the second delivery number of the delay duration increment to a power, calculate a third value by multiplying the initial value of the delay duration and the second value, calculate a fourth value by dividing the average processing time by the total time of the sliding time window, calculate a fifth value by dividing the average load occupancy rate by the preset load occupancy threshold, and determine the delay duration based on the third value, the fourth value and the fifth value.

[0220] In an embodiment, the response module 803 can also be configured to obtain a candidate delay duration based on the third value, the fourth value and the fifth value, obtain a preset delay duration threshold, determine the candidate delay duration as the delay duration in a case where the candidate delay duration is less than or equal to the delay duration threshold, and determine the delay duration threshold as the delay duration in a case where the candidate delay duration is greater than the delay duration threshold.

[0221] In an embodiment, the delivery module 802 can also be configured to perform segmentation processing on the key of the first key-value message to obtain a first delivery timestamp and a first delivery number of the delivery message, and re-deliver the delivery message to the message middleware in a case where the current time is greater than or equal to the first delivery timestamp and the first delivery number is less than the preset delivery number.

[0222] When the message middleware is unavailable due to a failure, a large number of delivery messages that fail to be delivered can be quickly and persistently stored in the embedded database. The embedded database can convert random modifications of the delivery messages into sequential writing, so that the embedded database can quickly write millions of messages and quickly query from the embedded database, thereby quickly re-delivering the messages and improving the timeliness of the business. By storing the delivery-failed messages in the embedded database, the database resources of the message middleware are not occupied, and the performance of the database is not affected by the delivery failure, thereby improving the database performance. In addition, re-delivering the messages according to the delivery timestamp determined by the delivery number can solve the problem of invalid delivery and reduce the failure rate of message re-delivery.

[0223] It should be noted that each module in the above apparatus embodiment and the corresponding steps in the method embodiment have the same examples and application scenarios, but are not limited to the contents disclosed in the above method embodiments. It should be noted that the above modules as part of the apparatus can be executed in a computer system such as a group of computer executable instructions.

[0224] Those skilled in the art can understand that each aspect of the present disclosure can be implemented in the form of a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system".

[0225] Based on the same inventive concept, the embodiments of the present disclosure further provide an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the message re-throw method of any one of the above by executing the executable instructions. Since the principle of solving problems of the electronic device embodiments is similar to that of the above method embodiments, the implementation of the electronic device embodiments can be referred to the implementation of the above method embodiments, and the repeated parts will not be described here.

[0226] The electronic device 900 according to this implementation of the present disclosure will be described below with reference to Figure 9 Figure 9 The display electronic device 900 is only one example and should not be taken as limiting the functionality and use of embodiments of the present disclosure.

[0227] As shown in Figure 9 The electronic device 900 is in the form of a general computing device. The components of the electronic device 900 can include, but are not limited to, the above-mentioned at least one processing unit 910, the above-mentioned at least one storage unit 920, and a bus 930 connecting different system components, including the storage unit 920 and the processing unit 910.

[0228] The storage unit stores program codes which can be executed by the processing unit 910, so that the processing unit 910 executes the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of the present specification.

[0229] The storage unit 920 can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 9201 and / or a cache memory unit 9202, and can further include a read-only memory (ROM) 9203.

[0230] The storage unit 920 can further include program / utility 9204 having a set of (at least one) program modules 9205, including but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof can include implementation of a network environment.

[0231] The bus 930 can represent one or more of several types of bus structures, including a storage unit bus or storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of a variety of bus structures.

[0232] ​The electronic device 900 can also communicate with one or more external devices 940 such as a keyboard or pointing device, a Bluetooth device, or a database, and / or one or more devices that enable a user to interact with the electronic device 900 and / or one or more devices (e.g., a router, a modem, a server, etc.) that enable the electronic device 900 to communicate with one or more other computing devices. Such communication can occur via an input / output (I / O) interface 950. Still yet, the electronic device 900 can communicate with one or more networks, such as one or more local area networks (LANs), wide area networks (WANs), and / or the Internet, through a network adapter 960. As depicted, the network adapter 960 communicates with the other components of the electronic device 900 via the bus 930. It should be appreciated that the network adapter 960 and / or the bus 930 can be implemented using one or more types of technology, including, but not limited to, Ethernet, Bluetooth, and / or any other technology.

[0233] From the above description of the embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, or the like) or a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to perform the methods according to the embodiments of the present disclosure.

[0234] In the disclosed example embodiments, a computer-readable storage medium is also provided, which can be a readable signal medium or a readable storage medium. Figure 10 A schematic diagram of a computer-readable storage medium in an embodiment of the present disclosure is shown in FIG. 10. As shown in FIG. 10, the computer-readable storage medium 1000 stores a program product capable of implementing the method described above. Figure 10

[0235] In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing an end device to perform the steps according to various example embodiments of the present disclosure described in the above “DETAILED DESCRIPTION” section of the present specification when the program product is run on the end device.

[0236] ​More specific examples of the computer-readable storage medium in the present disclosure can include but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination of the foregoing.

[0237] In the present disclosure, a computer-readable storage medium can include a data signal carrying the readable program code in a baseband or as part of a carrier wave propagating through the transmission medium. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. A computer-readable medium can also be any medium that can be read by a computer or a device that can be used to store or transport the program code means that can be used by or in connection with the instruction execution system, apparatus, or device.

[0238] Optionally, the program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0239] In specific implementation, the program code for performing the operations of the present disclosure can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, C++, etc., and a conventional procedural programming language such as "C" language or similar programming languages. The program code can be executed entirely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case involving a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, through the Internet by connecting to an Internet service provider).

[0240] Based on the same inventive concept, the present disclosure also provides a computer program product, including a computer program product, including: a computer program or instructions, which, when executed by a processor, implements the message retransmission method of any one of the above method embodiments. Since the principles of the computer program product embodiment for solving problems are similar to those of the above method embodiments, the implementation of the computer program product embodiment can be referred to the implementation of the above method embodiments, and the repeated parts will not be repeated.

[0241] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0242] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0243] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A message retransmission method, characterized in that, include: The embedded database is traversed to obtain the first key-value message. The value of the first key-value message includes a delivery message. The key of the first key-value message includes the first delivery timestamp and the first delivery count. The delivery message is a message indicating that delivery to the message middleware failed. The first key-value message is delivered to the message middleware according to the first delivery timestamp; In response to successful delivery, the first key-value message is deleted from the embedded database; In response to delivery failure, update the first delivery count to the second delivery count; A second delivery timestamp is determined based on the second delivery count, wherein the second delivery timestamp has an exponential relationship with the second delivery count; A second key-value message is generated based on the second delivery timestamp, the second delivery count, and the delivery message; The second key-value message is stored as a new first key-value message in the embedded database, and the first key-value message that failed to be delivered is deleted from the embedded database.

2. The method according to claim 1, characterized in that, Based on the second delivery number, the second delivery timestamp is determined, including: The delay duration is determined based on the second delivery count; The second delivery timestamp is determined based on the delay duration.

3. The method according to claim 2, characterized in that, Both the first delivery count and the second delivery count are exponentially related to the delay duration.

4. The method according to claim 2, characterized in that, The delay duration is determined based on the second delivery count, including: Obtain the fault type of the message middleware; The delay duration is determined based on the fault type and the second delivery count; Where the number of second deliveries is the same, the longer the fault recovery time of the fault type, the longer the determined delay duration.

5. The method according to claim 1, characterized in that, Based on the second delivery number, the second delivery timestamp is determined, including: Obtain the average load occupancy of the message middleware within the sliding time window; If the average load occupancy rate is less than the preset load occupancy threshold, the delay time is determined based on the preset initial delay time value, the preset delay time increment, and the second delivery count. The second delivery timestamp is determined based on the delay duration.

6. The method according to claim 5, characterized in that, The delay duration is determined based on a preset initial delay duration value, a preset delay duration increment, and the second delivery count, including: Calculate the second delivery count raised to the power of the delay duration increment to obtain the first value; The delay duration is obtained by multiplying the initial value of the delay duration by the first value.

7. The method according to claim 5, characterized in that, After obtaining the average load occupancy of the message middleware within the sliding time window, the method further includes: If the average load occupancy rate is greater than or equal to a preset load occupancy threshold, the total number of successful deliveries within the sliding time window is obtained, and the average processing time is obtained based on the total number of successful deliveries within the sliding time window and the total time of the sliding time window. The delay duration is determined based on the preset initial delay duration value, the preset delay duration increment, the second delivery count, the average processing time, and the average load occupancy rate. The second delivery timestamp is determined based on the delay duration.

8. The method according to claim 7, characterized in that, The delay duration is determined based on a preset initial delay duration value, a preset delay duration increment, the second delivery count, the average processing time, and the average load occupancy rate, including: Calculate the second delivery count raised to the power of the delay duration increment to obtain the second value; The third value is obtained by multiplying the initial value of the delay duration by the second value; The ratio of the average processing time to the total time of the sliding time window is calculated to obtain the fourth value; Calculate the ratio of the average load occupancy rate to the preset load occupancy threshold to obtain the fifth value; The delay duration is determined based on the third, fourth, and fifth values.

9. The method according to claim 8, characterized in that, Determining the delay duration based on the third, fourth, and fifth values ​​includes: Based on the third, fourth, and fifth values, candidate delay durations are obtained; Obtain the preset delay duration threshold; If the candidate delay duration is less than or equal to the delay duration threshold, the candidate delay duration is determined as the delay duration. If the candidate delay duration is greater than the delay duration threshold, the delay duration threshold is determined as the delay duration.

10. The method according to claim 1, characterized in that, Delivering the first key-value message to the message middleware according to the first delivery timestamp includes: The key of the first key-value message is segmented to obtain the first delivery timestamp and the first delivery number of the delivery message; If the current time is greater than or equal to the first delivery timestamp and the first delivery count is less than the preset delivery count, the delivery message is delivered to the message middleware.

11. A message retransmission device, characterized in that, include: The traversal module is used to traverse the embedded database to obtain the first key-value message; the value of the first key-value message includes a delivery message, and the key of the first key-value message includes a first delivery timestamp and a first delivery count; the delivery message is a message indicating that delivery to the message middleware failed. The delivery module is used to deliver the first key-value message to the message middleware according to the first delivery timestamp; The response module is used to delete the first key-value message from the embedded database in response to successful delivery; In response to delivery failure, the first delivery count is updated to a second delivery count; based on the second delivery count, a second delivery timestamp is determined, wherein the second delivery timestamp and the second delivery count have an exponential relationship; A second key-value message is generated based on the second delivery timestamp, the second delivery count, and the delivery message; the second key-value message is stored as a new first key-value message in the embedded database, and the first key-value message that failed to be delivered is deleted from the embedded database.

12. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the message retransmission method of any one of claims 1 to 10 by executing the executable instructions.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the message retransmission method according to any one of claims 1 to 10.

14. A computer program product comprising: A computer program or instruction, characterized in that, when executed by a processor, the computer program or instruction implements the message retransmission method according to any one of claims 1 to 10.

Images