Method, apparatus, device, storage medium, and program product for avoiding message loss

Abstract

The method, apparatus, device, storage medium, and program product for avoiding message loss provided by the present disclosure relate to message middleware technology, and include: when the consumption of a message sent by a message middleware fails, obtaining an offset from a database; the offset corresponds to the message for which the client consumption fails; the offset is obtained by the database from the message middleware; sending a first message retrieval instruction to the message middleware according to the offset, where the first message retrieval instruction includes the offset; and receiving the message corresponding to the offset sent by the message middleware. The solution provided by the present disclosure uses a database to record historical offsets. When the message consumption fails, the offset corresponding to the message for which the consumption fails can be obtained from the database, and according to the obtained offset, the message for which the consumption fails can be obtained from the message middleware, so that the message for which the consumption fails can be consumed again.

Description

Technical Field

[0001] The present disclosure relates to message middleware technology, and in particular to a method, apparatus, device, storage medium, and program product for avoiding message loss. Background Art

[0002] Kafka is an open-source stream processing platform developed by the Apache Software Foundation. Kafka is a high-throughput distributed publish-subscribe messaging system that can process all action stream data of consumers in a website.

[0003] In the prior art, in Kafka consumers, there is a very crucial mechanism, namely the offset mechanism. When resuming consumption next time, it is still possible to know where to start consuming. It is like a bookmark in a book. Each time through the bookmark (offset), it is possible to quickly find where to start reading (consuming). Kafka has two submission methods for offset: (1) automatic submission (the default submission method); (2) manual submission (which can flexibly control the offset).

[0004] However, both existing solutions have the problem that after a consumer reports a consumed message and then fails to consume the message, it is impossible to retrieve the offset of the message record that has been submitted, and thus impossible to find the message for which the consumption fails, making the message with consumption failure unable to be consumed again. Summary of the Invention

[0005] The present disclosure provides a method, apparatus, device, storage medium, and program product for avoiding message loss, so as to solve the problem in the prior art that after a consumer reports a consumed message and then fails to consume the message, it is impossible to retrieve the offset of the message record that has been submitted, and thus impossible to find the message for which the consumption fails, making the message with consumption failure unable to be consumed again.

[0006] According to a first aspect of the present disclosure, there is provided a method for avoiding message loss, the method comprising:

[0007] When the consumption of a message sent by a message middleware fails, obtaining an offset from a database; the offset corresponds to the message for which the client consumption fails; the offset is obtained by the database from the message middleware;

[0008] Sending a first message retrieval instruction to the message middleware according to the offset, the first message retrieval instruction including the offset;

[0009] Receiving the message corresponding to the offset sent by the message middleware.

[0010] According to a second aspect of the present disclosure, a method for avoiding message loss is provided. A plurality of offsets are stored in a database;

[0011] The method includes:

[0012] Receiving a message instruction sent by a client;

[0013] Obtaining a current offset from a message middleware, and updating the stored offset according to the current offset.

[0014] According to a third aspect of the present disclosure, a client device for avoiding message loss is provided. The client device includes:

[0015] An offset obtaining unit, configured to obtain an offset from a database when the consumption of a message sent by the received message middleware fails; the offset corresponds to the message for which the client consumption fails; the offset is obtained by the database from the message middleware;

[0016] A message request instruction sending unit, configured to send a first message request instruction to the message middleware according to the offset, where the first message request instruction includes the offset;

[0017] A receiving unit, configured to receive a message corresponding to the offset sent by the message middleware.

[0018] According to a fourth aspect of the present disclosure, a database device for avoiding message loss is provided. A plurality of offsets are stored in the database; the database device includes:

[0019] A receiving unit, configured to receive a message instruction sent by a client;

[0020] An offset updating unit, configured to obtain a current offset from a message middleware, and update the stored offset according to the current offset.

[0021] According to a fifth aspect of the present disclosure, an electronic device is provided, including a memory and a processor; wherein, the memory is configured to store a computer program; the processor is configured to read the computer program stored in the memory, and execute the method for avoiding message loss as described in the first aspect and the second aspect according to the computer program in the memory.

[0022] According to a sixth aspect of the present disclosure, a computer-readable storage medium is provided. A computer-executable instruction is stored in the computer-readable storage medium. When the processor executes the computer-executable instruction, the method for avoiding message loss as described in the first aspect and the second aspect is implemented.

[0023] According to the seventh aspect of the present disclosure, there is provided a computer program product including a computer program which, when executed by a processor, implements the method for avoiding message loss as described in the first aspect and the second aspect.

[0024] According to the eighth aspect of the present application, there is provided a system for avoiding message loss, including a client and a database; the client is configured to execute the method for avoiding message loss as described in the first aspect, and the database is configured to execute the method for avoiding message loss as described in the second aspect.

[0025] The method, device, equipment, storage medium, and program product for avoiding message loss provided by the present disclosure include: when the consumption of a message sent by a message middleware fails, obtaining an offset from a database; the offset corresponds to the message for which the client consumption fails; the offset is obtained by the database from the message middleware; sending a first message request instruction to the message middleware according to the offset, where the first message request instruction includes the offset; and receiving the message corresponding to the offset sent by the message middleware. The solution provided by the present disclosure uses a database to record historical offsets. When the message consumption fails, the offset corresponding to the message for which the consumption fails can be obtained from the database, and according to the obtained offset, the message for which the consumption fails can be obtained from the message middleware, so that the message for which the consumption fails can be consumed again. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic flowchart of the method for avoiding message loss shown in the first exemplary embodiment of the present disclosure;

[0027] Figure 2 It is a schematic flowchart of the method for avoiding message loss shown in the second exemplary embodiment of the present disclosure;

[0028] Figure 3 It is a schematic flowchart of the method for avoiding message loss shown in the third exemplary embodiment of the present disclosure;

[0029] Figure 4 It is a schematic flowchart of the method for avoiding message loss shown in the fourth exemplary embodiment of the present disclosure;

[0030] Figure 5 It is a structural diagram of a client device for avoiding message loss shown in an exemplary embodiment of the present disclosure;

[0031] Figure 6 It is a structural diagram of a client device for avoiding message loss shown in another exemplary embodiment of the present disclosure;

[0032] Figure 7 It is a structural diagram of a database device for avoiding message loss shown in an exemplary embodiment of the present disclosure;

[0033] Figure 8 Structural diagram of an electronic device shown in an exemplary embodiment of the present application. Detailed implementation manners

[0034] Kafka is an open-source stream processing platform developed by the Apache Software Foundation. Kafka is a high-throughput distributed publish-subscribe messaging system that can process all action stream data of consumers in a website. In the prior art, in Kafka consumers, there is a very crucial mechanism, namely the offset mechanism. When resuming consumption next time, it is still possible to know where to start consuming. It is like a bookmark in a book. Each time through the bookmark (offset), it is possible to quickly find where to start reading (consuming). Kafka has two submission methods for offset processing: (1) automatic submission (the default submission method); (2) manual submission (which can flexibly control the offset).

[0035] However, both existing solutions have the problem that after the consumer reports the consumed message, the consumption fails, and it is impossible to retrieve the offset of the already submitted message record, and thus impossible to find the message for which the consumption fails, making the message with consumption failure unable to be consumed again.

[0036] To solve the above technical problems, the solution provided by the present disclosure uses a database to record historical offsets. When the message consumption fails, the offset corresponding to the message with consumption failure can be obtained from the database, and based on the obtained offset, the message with consumption failure can be obtained from the message middleware, so that the message with consumption failure can be consumed again.

[0037] Figure 1 Flow schematic diagram of a method for avoiding message loss shown in a first exemplary embodiment of the present disclosure.

[0038] As Figure 1 shown, the method for avoiding message loss provided in this embodiment includes:

[0039] Step 101, when the consumption of the message sent by the message middleware fails, obtain the offset from the database; the offset corresponds to the message with consumption failure of the client; the offset is obtained by the database from the message middleware.

[0040] Among them, the method provided by the present disclosure can be executed by an electronic device with computing capabilities, such as a computer or other devices. This electronic device can obtain the offset from the database.

[0041] Among them, the message middleware can be Kafka.

[0042] Among them, the offset can be the offset in Kafka. Among them, the offset corresponds one-to-one with the message.

[0043] Among them, the database can be a Remote Dictionary Server (Redis). Redis is an open-source log-type, Key-Value database written in ANSI C language, supporting networking, memory-based or persistent, and providing application programming interfaces (APIs) in multiple languages.

[0044] Among them, the client can be a message consumer, which can read messages from Kafka or obtain offsets from the database.

[0045] Among them, a message refers to the data transmitted between applications. A message can be very simple, such as only containing a text string, or more complex, possibly containing embedded objects.

[0046] Among them, multiple offsets can be stored in the database, such as 2. Optionally, the queue of offsets stored in the database can adopt the first-in-first-out method, that is, the first obtained is the offset with an earlier storage time in the database. For example, if 2 offsets are stored in the database, the message corresponding to the offset with an earlier storage time in the database can be the message last consumed by the client. For example, when the client fails to consume a message last time and wants to obtain the message consumed last time again, it can first obtain the offset with an earlier storage time in the database from the database, and obtain the message from the message middleware according to this offset, and this message is the message that the client failed to consume last time.

[0047] Step 102, send a first message request instruction to the message middleware according to the offset, and the first message request instruction includes the offset.

[0048] Specifically, the client can send a first message request instruction including the offset to the message middleware to request the message corresponding to the offset from the message middleware.

[0049] Among them, after receiving the first message request instruction including the offset, the message middleware can send the message corresponding to the offset to the client.

[0050] Kafka has two submission methods for processing offsets. In the automatic submission method, the client can simultaneously send a first message request instruction including the offset to the message middleware and the database.

[0051] Specifically, after receiving the first message request instruction including the offset, the message middleware can also update the offset.

[0052] Specifically, after the database receives the first message request instruction including the offset, the database can obtain the latest offset from the message middleware, update the offset stored in the database, store the latest offset, and delete the offset with the earliest storage time.

[0053] Step 103: Receive the message corresponding to the offset sent by the message middleware.

[0054] Specifically, the client can receive the message corresponding to the offset sent by the message middleware. Further, the client can consume the message again.

[0055] Specifically, after the client receives the message, it can send a confirmation message to the message middleware, where the confirmation message is used to indicate that the client has received the message distributed by the message middleware.

[0056] Kafka has two submission methods for offset processing. In the manual submission method, the client can send confirmation messages to the message middleware and the database simultaneously. The message middleware can receive the confirmation message and update the offset according to the confirmation message. The database can receive the confirmation message, obtain the latest offset from the message middleware according to the confirmation message, update the stored offset, store the latest offset, and delete the offset with the earliest storage time.

[0057] Optionally, a confirmation message can be sent to the message middleware according to preset information; where the preset message is used to indicate the timing of sending the confirmation message to the message middleware. The preset information is pre-set information. For example, it can be pre-set to send a confirmation message to the message middleware after a specific stage of consuming the message is completed.

[0058] Further, after consuming a message, if there is no message that needs to be obtained and consumed again due to consumption failure, a second message request instruction can also be sent to the message middleware, where the second message request instruction may not include an offset.

[0059] Kafka has two submission methods for offset processing. In the automatic submission method, the client can send a second message request instruction that does not include an offset to the message middleware and the database. The second message request instruction can instruct the message middleware to send the next message to the client and update the offset. After the database receives the second message request instruction, it can obtain the latest offset from the message middleware and update the offset stored in the database, store the latest offset, and delete the offset with the earliest storage time. After the client receives the next message, it can send a confirmation message to the message middleware.

[0060] Kafka has two ways to commit offsets. In the manual commit method, after the client receives the next message, it can send confirmation messages to both the message middleware and the database simultaneously. The message middleware can receive the confirmation message and update the offset according to the confirmation message. The database can receive the confirmation message, obtain the latest offset from the message middleware according to the confirmation message, update the stored offset, store the latest offset, and delete the offset with the earliest storage time.

[0061] Optionally, confirmation messages can be sent to the message middleware according to preset information; among them, the preset message is used to indicate the timing of sending the confirmation message to the message middleware. Among them, the preset information is pre-set information. For example, it can be pre-set to send a confirmation message to the message middleware after a specific stage of consuming the message is completed.

[0062] The method for avoiding message loss provided by the present disclosure includes: when the consumption of the message sent by the received message middleware fails, obtaining the offset from the database; the offset corresponds to the message that the user terminal fails to consume; the offset is obtained by the database from the message middleware; sending a first message request instruction including the offset to the message middleware according to the offset; receiving the message corresponding to the offset sent by the message middleware. The solution provided by the present disclosure uses the database to record historical offsets. When the message consumption fails, the offset corresponding to the message that fails to be consumed can be obtained from the database, and according to the obtained offset, the message that fails to be consumed can be obtained from the message middleware, so that the message that fails to be consumed can be consumed again.

[0063] Figure 2 It is a schematic flowchart of the method for avoiding message loss shown in the second exemplary embodiment of the present disclosure.

[0064] As Figure 2 shown, the method for avoiding message loss provided in this embodiment includes:

[0065] Step 201, when the consumption of the message sent by the received message middleware fails, obtain the offset from the database; the offset corresponds to the message that the client fails to consume; the offset is obtained by the database from the message middleware.

[0066] Among them, the message middleware can be Kafka.

[0067] Among them, the offset can be the offset in Kafka. Among them, the offset corresponds one-to-one with the message.

[0068] Among them, the database can be Redis.

[0069] Among them, the client can be a message consumer, which can read messages from Kafka or obtain offsets from a database.

[0070] Among them, a message refers to data transmitted between applications. A message can be very simple, such as only containing a text string, or it can be more complex and may contain embedded objects.

[0071] Optionally, obtain the offset with an earlier storage time from the database.

[0072] Among them, multiple offsets can be stored in the database, such as 2. Optionally, the queue of offsets stored in the database can adopt the first-in-first-out method, that is, the first obtained is the offset with an earlier storage time in the database. For example, if 2 offsets can be stored in the database, the message corresponding to the offset with an earlier storage time in the database can be the message that the client consumed last time. For example, if the client failed to consume a message last time and wants to obtain the message consumed last time again, it can first obtain the offset with an earlier storage time in the database from the database and obtain the message from the message middleware according to this offset, and this message is the message that the client failed to consume last time.

[0073] Step 202: Send a first message request instruction to the message middleware according to the offset. The first message request instruction includes the offset.

[0074] Specifically, the client can send a first message request instruction including the offset to the message middleware to request the message corresponding to the offset from the message middleware.

[0075] Among them, after receiving the first message request instruction including the offset, the message middleware can send the message corresponding to the offset to the client.

[0076] Optionally, send a first message request instruction to the database. The first message request instruction is used to instruct the database to obtain the latest offset from the message middleware.

[0077] Kafka has two submission methods for offset. In the automatic submission method, the client can send a first message request instruction including the offset to the message middleware and the database at the same time.

[0078] Specifically, after receiving the first message request instruction including the offset, the message middleware can also update the offset.

[0079] Specifically, after receiving the first message request instruction including the offset, the database can obtain the latest offset from the message middleware, update the offset stored in the database, store the latest offset and delete the offset with the earliest storage time.

[0080] Step 203: Receive the message corresponding to the offset sent by the message middleware.

[0081] Specifically, the principle and implementation method of Step 203 are similar to those of Step 103, and will not be elaborated here.

[0082] Specifically, after Step 203, Step 204 can be executed, or Step 206 can be executed.

[0083] Step 204: After consuming a message, send a second message request instruction to the message middleware and the database. The second message request instruction does not include the offset. The second message request instruction is used to instruct the message middleware to send the next message and update the offset. The second message request instruction is used to instruct the database to obtain the latest offset from the message middleware. Receive the next message sent by the message middleware.

[0084] Specifically, after consuming a message, if there is no message that fails to be consumed and needs to be retrieved and consumed again, a second message request instruction can also be sent to the message middleware, where the second message request instruction may not include the offset.

[0085] Kafka has two submission methods for processing offsets. In the automatic submission method, the client can send a second message request instruction that does not include the offset to the message middleware and the database. Among them, the second message request instruction can instruct the message middleware to send the next message to the client and update the offset. After receiving the second message request instruction, the database can obtain the latest offset from the message middleware, update the offset stored in the database, store the latest offset, and delete the offset with the earliest storage time.

[0086] Step 205: Send an acknowledgment message to the message middleware. The acknowledgment message is used to indicate that the client has received the message distributed by the message middleware.

[0087] Specifically, after the client receives the next message, it can send an acknowledgment message to the message middleware. The message middleware can receive the acknowledgment message sent by the client.

[0088] Step 206: After consuming a message, send a second message request instruction to the message middleware. The second message request instruction does not include the offset. The second message request instruction is used to instruct the message middleware to send the next message to the client. Receive the next message sent by the message middleware.

[0089] Specifically, after consuming a message, if there is no message that fails to be consumed and needs to be retrieved and consumed again, a second message request instruction can also be sent to the message middleware, where the second message request instruction may not include the offset.

[0090] Step 207: Send confirmation messages to the message middleware and the database respectively. The confirmation message is used to indicate that the client has received the message distributed by the message middleware. The confirmation message is used to instruct the message middleware to update the offset. The confirmation message is used to instruct the database to obtain the latest offset from the message middleware.

[0091] Kafka has two ways to handle offsets. In the manual submission method, after the client receives the next message, it can send confirmation messages to the message middleware and the database simultaneously. The message middleware can receive the confirmation message and update the offset according to the confirmation message. The database can receive the confirmation message, obtain the latest offset from the message middleware according to the confirmation message, update the stored offset, store the latest offset, and delete the offset with the earliest storage time.

[0092] Optionally, send a confirmation message to the message middleware according to preset information. The preset message is used to indicate the timing of sending the confirmation message to the message middleware. The confirmation message is used to instruct the message middleware to update the offset.

[0093] The preset information is pre-set information. For example, it can be pre-set to send a confirmation message to the message middleware after a specific stage of consuming the message is completed.

[0094] Figure 3 It is a schematic flowchart of the method for avoiding message loss shown in the third exemplary embodiment of the present disclosure.

[0095] As Figure 3 shown, for the method for avoiding message loss provided in this embodiment, multiple offsets are stored in the database. The method includes:

[0096] Step 301: Receive the message instruction sent by the client.

[0097] The database can be a Remote Dictionary Server (Redis). Redis is an open-source key-value database written in ANSI C language, supporting networking, memory-based or persistent logging, and providing application programming interfaces (APIs) in multiple languages.

[0098] Specifically, multiple offsets can be stored in the database according to actual needs.

[0099] In one implementable manner, the database can receive the message request instruction sent by the client.

[0100] Kafka has two ways to handle offsets for submission. In the automatic submission method, the client can send a message request instruction to the message middleware and the database. Among them, the message request instruction can instruct the message middleware to send the next message to the client and update the offset. Among them, after receiving the message request instruction, the database can obtain the current offset from the message middleware. Among them, the current offset is the latest offset in the message middleware.

[0101] In another implementable way, the database can receive the confirmation message sent by the client. Kafka has two ways to handle offsets for submission. In the manual submission method, after receiving the next message, the client can send the confirmation message to the message middleware and the database at the same time. The message middleware can receive the confirmation message and update the offset according to the confirmation message. The database can receive the confirmation message and obtain the current offset from the message middleware according to the confirmation message. Among them, the current offset is the latest offset in the message middleware.

[0102] Step 302, obtain the current offset from the message middleware and update the stored offset according to the current offset.

[0103] Among them, the current offset is the latest offset in the message middleware. The database can update the stored offset according to the current offset. The database can store the obtained current offset and delete the offset with the earliest storage time.

[0104] Specifically, multiple offsets are stored in the database. For example, 2 offsets can be stored. Optionally, the queue of offsets stored in the database can adopt the first-in, first-out method, that is, the first obtained is the offset with the earliest storage time in the database. For example, if 2 offsets can be stored in the database, the message corresponding to the offset with the earliest storage time in the database can be the message that the client consumed last time. For example, if the client failed to consume the message last time and wants to obtain the message consumed last time again, it can first obtain the offset with the earliest storage time in the database from the database and obtain the message from the message middleware according to the offset. This message is the message that the client failed to consume last time.

[0105] Figure 4 It is a schematic flowchart of the method for avoiding message loss shown in the fourth exemplary embodiment of the present disclosure.

[0106] As Figure 4 shown, for the method for avoiding message loss provided in this embodiment, multiple offsets are stored in the database. The method includes:

[0107] Step 401, receive the message instruction sent by the client. The message instruction includes a message request instruction and / or a confirmation message.

[0108] Among them, the database can be Redis.

[0109] Specifically, the database can store multiple offsets according to actual requirements.

[0110] In one implementable manner, the database can receive a message request instruction sent by the client.

[0111] Kafka has two submission methods for processing offsets. In the automatic submission method, the client can send a message request instruction to the message middleware and the database. Among them, the message request instruction can instruct the message middleware to send the next message to the client and update the offset. Among them, after receiving the message request instruction, the database can obtain the current offset from the message middleware. Among them, the current offset is the latest offset in the message middleware.

[0112] In another implementable manner, the database can receive a confirmation message sent by the client. Kafka has two submission methods for processing offsets. In the manual submission method, after receiving the next message, the client can send a confirmation message to the message middleware and the database at the same time. The message middleware can receive the confirmation message and update the offset according to the confirmation message. The database can receive the confirmation message and obtain the current offset from the message middleware according to the confirmation message. Among them, the current offset is the latest offset in the message middleware.

[0113] Step 402: Obtain the current offset from the message middleware, delete the historical offset with the earliest storage time among the stored offsets, and store the obtained current offset.

[0114] Specifically, the database can obtain the current offset from the message middleware. Among them, the current offset is the latest offset in the message middleware. And update the stored offset according to the current offset. The database can store the obtained current offset and delete the offset with the earliest storage time.

[0115] Optionally, the number of offsets stored in the database is 2.

[0116] Specifically, multiple offsets are stored in the database. For example, 2 offsets can be stored. Optionally, the queue of offsets stored in the database can adopt the first-in, first-out method, that is, the first obtained is the offset with an earlier storage time in the database. For example, if 2 offsets can be stored in the database, the message corresponding to the offset with an earlier storage time in the database can be the message that the client consumed last time. For example, when the client failed to consume a message last time and wants to obtain the message consumed last time again, it can first obtain the offset with an earlier storage time in the database from the database, and obtain the message from the message middleware according to this offset, and this message is the message that the client failed to consume last time.

[0117] Figure 5 The structural diagram of the client device for avoiding message loss shown in an exemplary embodiment of the present disclosure.

[0118] As Figure 5 As shown, the client device 500 for avoiding message loss provided in this embodiment includes:

[0119] The offset acquisition unit 510 is configured to obtain an offset from the database when the message consumption sent by the received message middleware fails; the offset corresponds to the message that the client consumption fails; the offset is obtained by the database from the message middleware.

[0120] The message request instruction sending unit 520 is configured to send a first message request instruction to the message middleware according to the offset, and the first message request instruction includes the offset.

[0121] The receiving unit 530 is configured to receive the message corresponding to the offset sent by the message middleware.

[0122] Figure 6 The structural diagram of the client device for avoiding message loss shown in another exemplary embodiment of the present disclosure.

[0123] As Figure 6 As shown, on the basis of the above embodiment, in the client device 600 for avoiding message loss provided in this embodiment,

[0124] The offset acquisition unit 510 is specifically configured to obtain the offset with an earlier storage time from the database.

[0125] The message request instruction sending unit 520 is further configured to send a first message request instruction to the database, and the first message request instruction is used to instruct the database to obtain the latest offset from the message middleware.

[0126] The message fetching instruction sending unit 520 is further configured to send a second message fetching instruction to the message middleware and the database after consuming a message, where the second message fetching instruction does not include an offset; the second message fetching instruction is used to instruct the message middleware to send the next message and update the offset; the second message fetching instruction is used to instruct the database to obtain the latest offset from the message middleware.

[0127] The receiving unit 530 is further configured to receive the next message sent by the message middleware.

[0128] In the client device 600 for avoiding message loss provided in this embodiment, there is further included: a confirmation message sending unit 540, configured to send a confirmation message to the message middleware; the confirmation message is used to represent that the client has received the message distributed by the message middleware.

[0129] The message fetching instruction sending unit 520 is further configured to send a second message fetching instruction to the message middleware after consuming a message, where the second message fetching instruction does not include an offset; the second message fetching instruction is used to instruct the message middleware to send the next message to the client.

[0130] The confirmation message sending unit 540 is further configured to send confirmation messages to the message middleware and the database respectively; the confirmation message is used to represent that the client has received the message distributed by the message middleware; the confirmation message is used to instruct the message middleware to update the offset; the confirmation message is used to instruct the database to obtain the latest offset from the message middleware.

[0131] The confirmation message sending unit 540 is specifically configured to send a confirmation message to the message middleware according to preset information; where the preset message is used to indicate the timing of sending the confirmation message to the message middleware; the confirmation message is used to instruct the message middleware to update the offset.

[0132] Figure 7 It is a structural diagram of a database device for avoiding message loss shown in an exemplary embodiment of the present disclosure.

[0133] As Figure 7 shown, the database device 700 for avoiding message loss provided in this embodiment includes:

[0134] The receiving unit 710 is configured to receive a message instruction sent by the client.

[0135] The offset updating unit 720 is configured to obtain the current offset from the message middleware and update the stored offset according to the current offset.

[0136] Optionally, the message instruction includes a message fetching instruction and / or a confirmation message.

[0137] The offset update unit 720 is specifically configured to delete the historical offset with the earliest storage time from the stored offsets and store the obtained current offset.

[0138] Optionally, the number of offsets stored in the database is 2.

[0139] Figure 8 This is a structural diagram of an electronic device shown in an exemplary embodiment of the present application.

[0140] As Figure 8 shown, the electronic device provided in this embodiment includes:

[0141] A memory 801;

[0142] A processor 802; and

[0143] A computer program;

[0144] Wherein, the computer program is stored in the memory 801 and is configured to be executed by the processor 802 to implement any of the above methods for avoiding message loss.

[0145] This embodiment also provides a computer-readable storage medium, on which a computer program is stored,

[0146] The computer program is executed by the processor to implement any of the above methods for avoiding message loss.

[0147] This embodiment also provides a computer program product, including a computer program, which when executed by the processor, implements any of the above methods for avoiding message loss.

[0148] A system for avoiding message loss includes a client and a database; the client is used to execute any of the methods such as Figure 1 , Figure 2 , and the database is used to execute any of the methods such as Figure 3 , Figure 4 .

[0149] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disk, or optical disk that can store program code.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for avoiding message loss, characterized in that, The method includes: When the consumption of the message sent by the received message middleware fails, obtain the offset from the database; wherein, the database stores N offsets in a first-in-first-out policy, and N≥2; the offset corresponds to the message for which the client consumption fails; the offset is obtained by the database from the message middleware; Send a first message fetching instruction to the message middleware according to the offset, and the first message fetching instruction includes the offset; Receive the message corresponding to the offset sent by the message middleware; After consuming one message, send a second message fetching instruction to the message middleware and the database, and the second message fetching instruction does not include the offset; the second message fetching instruction is used to instruct the message middleware to send the next message and update the offset; the second message fetching instruction is used to instruct the database to obtain the latest offset from the message middleware; or, After consuming one message, send a second message fetching instruction to the message middleware, and the second message fetching instruction does not include the offset; the second message fetching instruction is used to instruct the message middleware to send the next message to the client; Receive the next message sent by the message middleware.

2. The method according to claim 1, wherein The obtaining the offset from the database includes: Obtain the offset with an earlier storage time from the database.

3. The method according to claim 1, characterized in that It further includes: Send the first message fetching instruction to the database, and the first message fetching instruction is used to instruct the database to obtain the latest offset from the message middleware.

4. The method according to claim 1, characterized in that After receiving the next message sent by the message middleware, it further includes: Send a confirmation message to the message middleware; the confirmation message is used to indicate that the client has received the message distributed by the message middleware.

5. The method according to claim 1, characterized in that After receiving the next message sent by the message middleware, it further includes: Send confirmation messages to the message middleware and the database respectively; the confirmation message is used to indicate that the client has received the message distributed by the message middleware; the confirmation message is used to instruct the message middleware to update the offset; the confirmation message is used to instruct the database to obtain the latest offset from the message middleware.

6. The method according to claim 5, wherein It further includes: Send a confirmation message to the message middleware according to the preset information; wherein, the preset message is used to indicate the timing of sending the confirmation message to the message middleware; The confirmation message is used to instruct the message middleware to update the offset.

7. A method for avoiding message loss, characterized in that, There are N offsets stored in the database in a first-in-first-out policy, and N≥2; the offset corresponds to the message for which the client consumption fails; The method includes: Receive the message instruction sent by the client; Obtain the current offset from the message middleware and send it to the client, so that the client sends a first message fetching instruction to the message middleware according to the offset and receives the message corresponding to the offset sent by the message middleware; the first message fetching instruction includes the offset; After the client consumes a message, a second message fetching instruction is sent to the message middleware and the database, and the offset is not included in the second message fetching instruction; the second message fetching instruction is used to instruct the message middleware to issue the next message and update the offset; the second message fetching instruction is used to instruct the database to obtain the latest offset from the message middleware; or, After the client consumes a message, a second message fetching instruction is sent to the message middleware, and the offset is not included in the second message fetching instruction; the second message fetching instruction is used to instruct the message middleware to issue the next message to the client; Update the stored offset according to the current offset.

8. The method according to claim 7, wherein The message instruction includes a message fetching instruction and / or an acknowledgement message.

9. The method according to claim 7, characterized in that, The updating the stored offset according to the current offset includes: Delete the historical offset with the earliest storage time from the stored offsets, and store the obtained current offset.

10. A client device for avoiding message loss, characterized in that, The client device includes: An offset obtaining unit, configured to obtain an offset from a database when a message sent by the received message middleware fails to be consumed; wherein, the database stores N offsets in a first-in-first-out policy, and N≥2; the offset corresponds to the message for which the client consumption fails; the offset is obtained by the database from the message middleware; A message fetching instruction sending unit, configured to send a first message fetching instruction to the message middleware according to the offset, and the offset is included in the first message fetching instruction; A receiving unit, configured to receive the message corresponding to the offset sent by the message middleware; The message fetching instruction sending unit is further configured to, after consuming a message, send a second message fetching instruction to the message middleware and the database, and the offset is not included in the second message fetching instruction; the second message fetching instruction is used to instruct the message middleware to issue the next message and update the offset; the second message fetching instruction is used to instruct the database to obtain the latest offset from the message middleware; or, after consuming a message, send a second message fetching instruction to the message middleware, and the offset is not included in the second message fetching instruction; the second message fetching instruction is used to instruct the message middleware to issue the next message to the client; The receiving unit is further configured to receive the next message sent by the message middleware.

11. A database device for avoiding message loss, characterized in that, N offsets are stored in the database in a first-in-first-out policy, and N≥2; the offset corresponds to the message for which the client consumption fails; the database device includes: A receiving unit, configured to receive a message instruction sent by the client; An offset updating unit, configured to obtain a current offset from the message middleware and send it to the client, so that the client sends a first message fetching instruction to the message middleware according to the offset and receives the message corresponding to the offset sent by the message middleware; the offset is included in the first message fetching instruction; After the client consumes a message, a second message fetching instruction is sent to the message middleware and the database, and the offset is not included in the second message fetching instruction; the second message fetching instruction is used to instruct the message middleware to send the next message and update the offset; the second message fetching instruction is used to instruct the database to obtain the latest offset from the message middleware; or, After the client consumes a message, a second message fetching instruction is sent to the message middleware, and the offset is not included in the second message fetching instruction; the second message fetching instruction is used to instruct the message middleware to send the next message to the client; The offset update unit is further configured to update the stored offset according to the current offset.

12. An electronic device, characterized in that, Comprising a memory and a processor; wherein, The memory is used to store computer programs; The processor is configured to read the computer program stored in the memory and execute the method according to any one of claims 1-6 or 7-9 above based on the computer program in the memory.

13. A computer-readable storage medium, characterized in that, Computer-executable instructions are stored in the computer-readable storage medium, and when the processor executes the computer-executable instructions, the method according to any one of claims 1-6 or 7-9 above is implemented.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the method according to any one of claims 1-6 or 7-9 above is implemented.

15. A system for avoiding message loss, characterized in that, Comprising a client and a database; the client is configured to execute the method according to any one of claims 1-6 above; the database is configured to execute the method according to any one of claims 7-9 above.

Images