Message processing method, device, equipment, medium and program product

Abstract

The invention provides a message processing method, which can be applied to the technical field of computers, and the message processing method comprises the following steps: obtaining pull requests of a plurality of consumers for messages in the same partition, the partition comprising a plurality of messages, the plurality of messages being divided into a plurality of message groups arranged according to a predetermined order, each message group corresponds to one consumption limiting lock, and the plurality of message groups comprise M allocated message groups and N to-be-allocated message groups; according to the sequence of the M allocated message groups and each pull request, the N to-be-allocated message groups are allocated to L target consumers, and L is smaller than or equal to N; and distributing the N consumption limiting locks corresponding to the N to-be-distributed message groups to the L target consumers. The invention further provides a message processing device, equipment, a storage medium and a program product.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more specifically, to a message processing method, apparatus, device, medium and program product. Background Art

[0002] In the current existing message queue technology, for the same subscription group, a partition can usually only be assigned to one consumer process for consumption. There are two main implementation methods. One is to periodically execute the load balancing algorithm to uniquely assign a partition to a surviving consumer. If the consumer goes offline, it will be assigned to other consumers during the next load balancing. Consumers only consume the partitions assigned to themselves, so that only one consumer consumes messages in a partition. The other is to lock the partition on the server (broker) to ensure that only one consumer can pull and consume messages at the same time. When this batch of messages is confirmed to be consumed successfully, failed, or timed out, other consumers are allowed to pull subsequent messages from this partition for consumption.

[0003] In the process of implementing the concept disclosed in the present invention, the inventors found that there are at least the following problems in the related technology: when the number of consumers in a consumer cluster is much larger than the number of partitions, it is possible that each consumer in a part of the consumers occupies a queue, while the remaining consumers cannot occupy the queue and fall into a waiting state, resulting in a decrease in the message consumption rate of the entire system and even a backlog of messages. Summary of the invention

[0004] In view of this, the present disclosure provides a message processing method, apparatus, device, medium and program product.

[0005] One aspect of the present disclosure provides a message processing method, including:

[0006] Get multiple consumers' pull requests for messages in the same partition, where the partition contains multiple messages, and the multiple messages are divided into multiple message groups arranged in a predetermined order, each message group corresponds to a restricted consumption lock, and the multiple message groups include: M allocated message groups and N to-be-allocated message groups; according to the order of the M allocated message groups and each pull request, allocate the N to-be-allocated message groups to L target consumers, L is less than or equal to N; at the same time, allocate N restricted consumption locks corresponding to the N to-be-allocated message groups to the L target consumers.

[0007] According to an embodiment of the present disclosure, N to-be-allocated message groups are allocated to L target consumers according to the order of the M allocated message groups and each pull request, including: determining X to-be-allocated message groups from the N to-be-allocated message groups according to the order of the M allocated message groups and a preset limited consumption quantity; and allocating the X to-be-allocated message groups to the L target consumers according to each pull request, wherein X is less than N.

[0008] According to an embodiment of the present disclosure, determining X message groups to be allocated from N message groups to be allocated according to the order of M allocated message groups and a preset consumption limit number includes: determining the starting position of the X message groups to be allocated according to the order of the M allocated message groups; and determining the X message groups to be allocated according to the starting position and the consumption limit number.

[0009] According to an embodiment of the present disclosure, the message processing method also includes: in response to multiple allocated message groups returning confirmed consumption messages, determining multiple consumed message group sets and multiple message group order intervals corresponding to the multiple consumed message group sets; determining a target message group set from the multiple consumed message group sets based on the multiple message group order intervals; and updating X message groups to be allocated based on the message group order intervals corresponding to the target message group set.

[0010] According to an embodiment of the present disclosure, a target message group set is determined from multiple consumed message group sets according to multiple message group order intervals, including: according to multiple message group order intervals, a consumed message group set with the smallest starting order is determined as the target message group set.

[0011] According to an embodiment of the present disclosure, updating X to-be-allocated message groups according to the message group order interval corresponding to the target message group set includes: updating the starting positions of the X to-be-allocated message groups according to the ending order of the message group order interval corresponding to the target message group set.

[0012] According to an embodiment of the present disclosure, the message processing method also includes: in response to multiple assigned message groups not returning confirmed consumption messages, determining multiple message groups to be confirmed, wherein the message groups to be confirmed are used to represent message groups that have been assigned but have not returned confirmed consumption messages; judging whether the message groups to be confirmed are expired according to predetermined time conditions; and in the case that the message groups to be confirmed are expired, determining the message groups to be confirmed as expired message groups.

[0013] According to an embodiment of the present disclosure, before allocating N to-be-allocated message groups to L target consumers according to the order of M allocated message groups and each pull request, the method also includes: determining whether there is an expired message group among the multiple message groups; if there is an expired message group, allocating the messages in the expired message group and the restricted consumption locks corresponding to the expired message group to the L target consumers.

[0014] Another aspect of the present disclosure provides a message processing device, including: obtaining pull requests from multiple consumers for messages in the same partition, wherein the partition contains multiple messages, and the multiple messages are divided into multiple message groups arranged in a predetermined order, each message group corresponds to a restricted consumption lock, and the multiple message groups include: M allocated message groups and N to-be-allocated message groups; according to the order of the M allocated message groups and each pull request, the N to-be-allocated message groups are allocated to L target consumers, L is less than or equal to N; and the N restricted consumption locks corresponding to the N to-be-allocated message groups are allocated to the L target consumers.

[0015] Another aspect of the present disclosure provides an electronic device, comprising:

[0016] one or more processors;

[0017] a memory for storing one or more programs,

[0018] When one or more programs are executed by one or more processors, the one or more processors implement the above method.

[0019] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0020] Another aspect of the present disclosure provides a computer program product, which includes computer executable instructions, and the instructions are used to implement the above method when executed.

[0021] According to the embodiments of the present disclosure, since the same partition is divided into multiple message groups, and a corresponding consumption restriction lock is set for each message group, multiple consumers can process data in different message groups in parallel, thereby improving the concurrency performance and throughput of the system, and improving the efficiency of message processing. The setting of the consumption restriction lock enables each consumer to consume independently when processing the messages in the message zone group assigned to it, ensuring the consistency of the messages when they are consumed and avoiding repeated processing of messages. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0023] Figure 1 An exemplary system architecture to which the message processing method according to an embodiment of the present disclosure can be applied is schematically shown;

[0024] Figure 2 The flowchart of the message processing method according to the embodiment of the present disclosure is schematically shown;

[0025] Figure 3A and Figure 3B A schematic diagram of a message group in a partition according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 4A and Figure 4B A schematic diagram schematically shows X message groups to be allocated according to an embodiment of the present disclosure;

[0027] Figure 5 The flowchart of a message processing method according to another embodiment of the present disclosure is schematically shown;

[0028] Fig. 6A and Figure 6B FIG. 2 schematically shows another schematic diagram of X message groups to be allocated according to an embodiment of the present disclosure;

[0029] Figure 7 A schematic diagram of a hollow message group according to an embodiment of the present disclosure is schematically shown;

[0030] Figure 8 The flowchart of a message processing method according to another embodiment of the present disclosure is schematically shown;

[0031] Fig. 9 A block diagram schematically shows a message processing device according to an embodiment of the present disclosure; and

[0032] Fig.10 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0033] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0034] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0035] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0036] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0037] In the embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security, and national security.

[0038] In the embodiments of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0039] When different consumers can only consume from different partitions, the upper limit of the number of messages consumed globally in parallel is the number of partitions. When the business traffic increases, the consumption rate cannot be flexibly increased by expanding the capacity of downstream nodes. For this problem, there are usually the following solutions:

[0040] 1. Expand the number of partitions to increase the number of messages processed in parallel at the same time.

[0041] However, due to the limitations of the message queue storage model, the number of partitioned queues cannot grow indefinitely. Too many partitions will lead to more random writes, more metadata, etc., which will have a significant negative impact on performance and management complexity.

[0042] 2. After the consumer instances are pulled in batches, they are internally distributed to multiple threads for processing.

[0043] However, there are two problems: a) Due to the hardware bottleneck of a single node, the processing capacity of a single node is limited, and the amount that can be distributed is limited. b) Batch consumption can only be confirmed to the broker after all consumption is completed, which increases the complexity of consumption behavior.

[0044] Furthermore, the traditional method of locking partitions is mainly based on the consideration that some messages have sequential dependencies, such as relying on messages to synchronize database logs, etc. However, for most business scenarios, a message represents a business, and there is no temporal dependency between messages, so they can be consumed out of order.

[0045] In view of this, an embodiment of the present disclosure provides a message processing method, including: obtaining pull requests from multiple consumers for messages in the same partition, wherein the partition contains multiple messages, and the multiple messages are divided into multiple message groups arranged in a predetermined order, each message group corresponds to a restricted consumption lock, and the multiple message groups include: M allocated message groups and N to-be-allocated message groups; according to the order of the M allocated message groups and each pull request, the N to-be-allocated message groups are allocated to L target consumers, L is less than or equal to N; and the N restricted consumption locks corresponding to the N to-be-allocated message groups are allocated to the L target consumers.

[0046] Figure 1 The exemplary system architecture 100 to which the message processing method according to the embodiment of the present disclosure can be applied is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0047] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0048] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).

[0049] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0050] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0051] It should be noted that the message processing method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the message processing system provided in the embodiment of the present disclosure can generally be set in the server 105. The message processing method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the message processing system provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Alternatively, the message processing method provided in the embodiment of the present disclosure can also be executed by the first terminal device 101, the second terminal device 102 or the third terminal device 103, or can also be executed by other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103. Accordingly, the message processing system provided by the embodiment of the present disclosure may also be arranged in the first terminal device 101, the second terminal device 102 or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103.

[0052] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0053] Figure 2 The flowchart of the message processing method according to the embodiment of the present disclosure is schematically shown. Figure 3A and Figure 3B The schematic diagram of the message group in the partition according to the embodiment of the present disclosure is schematically shown. Figure 2 , Figure 3A and Figure 3B Detailed description.

[0054] like Figure 2 As shown, the method includes operations S210 to S230.

[0055] In operation S210, pull requests from multiple consumers for messages in the same partition are obtained, wherein the partition contains multiple messages, and the multiple messages are divided into multiple message groups arranged in a predetermined order, each message group corresponds to a restricted consumption lock, and the multiple message groups include: M allocated message groups and N to-be-allocated message groups.

[0056] According to an embodiment of the present disclosure, before operation S210, the message sequence in a partition is grouped according to a preset rule to obtain a plurality of message groups arranged in a predetermined order, such as Figure 3A or Figure 3B The number of messages in each message group can be the same, for example, every 100 messages are divided into one message group; the number of messages in each message group can also be different, for example, according to experience or device hardware performance, a different number of messages is set for each message group, and when different consumers pull messages in batches, they are matched with message groups that match their consumption capabilities.

[0057] According to an embodiment of the present disclosure, a corresponding consumption restriction lock is configured for each message group to restrict multiple consumers from repeatedly consuming the same message group.

[0058] According to an embodiment of the present disclosure, the allocated message group is used to represent the message group that has been allocated to the consumer in response to the consumer pull request; the to-be-allocated message group is used to represent the message group that has not been allocated to the consumer. Wherein, M is greater than or equal to 0, N is greater than 0, such as Figure 3B As shown, M=0 means that no message group is currently allocated and all are message groups to be allocated.

[0059] In operation S220 , N to-be-allocated message groups are allocated to L target consumers according to the order of the M allocated message groups and each pull request, where L is less than or equal to N.

[0060] According to an embodiment of the present disclosure, a pull request includes the number of messages requested to be pulled by the consumer. A corresponding message group is allocated to the current consumer according to the order of the M allocated message groups and the number of messages requested to be pulled by the consumer. Specifically, the order of the N message groups to be allocated may be determined according to the maximum order of the M allocated message groups, so as to determine the N message groups to be allocated according to the number of messages requested to be pulled; or, a message group matching the number of messages to be pulled is randomly determined from the N message groups to be allocated other than the M allocated message groups and allocated to the L target consumers.

[0061] According to an embodiment of the present disclosure, all or part of the N to-be-allocated message groups are allocated to L target consumers according to the order of the M allocated message groups and each pull request. For example, if L=3 and N=15, all messages in the 15 message groups are allocated to 3 consumers according to the pull request, as long as the allocated messages do not exceed the number of messages pulled by the consumers in batches; or, according to the pull request, 3 message groups out of the 15 message groups are allocated to 3 consumers respectively, and the remaining 12 message groups are not allocated for the time being.

[0062] In operation S230, at the same time, N restricted consumption locks corresponding to the N to-be-allocated message groups are allocated to L target consumers.

[0063] According to an embodiment of the present disclosure, while N to-be-allocated message groups are allocated to target consumers, the N message groups are locked, that is, N consumption restriction locks corresponding to the N to-be-allocated message groups are marked to restrict other consumers from consuming messages in the N message groups.

[0064] For example, Figure 3A As shown, when consumer A requests to pull messages, the messages of message group 1 to be assigned are assigned to consumer A, and message group 1 is locked. When message group 1 is locked, other consumers cannot consume the messages of message group 1 to be assigned if they cannot obtain restricted consumption lock 1, so as to avoid simultaneous consumption and ensure data consistency and integrity.

[0065] According to the embodiments of the present disclosure, since the same partition is divided into multiple message groups, and a corresponding consumption restriction lock is set for each message group, multiple consumers can process data in different message groups in parallel, thereby improving the concurrency performance and throughput of the system, and improving the efficiency of message processing. The setting of the consumption restriction lock enables each consumer to consume independently when processing the messages in the message zone group assigned to it, ensuring the consistency of the messages when they are consumed and avoiding repeated processing of messages.

[0066] According to an embodiment of the present disclosure, N to-be-allocated message groups are allocated to L target consumers according to the order of M allocated message groups and each pull request, including steps 11 and 12.

[0067] In step 11, X message groups to be allocated are determined from the N message groups to be allocated according to the order of the M allocated message groups and the preset consumption limit quantity.

[0068] According to an embodiment of the present disclosure, X message groups to be allocated are determined from the N message groups to be allocated in a predetermined order according to the order of the M allocated message groups and the preset consumption limit quantity. For example, if the preset consumption limit quantity is X, the first X message groups in the N message groups to be allocated are determined as the X message groups to be allocated.

[0069] According to an embodiment of the present disclosure, before step 11, the number of restricted consumption message groups is determined according to a preset restricted consumption number. If the number of messages in each message group is the same, the number of restricted consumption message groups can be directly determined according to the preset restricted consumption number. For example, if the number of messages in each message group is 100 and the restricted consumption number is 1000, it can be determined that it is divided into 10 restricted consumption message groups. If the number of messages in each message group is different, the number and order of each restricted consumption message group are calculated according to the number of messages in different message groups and the preset restricted consumption message number. For example, there are 100 messages in message group 1, 150 messages in message group 2, 200 messages in message group 3... If the restricted consumption number is 1000, 5 message groups are calculated.

[0070] Figure 4A and Figure 4B A schematic diagram of X message groups to be allocated according to an embodiment of the present disclosure is schematically shown.

[0071] According to an embodiment of the present disclosure, in step 11, X message groups to be allocated are determined from N message groups to be allocated, such as Figure 4A As shown, the first X message groups among the N message groups to be allocated may be directly determined as the X message groups to be allocated; Figure 4B As shown, X message groups arranged consecutively may be selected from the N message groups to be allocated as the X message groups to be allocated; or X message groups may be randomly selected from the N message groups to be allocated as the X message groups to be allocated (not shown in the figure).

[0072] In step 12, X to-be-allocated message groups are allocated to L target consumers according to each pull request, where X is less than N.

[0073] According to an embodiment of the present disclosure, the pull request includes at least the number of messages pulled by each consumer, for example, consumer A requests to pull 100 messages, consumer B requests to pull 200 messages, and consumer C requests to pull 100 data. According to the number of messages requested to be pulled by each consumer, some or all of the messages in the X to-be-allocated message groups are allocated to each consumer.

[0074] According to an embodiment of the present disclosure, X message groups to be allocated are determined from N message groups to be allocated according to the order of the M allocated message groups and the preset consumption limit quantity, including step 21 and step 22.

[0075] In step 21, the starting positions of the X message groups to be allocated are determined according to the order of the M allocated message groups.

[0076] According to an embodiment of the present disclosure, based on the maximum order of the M allocated message groups, the next order of the message group to be allocated is determined as the starting position of the X message groups to be allocated. Figure 4A As shown, if the maximum order of the M allocated message groups is the allocated message group 1, then the starting position of the X to-be-allocated message groups is the to-be-allocated message group 1.

[0077] In step 22, X message groups to be allocated are determined according to the starting position and the limited consumption quantity.

[0078] According to an embodiment of the present disclosure, after the restricted consumption group is calculated according to the restricted consumption quantity, X message groups to be allocated are determined according to the starting position and the restricted consumption group. Figure 4A As shown, if the restricted consumption group is 35 groups and the starting position is the to-be-allocated message group 1, then the X to-be-allocated message groups are the to-be-allocated message groups with sequence numbers 1-35.

[0079] According to an embodiment of the present disclosure, when the system allocates the messages in the corresponding message group to the consumer in response to the pull request of the consumer, the system will record the allocated message group in the allocated module (noAckBlocks). Thus, the system can track in real time which messages in the message group have been acquired by the consumer but have not received the confirmation consumption information returned by the consumer.

[0080] Figure 5 The flowchart of a message processing method according to another embodiment of the present disclosure is schematically shown. Fig. 6A and Figure 6B FIG. 2 schematically shows another schematic diagram of X message groups to be allocated according to an embodiment of the present disclosure. Figure 5 , Fig. 6A and Figure 6B Provide detailed explanation.

[0081] like Figure 5 As shown, the message processing method also includes operations S510 to S530.

[0082] In operation S510, in response to a plurality of allocated message groups returning confirmed consumption messages, a plurality of consumed message group sets and a plurality of message group order intervals corresponding to the plurality of consumed message group sets are determined.

[0083] like Fig. 6A and Figure 6B As shown, a blank space represents a message group to be allocated, and a dotted frame represents X message groups to be allocated. For example, Fig. 6A and Figure 6B As shown, X = 100. By determining X message groups to be allocated according to a predetermined limited consumption quantity, the number of messages that can be consumed in parallel by multiple consumers is limited.

[0084] like Figure 6B As shown, striped grids are used to represent allocated message groups that have been allocated but not confirmed to be consumed, such as message groups 4 and 5; grids are used to represent consumed message groups that have been allocated and confirmed to be consumed, such as message groups 1~3, 6~9.

[0085] According to an embodiment of the present disclosure, after some messages in the X to-be-allocated message groups are allocated to consumers, the consumers consume the messages. After the consumption is completed, the consumers send a message confirming the consumption. Figure 6B When the consumer groups 1~3, 6~9 in the above table return confirmed consumption messages, multiple sets of consumed message groups and the order interval of the message groups corresponding to the sets of consumed message groups are determined based on the message groups consumed continuously. Figure 6B As shown, according to the continuously consumed message groups, it can be seen that the consumed message group sets are: consumed message group set 1 composed of message groups 1, 2, and 3, and the message group order interval is [1,3]; consumed message group set 2 composed of message groups 6, 7, 8, and 9, and the message group order interval is [6,9].

[0086] In operation S520, a target message group set is determined from a plurality of consumed message group sets according to a plurality of message group order intervals. Specifically, a consumed message group set with the smallest starting order is determined as the target message group set according to a plurality of message group order intervals.

[0087] like Figure 6B As shown, from the message group order interval [1,3] and the message group order interval [6,9], the consumed message group set with the smallest starting order is determined as the target message group set, that is, according to the starting orders 1 and 6, the consumed message group set 1 corresponding to the message group order interval [1,3] is determined as the target message group set.

[0088] In operation S530, the X message groups to be allocated are updated according to the message group order interval corresponding to the target message group set. Specifically, the starting positions of the X message groups to be allocated are updated according to the ending order of the message group order interval corresponding to the target message group set.

[0089] According to an embodiment of the present disclosure, the next message group in the ending order of the message group sequence interval corresponding to the target message group set is determined as the starting position of the X message groups to be allocated in the next update, and the sequence numbers of the X message groups to be allocated are postponed backward according to the consumption limit. Figure 6BAs shown, the end order of the message group sequence interval [1,3] is determined, that is, message group 3; the next message group of message group 3 (message group 4) is determined as the starting position of the X message groups to be allocated for the next update, and the sequence numbers of the X message groups to be allocated are postponed backward according to the preset limited consumption quantity, as shown in FIG. Figure 6B As shown, in order to maintain the number of X message groups to be allocated as 100, it is postponed to 103, and then it can be determined that the starting position of the X message groups to be allocated is 4, the ending order is 103, and X=100.

[0090] According to the embodiments of the present disclosure, the next starting position is determined according to the ending order of the message group order interval, which can accurately track which messages have been processed and which have not been processed. If this rule is not followed, such as randomly selecting the starting position, some messages may be repeatedly assigned to consumers for processing, or some messages may be omitted and not processed.

[0091] In the present disclosure, the distribution, consumption and management of messages are carried out with message groups as the basic unit level. However, it should be emphasized that this method of using message groups as units is only one of the many feasible methods of the present disclosure. This method is used here mainly to help understand the relevant content and serve as an example. In fact, in addition to using message groups as units, there is another way, which is to directly use each message as a unit level to perform message distribution, consumption and management operations. In addition, it needs to be further explained that in actual situations, each message may differ in the number of bytes. However, in order to maintain the simplicity of the explanation, in the present disclosure, the specific number of bytes of each message will not be discussed in detail, and the focus is still on the introduction of the distribution, consumption and management methods of messages.

[0092] According to the embodiments of the present disclosure, since different consumers have different message processing capabilities, different consumers return confirmed messages at different times. Figure 6B As shown, when consumer groups 1, 2, 3, 6, 7, 8, and 9 have all been confirmed to have been consumed, message groups 4 and 5 have not been consumed.

[0093] In order to avoid missing consumption and maintain the performance of parallel consumption, the starting position of the X message groups to be allocated is determined according to the message group order interval and the information of the consumed message groups; and the number of message groups to be allocated is maintained. Figure 6B As shown, in order to ensure that the messages in consumer groups 4 and 5 are not missed, only the starting order of the X message groups to be allocated is updated to message group 4 and then postponed to message group 103. Compared with updating the starting order of the X message groups to be allocated to the maximum value (message group) of continuous consumption, message omission can be effectively avoided.

[0094] According to the embodiments of the present disclosure, further, the information of the allocated message groups is stored in the memory, and the message group order intervals corresponding to each set of consumed message groups and the information of the consumed message groups are persistently stored. If the starting positions of the X message groups to be allocated are updated only based on the information of the allocated message groups in the memory, when the system encounters a failure, restart, etc., the information of the allocated message groups in the memory is easily lost. This information loss will result in the inability to accurately determine the next message group to be allocated, which will lead to the problem of missing or repeated allocation of messages.

[0095] According to an embodiment of the present disclosure, a method for updating the starting positions of X to-be-allocated message groups based on the message group order interval and the information of the consumed message groups is adopted. By persistently processing the information of the message group order interval and the consumed message groups, a more stable and reliable message consumption status recording system is constructed. Under this system, there is no problem of data loss caused by abnormal conditions such as system failure. Even if the system fails, the starting positions of X to-be-allocated message groups can still be accurately determined, thereby avoiding the problems of repeated consumption and missed consumption of messages at the same time, and ensuring the continuity confirmation of messages. Therefore, the present disclosure allows multiple consumers to consume messages in the same partition concurrently and confirm the message intervals they consume respectively. This can improve the efficiency and flexibility of message processing, and different consumers can confirm the consumption of different parts of messages at different time points. At the same time, by maintaining the order interval of the message group, it is possible to clearly understand which message groups have been confirmed to be consumed and which are still waiting for confirmation, which is convenient for the system to perform overall message management and tracking. Through a strict confirmation mechanism, it is ensured that messages will not be processed repeatedly or lost. Only when a message group is fully confirmed for consumption will the starting position of the X message groups to be allocated be postponed backwards. This ensures that each message is processed only once and confirmation will not be lost due to system failure or other reasons.

[0096] According to the embodiments of the present disclosure, from the perspective of the system as a whole, this orderly message allocation method helps to optimize the use of system resources. For example, in terms of storage resources, messages can be stored and read in order, reducing the overhead caused by random access and improving the efficiency of storage and reading. This precise control helps to avoid the problem of message backlog and insufficient processing.

[0097] According to an embodiment of the present disclosure, the message processing method further includes steps 31 to 33.

[0098] In step 31, in response to multiple allocated message groups not returning confirmed consumption messages, multiple message groups to be confirmed are determined, wherein the message groups to be confirmed are used to represent message groups that have been allocated but have not returned confirmed consumption messages.

[0099] In step 32, it is determined whether the message group to be confirmed is expired according to a predetermined time condition.

[0100] According to an embodiment of the present disclosure, the system sets a specific time limit condition. If after the consumer pulls the message, after the specified time, the consumer has not returned information to the system confirming that the message has been consumed, then the system will determine the to-be-confirmed message group as an expired message group.

[0101] In step 33, when the message group to be confirmed is expired, the message group to be confirmed is determined as an expired message group.

[0102] According to an embodiment of the present disclosure, a copy of the message group that has been allocated but not confirmed in the allocated module (noAckBlocks) is saved in the expired module (noAckExpiredBlocks). At this time, the expired message group in the allocated module (noAckBlocks) will not be deleted. This means that the allocated module (noAckBlocks) not only records all messages that have been pulled normally by consumers but have not yet been confirmed, but may also contain messages that have expired but have not yet been processed through subsequent processes (such as reallocation or delivery to a retry queue, etc.). This mechanism helps the system better manage the status of messages. For example, these records can be used to determine whether the message has been consumed normally, whether it has expired, and how to handle it after expiration.

[0103] According to the embodiments of the present disclosure, the predetermined time condition is set in advance to determine whether the group of messages to be confirmed that have been allocated but not confirmed for consumption has expired, starting from the time when the message is successfully allocated to the consumer. The predetermined time condition can be set based on experience, for example, it can be 20 seconds, 2 minutes, 2 hours, 2 days, etc. The predetermined time condition can be configured in the broker configuration file; or in the console to configure the consumption strategy-response timeout; or set in the consumer to send the consumption request object (GetMessage).

[0104] According to an embodiment of the present disclosure, the message transfer work of determining the to-be-confirmed message group as an expired message group can be completed by a dedicated background thread. This background thread will check at a certain time interval to determine whether the to-be-confirmed message group is expired. When the data times out (i.e. expires), the system will reallocate the expired message group to the consumer. Specifically, during the processing, the messages in the to-be-confirmed module (noAckExpiredBlocks) will be reallocated first. The main purpose of doing this is to ensure that the message group order interval is as continuous as possible. Because only when the message group order interval is continuous, the starting position of the X to-be-allocated message groups can be smoothly updated backwards, thereby ensuring the orderly progress of the entire message processing process and avoiding system confusion or data processing errors caused by data expiration problems.

[0105] According to an embodiment of the present disclosure, it is determined whether there is an expired message group among multiple message groups; if there is an expired message group, the messages in the expired message group and the restricted consumption locks corresponding to the expired message group are allocated to L target consumers.

[0106] According to an embodiment of the present disclosure, in a concurrent consumption scenario, multiple consumers simultaneously obtain and consume messages from the same partition. Under normal circumstances, after successfully consuming a message, the consumer should promptly return confirmation information of the consumed message to the broker. However, due to certain reasons, such as system failure, network delay, complex and time-consuming consumption process, etc., the consumer did not return confirmation information of the consumed message within the expected time. This causes interruptions in the order of the consumed message groups recorded on the broker side, and this interruption is called a "hole". If a consumer group has a "hole" phenomenon, we call it a "hole message group". This "hole message group" may cause confusion in message processing, such as messages may be consumed repeatedly or the consumption order may be wrong, because the message middleware cannot accurately determine whether these messages have been actually consumed.

[0107] Figure 7 A schematic diagram of a hollow message group according to an embodiment of the present disclosure is schematically shown.

[0108] According to the embodiment of the present disclosure, if the hole is caused by the expiration of the message group to be confirmed, then the empty message group is the expired message group, and the expired message group is processed according to steps 31 to 33. If the hole is caused by system failure, network delay, etc., the starting position of the X message groups to be allocated is re-determined according to the updated message group order interval and the set of consumed message groups to reallocate messages or message groups to consumers. In this way, message loss can be avoided, repeated consumption of messages can be prevented, the stability and concurrency of parallel consumption can be improved, and the efficiency of message processing can be improved.

[0109] Figure 8 The flowchart of the message processing method according to another embodiment of the present disclosure is schematically shown. Figure 8 As shown, the message processing method also includes operations S810 to S890.

[0110] In operation S810, a pull request for messages in the same partition is received from a consumer, and it is determined whether to enable parallel consumption according to a preset condition. If enabled, operation S820 is executed.

[0111] In operation S820, it is determined whether there is an expired message group in the expiration module (noAckExpiredBlocks). If so, operation S821 is executed, and according to the pull request, the expired message group that meets the pull request and its corresponding restricted consumption lock are preferentially allocated to the consumer from the expiration module. If not, operation S830 is executed.

[0112] In operation S830, it is determined whether the allocated message group order is 0. If the allocated message group order is 0, operation S831 is performed to allocate corresponding to-be-allocated messages and their corresponding restricted consumption locks to consumers in order from small to large according to the to-be-allocated message group order. If the allocated message group order is not 0, operation S832 is performed to allocate the messages in the next to-be-allocated message group adjacent to the allocated message group and their corresponding restricted consumption locks to consumers according to the allocated message group order.

[0113] In operation S840, it is detected whether each message group in the partition is an empty message group. If there is an empty message group, operation S841 is performed, and according to the pull request, the messages in the matching message group in the empty consumer group and the corresponding restricted consumption lock are preferentially allocated to the consumer. If there is no empty message group, operation S830 is performed.

[0114] Fig. 9 A block diagram of a message processing device according to an embodiment of the present disclosure is schematically shown.

[0115] like Fig. 9 As shown, the message processing device 900 includes an acquisition module 910 , a first allocation module 920 and a second allocation module 930 .

[0116] The acquisition module 910 is used to obtain pull requests from multiple consumers for messages in the same partition, wherein the partition contains multiple messages, and the multiple messages are divided into multiple message groups arranged in a predetermined order, each message group corresponds to a consumption restriction lock, and the multiple message groups include: M allocated message groups and N to-be-allocated message groups;

[0117] A first allocation module 920, configured to allocate N to-be-allocated message groups to L target consumers according to the order of the M allocated message groups and each pull request, where L is less than or equal to N;

[0118] The second allocation module 930 is used to allocate N consumption restriction locks corresponding to the N message groups to be allocated to L target consumers.

[0119] According to the embodiments of the present disclosure, since the same partition is divided into multiple message groups, and a corresponding restricted consumption lock is set for each message group, multiple consumers can process data in different message groups in parallel, thereby improving the concurrency performance and throughput of the system, and at the same time improving the efficiency of message processing. The setting of the restricted consumption lock enables each consumer to consume independently when processing the messages in the message zone group assigned to it, ensuring the consistency of the messages when they are consumed. Only consumers with the restricted consumption lock can process the corresponding messages, avoiding repeated processing of messages.

[0120] According to an embodiment of the present disclosure, N to-be-allocated message groups are allocated to L target consumers according to the order of the M allocated message groups and each pull request, including:

[0121] Determine X message groups to be allocated from N message groups to be allocated according to the order of the M allocated message groups and the preset consumption limit quantity;

[0122] According to each pull request, X to-be-allocated message groups are allocated to L target consumers, where X is less than N.

[0123] According to an embodiment of the present disclosure, according to the order of the M allocated message groups and the preset limited consumption quantity, X message groups to be allocated are determined from the N message groups to be allocated, including:

[0124] Determine the starting positions of the X message groups to be allocated according to the order of the M allocated message groups;

[0125] According to the starting position and the consumption limit, X message groups to be allocated are determined.

[0126] According to an embodiment of the present disclosure, the message processing device also includes: in response to multiple allocated message groups returning confirmed consumption messages, determining multiple consumed message group sets and multiple message group order intervals corresponding to the multiple consumed message group sets; determining a target message group set from the multiple consumed message group sets according to the multiple message group order intervals; and updating X message groups to be allocated according to the message group order intervals corresponding to the target message group set.

[0127] According to an embodiment of the present disclosure, a target message group set is determined from multiple consumed message group sets according to multiple message group order intervals, including: according to multiple message group order intervals, a consumed message group set with the smallest starting order is determined as the target message group set.

[0128] According to an embodiment of the present disclosure, updating X to-be-allocated message groups according to the message group order interval corresponding to the target message group set includes: updating the starting positions of the X to-be-allocated message groups according to the ending order of the message group order interval corresponding to the target message group set.

[0129] According to an embodiment of the present disclosure, the message processing device also includes: in response to multiple assigned message groups not returning confirmed consumption messages, determining multiple message groups to be confirmed, wherein the message groups to be confirmed are used to represent message groups that have been assigned but have not returned confirmed consumption messages; judging whether the message groups to be confirmed are expired according to predetermined time conditions; and in the case that the message groups to be confirmed are expired, determining the message groups to be confirmed as expired message groups.

[0130] According to an embodiment of the present disclosure, before allocating N to-be-allocated message groups to L target consumers according to the order of M allocated message groups and each pull request, the method also includes: determining whether there is an expired message group among the multiple message groups; if there is an expired message group, allocating the messages in the expired message group and the restricted consumption locks corresponding to the expired message group to the L target consumers.

[0131] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0132] For example, any multiple of the acquisition module 910, the first allocation module 920, and the second allocation module 930 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the acquisition module 910, the first allocation module 920, and the second allocation module 930 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 910, the first allocation module 920, and the second allocation module 930 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be performed.

[0133] Fig.10 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Fig.10 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0134] like Fig.10 As shown, the electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 to a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0135] In RAM 1003, various programs and data required for the operation of electronic device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0136] According to an embodiment of the present disclosure, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The electronic device 1000 may further include one or more of the following components connected to the input / output (I / O) interface 1005: an input portion 1006 including a keyboard, a mouse, etc.; an output portion 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk, etc.; and a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the input / output (I / O) interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage portion 1008 as needed.

[0137] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0138] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0139] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0140] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 1002 and / or the RAM 1003 described above and / or one or more memories other than the ROM 1002 and the RAM 1003 .

[0141] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains a program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the message processing method provided by the embodiment of the present disclosure.

[0142] When the computer program is executed by the processor 1001, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0143] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0144] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0145] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0146] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A message processing method, comprising: Obtain pull requests from multiple consumers for messages in the same partition, wherein the partition contains multiple messages, the multiple messages are divided into multiple message groups arranged in a predetermined order, each message group corresponds to a restricted consumption lock, and the multiple message groups include: M allocated message groups and N to-be-allocated message groups; According to the order of the M allocated message groups and each of the pull requests, the N to-be-allocated message groups are allocated to L target consumers, where L is less than or equal to N; At the same time, N restricted consumption locks corresponding to the N to-be-allocated message groups are allocated to the L target consumers.

2. The method according to claim 1, wherein: According to the order of the M allocated message groups and each of the pull requests, the N to-be-allocated message groups are allocated to the L target consumers, including: Determining X message groups to be allocated from the N message groups to be allocated according to the order of the M allocated message groups and the preset consumption limit quantity; According to the pull requests, the X to-be-allocated message groups are allocated to the L target consumers, where X is less than N.

3. The method according to claim 2, wherein: Determining X message groups to be allocated from the N message groups to be allocated according to the order of the M allocated message groups and the preset limited consumption quantity, including: Determine the starting positions of the X message groups to be allocated according to the order of the M allocated message groups; The X message groups to be allocated are determined according to the starting position and the limited consumption quantity.

4. The method according to claim 1 or 2, wherein: The method further comprises: In response to the multiple allocated message groups returning confirmed consumption messages, determining multiple consumed message group sets and multiple message group order intervals corresponding to the multiple consumed message group sets; Determining a target message group set from the plurality of consumed message group sets according to the plurality of message group order intervals; The X message groups to be allocated are updated according to the message group order interval corresponding to the target message group set.

5. The method according to claim 4, wherein: Determining a target message group set from the plurality of consumed message group sets according to the plurality of message group order intervals includes: According to the multiple message group order intervals, a consumed message group set with the smallest starting order is determined as a target message group set.

6. The method according to claim 4, wherein: Updating the X message groups to be allocated according to the message group order interval corresponding to the target message group set includes: According to the end order of the message group sequence interval corresponding to the target message group set, the starting positions of the X message groups to be allocated are updated.

7. The method according to claim 1 or 2, wherein: The method further comprises: In response to the fact that the multiple allocated message groups do not return confirmed consumption messages, determining multiple message groups to be confirmed, wherein the message groups to be confirmed are used to represent message groups that have been allocated but have not returned confirmed consumption messages; According to a predetermined time condition, determining whether the to-be-confirmed message group has expired; In the case that the to-be-confirmed message group is expired, the to-be-confirmed message group is determined as an expired message group.

8. The method according to claim 7, wherein: Before allocating the N to-be-allocated message groups to the L target consumers according to the order of the M allocated message groups and the respective pull requests, the method further includes: Determining whether the expired message group exists in the multiple message groups; In the case where the expired message group exists, the messages in the expired message group and the restricted consumption locks corresponding to the expired message group are allocated to the L target consumers.

9. A message processing device, comprising: An acquisition module, used to acquire pull requests from multiple consumers for messages in the same partition, wherein the partition contains multiple messages, and the multiple messages are divided into multiple message groups arranged in a predetermined order, each message group corresponds to a consumption restriction lock, and the multiple message groups include: M allocated message groups and N to-be-allocated message groups; A first allocation module, configured to allocate the N to-be-allocated message groups to L target consumers according to the order of the M allocated message groups and each of the pull requests, where L is less than or equal to N; The second allocation module is used to allocate N consumption restriction locks corresponding to the N message groups to be allocated to the L target consumers.

10. An electronic device, comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images