Message processing method, device, electronic device and storage medium

Abstract

The present application discloses a message processing method, apparatus, electronic device, and storage medium. The method includes: receiving a message request from a first consumer instance within a first consumer group; determining a first message partition based on a first quantity and a message access parameter of the first consumer instance; the first quantity representing the number of message partitions allocated to the first consumer group; extracting a corresponding first message from the first message partition, and returning the first message to the first consumer instance.

Description

Technical Field

[0001] The present application relates to the field of software technology, and in particular to a message processing method, device, electronic device and storage medium. Background Art

[0002] With message middleware, consumer instances in different consumer groups can simultaneously consume messages from a message partition. However, in related technologies, when a consumer group or a consumer instance within a consumer group changes, all consumer instances are temporarily unable to consume messages, leading to problems such as a large message backlog or duplicate consumption. Summary of the Invention

[0003] In view of this, the main purpose of the embodiments of the present application is to provide a message processing method, device, electronic device and storage medium to solve the problem of large-scale message backlog or repeated consumption in related technologies.

[0004] To achieve the above objectives, the technical solution of the embodiment of the present application is implemented as follows:

[0005] This embodiment of the present application provides a message processing method, the method comprising:

[0006] Receive a message request from a first consumer instance in a first consumer group;

[0007] Determining a first message partition based on a first number and a message access parameter of the first consumer instance, wherein the first number represents the number of message partitions allocated to the first consumer group;

[0008] Extract the corresponding first message from the first message partition, and return the first message to the first consumer instance.

[0009] In the above solution, before receiving the message request from the first consumer instance in the first consumer group, the method further includes:

[0010] Cache the first information corresponding to the first consumer group; wherein,

[0011] The first information at least represents the correspondence between the first consumer group and the allocated message partition.

[0012] In the above solution, determining the first message partition based on the first quantity and the message access parameter of the first consumer instance includes:

[0013] Taking the first quantity modulo the number of message accesses of the first consumer instance;

[0014] Among all message partitions allocated to the first consumer group, determine the message partition whose corresponding number is the same as the modulo result as the first message partition;

[0015] Among them, all message partitions allocated to the first consumer group are numbered sequentially.

[0016] In the above solution, extracting the corresponding first message from the first message partition includes:

[0017] Determine a first message location corresponding to the first consumer instance;

[0018] The first message is extracted from the first message partition according to the first message location.

[0019] In the above solution, before receiving the message request from the first consumer instance in the first consumer group, the method further includes:

[0020] Cache the first message location corresponding to the first consumer instance; wherein,

[0021] The message location represents the consumption progress of the message queue in the message partition by the corresponding consumer instance.

[0022] In the above solution, the method further includes:

[0023] When the first message is successfully received by the first consumer instance, the first message location corresponding to the first consumer instance is updated.

[0024] In the above solution, after returning the first message to the first consumer instance, the method further includes:

[0025] Creating a first thread, and saving the first message to the first thread;

[0026] If the second message is not monitored within the set time, the first message saved in the first thread is sent to the first consumer instance; wherein,

[0027] The second message represents a confirmation message returned by the first consumer instance regarding receipt of the first message.

[0028] In the above solution, the method further includes:

[0029] In a case where the second message is monitored within the set time period, the first message is deleted from the first thread.

[0030] The present invention also provides a message processing device, comprising:

[0031] A receiving unit, configured to receive a message request from a first consumer instance in a first consumer group;

[0032] a determining unit, configured to determine a first message partition based on a first quantity and a message access parameter of the first consumer instance, wherein the first quantity represents a number of message partitions allocated to the first consumer group;

[0033] A returning unit is used to extract the corresponding first message from the first message partition and return the first message to the first consumer instance.

[0034] An embodiment of the present application further provides an electronic device, comprising: a processor and a memory for storing a computer program that can be run on the processor, wherein:

[0035] The processor is configured to execute the steps of any of the above methods when running the computer program.

[0036] An embodiment of the present application further provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0037] In an embodiment of the present application, a message request from a first consumer instance in a first consumer group is received; a first message partition is determined based on a first quantity and a message access parameter of the first consumer instance; the first quantity represents the number of message partitions allocated to the first consumer group; the corresponding first message is extracted from the first message partition, and the first message is returned to the first consumer instance. In this way, a consumer instance in the consumer group can consume any message partition in the message partition corresponding to the consumer group according to the message access parameter. There is no fixed correspondence between the consumer instance and the message partition. In this way, even if the number of consumer instances in the consumer group changes, all consumer instances will not stop consuming messages, thereby avoiding the problem of large backlog or repeated consumption of messages in a short period of time, and improving the efficiency of message consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of the implementation flow of the message processing method provided in an embodiment of the present application;

[0039] Figure 2 A schematic diagram of a consumer instance pulling messages provided in an embodiment of the present application;

[0040] Figure 3 A schematic diagram of a consumer instance partition index provided in an embodiment of the present application;

[0041] Figure 4A schematic diagram of the storage status of a key-value database provided in an embodiment of the present application;

[0042] Figure 5 A flowchart of a message processing method provided in an embodiment of the present application;

[0043] Figure 6 Another flowchart of the message processing method provided by the application embodiment of the present application;

[0044] Figure 7 Another flowchart of the message processing method provided by the application embodiment of the present application;

[0045] Figure 8 Another flowchart of the message processing method provided by the application embodiment of the present application;

[0046] Figure 9 A schematic diagram of a message processing device provided in an embodiment of the present application;

[0047] Figure 10 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The message middleware system is highly reliable and is widely used in the company's internal systems, such as order systems, payment systems, and warehouse systems.

[0049] In message-based middleware, a consumer group can subscribe to multiple topics, each of which corresponds to multiple message partitions. Consumer instances in different consumer groups can simultaneously consume messages from a message partition. Consumed messages are not deleted from the message queue; only the message location of the consumer instance is updated. However, each consumer instance in a consumer group only consumes messages from a subset of message partitions, and each message partition can only be consumed by one consumer instance in the same consumer group. In other words, there is a specific correspondence between consumer instances and message partitions.

[0050] When a new consumer instance is added to a consumer group, a consumer instance leaves the consumer group, or a consumer instance goes down, a rebalance occurs. Rebalancing means redistributing the correspondence between each consumer instance and message partition in the consumer group.

[0051] When a rebalance occurs, all consumer instances cease consuming and resume consuming only after the rebalance is complete. While consumer instances cease consuming, a large message backlog accumulates within a short period of time, potentially impacting the performance of the message middleware system. Furthermore, the communication connection between the consumer instance and the message partition is interrupted. If the consumer instance needs to continue consuming, it must reestablish the connection with the message partition after the rebalance is complete, resulting in significant performance overhead. Furthermore, message locations are submitted asynchronously by consumer instances. If a rebalance occurs during this submission process, duplicate consumption can occur before the message location is successfully submitted.

[0052] In other words, the occurrence of rebalancing will cause a large backlog of messages in a short period of time and problems such as repeated consumption of messages.

[0053] Based on this, an embodiment of the present application provides a message processing method, device, electronic device and storage medium, which receives a message request from a first consumer instance in a first consumer group; determines a first message partition based on a first quantity and a message access parameter of the first consumer instance; the first quantity represents the number of message partitions allocated to the first consumer group; extracts the corresponding first message from the first message partition, and returns the first message to the first consumer instance. In this way, a consumer instance in the consumer group can consume any message partition in the message partition corresponding to the consumer group according to the message access parameter. There is no fixed correspondence between the consumer instance and the message partition. In this way, even if the number of consumer instances in the consumer group changes, all consumer instances will not stop consuming messages, thereby avoiding the problem of large backlog or repeated consumption of messages in a short period of time, and improving the efficiency of message consumption.

[0054] The present application will be described in further detail below with reference to the accompanying drawings and embodiments.

[0055] Figure 1 This is a schematic diagram of the implementation flow of the message processing method provided in the embodiment of the present application, and the method is applied to the message server. Figure 1 As shown, the method includes:

[0056] Step 101: Receive a message request from a first consumer instance in a first consumer group.

[0057] Here, a message request from a first consumer instance in a first consumer group is received, where the message request is used to request to consume a message in a message queue of a message partition.

[0058] Step 102: Determine a first message partition based on a first quantity and a message access parameter of the first consumer instance; the first quantity represents the number of message partitions allocated to the first consumer group.

[0059] Here, at least one message partition is assigned to the first consumer group, so the first number is greater than or equal to one. The message access parameter of the first consumer instance includes at least the current access count of the first consumer instance, which represents the number of times the first consumer instance has pulled messages from the message server. Based on the first number and the message access parameter of the first consumer instance, the message partition to be consumed by the first consumer instance is determined.

[0060] Figure 2 A schematic diagram of a consumer instance pulling messages provided in an embodiment of the present application, such as Figure 2 As shown:

[0061] In step 1, the consumer instance in the client pulls messages from the message server. In actual applications, each time the consumer instance pulls a message, the message server will increase the access count by one. When the number of messages pulled by the consumer instance reaches a certain number, the message server determines the access count of the consumer instance at this time, so that the consumer instance consumes messages based on the access count.

[0062] After the consumer instance obtains the current number of visits, the consumer instance sends a confirmation message to the message server to confirm that the current number of visits has been successfully obtained.

[0063] Step 103: Extract the corresponding first message from the first message partition, and return the first message to the first consumer instance.

[0064] Here, after determining that the message partition consumed by the first consumer instance is the first message partition, the first message corresponding to the message request is pulled from the first message partition, and the first message is returned to the first consumer instance.

[0065] In one embodiment, before receiving a message request from a first consumer instance in a first consumer group, the method further includes:

[0066] Cache the first information corresponding to the first consumer group; wherein,

[0067] The first information at least represents the correspondence between the first consumer group and the allocated message partition.

[0068] Here, before receiving the message request of the first consumer instance, the first information corresponding to the first consumer group is cached, that is, the correspondence between the first consumer group and the assigned message partition is cached. For example, if the first consumer group is assigned to four message partitions, the correspondence between the first consumer group and these four message partitions is cached.

[0069] In practical applications, messages have various topics. Topics represent the identification type of messages. A topic represents a topic type and is unique. A topic corresponds to multiple message partitions. Therefore, in one embodiment, the process further includes caching second information corresponding to the first consumer group. The second information represents the correspondence between the first consumer group and the topic to which the assigned message partition belongs.

[0070] By pre-caching the correspondence between the first consumer group and the assigned message partition, it is convenient to quickly select the appropriate message partition from the message partition corresponding to the first consumer group for consumption when receiving a message request from the first consumer instance, thereby improving consumption efficiency.

[0071] In one embodiment, determining the first message partition based on the first quantity and the message access parameter of the first consumer instance includes:

[0072] Taking the first quantity modulo the number of message accesses of the first consumer instance;

[0073] Among all message partitions allocated to the first consumer group, determine the message partition whose corresponding number is the same as the modulo result as the first message partition;

[0074] Among them, all message partitions allocated to the first consumer group are numbered sequentially.

[0075] Here, all message partitions assigned to the first consumer group are numbered sequentially. For example, if the first consumer group is assigned to four message partitions, these four message partitions are numbered sequentially as message partition 1, message partition 2, message partition 3, and message partition 4.

[0076] The total number of message partitions allocated to the first consumer group is moduloed by the number of message accesses of the first consumer instance, and the message partition with the same corresponding number as the modulo result is selected from all message partitions allocated to the first consumer group as the first message partition. For example, the number of message accesses of the first consumer instance is 5, and when the number of message partitions allocated to the first consumer group is 4, the modulo result obtained by taking the first number moduloed by the number of message accesses is 1. Therefore, among the message partitions allocated to the first consumer group, the message partition numbered 1, that is, message partition 1, is selected as the first message partition corresponding to the first consumer instance.

[0077] Figure 3 A schematic diagram of a consumer instance partition index provided in an embodiment of the present application, such as Figure 3 As shown:

[0078] A consumer group includes two consumer instances, Consumer A and Consumer B. The message partitions assigned to this consumer group are Partition 1-4.

[0079] In the case of rebalancing, there is a specific correspondence between consumer instances Consumer A and Consumer B and message partitions: Consumer A corresponds to Partition 1 and Partition 2, and Consumer B corresponds to Partition 3 and Partition 4. When consuming messages, Consumer A will only choose one of Partition 1 and Partition 2 for consumption, and Consumer B will only choose one of Partition 3 and Partition 4 for consumption.

[0080] In the absence of rebalancing, a consumer instance can consume from any partition assigned to the consumer group, depending on the number of times it accesses messages. Depending on the number of times it accesses messages, Consumer A can choose any partition from Partitions 1-4 for message consumption. Consumer B can also choose any partition from Partitions 1-4 for message consumption.

[0081] In actual applications, other load balancing algorithms may also be used to index the message partitions of the first consumer instance in the first number of message partitions.

[0082] By determining the first message partition based on the modulo result of the number of message accesses and the first quantity, the message partition can be flexibly selected for message consumption based on the number of message accesses of the first consumer instance, avoiding a fixed correspondence between the consumer instance and the message partition, thereby avoiding the occurrence of rebalancing.

[0083] In one embodiment, extracting the corresponding first message from the first message partition includes:

[0084] Determine a first message location corresponding to the first consumer instance;

[0085] The first message is extracted from the first message partition according to the first message location.

[0086] Here, when extracting the corresponding first message from the first message partition, it is necessary to first determine the first message location corresponding to the first consumer instance. The first message location represents the current consumption progress of the first consumer instance in the message queue of the message partition. For example, when the first message location is 4, it indicates that the first consumer instance has consumed the message numbered 4 in the message queue of the message partition.

[0087] According to the first message location, the message of the corresponding location in the first message partition is extracted as the first message. For example, when the first consumption location is 4, the message with location 4 is extracted from the message queue in the first message partition as the first message.

[0088] By extracting the corresponding first message in the first message partition according to the first message location corresponding to the first consumer instance, the corresponding message can be accurately extracted according to the current consumption progress of the first consumer instance, thereby improving the accuracy of message consumption.

[0089] In one embodiment, before receiving a message request from a first consumer instance in a first consumer group, the method further includes:

[0090] Cache the first message location corresponding to the first consumer instance; wherein,

[0091] The message location represents the consumption progress of the message queue in the message partition by the corresponding consumer instance.

[0092] Here, the first consumption site corresponding to the first consumer instance is pre-cached, and the consumption site represents the consumption progress of the corresponding consumer instance on the message queue in the message partition.

[0093] Before de-balancing, the consumer instance saves and manages its own corresponding message location. In an embodiment of the present application, the message locations of all consumer instances in a consumer group are saved in the message server, and the message server stores and manages the message locations corresponding to the consumer instances. The message server also asynchronously submits the stored message locations corresponding to the consumer instances to the disk for persistence processing.

[0094] In practical applications, a key-value database can be used to store the message locations corresponding to the consumer instances. This can store the message locations corresponding to all consumer instances within a consumer group and bind the message locations to the consumer group's ID. In a key-value database, the key consists of at least three parts: the consumer group ID, the topic, and the message partition number. The value can include the message location corresponding to the consumer instance.

[0095] Figure 4 A schematic diagram of the storage status of the key-value database provided in the embodiment of the present application is shown as follows: Figure 4 As shown:

[0096] The key part can include the format version of the message middleware, the group name of the consumer group, the topic, and the number of the message partition assigned to the consumer group.

[0097] The value part can include the format version of the message middleware, the message location corresponding to all consumer instances in the consumer group, the replica leader epoch value of the message location corresponding to the consumer instance, custom metadata, and timestamp.

[0098] By pre-saving the message location corresponding to the consumer instance, when receiving a message request from the consumer instance, the corresponding message can be quickly extracted and returned to the consumer instance, thereby improving the efficiency of message consumption.

[0099] In one embodiment, the method further comprises:

[0100] When the first message is successfully received by the first consumer instance, the first message location corresponding to the first consumer instance is updated.

[0101] Here, when the first message is successfully received by the first consumer instance, it indicates that the first consumer instance has successfully obtained the required message and the message request has ended. Therefore, it is necessary to update the consumption progress of the first consumer and therefore update the first message location corresponding to the first consumer instance.

[0102] By updating the corresponding message location after the consumer instance successfully consumes, it can be ensured that the consumer instance can subsequently consume accurately based on the correct message location.

[0103] In one embodiment, after returning the first message to the first consumer instance, the method further includes:

[0104] Creating a first thread, and saving the first message to the first thread;

[0105] If the second message is not monitored within the set time, the first message saved in the first thread is sent to the first consumer instance; wherein,

[0106] The second message represents a confirmation message returned by the first consumer instance regarding receipt of the first message.

[0107] Here, after returning the first message to the first consumer instance, a first thread is created and the first message is saved in the first thread. If no confirmation message regarding the received first message is received from the first consumer instance within a set time period, it indicates that the first message was not successfully received by the first consumer instance. In this case, the first message saved in the first thread is sent to the first consumer instance. In actual applications, a retry queue can be created and the first message saved in the first thread can be sent to the retry queue, which then sends the first message to the first consumer instance.

[0108] By creating the first thread and resending the first message when the first message is not successfully received, the success rate of receiving the first message can be increased through the retry mechanism, thereby improving consumption efficiency.

[0109] In one embodiment, the method further comprises:

[0110] In a case where the second message is monitored within the set time period, the first message is deleted from the first thread.

[0111] Here, if a confirmation message regarding the successful receipt of the first message is heard from the first consumer instance within the set time period, it indicates that the first message has been successfully received, and therefore there is no need to retransmit the first message to the first consumer instance. Therefore, in order to save the memory space of the first thread, the first message is deleted from the first thread.

[0112] By deleting the first message from the first thread when the second message is monitored, storage space can be saved.

[0113] In one embodiment, the method may further include: storing, by a first messaging middleware, a correspondence between the first consumer group and its assigned message partitions, and having a messaging server calculate messages corresponding to the message request of the first consumer instance. Here, storage and computation are separated on the messaging server, with the messaging server performing only computation and the first messaging middleware performing storage. This eliminates the correspondence between consumer instances and message partitions in the messaging server, thus avoiding the need for rebalancing. In practical applications, the first messaging middleware may be Pulsar.

[0114] Figure 5 A flow chart of a message processing method provided by an embodiment of the present application is shown in FIG. Figure 5 As shown:

[0115] The consumer instance pulls the request from the message server's request queue. The message server finds the corresponding message location for the consumer instance in the cached message locations, extracts the corresponding message from the message partition based on the found message location, returns the extracted message to the consumer instance, and then commits the message location update in the cached message location. A retry thread is created and the extracted message is sent to the retry thread. After the retry thread receives a confirmation message from the consumer instance, it deletes the saved extracted message. If the retry thread does not receive a confirmation message from the consumer instance, it puts the previously stored extracted message into the retry queue and enters the retry logic.

[0116] Figure 6 Another flowchart of the message processing method provided by the application embodiment of the present application is as follows Figure 6 As shown:

[0117] The consumer instance initiates a request to the request queue, and the request queue pulls the request from the processing thread. After receiving the message request, the processing thread queries the message location corresponding to the consumer instance in the cached message location, and selects a message partition in the message partition corresponding to the consumer group to which the consumer instance belongs. After querying the corresponding message in the message partition, the message is returned to the processing thread. After receiving the returned message, the processing thread updates the message location corresponding to the consumer instance in the cached message location. The processing thread then returns the message to the consumer instance. A retry thread is added. If the retry thread does not listen to the confirmation message returned by the consumer instance within the set time, the message returned to the consumer instance will be sent to the retry queue, which will resend it.

[0118] Figure 7 Another flowchart of the message processing method provided by the application embodiment of the present application is as follows Figure 7 As shown:

[0119] After the processing thread starts executing, it detects whether there is a message request for message consumption. If there is a message request, it creates a message collection, finds the corresponding message according to the message request, adds a retry thread, updates the message location corresponding to the consumer instance, and returns the found message to the consumer instance.

[0120] Figure 8 Another flowchart of the message processing method provided by the application embodiment of the present application is as follows Figure 8 As shown:

[0121] When the message server receives message requests from multiple consumer instances at the same time, the processing thread creates a request queue and coordinates resources between consumer instances at the end of the request queue. In an embodiment of the present application, a spin lock (CAS, CompareAndSet) method is used to sort consumer instances accessing the message server. The CAS method can implement lock-free programming. When multiple consumer instances use the CAS method to insert into the end of the request queue at the same time, only one consumer instance will be successfully inserted, and the remaining consumer instances will fail. Because the CAS method corresponds to 3 operands, the memory value V, the old expected value A, and the new expected value B that needs to be modified, the old expected value A is compared with the memory value V. If the two are equal, it means that the old expected value has not been modified. At this time, the new expected value B is assigned to the memory value V.

[0122] It's important to note that when a message server receives a message request from a consumer instance, the relevant thread performs the necessary checks on the message request and then places it in the request queue. The time required from receiving a message request to placing it in the request queue is very short, so even if the message server handles a large number of message requests, it won't overwhelm threads.

[0123] It should be noted that before de-balancing, the consumer instance needs to submit its own corresponding message location to the message server after successful consumption. The consumer instance generally submits the message location synchronously. However, when submitting the message location synchronously, the consumer instance program will be in a blocked state, and the blocked state will continue until the message server returns the submission result. In any system, blocking caused by program rather than resource limitations will greatly reduce the performance of the system and will also affect the system throughput TPS of the entire system. If the consumer instance submits its own corresponding message location asynchronously, if the message location submission fails, if the message location submission is automatically retried, the message location submitted during the retry may have expired or is not the current latest message location. Therefore, retrying the submission in the case of asynchronous submission is meaningless, so asynchronous submission will not automatically retry the submission. In the embodiment of the present application, the consumer instance does not need to submit its own corresponding message location after successful consumption, because the corresponding message location is stored and submitted for update by the message server. After successful consumption, the consumer instance only needs to asynchronously submit a confirmation message about the successfully consumed message to the message server. The message server creates a retry logic for messages returned to consumer instances. If a confirmation message is received from the consumer instance, the message stored in the retry thread is deleted. If no confirmation message is received from the consumer instance, the message stored in the retry thread is sent to the retry queue, which then executes the retry logic. This avoids the problem of automatic retry failure when the consumer instance asynchronously submits a message.

[0124] In an embodiment of the present application, a message request from a first consumer instance in a first consumer group is received; a first message partition is determined based on a first quantity and a message access parameter of the first consumer instance; the first quantity represents the number of message partitions allocated to the first consumer group; the corresponding first message is extracted from the first message partition, and the first message is returned to the first consumer instance. In this way, a consumer instance in the consumer group can consume any message partition in the message partition corresponding to the consumer group according to the message access parameter. There is no fixed correspondence between the consumer instance and the message partition. In this way, even if the number of consumer instances in the consumer group changes, all consumer instances will not stop consuming messages, thereby avoiding the problem of large backlog or repeated consumption of messages in a short period of time, and improving the efficiency of message consumption.

[0125] In order to implement the method of the embodiment of the present application, the embodiment of the present application also provides a message processing device, Figure 9 For a schematic diagram of the message processing device provided in the embodiment of the present application, please refer to Figure 9 , the device comprises:

[0126] A receiving unit 901 is configured to receive a message request from a first consumer instance in a first consumer group;

[0127] A determining unit 902 is configured to determine a first message partition based on a first quantity and a message access parameter of the first consumer instance, wherein the first quantity represents a number of message partitions allocated to the first consumer group;

[0128] The returning unit 903 is configured to extract the corresponding first message from the first message partition and return the first message to the first consumer instance.

[0129] In one embodiment, the device further includes: a first cache unit, configured to cache first information corresponding to the first consumer group; wherein,

[0130] The first information at least represents the correspondence between the first consumer group and the allocated message partition.

[0131] In one embodiment, the determining unit 902 is further configured to modulo the first quantity by the number of message accesses of the first consumer instance;

[0132] Among all message partitions allocated to the first consumer group, determine the message partition whose corresponding number is the same as the modulo result as the first message partition;

[0133] Among them, all message partitions allocated to the first consumer group are numbered sequentially.

[0134] In one embodiment, the returning unit 903 is further configured to determine a first message location corresponding to the first consumer instance;

[0135] The first message is extracted from the first message partition according to the first message location.

[0136] In one embodiment, the apparatus further includes: a second cache unit configured to cache a first message location corresponding to the first consumer instance; wherein,

[0137] The message location represents the consumption progress of the message queue in the message partition by the corresponding consumer instance.

[0138] In one embodiment, the apparatus further includes: an updating unit, configured to update a first message location corresponding to the first consumer instance when the first message is successfully received by the first consumer instance.

[0139] In one embodiment, the apparatus further includes: a creating unit, configured to create a first thread and save the first message into the first thread;

[0140] If the second message is not monitored within the set time, the first message saved in the first thread is sent to the first consumer instance; wherein,

[0141] The second message represents a confirmation message returned by the first consumer instance regarding receipt of the first message.

[0142] In one embodiment, the apparatus further includes: a deleting unit, configured to delete the first message from the first thread if the second message is monitored within the set time period.

[0143] In actual application, the receiving unit 901, the determining unit 902, the returning unit 903, the first cache unit, the second cache unit, the updating unit, the creating unit, and the deleting unit can be implemented by a processor in the terminal, such as a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU), or a programmable gate array (FPGA).

[0144] It should be noted that the message processing device provided in the above embodiment only uses the division of the above-mentioned program modules as an example to illustrate information display. In actual applications, the above-mentioned processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the above-described processing. In addition, the message processing device provided in the above embodiment and the message processing method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0145] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present application, the embodiment of the present application also provides an electronic device. Figure 10 A schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application is shown in FIG. Figure 10 As shown, the electronic equipment includes:

[0146] Communication interface 1001, capable of exchanging information with other devices such as network devices;

[0147] The processor 1002 is connected to the communication interface 1001 to implement information exchange with other devices and is used to execute the methods provided by one or more technical solutions on the terminal side when running a computer program. The computer program is stored in the memory 1003.

[0148] Specifically, the processor 1002 is used to receive a message request from a first consumer instance in a first consumer group; determine a first message partition based on a first quantity and a message access parameter of the first consumer instance; the first quantity represents the number of message partitions allocated to the first consumer group; extract the corresponding first message from the first message partition, and return the first message to the first consumer instance.

[0149] In one embodiment, before receiving the message request of the first consumer instance in the first consumer group, the processor 1002 is further configured to cache the first information corresponding to the first consumer group; wherein,

[0150] The first information at least represents the correspondence between the first consumer group and the allocated message partition.

[0151] In one embodiment, the processor 1002 is further configured to modulo the first quantity by the number of message accesses of the first consumer instance;

[0152] Among all message partitions allocated to the first consumer group, determine the message partition whose corresponding number is the same as the modulo result as the first message partition;

[0153] Among them, all message partitions allocated to the first consumer group are numbered sequentially.

[0154] In one embodiment, the processor 1002 is further configured to determine a first message location corresponding to the first consumer instance;

[0155] The first message is extracted from the first message partition according to the first message location.

[0156] In one embodiment, before receiving the message request of the first consumer instance in the first consumer group, the processor 1002 is further configured to cache the first message location corresponding to the first consumer instance; wherein,

[0157] The message location represents the consumption progress of the message queue in the message partition by the corresponding consumer instance.

[0158] In one embodiment, the processor 1002 is further configured to update a first message location corresponding to the first consumer instance when the first message is successfully received by the first consumer instance.

[0159] In one embodiment, after returning the first message to the first consumer instance, the processor 1002 is further configured to create a first thread and save the first message to the first thread;

[0160] If the second message is not monitored within the set time, the first message saved in the first thread is sent to the first consumer instance; wherein,

[0161] The second message represents a confirmation message returned by the first consumer instance regarding receipt of the first message.

[0162] In one embodiment, the processor 1002 is further configured to delete the first message from the first thread if the second message is monitored within the set time period.

[0163] Of course, in actual application, the various components in the electronic device are coupled together through the bus system 1004. It is understood that the bus system 1004 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1004 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 10 Various buses are labeled as bus system 1004.

[0164] The memory 1003 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program used to operate on the electronic device.

[0165] It is understood that the memory 1003 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a magnetic disk memory or a magnetic tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memory 1003 described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memories.

[0166] The methods disclosed in the above embodiments of the present application can be applied to or implemented by processor 1002. Processor 1002 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 1002 or by software instructions. The above processor 1002 may be a general-purpose processor, a DSP, or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. Processor 1002 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium located in memory 1003. Processor 1002 reads the program in memory 1003 and, in conjunction with its hardware, completes the steps of the above method.

[0167] When the processor 1002 executes the program, the corresponding processes in the various methods of the embodiments of the present application are implemented.

[0168] In an exemplary embodiment, the present application also provides a storage medium, namely, a computer storage medium, specifically a computer-readable storage medium, including, for example, a memory 1003 storing a computer program. The computer program can be executed by the processor 1002 to perform the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.

[0169] In the several embodiments provided in this application, it should be understood that the disclosed devices, terminals and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0170] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0171] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above-mentioned integrated units can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0172] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, ROM, RAM, disks or optical disks, etc. Various media that can store program codes.

[0173] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0174] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A message processing method, characterized in that: Applied to a message server, the method includes: Receive a message request from a first consumer instance in a first consumer group; Determining a first message partition based on a first quantity and a message access parameter of the first consumer instance; the first quantity represents the number of message partitions allocated to the first consumer group; the message access parameter includes at least a current access count of the first consumer instance; there is no fixed correspondence between the first consumer instance and the message partition; Extracting the corresponding first message from the first message partition, and returning the first message to the first consumer instance; Determine a first message partition based on a first quantity and a message access parameter of the first consumer instance, including: taking a modulus of the first quantity by the number of message accesses of the first consumer instance; among all message partitions assigned to the first consumer group, determine a message partition with a corresponding number identical to the modulo result as the first message partition; wherein all message partitions assigned to the first consumer group are numbered sequentially.

2. The message processing method according to claim 1, wherein: Before receiving the message request from the first consumer instance in the first consumer group, the method further includes: Cache the first information corresponding to the first consumer group; wherein, The first information at least represents the correspondence between the first consumer group and the allocated message partition.

3. The message processing method according to claim 1, wherein: The extracting the corresponding first message from the first message partition includes: Determine a first message location corresponding to the first consumer instance; The first message is extracted from the first message partition according to the first message location.

4. The message processing method according to claim 3, wherein: Before receiving the message request from the first consumer instance in the first consumer group, the method further includes: Cache the first message location corresponding to the first consumer instance; wherein, The message location represents the consumption progress of the message queue in the message partition by the corresponding consumer instance.

5. The message processing method according to claim 4, characterized in that: The method further comprises: When the first message is successfully received by the first consumer instance, the first message location corresponding to the first consumer instance is updated.

6. The message processing method according to claim 1, wherein: After returning the first message to the first consumer instance, the method further includes: Creating a first thread, and saving the first message to the first thread; If the second message is not monitored within the set time, the first message saved in the first thread is sent to the first consumer instance; wherein, The second message represents a confirmation message returned by the first consumer instance regarding receipt of the first message.

7. The method according to claim 6, characterized in that The method further comprises: In a case where the second message is monitored within the set time period, the first message is deleted from the first thread.

8. A message processing device, characterized in that: The device comprises: A receiving unit, configured to receive a message request from a first consumer instance in a first consumer group; a determining unit, configured to determine a first message partition based on a first quantity and a message access parameter of the first consumer instance; the first quantity representing the number of message partitions allocated to the first consumer group; the message access parameter including at least a current access count of the first consumer instance; and there is no fixed correspondence between the first consumer instance and the message partition; a returning unit, configured to extract the corresponding first message from the first message partition and return the first message to the first consumer instance; The determination unit is further used to determine the first message partition based on the first quantity and the message access parameter of the first consumer instance, including: taking the modulus of the first quantity by the number of message accesses of the first consumer instance; among all message partitions allocated to the first consumer group, determining the message partition whose corresponding number is the same as the modulo result as the first message partition; wherein all message partitions allocated to the first consumer group are numbered sequentially.

9. An electronic device, characterized in that: include: A processor and a memory for storing a computer program capable of being executed on the processor, wherein When the processor is used to run the computer program, the processor performs the steps of the method according to any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images