Consumption progress management method and device, equipment, medium and product

Abstract

The invention relates to the technical field of computers, and particularly provides a consumption progress management method and device, equipment, a medium and a product. In the disclosure, a pulling request from a consumption end is received, and the pulling request carries consumer group information; obtaining at least one consumption request message corresponding to the pull request from a storage message log; the consumption request messages are filtered based on at least one interval object maintained in the consumption progress queue, and the interval object is used for representing the consumption progress of each consumption request message in an interval determined based on the consumer group information; and returning the filtered consumption request message to the consumption end. The consumption progress of the consumption group on the consumption queue is represented through the opening and closing interval, that is to say, the confirmed range of the consumption message can be determined. Therefore, the consumption condition of the whole queue can be expressed by using a small number of intervals, each message does not need to be recorded independently, the condition of repeated consumption is avoided, and the complexity of consumption progress management is reduced.

Description

Technical Field

[0001] This disclosure belongs to the field of computer technology, and specifically relates to a consumption progress management method, apparatus, equipment, medium and product. Background Technology

[0002] RocketMQ (Apache RocketMQ) is a distributed message queue with a highly available distributed deployment architecture. It can meet the requirement of at least once message delivery and has few dependencies. In RocketMQ, messages are stored in multiple queues within a specified topic in the order they arrive at the server. Each message has a unique Long type coordinate (message position or offset) in the queue. RocketMQ's consumption progress management ensures that consumers can accurately continue processing unprocessed messages in different states (such as normal operation, restart, abnormal shutdown, etc.).

[0003] In related technologies, the consumption progress manager typically records the consumption position of the first unconsumed consumption message in the consumption queue. This management method, when a consumer instance suddenly exits and the consumption queue is reassigned to a new consumer instance, will leave the consumption progress at the first unconsumed message, potentially leading to duplicate consumption of already consumed information in the queue. Therefore, this increases the complexity of consumption progress management. Summary of the Invention

[0004] This disclosure addresses some deficiencies mentioned in the background art by providing a consumption schedule management method, apparatus, equipment, medium, and product that can improve the efficiency of consumption schedule management.

[0005] In a first aspect, embodiments of this disclosure provide a consumption progress management method, comprising: Receive a pull request from the consumer, the pull request carrying consumer group information; Obtain at least one request consumption message corresponding to the pull request from the stored message log; Based on at least one interval object maintained in the consumption progress queue, the at least one request consumption message is filtered, wherein the interval object is used to characterize the consumption progress of each request consumption message in the interval determined based on the consumer group information. The filtered request consumption message is returned to the consumer.

[0006] Optionally, the interval object includes multiple bit set objects, each bit set object including at least one key-value mapping relationship, wherein the key of the bit set object corresponds to the filename of the consumption progress queue, and the value of the bit set object corresponds to the status of the consumed message in the consumption progress queue.

[0007] Optionally, the pull request carries at least one of the following information: the topic of the requested message to be consumed, the queue number, and the sequence number of the message to be consumed.

[0008] Optionally, obtaining at least one request consumption message corresponding to the pull request from the stored message log includes: Based on the topic and the queue number, obtain the corresponding consumption progress queue; Based on the consumption message sequence number and the consumption progress queue, the corresponding request consumption message is retrieved from the stored message log.

[0009] Optionally, filtering the request consumption message based on at least one interval object maintained in the consumption progress queue includes: The consumption status of the request consumption message is determined based on the value of the bit set object in the interval object; Based on the consumption status and the consumption message sequence number, determine whether the request to consume message is a consumption message that has already been consumed; If the request to consume message is a message that has already been consumed, then the request to consume message is filtered out.

[0010] Optionally, the method further includes: When all consumed messages corresponding to the first bit set object of the interval object have been consumed, the first bit set object is deleted. When the retention time of all consumed messages in the stored message log exceeds the first threshold, the consumption progress queue and bit set object corresponding to the consumed message are deleted.

[0011] Optionally, the method further includes: Receive feedback information sent by the consumer, the feedback information carrying the consumption message sequence number of the message that has been consumed; Update the interval object based on the consumed message sequence number.

[0012] In a second aspect, embodiments of this disclosure provide a consumption progress management device, comprising: The first receiving module is used to receive a pull request from the consumer, the pull request carrying consumer group information; The acquisition module is used to acquire at least one request consumption message corresponding to the pull request from the stored message log; A filtering module is used to filter the at least one request consumption message based on at least one interval object maintained in the consumption progress queue, wherein the interval object is used to characterize the consumption progress of each request consumption message in the interval determined based on the consumer group information. The sending module is used to return the filtered request consumption message to the consumer.

[0013] In a third aspect, embodiments of this disclosure provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described consumption progress management method.

[0014] In a fourth aspect, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the above-described consumption progress management method.

[0015] In a fifth aspect, embodiments of this disclosure provide a computer program product including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the above-described consumption progress management method.

[0016] In this disclosure, a pull request is received from the consumer, carrying consumer group information. At least one request consumption message corresponding to the pull request is retrieved from the stored message log. Based on at least one interval object maintained in the consumption progress queue, the request consumption messages are filtered. The interval object represents the consumption progress of each request consumption message within an interval determined by the consumer group information. The filtered request consumption messages are then returned to the consumer. Open and closed intervals represent the consumption progress of the consumer group in the consumption queue, meaning the range of acknowledged messages can be determined. This allows the consumption status of the entire queue to be represented by a small number of intervals, eliminating the need to record each message individually. During a pull request, request consumption messages can be filtered to check the consumption status of messages within an interval, sending only unacknowledged request consumption messages to the consumer, thus avoiding duplicate consumption. Therefore, the complexity of consumption progress management is reduced.

[0017] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the prior art of recording consumption progress as provided in this disclosure.

[0019] Figure 2A flowchart of a consumption progress management method provided in this disclosure.

[0020] Figure 3 This is a schematic diagram of the structure of the key value of the bit set object provided in this disclosure.

[0021] Figure 4 This is a schematic diagram of a consumption progress management device provided in this disclosure.

[0022] Figure 5 This is a hardware block diagram of an electronic device provided in this disclosure.

[0023] Figure 6 This is a schematic diagram of a computer program product provided in this disclosure. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solution of this application, the application scenario of this application will be described first below.

[0025] RocketMQ (Apache RocketMQ) is a distributed message queue. It features low latency, high concurrency, high availability, and high reliability. Written in Java, RocketMQ supports transactional messages, sequential messages, batch messages, timed messages, and message backtracking. Furthermore, it supports publish / subscribe (Pub / Sub) and peer-to-peer (P2P) messaging models, as well as pull and push messaging modes. RocketMQ boasts a distributed, highly available deployment architecture, meeting at least-once message delivery requirements with minimal dependencies. As a core infrastructure for implementing distributed services, RocketMQ plays a crucial role in module decoupling, traffic smoothing, and asynchronous processing, holding a pivotal position in modern application architecture design. It is widely used in traditional industries such as transportation, healthcare, and education to improve system stability and efficiency, and also provides strong support for emerging technologies such as cloud computing and the Internet of Things, driving their rapid development.

[0026] Regarding RocketMQ consumption, all message consumers within the same consumer group share all messages under the same message topic. The same message (which is also the same queue) will only be consumed by one consumer within the group at a time, and its consumption is reloaded as the consumption queue dynamically changes. Therefore, consumption progress needs to be stored in a place accessible to every consumer (the message server broker). Consumption progress refers to the progress of each consumer group in consuming a specific queue, i.e., the position (ConsumerOffset) of the latest consumed message. In RocketMQ, messages are stored in multiple queues of a specified topic in the order they arrive at the server. Each message has a unique Long type coordinate (message position or Offset) in the queue. Managing consumption progress in RocketMQ ensures that consumers can accurately continue processing unprocessed messages in different states (such as normal operation, restart, abnormal shutdown, etc.).

[0027] In related technologies, RocketMQ's current consumption progress is stored in the Broker's ConsumerOffset.json file, which records the topics consumed by the consumer group and the consumption progress of each queue within those topics. After the Broker restarts, consumers retrieve the consumption progress of each queue from this configuration file through the Broker's interface and begin consuming from that progress. After a consumer finishes consuming a batch of messages, it saves the consumption progress to the local memory of its own progress manager. First, it removes messages from the queue consumption snapshot, returns the first offset of the consumption snapshot msgTreeMap, and then calls the consumption message progress manager offsetStore to update the consumption progress. This consumption progress is periodically and asynchronously reported to the server-side Broker.

[0028] The following is a method for updating the consumption position offset in the consumption queue in related technologies. Figure 1 This is a schematic diagram of the prior art recording consumption progress provided in this disclosure, such as... Figure 1 As shown: Scenario 1: In the snapshot, messages 1001 (message 1) and 1010 (message 10) have been consumed. There are no more messages in the snapshot. Return the maximum offset of the consumed messages + 1, which is 1011.

[0029] Scenario 2: In the snapshot, messages 1001 (message 1) to 1008 (message 8) have been consumed, and only two messages remain in the snapshot. Return the smallest offset, 1009.

[0030] Scenario 3: In the snapshot, message 1001 (message 1) has not been consumed in the consumer object for some reason, even though subsequent messages 1005-1010 have been consumed. The minimum offset returned is 1001.

[0031] Regarding scenario 3 above, to ensure messages are successfully consumed, RocketMQ maintains its consumption progress at 1001 (message 1) until all messages at 1001 are consumed, at which point the local consumption progress is updated to 1011. If the consumer instance suddenly exits before message 1001 (message 1) is fully consumed (e.g., due to a power outage), there is a risk of duplicate consumption. Because the queue's consumption progress remains at 1001, when the queue is reassigned to a new consumer instance, the new instance will still receive a consumption progress of 1001 from the Broker. This will cause it to start consuming from 1001 again, while messages 1005-1010 have already been consumed.

[0032] In other words, the consumption progress manager typically records the consumption position of the first unconsumed message in the consumption queue. With this management method, if a consumer instance suddenly exits and the consumption queue is reassigned to a new consumer instance, the consumption progress will remain at the first unconsumed message. This could lead to duplicate consumption of messages already consumed in the queue, thus increasing the complexity of consumption progress management.

[0033] To address the aforementioned technical issues, this disclosure provides an inventive concept: In the message server broker, a range-based consumption progress file, `consumerRangeOffset`, is designed. Instead of using a single number to represent the consumption progress of a consumer group within a queue, this file uses open and closed ranges to indicate the consumption progress of the consumer group within the queue. This means the range of acknowledged messages can be determined. This allows the consumption status of the entire queue to be represented by a small number of ranges, eliminating the need to record each message individually. When fetching requests, the broker retrieves messages from disk / memory, filters the requested consumption messages, checks the consumption status of messages within the specified ranges, and only sends unacknowledged requests to the consumer, preventing duplicate consumption. Therefore, the complexity of consumption progress management is reduced.

[0034] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present disclosure and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the drawings, not the entire structure.

[0035] Figure 2 A flowchart illustrating a consumption progress management method provided in this disclosure. Figure 2 As shown, the method includes: S201: Receive a pull request from the consumer.

[0036] Specifically, in this embodiment, there are business scenarios involving user interaction between the consumer and the server-side of consumption progress management, such as payment notifications and order status updates. When executing these business processes, the server receives pull requests from the consumer, whereby the pull request is a request to retrieve consumption messages, which carries the consumer group information corresponding to the consumption messages.

[0037] S202: Retrieve at least one request consumption message corresponding to the pull request from the stored message log.

[0038] Specifically, the server stores the message logs of consumed messages on its disk / memory. The commit log stores the consumption progress of the requested messages. The requested messages are the messages that the user needs to consume when performing business operations, such as the nth message. The retrieval request may include multiple requested messages.

[0039] S203: Filter the request consumption messages based on at least one range object maintained in the consumption progress queue.

[0040] Specifically, in related technologies, message acknowledgment only requires confirming one message, and the consumption progress can be moved forward by multiple messages. For example, if consumer group 1 confirms message 4, then messages from 0 to 4 have all been confirmed, meaning messages are confirmed one by one. In actual RocketMQ usage, the consumer inevitably needs updates and iterations, requiring updates and restarts; or the consumption logic may trigger bugs, causing the consumption of a certain message to "get stuck"; or consumers may be added or removed. In these scenarios, some messages may have actually been consumed, but because the preceding messages have not yet been confirmed, the confirmation progress of these messages cannot be submitted, resulting in meaningless duplicate consumption.

[0041] In this disclosure, by retrieving data from the request, a corresponding consumption progress queue (consumequeue) can be obtained. After obtaining the corresponding consumptionqueue, the consumption progress of the requested messages can be determined, thereby filtering out messages that have already been consumed and obtaining the unconsumed requested messages. The consumption progress queue is used to maintain interval objects, which represent the consumption progress of each requested message within an interval determined based on consumer group information. This allows for the expression of whether a massive number of messages have been acknowledged with minimal storage space. Furthermore, the design of interval consumption progress file management avoids excessive storage resource consumption in scenarios with high business volume.

[0042] S204: Return the filtered request consumption message to the consumer.

[0043] Specifically, the filtered, unconsumed request messages are returned to the consumer for users to use.

[0044] In this disclosure, a pull request is received from the consumer, carrying consumer group information. At least one request consumption message corresponding to the pull request is retrieved from the stored message log. Based on at least one interval object maintained in the consumption progress queue, the request consumption messages are filtered. The interval object represents the consumption progress of each request consumption message within an interval determined by the consumer group information. The filtered request consumption messages are then returned to the consumer. Open and closed intervals represent the consumption progress of the consumer group in the consumption queue, meaning the range of acknowledged messages can be determined. This allows the consumption status of the entire queue to be represented by a small number of intervals, eliminating the need to record each message individually. During a pull request, request consumption messages can be filtered to check the consumption status of messages within an interval, sending only unacknowledged request consumption messages to the consumer, thus avoiding duplicate consumption. Therefore, the complexity of consumption progress management is reduced.

[0045] In one possible implementation, the interval object includes multiple bit set objects, each bit set object including at least one key-value mapping, wherein the key of the bit set object corresponds to the filename in the consumption progress queue, and the value of the bit set object corresponds to the status of the consumed message in the consumption progress queue.

[0046] In related technologies, the consumption progress offsetTable is a Map object, which is periodically persisted to the consumerOffset.json file. Its data structure for storing consumed messages is shown below: ConcurrentHashMap <String,ConcurrentMap<Integer,Long> >(); In this structure, the key of the object is the topic name@consumer group name, and the value is {queue number, consumption progress}, for example, "topicA@consumerGroupA":{0:3}. The above example structure indicates that consumer group consumerGroupA has consumed messages 0, 1, and 2 in queue 0 of topicA, while message 3 has not yet been acknowledged. In other words, it records the first unconsumed message, while the consumption status of messages after message 3 in the queue cannot be confirmed.

[0047] In the embodiments disclosed herein, the consumption progress data structure is improved to an interval consumption progress. The interval objects in the interval consumption progress file consumerRangeOffset include multiple bit set objects, where the bit set objects are BitSet objects. Taking a BitSet object as an example, a BitSet object includes the correspondence between the consumption progress queue and the status of the consumed messages it stores. Typically, the value of a BitSet object can correspond to the status of 8 consumed messages; for example, 1 indicates that it has been acknowledged, and 0 indicates that it has not been acknowledged.

[0048] The BitSet data structure used to record whether each message has been acknowledged is shown below: ConcurrentHashMap <String,ConcurrentMap<Integer,Map<Long,BitSet> >>(); The key of the interval object is the filename of the consumequeue. The interval object includes a BitSet object, which records which messages in this consumequeue have been acknowledged (for example, 1 indicates acknowledged and 0 indicates unacknowledged).

[0049] For example, “topicA@consumerGroupA”:{0:{00001111:1110111000}}. Here, 0 represents the consumption queue; a topic has four queues by default: 0, 1, 2, and 3; 00001111 is the filename of the consumption progress queue; and 1110111000 indicates the status of the message consumption request. Figure 3 This is a schematic diagram of the structure of the key-value pairs of the bit set object provided in this disclosure, as shown below. Figure 3 As shown, an 8-bit BitSet indicates that messages 0, 1, 2, 4, and 5 have been confirmed, while messages 3, 6, and 7 have not yet been confirmed.

[0050] Typically, memory addresses are addressed in bytes, meaning even the smallest data types (like bool in some programming languages) occupy at least one byte, even though the actual information they store (0 or 1) can be represented by just one bit. BitSet, through precise control of memory usage, allows the eight bits of a single byte to be set and checked independently, so each bit can store a 0 or 1 value. This makes BitSet significantly more space-efficient than traditional boolean arrays or sets when storing large amounts of boolean values ​​(i.e., 0 and 1). Specifically, BitSet efficiently manages and manipulates these bits through bitwise operations. It allows users to logically access and manipulate individual bits without allocating a full byte for each bit, thus saving memory space.

[0051] Furthermore, the consumption progress data structure can be improved to an interval consumption progress data structure, and the data structure can also be transformed into a List data structure, as shown below: ConcurrentHashMap <String,ConcurrentMap<Integer,List <range>>>(); The object's key is the topic name @ consumer group name, and the object's value is {queue number, range List}. For example, "topicA@consumerGroupA":{0:[0,2],(3,6)} means that consumer group consumerGroupA has consumed messages 0, 1, 2, 4, and 5 in queue 0 of topicA, and message 3 has not yet been consumed and confirmed.

[0052] In one possible implementation, the pull request includes at least one of the following: a topic for requesting to consume messages, a queue number, and a message sequence number; an exemplary method for obtaining at least one message for requesting to consume corresponding to the pull request from the stored message log includes: Based on the topic and queue number, retrieve the corresponding consumption progress queue; based on the consumption message sequence number and consumption progress queue, retrieve the corresponding request consumption message from the stored message log.

[0053] Specifically, the message log stores the consumption progress queue for each consumer group's corresponding topic. Based on the topic and queue number carried in the pull request, the corresponding consumption progress queue can be retrieved. Then, based on the sequence number of the requested message carried in the pull request, and combined with the retrieved consumption progress queue, the content of the corresponding requested message is retrieved from the message log.

[0054] In one possible implementation, an exemplary method for filtering request consumption messages based on at least one interval object maintained in the consumption progress queue includes: Based on the value of the bit set object in the interval object, determine the consumption status of the requested message; based on the consumption status and the message sequence number, determine whether the requested message is a message that has already been consumed; if the requested message is a message that has already been consumed, then filter out the requested message.

[0055] Specifically, in related technologies, the Broker, based on the message pull request from the consumer, retrieves the corresponding consumption progress queue according to the topic and queue number. Then, based on the message number to be pulled, it obtains the physical location of the message in the message log storage from this consumption progress queue, and finally sends the message to the consumer. This process cannot determine whether the requested message has already been consumed, potentially leading to duplicate consumption.

[0056] In the embodiments disclosed herein, the Broker, upon receiving a message pull request from the consumer, retrieves the corresponding consumption progress queue based on the topic and queue number. After retrieving the corresponding consumption progress queue, before pulling the message, it first checks the value of the BitSet corresponding to the interval object to see if the message has already been acknowledged. In other words, it obtains the consumption message status through the consumption progress queue; if the status is "consumed," the request to consume the message is filtered out. Then, it can select unfiltered message requests to send to the consumer.

[0057] In one possible implementation, the method further includes: When all consumed messages corresponding to the first bit set object of the interval object have been consumed, delete the first bit set object.

[0058] Specifically, since each consumer group's corresponding topic's consumption progress queue is associated with a BitSet object, this can consume a significant amount of storage space in scenarios with a large number of consumer groups and topics. In this case, the BitSet object can be deleted after all consumed messages in the head BitSet of the queue have been confirmed, thus avoiding the consumption of storage space.

[0059] When the retention time of all consumed messages in the message log exceeds the first threshold, delete the consumption progress queue and bit set object corresponding to the consumed message.

[0060] Specifically, when the retention time of all consumed messages in the stored message log exceeds the first threshold—that is, after the preset storage period is met—the stored message log expires, triggering RocketMQ's deletion mechanism for the storage file. In this embodiment, RocketMQ will simultaneously delete the corresponding consumption progress queue and the corresponding bit set object.

[0061] In addition, in this embodiment, BitSets corresponding to consumption progress queues that have not yet been consumed by the consumer group are not created in advance to avoid occupying storage space.

[0062] In one possible implementation, the method further includes: Receive feedback information sent by the consumer; update the range object based on the sequence number of the consumed message.

[0063] Specifically, in related technologies, after the consumer retrieves messages from the server-side broker, it stores them in a TreeMap data structure. The key is the sequence number of the consumed message, and the value is the content of the consumed message. After each message is consumed, the consumer deletes the message from the TreeMap and periodically (e.g., every 5 seconds) uploads the consumption progress to the server-side broker. The consumption progress is represented by the key of the first element of the TreeMap, which is the sequence number of the consumed message. Upon receiving the sequence number, the server asynchronously updates the consumption progress file.

[0064] In this disclosure, after each message is consumed, the consumer records the sequence number of the consumed message and periodically (e.g., every 5 seconds) sends a consumption completion feedback message to the server broker, uploading the merged consumption progress. The feedback message carries the sequence number of the consumed message that has been consumed. Upon receiving the sequence number of the consumed message, the server asynchronously updates the interval object of the interval consumption progress queue. Without storing the messages in a TreeMap, the server retrieves the key of the first element and can automatically merge the consumption progress on its own.

[0065] This disclosure improves data quality and ensures business accuracy, which is crucial in critical business scenarios such as finance and e-commerce. Message queues effectively avoid data errors, duplicate deductions, and abnormal order statuses caused by duplicate message processing by reducing duplicate consumption, thus ensuring the accuracy and consistency of business data. This reduces customer disputes caused by data errors and improves customer satisfaction. It also improves system stability and reduces the risk of failure. Duplicate consumption in message queues can lead to decreased system performance, wasted resources, and even system crashes. By reducing duplicate consumption, the system can operate more stably, reducing the risk of business interruptions and failures due to system problems. This improves system reliability and stability, reducing business losses and maintenance costs caused by system problems. Furthermore, it optimizes resource utilization and reduces operating costs. Duplicate consumption leads to the same business logic being executed multiple times, wasting computing and bandwidth resources. By reducing duplicate consumption, the system can process messages more efficiently, reducing unnecessary resource waste. This lowers enterprise operating costs and improves resource utilization and return on investment. Finally, it simplifies error handling processes and reduces maintenance costs. In traditional message queues, developers need to manually handle duplicate messages, increasing system complexity and maintenance costs. Message queues reduce duplicate consumption through built-in mechanisms, simplifying error handling and lowering system maintenance costs. This reduces developer workload and maintenance pressure, improving development efficiency and quality. It can also enhance user experience and strengthen brand influence. In scenarios involving user interaction, such as payment notifications and order status updates, duplicate consumption can lead to users receiving repeated notifications or status updates, causing inconvenience. By reducing duplicate consumption, message queues ensure that users receive notifications or status updates only once, thereby improving user experience.

[0066] Figure 4 This is a schematic diagram of a consumption progress management device provided in this disclosure. Figure 4 As shown, the device 400 includes: a first receiving module 410, an acquisition module 420, a filtering module 430, and a sending module 440.

[0067] The first receiving module 410 is used to receive a pull request from the consumer end, the pull request carrying consumer group information; The acquisition module 420 is used to acquire at least one request consumption message corresponding to the pull request from the stored message log; The filtering module 430 is used to filter the at least one request consumption message based on at least one interval object maintained in the consumption progress queue, wherein the interval object is used to characterize the consumption progress of each request consumption message in the interval determined based on the consumer group information. The sending module 440 is used to return the filtered request consumption message to the consumer.

[0068] Optionally, the interval object includes multiple bit set objects, each bit set object including at least one key-value mapping relationship, wherein the key of the bit set object corresponds to the filename of the consumption progress queue, and the value of the bit set object corresponds to the status of the consumed message in the consumption progress queue.

[0069] Optionally, the pull request carries at least one of the following information: the topic of the requested message to be consumed, the queue number, and the sequence number of the message to be consumed.

[0070] Optionally, the acquisition module includes: The first acquisition submodule is used to acquire the corresponding consumption progress queue based on the topic and the queue number; The second acquisition submodule is used to retrieve the corresponding request consumption message from the stored message log based on the consumption message sequence number and the consumption progress queue.

[0071] Optionally, the filtering module includes: The determination submodule is used to determine the consumption status of the request consumption message based on the value of the bit set object in the interval object; The judgment submodule is used to determine whether the requested consumption message is a consumption message that has already been consumed, based on the consumption status and the consumption message sequence number. The filtering submodule is used to filter out the request to consume message if the request to consume message is a message that has already been consumed.

[0072] Optionally, the device further includes: The first deletion module is used to delete the first bit set object when all consumed messages corresponding to the first bit set object of the interval object have been consumed. The second deletion module is used to delete the consumption progress queue and bit set object corresponding to the consumption message when the retention time of all consumed messages in the stored message log exceeds the first threshold.

[0073] Optionally, the device further includes: The receiving module is used to receive feedback information sent by the consumer, the feedback information carrying the consumption message sequence number of the message that has been consumed. The update module is used to update the interval object based on the consumed message sequence number.

[0074] This application also provides an electronic device for executing the above-described consumption progress management method. Please refer to... Figure 5 It illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 5 As shown, the electronic device 50 includes: a processor 500, a memory 501, a bus 502, and a communication interface 503. The processor 500, the communication interface 503, and the memory 501 are connected via the bus 502. The memory 501 stores a computer program that can run on the processor 500. When the processor 500 runs the computer program, it executes the consumption progress management method provided in any of the foregoing embodiments of this application.

[0075] The memory 501 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 503 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0076] Bus 502 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 501 is used to store programs. After receiving an execution instruction, the processor 500 executes the program. The consumption progress management method disclosed in any of the foregoing embodiments of this application can be applied to the processor 500, or implemented by the processor 500.

[0077] The processor 500 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 500 or by instructions in software form. The processor 500 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 501. The processor 500 reads the information in memory 501 and, in conjunction with its hardware, completes the steps of the above method.

[0078] The electronic device provided in this application embodiment and the consumption progress management method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0079] This application also provides a computer-readable storage medium corresponding to the consumption progress management method provided in the foregoing embodiments. The computer-readable storage medium shown can be an optical disc, on which a computer program is stored. When the computer program is run by a processor, it executes the consumption progress management method provided in any of the foregoing embodiments.

[0080] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0081] The computer-readable storage medium provided in the above embodiments of this application and the consumption progress management method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0082] This application also provides a computer program product 600, such as... Figure 6 As shown. The computer program product carries a computer program 601, the instructions of which can be used to execute the steps of the consumption progress management method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0083] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0084] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0085] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0086] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0087] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0088] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0089] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0090] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.< / range>

Claims

1. A consumption progress management method, characterized in that, include: Receive a pull request from the consumer, the pull request carrying consumer group information; Obtain at least one request consumption message corresponding to the pull request from the stored message log; Based on at least one interval object maintained in the consumption progress queue, the at least one request consumption message is filtered, wherein the interval object is used to characterize the consumption progress of each request consumption message in the interval determined based on the consumer group information. The filtered request consumption message is returned to the consumer.

2. The method according to claim 1, characterized in that, The interval object includes multiple bit set objects, each bit set object including at least one key-value mapping relationship, wherein the key of the bit set object corresponds to the filename of the consumption progress queue, and the value of the bit set object corresponds to the status of the consumed message in the consumption progress queue.

3. The method according to claim 1, characterized in that, The pull request carries at least one of the following information: the subject of the requested message to be consumed, the queue number, and the sequence number of the message to be consumed.

4. The method according to claim 3, characterized in that, The step of retrieving at least one request consumption message corresponding to the pull request from the stored message log includes: Based on the topic and the queue number, obtain the corresponding consumption progress queue; Based on the consumption message sequence number and the consumption progress queue, the corresponding request consumption message is retrieved from the stored message log.

5. The method according to claim 3, characterized in that, The filtering of the request consumption message based on at least one interval object maintained in the consumption progress queue includes: The consumption status of the request consumption message is determined based on the value of the bit set object in the interval object; Based on the consumption status and the consumption message sequence number, determine whether the request to consume message is a consumption message that has already been consumed; If the request to consume message is a message that has already been consumed, then the request to consume message is filtered out.

6. The method according to claim 2, characterized in that, The method further includes: When all consumed messages corresponding to the first bit set object of the interval object have been consumed, the first bit set object is deleted. When the retention time of all consumed messages in the stored message log exceeds the first threshold, the consumption progress queue and bit set object corresponding to the consumed message are deleted.

7. The method according to claim 1, characterized in that, The method further includes: Receive feedback information sent by the consumer, the feedback information carrying the consumption message sequence number of the message that has been consumed; Update the interval object based on the consumed message sequence number.

8. A consumption progress management device, characterized in that, include: The first receiving module is used to receive a pull request from the consumer, the pull request carrying consumer group information; The acquisition module is used to acquire at least one request consumption message corresponding to the pull request from the stored message log; A filtering module is used to filter the at least one request consumption message based on at least one interval object maintained in the consumption progress queue, wherein the interval object is used to characterize the consumption progress of each request consumption message in the interval determined based on the consumer group information. The sending module is used to return the filtered request consumption message to the consumer.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1-7.

Images