A message processing method and device

Abstract

The present invention discloses a message processing method and device, and relates to the field of computer technology. A specific implementation of the method includes: the message middleware finds out the partition with the same message type as the acquisition request according to the received acquisition request; wherein the acquisition request includes the user identification and the message type requested to be acquired, and the partition stores the storage location information of the message; the message middleware selects a segment from the partition according to the consumption location information of the partition; wherein the partition includes at least one segment; the message middleware outputs the message corresponding to the segment to the message user according to the user identification and the storage location information. This implementation method divides each partition into segments, and processes messages in parallel between different segments, thereby doubling the number of messages processed without increasing the number of partitions, thereby reducing the message transmission delay.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a message processing method and device. Background Art

[0002] At present, in order to achieve the purpose of application decoupling and other purposes, a message middleware is introduced for data interaction between different applications. Suppose an application A is deployed in computer room A as the message generator; and an application B is deployed in computer room B as the message consumer. The message generator sends the generated message to the Broker of the message middleware (Broker is an application that receives, saves, and distributes messages, and serves as a proxy for message processing) for temporary storage; the message consumer obtains the message from the Broker and then executes the corresponding business logic. In the above message processing process, on the one hand, the message generator requires that the message can be sent to the Broker quickly and reliably and stored, and on the other hand, the message consumer requires that the message can be pulled from the Broker at the fastest speed and then consumed. Therefore, whether it is slow to send messages or pull messages, it will cause delays in message delivery.

[0003] The existing technology uses ROKETMQ (a distributed message middleware with only one publish-subscribe mode) to process messages, which provides the functions of synchronous writing of a single log file (Journal) and asynchronous indexing of multiple partitions (Partition). The synchronous writing of a single log file ensures that the messages sent by the message generator can be written to the Broker quickly and reliably; the asynchronous indexing of multiple partitions improves the speed of concurrently pulling messages by the message consumer to a certain extent.

[0004] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0005] (1) Although index files are written to disk asynchronously, a disk flushing operation is performed during writing. If there are too many partitions, random disk writes will occur, affecting the synchronous writing of log files and thus causing a decrease in message sending performance.

[0006] (2) Since the number of partitions is fixed and the message processing capacity of each thread of each message consumer is limited, when a large backlog of messages occurs, the backlog of messages cannot be processed in time. Summary of the invention

[0007] In view of this, an embodiment of the present invention provides a message processing method and device, which divides each partition into segments and processes messages in parallel between different segments. This doubles the number of processed messages without increasing the number of partitions, thereby reducing the message transmission delay.

[0008] To achieve the above objective, according to one aspect of an embodiment of the present invention, a message processing method is provided.

[0009] A message processing method according to an embodiment of the present invention comprises: the message middleware searches out a partition having the same message type as the acquisition request according to a received acquisition request; wherein the acquisition request comprises a user identifier and the message type requested to be acquired, and the partition stores storage location information of the message; the message middleware selects a segment from the partition according to the consumption location information of the partition; wherein the partition comprises at least one segment; and the message middleware outputs a message corresponding to the segment to the message user according to the user identifier and the storage location information.

[0010] Optionally, before the message middleware searches out the partition with the same message type as the acquisition request based on the received acquisition request, it also includes: setting at least one segment for each partition; wherein the number of segments on a partition and the number of messages corresponding to a segment are determined according to demand, and the number of segments of all partitions is greater than the number of all partitions.

[0011] Optionally, after the step of outputting the message corresponding to the segment to the message user according to the user identifier and the storage location information, the message middleware also includes: receiving confirmation information corresponding to the message within a predetermined timeout period, if the location information of the confirmation information is continuous within the segment and is continuous within the partition starting from the head of the partition corresponding to the segment, then using the last location information of the continuous confirmation as the confirmation location information of the partition; and updating the consumption location information according to the confirmation location information of the partition.

[0012] Optionally, after the step of outputting the message corresponding to the segment to the message user according to the user identifier and the storage location information, the message middleware further includes: if no confirmation information corresponding to the message is received within the timeout period, re-outputting the message.

[0013] Optionally, after the step of finding out the partition with the same message type as the acquisition request and before the step of the message middleware selecting a segment from the partition according to the consumption location information of the partition, it also includes: if all messages corresponding to all segments in the partition have been consumed, resetting the segment in the partition.

[0014] To achieve the above objective, according to another aspect of an embodiment of the present invention, a message processing device is provided.

[0015] A message processing device according to an embodiment of the present invention comprises: a search module, used to search for a partition with the same message type as the acquisition request according to a received acquisition request; wherein the acquisition request comprises a user identifier and the message type requested to be acquired, and the partition stores storage location information of the message; a selection module, used to select a segment from the partition according to the consumption location information of the partition; wherein the partition comprises at least one of the segments; and an output module, used to output a message corresponding to the segment to a message user according to the user identifier and the storage location information.

[0016] Optionally, the device also includes: a setting module, used to set at least one segment for each partition; wherein the number of segments on a partition and the number of messages corresponding to a segment are determined according to demand, and the number of segments of all partitions is greater than the number of all partitions.

[0017] Optionally, the device also includes: an update module, used to receive confirmation information corresponding to the message within a predetermined timeout period, if the location information of the confirmation information is continuous within the segment and is continuous within the partition starting from the head of the partition corresponding to the segment, then the last location information of the continuous confirmation is used as the confirmation location information of the partition; and the consumption location information is updated according to the confirmation location information of the partition.

[0018] Optionally, the device further comprises: a re-output module, configured to re-output the message if confirmation information corresponding to the message is not received within the timeout period.

[0019] Optionally, the device further comprises: a resetting module, configured to reset the segments in the partition if all messages corresponding to all segments in the partition have been consumed.

[0020] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device is provided.

[0021] An electronic device according to an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement a message processing method according to an embodiment of the present invention.

[0022] To achieve the above objective, according to another aspect of the embodiments of the present invention, a computer readable medium is provided.

[0023] A computer-readable medium according to an embodiment of the present invention stores a computer program, and when the program is executed by a processor, a message processing method according to an embodiment of the present invention is implemented.

[0024] An embodiment of the above invention has the following advantages or beneficial effects: by segmenting each partition and processing messages in parallel between different segments, the number of messages processed is doubled without increasing the number of partitions, thereby reducing the message transmission delay; the number of segments of each partition and the number of messages corresponding to each segment are flexibly set as needed, which supports both high-performance and high-reliability message writing and high-performance message pulling; the consumption location information of the partition and the timeout confirmation mechanism are updated according to the confirmation information, which can ensure that messages will not be lost when supporting high concurrency; by dynamically adjusting the number of segments of each partition, the overall message processing speed is improved.

[0025] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0027] Figure 1 is a schematic diagram of main steps of a message processing method according to an embodiment of the present invention;

[0028] Figure 2 It is an architectural diagram of a message processing method in the prior art;

[0029] Figure 3 is an architecture diagram of a message processing method according to an embodiment of the present invention;

[0030] Figure 4 is a schematic diagram of the main flow of the message processing method according to an embodiment of the present invention;

[0031] Figure 5 is a schematic diagram of main modules of a message processing device according to an embodiment of the present invention;

[0032] Figure 6 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0033] Figure 7 It is a schematic diagram of the structure of a computer device suitable for implementing the electronic device of the embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0035] The following is an explanation of the terms involved in the embodiments of the present invention.

[0036] Message generator: generates messages and sends them to Broker's application.

[0037] Message consumer: An application that subscribes to messages, obtains subscribed messages from the Broker, and processes them.

[0038] Message: Data passed between a message producer and a message consumer.

[0039] Figure 1 FIG. 1 is a schematic diagram of the main steps of the message processing method according to an embodiment of the present invention. Figure 1 As shown, the message processing method of the embodiment of the present invention mainly includes the following steps:

[0040] Step S101: The message middleware finds the partition with the same message type as the acquisition request based on the received acquisition request; wherein the acquisition request includes a user identifier and the type of message requested, and the partition stores the storage location information of the message. The user identifier is used to uniquely identify the message user. After the message generator generates the message, it sends the message to the message middleware; the message middleware stores the message and saves the storage location information of the message. The message middleware receives the acquisition request from the message user, and finds the corresponding partition based on the message type in the request. In the embodiment, the message type is a topic.

[0041] Step S102: The message middleware selects a segment from the partition according to the consumption location information of the partition; wherein the partition includes at least one of the segments. The message middleware pre-sets at least one segment for each partition, and the number of segments on each partition and the number of messages corresponding to each segment are determined according to actual needs. The consumption location information of the partition is used to indicate the messages that have been consumed by the partition (i.e., the messages that have been pulled and processed by the message user), and to select an idle segment (i.e., a segment that is not being consumed) from the unconsumed messages of the partition.

[0042] Step S103: The message middleware outputs the message corresponding to the segment to the message user according to the user identifier and the storage location information. The message middleware obtains all or part of the message corresponding to the segment according to the storage location information, and then outputs it to the corresponding message user.

[0043] Figure 2 It is an architecture diagram of a message processing method in the prior art. Figure 2 As shown in the figure, the message processing method is implemented by Broker. The message generator writes the generated message synchronously to the Broker's Journal. Broker asynchronously creates an index file, which includes 5 partitions, Partition 1 to Partition 5; each small block stores the location information of a message in the Journal. The message consumer pulls messages from Broker and processes them through 5 threads (thread 1 to thread 5).

[0044] Figure 3 is an architectural diagram of a message processing method according to an embodiment of the present invention. Figure 3 As shown, the message processing method of the embodiment of the present invention is implemented by the Broker of the message middleware. The message generator writes the generated message synchronously to the Journal of the Broker. The Broker asynchronously creates an index file, which includes 5 partitions, namely Partition1 to Partition5; each partition is set with 2 segments, the segments of Partition1 are P1_segment 1 and P1_segment 2, the segments of Partition2 are P2_segment 1 and P2_segment 2, the segments of Partition3 are P3_segment 1 and P3_segment 2, the segments of Partition4 are P4_segment 1 and P4_segment 2, and the segments of Partition5 are P5_segment 1 and P5_segment 2; each small square stores the storage location information of a message in the Journal. The message consumer pulls and processes messages from the Broker through 10 threads (thread 1 to thread 10). In the embodiment of the present invention, without increasing the number of partitions, by setting segments, message users can use more threads to pull messages in the same time. Figure 2 The speed of pulling messages is doubled.

[0045] Figure 4 FIG. 1 is a schematic diagram of the main flow of the message processing method according to an embodiment of the present invention. Figure 4 As shown, the message processing method of the embodiment of the present invention mainly includes the following steps:

[0046] Step S401: The message generator connects to the Broker, and then writes each generated message synchronously to the Journal. The original content of the message is saved in the Journal. In the embodiment of the present invention, synchronization means that the next thread can be executed only after one thread is executed; asynchronous means that another thread can be executed while one thread is executed. This method makes full use of the sequential write characteristics of the disk and realizes reliable and low-latency message writing. The message includes topics and partitions. Among them, Topic is the logical classification of the message. For example, if the message includes inventory messages and order messages, Topic1 can be used to store inventory messages and Topic2 can be used to store order messages.

[0047] Step S402: Broker asynchronously creates an index file. The index file stores the storage location information of each message in the Journal. The index file includes at least one Topic, and each Topic includes at least one partition. The storage location information of each message in the Journal is written into the partition of the specified Topic.

[0048] Step S403: Broker sets at least one segment for each partition in advance. The number of segments on each partition and the number of messages corresponding to each segment are determined according to demand, and the number of segments of all partitions is greater than the number of all partitions. Messages can be processed in parallel between different segments. Taking 2 segments set for each partition as an example, the number of concurrently processed messages can be doubled without increasing the number of partitions. The number of segments on each partition and the number of messages corresponding to each segment can be adjusted dynamically. Taking orders as an example, if the merchant conducts promotions within a certain period of time, the number of orders will increase significantly, then the number of segments and the number of messages in the segments can be increased, and the specific values ​​are manually specified.

[0049] Step S404: The message consumer connects to the Broker and sends a request to the Broker through one or more threads. The request includes the consumer identifier and the subject of the message requested. The consumer identifier in the request can be Consumer1, Consumer2, and the subject can be Topic1, Topic2. When there are multiple message consumers, each message consumer connects to the Broker separately and sends a request to the Broker through its own thread.

[0050] Step S405: After receiving the acquisition request, the Broker searches for the partition with the same topic as the acquisition request. Assuming that the partitions corresponding to Topic 1 are Partition 1 to Partition 5, and the partitions corresponding to Topic 2 are Partition 6 to Partition 10, the topic in the acquisition request sent by message consumer 1 to Broker through threads 1 to 10 is Topic 1, and the topic in the acquisition request sent by message consumer 2 to Broker through threads 11 to 15 is Topic 2, then for message consumer 1, the partitions found by Broker are Partition 1 to Partition 5, and for message consumer 2, the partitions found by Broker are Partition 6 to Partition 10.

[0051] Step S406: Broker selects a segment from the partition according to the consumption location information of the partition. Broker maintains a consumption location information for each message consumer in each Topic partition, which is used to record the messages that each message consumer has consumed. When there are multiple partitions in the Topic where the message requested by the message consumer is located, the Broker randomly selects a partition. Assuming that the consumption location information of Partition1 of Topic1 (the consumption location information is Offset in the embodiment) is 4, an idle segment can be selected starting from the 5th location information of Partition1, which is P1_Segment 2 in the embodiment.

[0052] Step S407: Broker outputs the message corresponding to the segment to the message consumer according to the consumer identifier and the storage location information. Taking message consumer 1 as an example, Broker obtains all or part of the messages corresponding to P1_segment 2 according to the message transmission capability and storage location information of thread 2, and then outputs it to message consumer 1.

[0053] Step S408: After receiving the message, the message consumer processes it according to the corresponding business logic, and after processing, the confirmation information is fed back to the Broker. After each message consumer receives the message and completes the processing, it returns the confirmation information of the message to the Broker. The confirmation information includes the topic, partition number and location information. Taking message consumer 1 as an example, there are 5 confirmation information. Taking the confirmation information of the message corresponding to the 5th storage location information as an example, it includes Topic1, Partition1 and 5.

[0054] Step S409: The Broker determines whether it has received confirmation information corresponding to the message within a predetermined timeout period. If it has, it executes step S410; if it has not, it re-outputs the message and executes step S408. The Broker sets a timeout period in advance. If it does not receive confirmation information for a message within the period, it outputs the message through other threads for processing.

[0055] Step S410: Broker determines whether the location information of the confirmation information is continuous within the segment and within the partition starting from the head of the partition corresponding to the segment. If it is continuous, execute step S411; otherwise, do nothing. Assuming that Broker outputs the messages corresponding to the 5th to 7th storage location information to message consumer 1, and receives confirmation information for these three messages, it means that the confirmation within the segment is continuous; then determine whether Partition1 forms a continuous confirmation starting from the 1st storage location information (if confirmation information for the messages corresponding to the 1st storage location information to the 4th storage location information is received, it means that the confirmation within the partition is continuous). If only confirmation information for the messages corresponding to the 5th and 7th storage location information is received, the confirmation within the segment is discontinuous, and wait for confirmation information for the message corresponding to the 6th storage location information.

[0056] Step S411: Broker uses the last location information of continuous confirmation as the confirmed location information of the partition, updates the consumption location information according to the confirmed location information of the partition, and ends this process. Assuming that the confirmation information received by Broker forms a continuous confirmation from the 1st storage location information to the 7th storage location information, 7 is used as the confirmed location information of Partition1, and Broker's consumption location information is updated from 4 to 7.

[0057] In a preferred embodiment, between step S405 and step S406, it is also necessary to determine whether all messages corresponding to all segments in the partition have been consumed. If all messages corresponding to all segments in all partitions of the Topic corresponding to the message to be pulled by the message consumer have been consumed, it is necessary to reset the segments in the partition of the topic. Assuming that all messages corresponding to all segments of Topic1 have been consumed, it is necessary to set new segments in the 5 partitions of Topic1.

[0058] It can be seen from the message processing method of the embodiment of the present invention that by segmenting each partition and processing messages in parallel between different segments, the number of processed messages is doubled without increasing the number of partitions, thereby reducing the message transmission delay; the number of segments of each partition and the number of messages corresponding to each segment are flexibly set as needed, which not only supports high-performance and high-reliability message writing, but also ensures high-performance message pulling; the consumption location information of the partition and the timeout confirmation mechanism are updated according to the confirmation information, which can ensure that messages will not be lost when supporting high concurrency; by dynamically adjusting the number of segments of each partition, the overall message processing speed is improved.

[0059] Figure 5 Schematic diagram of main modules of a message processing device according to an embodiment of the present invention. Figure 5 As shown, the message processing device 500 (ie, message middleware) of the embodiment of the present invention mainly includes:

[0060] The search module 501 is used to find the partition with the same message type as the acquisition request according to the received acquisition request; wherein the acquisition request includes the user identification and the message type requested to be acquired, and the partition stores the storage location information of the message. After the message generator generates the message, it sends the message to the message middleware; the message middleware stores the message and saves the storage location information of the message. The message middleware receives the acquisition request from the message user and finds the corresponding partition according to the message type in the request.

[0061] The selection module 502 is used to select a segment from the partition according to the consumption position information of the partition; wherein the partition includes at least one of the segments. The message middleware pre-sets at least one segment for each partition, and the number of segments on each partition and the number of messages corresponding to each segment are determined according to actual needs. The consumption position information of the partition is used to indicate the messages that have been consumed by the partition, and an idle segment (i.e., a segment that is not being consumed) is selected from the messages that have not been consumed by the partition.

[0062] The output module 503 is used to output the message corresponding to the segment to the message user according to the user identifier and the storage location information. The message middleware obtains all or part of the message corresponding to the segment according to the storage location information, and then outputs it to the corresponding message user.

[0063] In addition, the message processing device 500 of the embodiment of the present invention may also include: a setting module, an update module, a re-output module and a reset module (not shown in the figure). The setting module is used to set at least one segment for each partition; wherein the number of segments on a partition and the number of messages corresponding to a segment are determined according to demand, and the number of segments of all partitions is greater than the number of all partitions. The update module is used to receive the confirmation information corresponding to the message within a predetermined timeout period, if the position information of the confirmation information is continuous in the segment and is continuous in the partition starting from the head of the partition corresponding to the segment, then the last position information of the continuous confirmation is used as the confirmation position information of the partition; and the consumption position information is updated according to the confirmation position information of the partition. The re-output module is used to re-output the message if the confirmation information corresponding to the message is not received within the timeout period. The reset module is used to reset the segment in the partition if all messages corresponding to all segments in the partition have been consumed.

[0064] From the above description, it can be seen that by segmenting each partition and processing messages in parallel between different segments, the number of messages processed can be doubled without increasing the number of partitions, thereby reducing the message transmission latency; the number of segments for each partition and the number of messages corresponding to each segment can be flexibly set as needed, which not only supports high-performance and high-reliability message writing, but also ensures high-performance message pulling; the consumption location information of the partition is updated according to the confirmation information, and the timeout confirmation mechanism can ensure that messages will not be lost in the case of high concurrency; by dynamically adjusting the number of segments for each partition, the overall message processing speed is improved.

[0065] Figure 6 An exemplary system architecture 600 is shown to which the message processing method or message processing apparatus according to the embodiment of the present invention can be applied.

[0066] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, 603, network 604 and server 605. Network 604 is used to provide a medium for communication links between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0067] Users can use terminal devices 601, 602, and 603 to interact with server 605 through network 604 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 601, 602, and 603, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0068] The terminal devices 601 , 602 , and 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0069] The server 605 may be a server that provides various services, such as a backend management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 601, 602, and 603. The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information, product information - only an example) to the terminal device.

[0070] It should be noted that the message processing method provided in the embodiment of the present application is generally executed by the server 605, and accordingly, the message processing device is generally set in the server 605.

[0071] It should be understood that Figure 6 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0072] According to an embodiment of the present invention, the present invention also provides an electronic device and a computer-readable medium.

[0073] The electronic device of the present invention comprises: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement a message processing method of an embodiment of the present invention.

[0074] The computer-readable medium of the present invention stores a computer program, and when the program is executed by a processor, a message processing method of an embodiment of the present invention is implemented.

[0075] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system 700 suitable for implementing an electronic device of an embodiment of the present invention. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0076] like Figure 7As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the computer system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0077] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0078] In particular, according to the embodiments disclosed in the present invention, the process described in the main step diagram above can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the main step diagram. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.

[0079] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0080] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a unit, a program segment, or a part of a code, and the above-mentioned unit, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0081] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes a search module, a selection module and an output module. The names of these modules do not constitute a limitation on the modules themselves in some cases, for example, the search module may also be described as "a module for finding a partition with the same message type as the acquisition request according to the received acquisition request".

[0082] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device includes: the message middleware searches for a partition with the same message type as the acquisition request according to the received acquisition request; wherein the acquisition request includes a user identifier and the requested message type, and the partition stores the storage location information of the message; the message middleware selects a segment from the partition according to the consumption location information of the partition; wherein the partition includes at least one segment; the message middleware outputs the message corresponding to the segment to the message user according to the user identifier and the storage location information.

[0083] From the above description, it can be seen that by segmenting each partition and processing messages in parallel between different segments, the number of messages processed can be doubled without increasing the number of partitions, thereby reducing the message transmission latency; the number of segments for each partition and the number of messages corresponding to each segment can be flexibly set as needed, which not only supports high-performance and high-reliability message writing, but also ensures high-performance message pulling; the consumption location information of the partition is updated according to the confirmation information, and the timeout confirmation mechanism can ensure that messages will not be lost in the case of high concurrency; by dynamically adjusting the number of segments for each partition, the overall message processing speed is improved.

[0084] The above product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0085] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A message processing method, characterized in that: include: The message middleware searches for a partition having the same message type as the acquisition request according to the received acquisition request; wherein the acquisition request includes a user identifier and the type of message requested to be acquired, and the partition stores storage location information of the message; The message middleware selects a segment from the partition according to the consumption location information of the partition; wherein the partition includes at least one of the segments; The message middleware outputs the message corresponding to the segment to the message consumer according to the consumer identifier and the storage location information.

2. The method according to claim 1, characterized in that Before the message middleware searches out a partition with the same message type as the acquisition request based on the received acquisition request, it also includes: setting at least one segment for each partition; wherein the number of segments on a partition and the number of messages corresponding to a segment are determined according to demand, and the number of segments of all partitions is greater than the number of all partitions.

3. The method according to claim 1, characterized in that: After the step of outputting the message corresponding to the segment to the message consumer according to the consumer identifier and the storage location information, the message middleware further includes: receiving confirmation information corresponding to the message within a predetermined timeout period, and if the position information of the confirmation information is continuous within the segment and continuous within the partition starting from the head of the partition corresponding to the segment, taking the last position information of the continuous confirmation as the confirmation position information of the partition; The consumption location information is updated according to the confirmed location information of the partition.

4. The method according to claim 3, characterized in that: After the message middleware outputs the message corresponding to the segment to the message user according to the user identifier and the storage location information, the method further includes: if no confirmation information corresponding to the message is received within the timeout period, re-outputting the message.

5. The method according to claim 1, characterized in that After the step of finding the partition with the same message type as the acquisition request and before the step of the message middleware selecting a segment from the partition according to the consumption location information of the partition, it also includes: if all messages corresponding to all segments in the partition have been consumed, resetting the segment in the partition.

6. A message processing device, characterized in that: include: A search module, used to search for a partition with the same message type as the acquisition request according to the received acquisition request; wherein the acquisition request includes a user identifier and the type of message requested to be acquired, and the partition stores storage location information of the message; A selection module, configured to select a segment from the partition according to the consumption location information of the partition; wherein the partition includes at least one of the segments; An output module is used to output the message corresponding to the segment to the message user according to the user identifier and the storage location information.

7. The device according to claim 6, characterized in that The device also includes: a setting module, used to set at least one segment for each partition; wherein the number of segments on a partition and the number of messages corresponding to a segment are determined according to demand, and the number of segments of all partitions is greater than the number of all partitions.

8. The device according to claim 6, characterized in that The apparatus further includes: an updating module, configured to receive confirmation information corresponding to the message within a predetermined timeout period, and if the position information of the confirmation information is continuous within the segment and continuous within the partition starting from the head of the partition corresponding to the segment, use the last position information of the continuous confirmation as the confirmation position information of the partition; and The consumption location information is updated according to the confirmed location information of the partition.

9. The device according to claim 8, characterized in that The device further comprises: a re-output module, configured to re-output the message if confirmation information corresponding to the message is not received within the timeout period.

10. The device according to claim 6, characterized in that The device also includes: a resetting module, configured to reset the segments in the partition if all messages corresponding to all segments in the partition have been consumed.

11. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

12. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images