Message queue processing methods, devices, electronic equipment, and storage media

Abstract

The embodiments of the present application provide a method, device, electronic device and storage medium for processing a message queue, and relate to the field of computer technology. Obtain a message query request for the message queue; determine a target message storage block from the message queue according to the query time period; determine at least one first target message storage unit corresponding to the query time period from the target message storage block, and query the target message from the first target message storage unit according to the query time period; send a message query response based on the message content of the target message. Using the above-mentioned technical approach, by storing the message queue as multiple message storage blocks arranged in time field order, each message storage block includes multiple message storage units arranged in time field order, and by constructing a two-level index for message query, the access efficiency of the message queue is improved, and the performance of message query retrieval is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a message queue processing method, device, electronic device and storage medium. Background Art

[0002] MA (Microservice Architecture) is an architectural concept that is different from the traditional monolithic service. It decouples the solution by decomposing the functions into discrete services, so that when the function is changed, the scope of change can be reduced and the risk of change can be reduced. Message queue middleware is an important component in the microservice architecture and plays an important role in some core scenarios such as big data analysis and transaction payment.

[0003] The current traditional message queue provides a high-throughput and high-availability message processing mechanism. For example, Kafka and Pulsar provide high-throughput message production, message consumption with a delay of seconds, and message order guarantee, etc. However, the performance in message search and query is poor. In the existing solution, message data is arranged on the disk in chronological order, and each retrieval requires traversing the disk, which is extremely inefficient. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a message queue processing method, device, electronic device and storage medium.

[0005] To solve the above technical problems, the embodiments of the present application are implemented through the following aspects.

[0006] According to a first aspect of an embodiment of the present disclosure, a method for processing a message queue is provided, wherein the message queue includes a plurality of message storage blocks, each of the message storage blocks includes a plurality of message storage units for storing messages, the message includes a time field and a message content, different message storage blocks are sorted according to the time field of the message, and the message storage units in each of the message storage blocks are sorted according to the time field of the message, and the method includes:

[0007] Obtaining a message query request for the message queue, wherein the message query request includes a query time period;

[0008] Determining a target message storage block from the message queue according to the query time period, the target message storage block comprising one target message storage block or a plurality of continuous target message storage blocks;

[0009] Determine at least one first target message storage unit corresponding to the query time period from the target message storage block, and query the target message from the first target message storage unit according to the query time period, wherein the query time period includes a time field of the target message;

[0010] A message query response is sent based on the message content of the target message.

[0011] According to a second aspect of an embodiment of the present disclosure, a message queue processing device is provided, wherein the message queue includes multiple message storage blocks, each of the message storage blocks includes multiple message storage units for storing messages, the message includes a time field and a message content, different message storage blocks are sorted according to the time field of the message, and the message storage units in each message storage block are sorted according to the time field of the message, and the device includes:

[0012] An acquisition module is configured to acquire a message query request for the message queue, wherein the message query request includes a query time period;

[0013] a determination module, configured to determine a target message storage block from the message queue according to the query time period, the target message storage block comprising one target message storage block or a plurality of continuous target message storage blocks;

[0014] A query module, configured to determine at least one first target message storage unit corresponding to the query time period from the target message storage block, and query a target message from the first target message storage unit according to the query time period, wherein the query time period includes a time field of the target message;

[0015] The response module is configured to send a message query response based on the message content of the target message.

[0016] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a memory, a processor, and computer executable instructions stored on the memory and executable on the processor, wherein the computer executable instructions implement the following steps when executed by the processor: obtaining a message query request for the message queue, the message query request including a query time period; determining a target message storage block from the message queue according to the query time period, the target message storage block including one target message storage block or multiple consecutive target message storage blocks; determining at least one first target message storage unit corresponding to the query time period from the target message storage block, querying a target message from the first target message storage unit according to the query time period, the query time period including a time field of the target message; and sending a message query response based on the message content of the target message.

[0017] According to a fourth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium, the computer-readable storage medium storing computer-executable instructions, which implement the following steps when executed by a processor: obtaining a message query request for the message queue, the message query request including a query time period; determining a target message storage block from the message queue according to the query time period, the target message storage block including one target message storage block or multiple consecutive target message storage blocks; determining at least one first target message storage unit corresponding to the query time period from the target message storage block, querying a target message from the first target message storage unit according to the query time period, the query time period including a time field of the target message; and sending a message query response based on the message content of the target message.

[0018] The technical solution provided by the embodiments of the present disclosure is as follows: obtaining a message query request for the message queue; determining a target message storage block from the message queue according to the query time period; determining at least one first target message storage unit corresponding to the query time period from the target message storage block, and querying the target message from the first target message storage unit according to the query time period; and sending a message query response based on the message content of the target message. By adopting the above-mentioned technical approach, by storing the message queue as a plurality of message storage blocks arranged in the order of the time field, each message storage block includes a plurality of message storage units arranged in the order of the time field, and by constructing a two-level index for message query, the access efficiency of the message queue is improved, and the performance of message query retrieval is improved.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

[0020] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0022] Figure 1 A flow chart showing a method for processing a message queue provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram showing the structure of a message queue provided in an embodiment of the present application is shown;

[0024] Figure 3 A schematic diagram showing a target message storage block in a message queue processing method provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram showing a first target message storage unit in the message queue processing method provided in an embodiment of the present application;

[0026] Figure 5 Another flowchart of a method for processing a message queue provided in an embodiment of the present application is shown;

[0027] Figure 6 A schematic diagram showing another first target message storage unit in the message queue processing method provided in an embodiment of the present application;

[0028] Figure 7 A schematic diagram showing a target message in a message queue processing method provided in an embodiment of the present application;

[0029] Figure 8 Another flowchart of the message queue processing method provided by the embodiment of the present application is shown;

[0030] Fig. 9 A schematic diagram showing another target message in the message queue processing method provided in an embodiment of the present application;

[0031] Fig.10 Another flowchart of the message queue processing method provided by the embodiment of the present application is shown;

[0032] Fig.11 Another flowchart of the message queue processing method provided by the embodiment of the present application is shown;

[0033] Fig.12 A block diagram showing a message queue processing device provided by an embodiment of the present application;

[0034] Fig.13 A block diagram showing another message queue processing device provided in an embodiment of the present application;

[0035] Fig.14 A schematic diagram of the hardware structure of an electronic device for executing the message queue processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

[0037] Figure 1 A flow chart of a method for processing a message queue provided in an embodiment of the present application is shown. Figure 1 As shown, the method may include the following steps.

[0038] In step S101, a message query request for a message queue is obtained.

[0039] Among them, the message queue includes multiple message storage blocks, each message storage block includes multiple message storage units for storing messages, the message includes a time field and message content, different message storage blocks are sorted according to the time field of the message, the message storage units in each message storage block are sorted according to the time field of the message, and the message query request includes a query time period.

[0040] Figure 2 A schematic diagram of the structure of a message queue provided in an embodiment of the present application is shown as follows: Figure 2 As shown, the message queue can be composed of multiple message storage blocks (represented by Block in this application), each message storage block can be composed of 32 message storage units (represented by Cell in this application), each message storage unit can store 512 messages (represented by Msg in this application), that is, 16384 messages are stored in each message storage block. Those skilled in the art can understand that the number of message storage units in each message storage block and the number of messages stored in each message storage unit can also be set to other numbers as needed, and this application does not impose any restrictions on this.

[0041] Each message may include a time field and message content. The time field may indicate the time when the message enters the message queue, and the message content may be the body of the message stored in a byte array. In some possible implementations, different message storage blocks are sorted in ascending order according to the time field of the messages stored in the message storage blocks, and the message storage units in each message storage block are sorted according to the time field of the messages stored in the message storage units. Figure 2As shown, the time field of any message stored in Block1 is greater than or equal to the time field of any message stored in Block0. Correspondingly, the time field of any message stored in Cell1 of Block1 is greater than or equal to the time field of any message in Cell0 of Block1.

[0042] In step S102, a target message storage block is determined from the message queue according to the query time period.

[0043] The target message storage block includes one target message storage block or a plurality of continuous target message storage blocks.

[0044] In some possible implementations, the time period composed of the first minimum time field and the first maximum time field of the message in each message storage block can be matched with the query time period. When the time period composed of the first minimum time field and the first maximum time field overlaps with the query time period, the message storage block is determined as the target storage block. The first minimum time field and the first maximum time field corresponding to each message storage block and the identification information of the message storage block can be saved in an index file as an index of the message storage block. When performing a query, the first minimum time field and the first maximum time field can be directly obtained from the index, which can further improve the efficiency of the query.

[0045] Figure 3 A schematic diagram of a target message storage block in a message queue processing method provided in an embodiment of the present application is shown. Taking the message storage block including 6 message storage units as an example, since the message storage block is sorted based on the time field of the stored messages (for example, in ascending order), the target storage block can be as follows: Figure 3 (a) shows a target message storage block, which can also be Figure 3 (b) shows multiple consecutive target message storage blocks.

[0046] In step S103, at least one first target message storage unit corresponding to the query time period is determined from the target message storage block, and the target message is queried from the first target message storage unit according to the query time period.

[0047] The query time period includes the time field of the target message. In other words, the time field of the target message is included in the query time period.

[0048] In some possible implementations, the time period composed of the second minimum time field and the second maximum time field of the message in each message storage unit can be matched with the query time period. When there is a time overlap between the time period composed of the second minimum time field and the second maximum time field and the query time period, the message storage unit is determined as the first target message storage unit.

[0049] In another possible implementation, the second minimum time field and the second maximum time field corresponding to each message storage unit and the identification information of the message storage unit can be saved in an index file as an index of the message storage unit. When performing a query, the second minimum time field and the second maximum time field can be directly obtained from the index, which can further improve the query efficiency.

[0050] Figure 4 A schematic diagram of the first target message storage unit in the message queue processing method provided by an embodiment of the present application is shown. Taking the message storage block including 6 message storage units as an example, since the message storage unit is sorted based on the time field of the stored message (for example, in ascending order), the first target message storage unit can be as follows: Figure 4 (a) shows a first target message storage unit, which may also be as follows Figure 4 (b) The plurality of consecutive first target message storage units shown may also be as follows Figure 4 (c) shows a plurality of consecutive first target message storage units in a plurality of target message storage blocks.

[0051] After determining at least one first target message storage unit, the target message can be queried from the first target message storage unit according to the query time period. The message query request can also flexibly set other query conditions to flexibly query the target message from the first target message storage unit.

[0052] In step S104, a message query response is sent based on the message content of the target message.

[0053] By way of example, the message content of the target message can be flexibly processed according to the message query request. For example, the message content of the target message (or a specific part of the message content) can be used as a message query response, or the message content of the target message (or a specific part of the message content) can be aggregated and calculated, and the aggregated calculation result can be used as a message query response. The above-mentioned aggregated calculation can be, for example, a data statistical method such as a mean or a maximum, and the present application does not impose any restrictions on this.

[0054] By adopting the above-mentioned technical method, the message queue is stored as multiple message storage blocks arranged in the order of time fields, each message storage block includes multiple message storage units arranged in the order of time fields, and a two-level index is constructed for message query, so as to improve the access efficiency of the message queue and the performance of message query and retrieval.

[0055] Figure 5 Another flow chart of the message queue processing method provided in the embodiment of the present application is shown as follows: Figure 5 As shown, step S103 may include the following steps:

[0056] In step S1031, at least one first target message storage unit corresponding to the query time period and the query service field set is determined from the target message storage block.

[0057] The message also includes a business field. The business field included in each message is used for query and indexing. The message query request also includes a query business field set. The messages in each message storage unit are sorted according to the business fields of the stored messages.

[0058] The query service field set may include one query service field or multiple query service fields, which is not limited in the present application.

[0059] In some possible implementations, the time period composed of the second minimum time field and the second maximum time field of the message in each message storage unit can be matched with the query time period, and the business field set composed of the business field of the message in each message storage unit can be matched with the above-mentioned query business field set. When the time period composed of the second minimum time field and the second maximum time field overlaps with the query time period, and the business field set has the same business field as the above-mentioned query business field set, the message storage unit is determined as the first target message storage unit.

[0060] In another possible implementation, the second minimum time field, the second maximum time field corresponding to each message storage unit, the business field set consisting of the business fields of the message in the message storage unit, and the identification information of the message storage unit can be saved in an index file as an index of the message storage unit. When performing a query, the above information can be directly obtained from the index, which can further improve the query efficiency.

[0061] Figure 6 A schematic diagram of another first target message storage unit in the message queue processing method provided by an embodiment of the present application is shown. Taking the message storage block including 6 message storage units as an example, since the message storage unit is sorted based on the time field of the stored message (for example, in ascending order) and the business field set of the business composition of the message stored in the first target message storage unit and the set of query business fields have the same business fields, the first target message storage unit can be as follows: Figure 6 (a) shows a first target message storage unit, which may also be as follows Figure 6 (b) The plurality of continuous or discontinuous first target message storage units shown may also be as follows Figure 6 (c) shows a plurality of continuous or discontinuous first target message storage units in the plurality of target message storage blocks.

[0062] In step S1032, a target message is determined from the first target message storage unit according to the query time period and the query service field set.

[0063] The query time period includes the time field of the target message, and the query service field set includes the service field of the target message.

[0064] Figure 7 A schematic diagram showing a target message in a message queue processing method provided in an embodiment of the present application is shown, such as Figure 7 As shown, taking the first target message storage unit including 6 messages as an example, in some possible implementations, the messages in the first target message storage unit are arranged in ascending order according to the corresponding business fields, for example, Figure 7 In the example, the service field of the i-th message is less than or equal to the service field of the i+1-th message. Since the messages in the first target message storage unit are sorted according to the service field (for example, in ascending order), the target message can be as follows: Figure 7 (a) shows a target message, or it can be Figure 7 (b) shows multiple consecutive target messages.

[0065] Compared with a traditional message queue without indexing, the time saved by a message query request can be determined by the following formula 1.

[0066] (Formula 1)

[0067] Where n is the number of target messages. Based on the practical experience of this application, the overall query time is saved by about 48%.

[0068] By adopting the above-mentioned technical method, the message queue is stored as multiple message storage blocks arranged in the order of time fields, each message storage block includes multiple message storage units arranged according to the time field, and a two-level index is constructed for message query, so as to improve the access efficiency of the message queue and improve the performance of message query and retrieval.

[0069] Figure 8 Another flow chart of the message queue processing method provided in the embodiment of the present application is shown as follows: Figure 8 As shown, the message query request may be an aggregate calculation query, and step S104 may include the following steps:

[0070] In step S1041, when the cache includes the cache aggregation result of any second target message storage unit, aggregation calculation is performed according to the cache aggregation result and the message contents of the remaining target messages to obtain a calculated aggregation result.

[0071] The remaining target messages are target messages other than the target messages contained in the second target message storage unit, and each message in the second target message storage unit is a target message.

[0072] Fig. 9 A schematic diagram showing another target message in the message queue processing method provided in an embodiment of the present application is shown, such as Fig. 9 As shown, in the three first target message storage units Cell1, Cell2, and Cell3, each message in Cell2 is a target message, and Cell2 is a second target message storage unit. Correspondingly, the messages Msg3 and Msg4 in the first target message storage unit Cell1, and Msg2, Msg3, and Msg4 in the first target message storage unit Cell3 are all remaining target messages. In some possible implementations, when the cache includes the cache aggregation result corresponding to the second target message storage unit, an aggregation calculation can be performed based on the cache aggregation result and the message content of the remaining target messages to obtain a calculated aggregation result.

[0073] For example, if the message query request is to perform an average value aggregation calculation query on a specific field in the message content of the target message, the calculation aggregation result can be determined according to the following formula 2.

[0074] (Formula 2)

[0075] Among them, R is the calculation aggregation result, is the average value cache aggregation result corresponding to the specific field of the i-th second target message storage unit, L is the number of messages in the second target message storage unit, N is the number of second target message storage units, is the value of the specific field corresponding to the jth remaining target message, and M is the number of remaining target messages.

[0076] In step S1042, when the aggregate settlement result of the second target message storage unit is not included in the cache, an aggregate calculation is performed according to the message content of the target message to obtain a calculated aggregate result.

[0077] In step S1043, a message query response including the calculation aggregation result is sent.

[0078] In some embodiments, when the cache does not include the aggregate settlement result of the second target message storage unit, after performing aggregation calculation based on the message content of the target message and obtaining the calculated aggregation result, the calculated aggregation result of the target message of the second target message storage unit can also be saved in the cache as the cache calculation result of the specific field corresponding to the second target message storage unit.

[0079] By adopting the above-mentioned technical solution, by saving the calculation aggregation results of the second target storage unit in the cache, the aggregation calculation can be directly performed based on the cache aggregation results and the message content of the remaining target messages to obtain the calculation aggregation results. There is no need to repeatedly calculate the calculation aggregation results of the second target storage unit, thereby reducing the number of disk IO and further improving the query efficiency of the message queue.

[0080] In a scenario where aggregate calculation is performed on target messages within a query time period, after a first target message storage unit is hit by a query, the probability of being queried again will increase, and the probability of adjacent message storage units being queried will also increase. Fig.10 Another flow chart of the message queue processing method provided in the embodiment of the present application is shown as follows: Fig.10 As shown, the method may also include the following steps:

[0081] In step S105, the hit probability of the candidate message storage unit is obtained.

[0082] The candidate message storage unit is a message storage unit whose distance from any first target message storage unit is less than a preset distance threshold.

[0083] In some possible implementations, the hit probability of the candidate message storage unit may be determined by the following formula three.

[0084] (Formula 3)

[0085] in, is the hit probability of the candidate message storage unit, z is the distance between the candidate message storage unit and the first target message storage unit, is the initial probability, which can be set to 0, is the preset adjustment variable, The average of the hit probabilities of all candidate message storage units.

[0086] In step S106, the calculation aggregation result of the target candidate message storage unit is determined and stored in the cache as the cache aggregation result of the target candidate message storage unit.

[0087] Among them, the target candidate message storage unit is a candidate message storage unit whose hit probability is greater than or equal to a preset probability threshold. In some possible implementations, the preset probability threshold may be 50%.

[0088] In some possible implementations, when the cached aggregation result of the target candidate message storage unit does not exist in the cache, the calculation aggregation result of the target candidate message storage unit can be determined and stored in the cache as the cached aggregation result of the target candidate message storage unit. For candidate message storage units with cached calculation results in the cache and a hit probability less than the above-mentioned preset probability threshold, the corresponding cached aggregation result can be removed from the cache to save cache capacity.

[0089] By adopting the above technical method, the hit probability of the message storage unit can be determined in real time, and the cache aggregation results of the target candidate message storage unit can be determined in time and saved in the cache, so that in subsequent message query requests, the cache calculation results can be used to further improve the query efficiency of the message queue.

[0090] When storing the queue, if the time field corresponding to each message is stored in a timestamp format, a large storage space will be occupied. In some embodiments, the time field of the message can be stored in a compressed time field format. The following steps can be taken to determine the target message from the first target message storage unit according to the query time period.

[0091] In step 10, when the type of the time field of the message in the first target message storage unit is a compressed field, the time field of the message in the first target message storage unit is restored to a timestamp.

[0092] The specific technical solution for timestamp restoration can refer to the description of the specific implementation example when storing messages.

[0093] In step 11, a target message is determined from the first target message storage unit according to the timestamp of the message in the first target message storage unit and the query time period.

[0094] For example, when the query period includes a timestamp of the message, the message may be taken as the target message.

[0095] By adopting the above technical approach, the time field can be compressed, thereby further compressing the storage space of the time field corresponding to the message, thereby saving the storage space of the entire message queue.

[0096] Fig.11 Another flow chart of the message queue processing method provided in the embodiment of the present application is shown as follows: Fig.11 As shown, the method may also include the following steps.

[0097] In step S107, the current time of the received message is set as the time field of the received message, and the received message and the corresponding time field are cached in a cache message storage block.

[0098] In step S108, when the number of messages in the cache message storage block is greater than or equal to the capacity of the message storage block, the messages in the cache message storage block are sorted in chronological order and saved as a new message storage block.

[0099] For example, the capacity of the message storage block may be 16384. When the number of messages in the cache message storage block is greater than or equal to 16384, the messages in the cache message storage block may be sorted in ascending chronological order, and the first 16384 messages may be saved as a new message storage block.

[0100] In some embodiments, the time fields corresponding to the head message and the tail message of the messages sorted in chronological order may be saved in an index file to characterize the range of the time fields of the messages stored in the new message storage block, thereby improving the efficiency of message query.

[0101] In step S109, the new message storage block is divided into a plurality of new message storage units arranged according to the time fields of the messages.

[0102] For example, the capacity of the message storage block may be 16384, and the sorted 16384 messages may be divided into 32 new message storage units, each of which may store 512 messages.

[0103] In step S110, the messages in each new message storage unit are sorted according to the service fields of the messages.

[0104] It should be noted that the present application does not limit the order of the message queue storage steps shown in steps S107-S110 and the message queue query steps shown in steps S101-S104. The message queue storage steps may be before, after or in parallel with the message queue query steps.

[0105] By adopting the above-mentioned technical method, the message queue is stored as multiple message storage blocks arranged in the order of time fields, each message storage block includes multiple message storage units arranged according to the time field, and each message storage unit includes multiple messages arranged according to the business field. By building a two-level index for message query, the access efficiency of the message queue is improved, and the performance of message query and retrieval is improved.

[0106] In some embodiments, after executing step S110, the business fields corresponding to the head message and the tail message of the sorted messages can also be saved in the index file to represent the range of the business fields of the messages stored in the new message storage unit. The range of the time field of the messages stored in the new message storage unit can also be saved in the index file to improve the efficiency of message query.

[0107] In some embodiments, after executing step S109 and before executing step S110, the time field of the message in the new message storage unit can also be replaced by the timestamp growth change field to compress the time field. The bit width of the timestamp growth change field can be flexibly set to 8 bits or 32 bits as needed, and the present application does not impose any restrictions on this.

[0108] For example, the time field of the message in the new message storage unit may be compressed and stored using the following formula 4.

[0109] (Formula 4)

[0110] Accordingly, the following formula 5 can be used to restore the timestamp growth change field to the timestamp .

[0111] (Formula 5)

[0112] in, , and These are the time fields corresponding to the nth, n-1th, and n-2th messages respectively. The timestamp change value of the nth message, used to replace The time field of the first message is saved as the time field of the message to compress the time field and save the storage space of the message queue. You can calculate the timestamp increment change field of the second message without compression hour, You can increase the average value of the change field for the timestamp set according to the empirical value. Other than It can be calculated It is then removed from storage, saving storage space in the message queue.

[0113] By adopting the above technical approach, the time field can be compressed, thereby further compressing the storage space of the time field corresponding to the message, thereby saving the storage space of the message queue.

[0114] Fig.12A block diagram of a message queue processing device provided by an embodiment of the present application is shown, wherein the message queue includes multiple message storage blocks, each message storage block includes multiple message storage units for storing messages, the message includes a time field and a message content, different message storage blocks are sorted according to the time field of the message, and the message storage units in each message storage block are sorted according to the time field of the message, such as Fig.12 As shown, the message queue processing device 100 includes:

[0115] The acquisition module 110 is configured to acquire a message query request for a message queue, wherein the message query request includes a query time period;

[0116] The determination module 120 is configured to determine a target message storage block from the message queue according to the query time period, wherein the target message storage block includes one target message storage block or multiple consecutive target message storage blocks;

[0117] The query module 130 is configured to determine at least one first target message storage unit corresponding to a query time period from the target message storage block, and query the target message from the first target message storage unit according to the query time period, wherein the query time period includes a time field of the target message;

[0118] The response module 140 is configured to send a message query response based on the message content of the target message.

[0119] Optionally, the message further includes a business field, the message query request further includes a query business field set, the messages in each message storage unit are sorted according to the business field of the stored messages, and the query module 130 is further configured to:

[0120] Determine at least one first target message storage unit corresponding to the query time period and the query service field set from the target message storage block;

[0121] The target message is determined from the first target message storage unit according to a query time period and a query service field set, the query time period includes a time field of the target message, and the query service field set includes a service field of the target message.

[0122] Optionally, the message query request is an aggregate calculation query, and the response module 140 is further configured to:

[0123] In the case where the cache includes a cache aggregation result of any second target message storage unit, an aggregation calculation is performed based on the cache aggregation result and the message contents of the remaining target messages to obtain a calculated aggregation result, the remaining target messages are other target messages other than the target messages contained in the second target message storage unit, and each message in the second target message storage unit is a target message; or

[0124] In the case where the aggregate settlement result of the second target message storage unit is not included in the cache, performing aggregate calculation according to the message content of the target message to obtain a calculated aggregate result;

[0125] Send a message query response including the calculated aggregate result.

[0126] Optionally, the response module 140 is further configured to:

[0127] The calculation aggregation result of the target message of the second target message storage unit is stored in the cache.

[0128] Optionally, the response module 140 is further configured to:

[0129] Obtaining a hit probability of a candidate message storage unit, where the candidate message storage unit is a message storage unit whose distance from any first target message storage unit is less than a preset distance threshold;

[0130] The calculation aggregation result of the target candidate message storage unit is determined and stored in the cache as the cache aggregation result of the target candidate message storage unit, where the target candidate message storage unit is a candidate message storage unit with a hit probability greater than or equal to a preset probability threshold.

[0131] Optionally, the query module 130 is further configured to:

[0132] In the case where the type of the time field of the message in the first target message storage unit is a compressed field, restoring the time field of the message in the first target message storage unit to a timestamp;

[0133] The target message is determined from the first target message storage unit according to the timestamp of the message in the first target message storage unit and the query time period.

[0134] The device 100 provided in the embodiment of the present application can execute each method in the foregoing method embodiment and realize the functions and beneficial effects of each method in the foregoing method embodiment, which will not be repeated here.

[0135] Fig.13 A block diagram of another message queue processing device provided in an embodiment of the present application is shown. Fig.13 As shown, the message queue processing device 100 further includes:

[0136] The storage module 150 is configured as follows:

[0137] The current time of the received message is set as the time field of the received message, and the received message and the corresponding time field are cached in the cache message storage block;

[0138] When the number of messages in the cache message storage block is greater than or equal to the capacity of the message storage block, the messages in the cache message storage block are sorted in chronological order and saved into a new message storage block;

[0139] Dividing the new message storage block into a plurality of new message storage units arranged according to the time fields of the messages;

[0140] The messages in each new message storage unit are sorted according to the business fields of the messages.

[0141] Optionally, the storage module 150 is further configured to:

[0142] The time field of the message in the new message storage unit is compressed and stored.

[0143] The device 100 provided in the embodiment of the present application can execute each method in the foregoing method embodiment and realize the functions and beneficial effects of each method in the foregoing method embodiment, which will not be repeated here.

[0144] Fig.14 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application is shown. Referring to the figure, at the hardware level, the electronic device includes a processor, and optionally, an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage, etc. Of course, the electronic device may also include hardware required for other services. The processor, the network interface, and the memory may be interconnected through an internal bus, and the internal bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0145] The memory stores the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0146] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a device for locating the target user at the logical level. The processor executes the program stored in the memory and specifically performs: the method disclosed in the first aspect embodiment and realizes the functions and beneficial effects of each method described in the method embodiment above, which will not be repeated here.

[0147] The method disclosed in the first aspect of the present application can be implemented in a processor or by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in software form. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0148] The electronic device can also execute the methods described in the foregoing method embodiments, and realize the functions and beneficial effects of the methods described in the foregoing method embodiments, which will not be repeated here.

[0149] Of course, in addition to software implementation methods, the electronic device of the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0150] An embodiment of the present application also proposes a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple applications, the electronic device executes the method disclosed in the embodiment of the first aspect and realizes the functions and beneficial effects of the various methods described in the previous method embodiments, which will not be repeated here.

[0151] The computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0152] Furthermore, an embodiment of the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the following process is implemented: the method disclosed in the embodiment of the first aspect and the functions and beneficial effects of the various methods described in the previous method embodiments are implemented, which will not be repeated here.

[0153] In short, the above is only a preferred embodiment of the present application and does not limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0154] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0155] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0156] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0157] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A message queue processing method, characterized in that: The message queue includes a plurality of message storage blocks, each of which includes a plurality of message storage units for storing messages, the message includes a time field and a message content, different message storage blocks are sorted according to the time field of the message, and the message storage units in each message storage block are sorted according to the time field of the message, and the method includes: Obtaining a message query request for the message queue, wherein the message query request includes a query time period, and the message query request is an aggregate calculation query; Determining a target message storage block from the message queue according to the query time period, the target message storage block comprising one target message storage block or a plurality of continuous target message storage blocks; Determine at least one first target message storage unit corresponding to the query time period from the target message storage block, and query the target message from the first target message storage unit according to the query time period, wherein the query time period includes a time field of the target message; Sending a message query response based on the message content of the target message; The sending of a message query response based on the message content of the target message includes: In the case where the cache includes the cache aggregation result of any second target message storage unit, aggregation calculation is performed based on the cache aggregation result and the message content of the remaining target messages to obtain a calculated aggregation result, and a message query response including the calculated aggregation result is sent; wherein the remaining target messages are other target messages other than the target messages contained in the second target message storage unit, and each message in the second target message storage unit is the target message.

2. The method according to claim 1, characterized in that The message further includes a business field, the message query request further includes a query business field set, the messages in each of the message storage units are sorted according to the business fields of the stored messages, the determining at least one first target message storage unit corresponding to the query time period from the target message storage block, and querying the target message from the first target message storage unit according to the query time period, comprising: Determine at least one first target message storage unit corresponding to the query time period and the query service field set from the target message storage block; A target message is determined from the first target message storage unit according to the query time period and the query service field set, wherein the query time period includes a time field of the target message and the query service field set includes a service field of the target message.

3. The method according to claim 1, characterized in that The sending of a message query response based on the message content of the target message also includes: In the case that the aggregate settlement result of the second target message storage unit is not included in the cache, performing aggregate calculation according to the message content of the target message to obtain a calculated aggregate result; Sending a message query response including the calculated aggregation result.

4. The method according to claim 3, characterized in that In the case where the aggregate settlement result of the second target message storage unit is not included in the cache, the method further includes: The calculation aggregation result of the target message of the second target message storage unit is stored in the cache.

5. The method according to claim 3, characterized in that: The method further comprises: Obtaining a hit probability of a candidate message storage unit, wherein the candidate message storage unit is a message storage unit whose distance from any of the first target message storage units is less than a preset distance threshold; Determine the calculation aggregation result of the target candidate message storage unit and save it in the cache as the cache aggregation result of the target candidate message storage unit, wherein the target candidate message storage unit is a candidate message storage unit whose hit probability is greater than or equal to a preset probability threshold.

6. The method according to claim 1, characterized in that The determining the target message from the first target message storage unit according to the query time period includes: In the case where the type of the time field of the message in the first target message storage unit is a compressed field, restoring the time field of the message in the first target message storage unit to a timestamp; A target message is determined from the first target message storage unit according to the timestamp of the message in the first target message storage unit and the query time period.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The current time of the received message is set as the time field of the received message, and the received message and the corresponding time field are cached in the cache message storage block; When the number of messages in the cache message storage block is greater than or equal to the capacity of the message storage block, the messages in the cache message storage block are sorted in chronological order and saved into a new message storage block; Dividing the new message storage block into a plurality of new message storage units arranged according to the time fields of the messages; The messages in each of the new message storage units are sorted according to the service fields of the messages.

8. The method according to claim 7, characterized in that The method further comprises: The time field of the message in the new message storage unit is compressed and stored.

9. A message queue processing device, characterized in that: The message queue includes a plurality of message storage blocks, each of which includes a plurality of message storage units for storing messages, the message includes a time field and a message content, different message storage blocks are sorted according to the time field of the message, and the message storage units in each message storage block are sorted according to the time field of the message, and the device includes: an acquisition module, configured to acquire a message query request for the message queue, wherein the message query request includes a query time period, and the message query request is an aggregate calculation query; a determination module, configured to determine a target message storage block from the message queue according to the query time period, the target message storage block comprising one target message storage block or a plurality of continuous target message storage blocks; A query module, configured to determine at least one first target message storage unit corresponding to the query time period from the target message storage block, and query a target message from the first target message storage unit according to the query time period, wherein the query time period includes a time field of the target message; A response module, configured to send a message query response based on the message content of the target message; The response module is also configured to: in the case that the cache includes a cache aggregation result of any second target message storage unit, perform aggregation calculation based on the cache aggregation result and the message content of the remaining target messages to obtain a calculated aggregation result, and send a message query response including the calculated aggregation result; wherein the remaining target messages are other target messages other than the target messages contained in the second target message storage unit, and each message in the second target message storage unit is the target message.

10. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor is used to perform the steps of the message queue processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, enables the electronic device to execute the steps of the message queue processing method according to any one of claims 1 to 8.

Images