High-timeliness message processing method and device, electronic equipment and storage medium
By adopting hash partitioning and service priority queue in the message processing mode, the problems of performance reduction and resource waste in the traditional message processing mode are solved, and efficient and reliable message processing and transmission are achieved.
Patent Information
- Application Number
- CN202510311546.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-17
- Publication Date
- 2025-06-06
- Estimated Expiration
- 2045-03-17
AI Technical Summary
After the traditional message processing model accumulates the database data volume, the loading, sending and querying performance of messages is significantly reduced, resulting in waste of system resources, delay in message reception, low data cleaning efficiency and lock competition problems.
The message processing mode of hash partitioning and business priority queue is adopted. By hash partitioning the sending data table, the message entry data is evenly distributed, and the basic rules for message query and loading in the partition are set according to the business priority, the polled partition jump and priority message loading are realized.
It significantly improves the performance and efficiency of message processing, avoids data accumulation and lock competition problems, ensures the efficiency and reliability of message processing, and improves the system resource utilization rate and overall message sending efficiency.
Smart Images

Figure CN120104672A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computers, and in particular to a method, device, electronic device and storage medium for high-timeliness message processing. Background Art
[0002] In the process of message sending and processing, how to balance different business needs and improve processing efficiency has always been a technical challenge. In traditional message processing modes, such as SMS sending scenarios, as the amount of database data accumulates, the message loading and sending performance is significantly reduced, especially in distributed databases.
[0003] Most existing message processing modes are based on traditional B+ tree structured non-distributed databases, which have relatively balanced query and insertion performance. Traditional message processing uses a global loop table scan method. To improve loading performance, lists are usually partitioned based on the sending status, allowing messages with different sending statuses to be searched in specific partitions to speed up query performance. However, the database performance was not considered, resulting in the following problems in the distributed database: 1. System resource waste, slow message query, slow loading of message content to the sending queue, resulting in a large proportion of idle time in the sending queue, and failure to make good use of system resources; 2. Message reception delay. Due to the slow loading of messages, a large number of messages to be sent are accumulated in the database, resulting in a long waiting time for customers to receive messages, especially in the scenario of SMS verification codes, which seriously affects business processing; 3. Data cleaning abnormalities. In order to ensure the performance of message sending, the messages in the table are generally cleaned up globally according to certain rules every day. Under the original mechanism, the distributed database has low efficiency in batch deletion at the end of the day, resulting in a large number of messages to be cleaned up and unable to be deleted quickly, resulting in long transactions, accompanied by timeouts and deadlocks, which in turn affects query performance; 4. Lock competition: In order to avoid repeated message loading, the traditional message sending design will add row locks (such as the skip locked statement in Oracle) to avoid obtaining duplicate data when multi-threaded queries are performed; in the traditional global scanning method, when the number of threads for query services is large or the number of messages queried at one time is large, the lock competition will be greatly increased, thereby reducing query performance.
[0004] Therefore, there is an urgent need to develop a high-timeliness message processing solution that can significantly improve query performance and avoid the problem of large-scale accumulation caused by uneven distribution of messages of different priorities in partitions. Summary of the invention
[0005] The technical problem to be solved by the present invention is that in the traditional message processing mode, in the SMS sending and query scenarios, as the amount of database data accumulates, the message loading, sending and query performance is significantly reduced.
[0006] To solve the above technical problems, according to one aspect of the present invention, a method for high-timeliness message processing is provided, which includes the following steps: data table setting, canceling the list partitioning method in the traditional mode, performing hash partitioning on the sending data table according to the unique primary key id, and dividing it into multiple hash partitions. After all the message data is stored, it will be evenly inserted into multiple hash partition libraries; business priority queue setting, setting the priority order in the default state, which is designed according to the priority order from the first to the last in the queue array by default, or flexibly adopting the adjustment of the queue size or the number of messages in the queue to change the order; designing the priority configuration method, reading the configuration in the way of environment variables in the default state, if the message queue is set, it is read according to the configuration in the message queue first; sending logic setting, including the steps of: setting the basic jump rule of the partition, following the polling method to repeatedly load the short message to be sent from the first partition according to the rules of the message query in the partition until the last partition is traversed; when the query of the last partition is completed, restart the query from the first partition; setting the basic rules for querying and loading messages in the partition, in the current partition When querying, set query rules and priority queues, define the maximum query quantity, and query each time according to the basic order rule from the highest priority to the lowest priority. Load the maximum query quantity of messages to be sent of a certain priority into the message sending queue; if the query result is less than the maximum query quantity, query the next priority message to be sent in the current partition until the lowest priority message to be sent in the current partition is queried, and then enter the next partition; if the query result is full of the maximum query quantity, jump to the next partition; set the single query threshold and partition jump rule, and query the maximum query quantity in the current partition each time. If the number of messages to be sent of a certain priority is the maximum query quantity for three consecutive queries, it means that the number of messages of this priority is large in the current partition, then determine whether the priority is the first priority, and then execute according to the following situations: For the first priority, continue to query and load the messages to be sent of this priority in the current partition until the number of messages to be sent is less than the maximum query quantity in a certain query, and then query the next priority message of the current partition; if it is not the first priority, jump directly to the next partition and start querying from the highest priority again, and so on.
[0007] According to an embodiment of the present invention, in the data table setting, the hash partition can be divided into 32 hash partitions.
[0008] According to an embodiment of the present invention, the sending logic setting may also include the steps of: setting a method for handling blocking situations in the sending queue; when there are a large number of messages to be sent in the sending queue, it will jump to the next partition after a one-second rest and process the messages to be sent in the order of high priority to low priority according to the above rules.
[0009] According to an embodiment of the present invention, in the sending logic setting, the maximum number of queries can be defined as 100.
[0010] According to a second aspect of the present invention, a device for high-timeliness message processing is provided, which includes: a data table setting module, which is used to perform hash partitioning on the sending data table according to a unique primary key id, and divide it into multiple hash partitions. After all the message data is stored, it will be evenly inserted into multiple hash partition libraries; a business priority queue setting module, which is used to set the priority order in the default state, which is designed by default according to the priority order from the first to the last in the queue array, or flexibly adopts the adjustment of the queue size or the number of messages in the queue to change the order; the priority configuration method is designed, and the configuration is read in the form of environment variables in the default state. If a message queue is set, it is read preferentially according to the configuration in the message queue; a sending logic setting module is used to set the sending logic, and performs the following operations: setting the basic jump rule of the partition, and repeatedly loading the short message to be sent from the first partition according to the rules of the message query in the partition in a polling manner until the last partition is traversed; when the query of the last partition is completed, the query is restarted from the first partition; setting the basic rules for querying and loading messages in the partition, when querying in the current partition, set Set query rules and priority queues, define the maximum query number, and query a maximum of a certain priority to be sent messages each time according to the basic order rule from the highest priority to the lowest priority, and load them into the message sending queue; if the query result is less than the maximum query number, query the next priority to be sent messages in the current partition until the lowest priority to be sent messages in the current partition are queried, and then enter the next partition; if the query result is full of the maximum query number, jump to the next partition; set the single query threshold and partition jump rule, in the current partition, query the maximum query number each time, if three consecutive queries, the number of messages to be sent of a certain priority is the maximum query number, it means that the number of messages of this priority is large in the current partition, then determine whether the priority is the first priority, and then execute according to the following situations: first priority, continue to query and load the messages to be sent of this priority in the current partition, until the number of messages to be sent is less than the maximum query number in a certain query, and then query the next priority message of the current partition; if it is not the first priority, jump directly to the next partition and start querying from the highest priority again, and so on.
[0011] According to an embodiment of the present invention, in the data table setting module, the hash partition can be divided into 32 hash partitions.
[0012] According to an embodiment of the present invention, the following operations can also be performed in the sending logic setting module: setting a method for handling blocking situations in the sending queue; when there are a large number of messages to be sent in the sending queue, rest for one second before jumping to the next partition and processing the messages to be sent in the order of high priority to low priority according to the above rules.
[0013] According to an embodiment of the present invention, in the sending logic setting module, the maximum number of queries can be defined as 100.
[0014] According to a third aspect of the present invention, there is provided an electronic device, comprising: a memory, a processor, and a high-timeliness message processing program stored in the memory and executable on the processor, wherein the high-timeliness message processing program implements the steps of the above-mentioned high-timeliness message processing method when executed by the processor.
[0015] According to a fourth aspect of the present invention, a computer storage medium is provided, wherein a high-timeliness message processing program is stored on the computer storage medium, and when the high-timeliness message processing program is executed by a processor, the steps of the above-mentioned high-timeliness message processing method are implemented.
[0016] Compared with the prior art, the technical solution provided by the embodiments of the present invention can at least achieve the following beneficial effects:
[0017] The method of high-timeliness message processing of the present invention proposes a message processing mode based on business priority and hash partitioning. By flexibly configuring business priority, the specific needs of different businesses for message sending are met, and the pertinence and response speed of the business are improved; at the same time, by using hash partition database design and specific partition jump algorithm, the data processing pressure is effectively dispersed, the performance of message processing is greatly improved, and the efficiency and reliability of message processing are ensured.
[0018] The method for processing high-timeliness messages of the present invention can flexibly configure service priorities and effectively manage service priorities, so that emergency service messages can be delivered in a timely manner, bringing benefits to enterprises and users.
[0019] The high-timeliness message processing method of the present invention avoids the degradation of query performance caused by a large amount of data accumulation, and improves the efficiency of message push; it does not cause the problem of long transaction time for cleaning up large batches of data, and avoids the situation of timeout and deadlock.
[0020] The method and device for high-timeliness message processing of the present invention greatly improves the efficiency of querying and loading messages in message sending scenarios by using hash partitions to jump to partition queries according to business priorities and certain rules, while solving abnormal situations such as data cleaning and alleviating lock competition.
[0021] The method and device for processing high-timeliness messages of the present invention enhance flexibility under complex business scenario requirements, flexibly configure business priorities, and can meet various complex business scenarios. Compared with the traditional sending mode, it is more flexible.
[0022] The method and device for high-timeliness message processing of the present invention improves system resource utilization, solves the problem of message reception delay, improves SQL query performance, and further improves the utilization of the message sending channel and the overall efficiency of message sending.
[0023] The method and device for high-timeliness message processing of the present invention solve the problem of data cleaning anomalies, reduce the amount of data by deleting within the partition, greatly improve the deletion performance, and avoid anomalies such as timeouts and deadlocks caused by long transactions.
[0024] The high-timeliness message processing method and device of the present invention reduces lock contention. Under partition mode query, each partition is independent, which will greatly reduce lock contention in the case of multiple threads and multiple copies, and is more friendly to server expansion in high concurrency scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present invention, but are not intended to limit the present invention.
[0026] Figure 1 is a schematic diagram showing partition jump of high-timeliness message processing according to an embodiment of the present invention;
[0027] Figure 2 is a flow chart showing partition jump of high timeliness message processing according to an embodiment of the present invention;
[0028] Figure 3a~3d is a pseudo code illustrating a method for processing a high-timeliness message according to an embodiment of the present invention;
[0029] Figure 4a It is a schematic diagram of message loading speed in the prior art;
[0030] Figure 4b It is a schematic diagram showing the message loading speed in a partition design mode of high-timeliness message processing according to an embodiment of the present invention. DETAILED DESCRIPTION
[0031] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0032] Unless otherwise defined, the technical or scientific terms used herein shall have the common meanings understood by persons with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the patent application specification and claims of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "a" do not indicate a quantity limitation, but indicate the existence of at least one.
[0033] According to one or some embodiments of the present invention, a method for processing high-timeliness messages includes the following steps: data table setting, service priority queue setting, and sending logic setting.
[0034] In the data table settings, the traditional list partitioning method is cancelled, and the sending data table is hash partitioned according to the unique primary key id, which is divided into multiple hash partitions. All message data will be evenly inserted into multiple hash partition libraries after being stored.
[0035] Business priority queue settings, set the priority order in the default state, the default design is based on the priority order from the first to the last in the queue array, or flexibly adopt methods including adjusting the queue size or the number of messages in the queue to change the order; design the priority configuration method, read the configuration in the form of environment variables in the default state, if the message queue is set, it will be read according to the configuration in the message queue.
[0036] Figure 1 is a schematic diagram showing partition jump of high-timeliness message processing according to an embodiment of the present invention; Figure 2 It is a partition jump flow chart showing high-timeliness message processing according to an embodiment of the present invention.
[0037] like Figure 1 and Figure 2 As shown, the sending logic setting includes the steps of: setting the basic jump rule of the partition, setting the basic rules for querying and loading messages within the partition, setting the single query threshold and the partition jump rule.
[0038] Set the partition-based jump rule, and repeatedly load the short messages to be sent from the first partition according to the message query rules within the partition in a polling manner until the last partition is traversed; when the query of the last partition is completed, restart the query from the first partition.
[0039] Set the basic rules for querying and loading messages within the partition. When querying within the current partition, set the query rules and priority queues, define the maximum number of queries, and query each time according to the basic order rule from the highest priority to the lowest priority. Load the maximum number of messages to be sent of a certain priority into the message sending queue; if the query result is less than the maximum number of queries, query the next priority message to be sent in the current partition until the lowest priority message to be sent in the current partition is queried, and then enter the next partition; if the query result reaches the maximum number of queries, jump to the next partition.
[0040] Set the single query threshold and partition jump rule. In the current partition, the maximum number of queries is set each time. If the number of messages to be sent for a certain priority level reaches the maximum number of queries for three consecutive queries, it means that the number of messages of this priority level is large in the current partition. In this case, determine whether this priority level is the first priority level, and then execute according to the following situations.
[0041] For the first priority, continue to query and load the pending messages of this priority in the current partition until the maximum number of messages is less than the query quantity at a certain time, and then query the next priority message of the current partition; for non-first priority, directly jump to the next partition and start querying again from the highest priority, and so on.
[0042] The method of high-timeliness message processing of the present invention proposes a message processing mode based on business priority and hash partitioning. By flexibly configuring business priority, the specific needs of different businesses for message sending are met, and the pertinence and response speed of the business are improved; at the same time, by using hash partition database design and specific partition jump algorithm, the data processing pressure is effectively dispersed, the performance of message processing is greatly improved, and the efficiency and reliability of message processing are ensured.
[0043] According to one or some embodiments of the present invention, in the data table setting, the hash partition is divided into 32 hash partitions.
[0044] The method and device for high-timeliness message processing of the present invention greatly improves the efficiency of querying and loading messages in message sending scenarios by using hash partitions to jump to partition queries according to business priorities and certain rules, while solving abnormal situations such as data cleaning and alleviating lock competition.
[0045] According to one or some embodiments of the present invention, the sending logic setting also includes the steps of: setting a method for handling blocking situations in the sending queue; when the sending queue currently has a large number of messages to be sent, it will jump to the next partition after a one-second rest and process the messages to be sent in the order of high priority to low priority according to the above rules.
[0046] The high-timeliness message processing method of the present invention avoids the degradation of query performance caused by a large amount of data accumulation, and improves the efficiency of message push; it does not cause the problem of long transaction time for cleaning up large batches of data, and avoids the situation of timeout and deadlock.
[0047] According to one or some embodiments of the present invention, in the sending logic setting, the maximum number of queries is defined as 100.
[0048] The method and device for high-timeliness message processing of the present invention improves system resource utilization, solves the problem of message reception delay, improves SQL query performance, and further improves the utilization of the message sending channel and the overall efficiency of message sending.
[0049] According to a second aspect of the present invention, there is provided a device for processing high-timeliness messages, which includes: a data table setting module, a service priority queue setting module, and a sending logic setting module.
[0050] The data table setting module is used to perform hash partitioning on the sending data table according to the unique primary key id, and divide it into multiple hash partitions. After all message data is entered into the database, it will be evenly inserted into multiple hash partition libraries.
[0051] The business priority queue setting module is used to set the priority order in the default state. The default is designed according to the priority order from the first to the last in the queue array, or flexibly adopts methods including adjusting the queue size or the number of messages in the queue to change the order; the priority configuration method is designed. The configuration is read in the form of environment variables in the default state. If a message queue is set, it is read according to the configuration in the message queue.
[0052] The sending logic setting module is used to set the sending logic and perform operations: setting the basic jump rules for partitions, setting the basic rules for querying and loading messages within partitions, setting the single query threshold and partition jump rules.
[0053] Set the partition-based jump rule, and repeatedly load the short messages to be sent from the first partition according to the message query rules within the partition in a polling manner until the last partition is traversed; when the query of the last partition is completed, restart the query from the first partition.
[0054] Set the basic rules for querying and loading messages within the partition. When querying within the current partition, set the query rules and priority queues, define the maximum number of queries, and query each time according to the basic order rule from the highest priority to the lowest priority. Load the maximum number of messages to be sent of a certain priority into the message sending queue; if the query result is less than the maximum number of queries, query the next priority message to be sent in the current partition until the lowest priority message to be sent in the current partition is queried, and then enter the next partition; if the query result reaches the maximum number of queries, jump to the next partition.
[0055] Set the single query threshold and partition jump rule. In the current partition, the maximum number of queries is set each time. If the number of pending messages of a certain priority level reaches the maximum number of queries for three consecutive queries, it means that the number of messages of this priority level is large in the current partition. Then determine whether this priority level is the first priority level, and then execute according to the following situations: For the first priority level, continue to query and load pending messages of this priority level in the current partition until the number of pending messages is less than the maximum number of queries in a certain query, and then query the next priority message of the current partition; For non-first priority level, directly jump to the next partition and start querying again from the highest priority level, and so on.
[0056] The high-timeliness message processing method and device of the present invention reduces lock contention. Under partition mode query, each partition is independent, which will greatly reduce lock contention in the case of multiple threads and multiple copies, and is more friendly to server expansion in high concurrency scenarios.
[0057] According to one or some embodiments of the present invention, in the data table setting module, the hash partition is divided into 32 hash partitions.
[0058] According to one or some embodiments of the present invention, in the sending logic setting module, the following operations are further performed: the sending queue blocking situation processing method is set. When the sending queue currently has a large number of messages to be sent, it will jump to the next partition after a one-second rest and process the messages to be sent in the order of high priority to low priority according to the above rules.
[0059] The method and device for high-timeliness message processing of the present invention solve the problem of data cleaning anomalies, reduce the amount of data by deleting within the partition, greatly improve the deletion performance, and avoid anomalies such as timeouts and deadlocks caused by long transactions.
[0060] According to one or some embodiments of the present invention, in the sending logic setting module, the maximum number of queries is defined as 100.
[0061] The method and device for processing high-timeliness messages of the present invention enhance flexibility under complex business scenario requirements, flexibly configure business priorities, and can meet various complex business scenarios. Compared with the traditional sending mode, it is more flexible.
[0062] Figure 3a to 3d The pseudo code of the method for processing a high-timeliness message according to an embodiment of the present invention is shown to express the concept and logical structure of the method for processing a high-timeliness message according to an embodiment of the present invention.
[0063] Figure 4a It is a schematic diagram of message loading speed in the prior art; Figure 4b It is a schematic diagram showing the message loading speed in a partition design mode of high-timeliness message processing according to an embodiment of the present invention.
[0064] like Figure 4a and 4b As shown, when used, performance analysis is performed:
[0065] By observing the actual performance and partition jump process in the production environment, in the case of single-point execution (single server), it takes about 50-100 milliseconds to scan a partition, and the total time required to complete a polling scan of all partitions is 0.5 to 1.5 seconds.
[0066] According to the measurement and actual operation results, there is no situation where data cannot be processed in a timely manner due to too many partitions.
[0067] Performance comparison, using 10 million data volumes to compare the two modes, the loading conditions are as follows Figure 4a and 4b shown.
[0068] The results of the two modes show that the new message processing mode significantly improves the efficiency of message sending, and the average sending time is reduced from 80 milliseconds to about 5 milliseconds. In the actual production environment, the receiving rate of important business messages such as verification codes in the SMS system has increased significantly, and customers can quickly receive important notifications even during business peak periods.
[0069] The method for processing high-timeliness messages of the present invention can flexibly configure service priorities and effectively manage service priorities, so that emergency service messages can be delivered in a timely manner, bringing benefits to enterprises and users.
[0070] According to another aspect of the present invention, a device for high-timeliness message processing is provided, comprising: a memory, a processor, and a high-timeliness message processing program stored in the memory and executable on the processor, wherein the high-timeliness message processing program implements the steps of the above-mentioned high-timeliness message processing method when executed by the processor.
[0071] According to the present invention, a computer storage medium is also provided.
[0072] A high-timeliness message processing program is stored on the computer storage medium, and when the high-timeliness message processing program is executed by the processor, the steps of the high-timeliness message processing method described above are implemented.
[0073] Among them, the method implemented when the high-timeliness message processing program running on the processor is executed can refer to the various embodiments of the high-timeliness message processing method of the present invention, and will not be repeated here.
[0074] The present invention also provides a computer program product.
[0075] The computer program product of the present invention comprises a high-timeliness message processing program, and when the high-timeliness message processing program is executed by a processor, the steps of the high-timeliness message processing method as described above are implemented.
[0076] Among them, the method implemented when the high-timeliness message processing program running on the processor is executed can refer to the various embodiments of the high-timeliness message processing method of the present invention, and will not be repeated here.
[0077] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.
[0078] The above description is merely an exemplary embodiment of the present invention and is not intended to limit the protection scope of the present invention. The protection scope of the present invention is determined by the appended claims.
Claims
1. A method for processing a high-timeliness message, comprising the following steps: In the data table settings, the traditional list partitioning method is cancelled, and the sending data table is hash partitioned according to the unique primary key ID, which is divided into multiple hash partitions. After all the message data is stored, it will be evenly inserted into multiple hash partition libraries; Business priority queue settings, set the priority order in the default state, and design it in the order from the first to the last in the queue array by default, or flexibly adopt methods including adjusting the queue size or the number of messages in the queue to change the order; design the priority configuration method, and read the configuration in the way of environment variables in the default state. If a message queue is set, it will be read according to the configuration in the message queue first; Send logic settings, including the following steps: Set the basic jump rule of the partition, and repeat the process of loading the short messages to be sent from the first partition according to the rules of message query in the partition in a round-robin manner until the last partition is traversed; after the query of the last partition is completed, restart the query from the first partition; Set the basic rules for querying and loading messages within the partition. When querying within the current partition, set the query rules and priority queues, define the maximum number of queries, and query the maximum number of messages of a certain priority to be sent each time according to the basic order rule from the highest priority to the lowest priority and load them into the message sending queue; If the query result is less than the maximum query number, the next priority message to be sent in the current partition will be queried until the lowest priority message to be sent in the current partition is queried, and then the next partition will be entered; if the query result is full of the maximum query number, jump to the next partition; Set the single query threshold and partition jump rule. In the current partition, the maximum number of queries is set for each query. If the number of messages to be sent for a certain priority level reaches the maximum number of queries for three consecutive queries, determine whether the priority level is the first priority level, and then execute according to the following situations: First priority: Continue to query and load the messages to be sent of this priority in the current partition until the maximum number of queries is less than the maximum number of queries, and then query the next priority message of the current partition; If it is not the first priority, it will jump directly to the next partition and start the query again from the highest priority, and so on.
2. The method for processing a time-sensitive message according to claim 1, wherein: In the data table settings, the hash partition is divided into 32 hash partitions.
3. The method for processing time-sensitive messages according to claim 1, wherein: The sending logic setting also includes the following steps: Set the method for handling blocked sending queues. When there are many messages to be sent in the sending queue, it will jump to the next partition after a one-second rest and process the messages to be sent in the order of high priority to low priority according to the above rules.
4. The method for processing time-sensitive messages according to claim 1, wherein: In the sending logic settings, define the maximum number of queries as 100.
5. A device for processing high-timeliness messages, comprising: The data table setting module is used to perform hash partitioning on the sending data table according to the unique primary key ID, and divide it into multiple hash partitions. After all the message data is stored, it will be evenly inserted into multiple hash partition libraries; The business priority queue setting module is used to set the priority order in the default state. By default, the priority order is designed from the first to the last in the queue array, or the order can be changed flexibly by adjusting the queue size or the number of messages in the queue; the priority configuration method is designed. By default, the configuration is read in the form of environment variables. If a message queue is set, the configuration in the message queue is read first; The sending logic setting module is used to set the sending logic and perform the following operations: Set the basic jump rule of the partition, and repeat the process of loading the short messages to be sent from the first partition according to the rules of message query in the partition in a round-robin manner until the last partition is traversed; after the query of the last partition is completed, restart the query from the first partition; Set the basic rules for querying and loading messages within the partition. When querying within the current partition, set the query rules and priority queues, define the maximum number of queries, and query the maximum number of messages of a certain priority to be sent each time according to the basic order rule from the highest priority to the lowest priority and load them into the message sending queue; If the query result is less than the maximum query number, the next priority message to be sent in the current partition will be queried until the lowest priority message to be sent in the current partition is queried, and then the next partition will be entered; if the query result is full of the maximum query number, jump to the next partition; Set the single query threshold and partition jump rule. In the current partition, the maximum number of queries is set for each query. If the number of messages to be sent for a certain priority level reaches the maximum number of queries for three consecutive queries, determine whether the priority level is the first priority level, and then execute according to the following situations: First priority: Continue to query and load the messages to be sent of this priority in the current partition until the maximum number of queries is less than the maximum number of queries, and then query the next priority message of the current partition; If it is not the first priority, it will jump directly to the next partition and start the query again from the highest priority, and so on.
6. The device for processing high-timeliness messages as claimed in claim 5, wherein: In the data table setting module, hash partitions are divided into 32 hash partitions.
7. The device for processing high-timeliness messages according to claim 5, wherein: In the sending logic setting module, the following operations are also performed: Set the method for handling blocked sending queues. When there are many messages to be sent in the sending queue, it will jump to the next partition after a one-second rest and process the messages to be sent in the order of high priority to low priority according to the above rules.
8. The device for processing high-timeliness messages as claimed in claim 5, wherein: In the sending logic setting module, the maximum number of queries is defined as 100.
9. An electronic device, comprising: A memory, a processor, and a high-timeliness message processing program stored in the memory and executable on the processor, wherein the high-timeliness message processing program, when executed by the processor, implements the steps of the high-timeliness message processing method as described in any one of claims 1 to 4.
10. A computer storage medium, wherein: The computer storage medium stores a high-timeliness message processing program, and when the high-timeliness message processing program is executed by the processor, the steps of the high-timeliness message processing method as described in any one of claims 1 to 4 are implemented.
Citation Information
Patent Citations
Chat message processing method and device, computer equipment and storage medium
CN115633187A
Data hierarchical storage and query processing method and device, equipment and medium
CN119415568A
Prioritized lock requests to reduce blocking
US20140181342A1
Dynamic resource allocation for common storage query
US20180089262A1
Method and apparatus for processing message between processors
WO2014025145A1