Message processing method and device, equipment and medium

Abstract

The invention discloses a message processing method and device, equipment and a medium, and belongs to the technical field of data processing, and the method comprises the steps: receiving a to-be-processed message of a client; respectively sending the to-be-processed message to message processing subunits of a plurality of service units; determining a message dequeuing priority of the to-be-processed message through each message processing subunit; in response to the fact that one message processing subunit preferentially calculates and completes the message out-of-column priority of the message to be processed, the message out-of-column priority calculated by the message processing subunit serves as the target message out-of-column priority of the message to be processed; sending the to-be-processed message to a plurality of service units according to the target message dequeuing priority; and processing the to-be-processed message according to the message type. According to the method and the device, the problems of message retention and untimely message processing in the prior art are solved, the usability of the system is improved, and better use experience is brought to a user.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a message processing method, device, equipment and medium. Background Art

[0002] With the maturity of cloud computing technology, more and more businesses are no longer running on physical servers, but cloud hosts instead, in which cloud platforms play an important role; cloud platforms mainly manage virtualized resources from underlying physical machines, such as virtual machines, networks, virtual disks, etc., and display and manage these resources in the cloud platform. The consistency between the upper display and the underlying data is the main problem affecting the usability of the product; the cloud platform is based on long-connected websocket messages to ensure the synchronization of underlying and cloud platform resources. When the underlying resources change, websocket messages will be sent to notify the upper cloud platform to update. As more businesses are put on the cloud, the cloud platform's business is becoming more and more complex. Virtual resources are not only virtual machines, networks, and storage, but also bare metal, load balancing, custom networks and other resources. Therefore, more and more websocket messages are released, and the logic of processing messages is becoming more and more complex.

[0003] The related approach is to have a unified module receive websocket messages, put them into a queue, then get messages in time order, and distribute them to other modules for processing according to message types. This solution is fine for general scenarios, but if a module receives more messages than other modules in a short period of time, it will cause other modules to receive messages untimely. Or if a module is damaged, the messages of this module will not be processed, resulting in message backlogs, and new messages will not be processed, which will cause all messages to not be processed and cause message congestion. Summary of the invention

[0004] In order to overcome the above technical defects, the purpose of this application is to provide a message processing method, device, equipment and medium, the method comprising: receiving the pending messages of the client through the several business units; sending the pending messages to the message processing subunits of several business units respectively; determining the message dequeue priority of the pending messages through each message processing subunit; in response to one of the message processing subunits preferentially calculating the message dequeue priority of the pending message, the message dequeue priority calculated by the message processing subunit is used as the target message dequeue priority of the pending message; sending the pending message to the cache queue according to the target message dequeue priority; the cache queue sends the pending message to several business units according to the target message dequeue priority; the several business units process the pending messages according to the message type. This application solves the problems of message retention and untimely message processing in the original technology, improves the ease of use of the system, and brings a better user experience to users.

[0005] The specific technical solutions provided by the embodiments of this application are as follows:

[0006] In a first aspect, the present application provides a message processing method, which is applied to a message processing system, wherein the system includes a plurality of service units, a cache queue, and a client, each service unit includes a message processing subunit, and the method includes:

[0007] Receiving the to-be-processed messages from the client through the plurality of business units;

[0008] Sending the to-be-processed messages to the message processing subunits of several business units respectively;

[0009] Determining the message dequeue priority of the to-be-processed message by each message processing subunit;

[0010] In response to one of the message processing subunits preferentially calculating the message dequeue priority of the message to be processed, the message dequeue priority calculated by the message processing subunit is used as the target message dequeue priority of the message to be processed;

[0011] Sending the to-be-processed message to the cache queue according to the dequeuing priority of the target message;

[0012] The cache queue sends the to-be-processed message to a plurality of business units according to the dequeuing priority of the target message;

[0013] The plurality of service units process the messages to be processed according to the message types.

[0014] In one embodiment, before determining the message dequeue priority of the to-be-processed message by each message processing subunit, the process includes:

[0015] Deduplication processing is performed on the messages to be processed;

[0016] The deduplication processing of the to-be-processed message includes:

[0017] Create a message data collection;

[0018] Calculating a hash value of the message to be processed;

[0019] Determine whether the hash value of the message to be processed exists in the message data set;

[0020] If yes, delete the pending message; if no, add the hash value of the pending message to the message data set;

[0021] The deduplication processing of the to-be-processed message further includes:

[0022] Create an identifier for each message to be processed;

[0023] Determining a distributed lock according to the identifier of the message to be processed, wherein the distributed lock includes a database-based lock, a distributed application-based lock, and a distributed key-value-based lock;

[0024] When a distributed lock can be determined according to the identifier of the message to be processed, releasing the distributed lock of the message to be processed;

[0025] When the distributed lock cannot be determined according to the identifier of the message to be processed, the message corresponding to the identifier is deleted.

[0026] In one of the embodiments, determining the message dequeue priority of the to-be-processed message by each message processing subunit includes:

[0027] Get the message weight coefficient n, the number of times the message is processed m, and the current time t now , the time t when the message is initially placed into the cache queue start ;

[0028] Get the retention time f corresponding to the maximum priority score and the maximum priority score g;

[0029] By formula: A message dequeue priority score of the to-be-processed message is calculated.

[0030] In one embodiment, the plurality of business units process the to-be-processed messages according to the message types, including:

[0031] Classifying the messages to be processed;

[0032] Get the correspondence table between message types and business units;

[0033] Determine the target processing service unit corresponding to the message to be processed according to the message type of the message to be processed and the correspondence table between the message type and the service unit;

[0034] Determining whether the business unit is a target processing business unit;

[0035] If so, the message is processed by the message processing subunit of the business unit; if not, the message dequeue priority of the pending message is recalculated by each message processing subunit, and the pending message is stored in the cache queue according to the corresponding message dequeue priority calculated in priority.

[0036] In one embodiment, the method comprises:

[0037] Setting the message dequeue priority of the to-be-processed message to increase first and then decrease as the retention time in the cache queue;

[0038] When the retention time of any to-be-processed message in the cache queue is greater than a first threshold, the message is deleted from the cache queue.

[0039] In one embodiment, the method further comprises:

[0040] In response to the remaining message processing subunits not preferentially calculating the message dequeue priorities of the to-be-processed messages, the to-be-processed messages in the remaining message processing subunits and the calculated priority scores are deleted.

[0041] In one of the embodiments, determining the message dequeue priority of the to-be-processed message by each message processing subunit includes:

[0042] Classifying the to-be-processed messages by the four-quadrant rule, wherein the classification types include urgent and important messages, important but not urgent messages, urgent but not important messages, and neither urgent nor important messages;

[0043] Determine the message dequeue priority according to the classification type of the pending message, and process the pending message;

[0044] The step of determining the message dequeue priority of the to-be-processed message by each message processing subunit further includes:

[0045] Score and sort the messages to be processed according to their importance;

[0046] The message dequeue priority is determined according to the score of the message to be processed, and the message to be processed is processed.

[0047] In a second aspect, the present application further provides a message processing device, which is applied to a message processing system, wherein the system includes a plurality of service units, a cache queue, and a client, each service unit includes a message processing subunit, and the device includes:

[0048] A receiving module, used for receiving the to-be-processed messages of the client through the plurality of business units;

[0049] A sending module, used to send the pending messages to the message processing subunits of several business units respectively; send the pending messages to the cache queue according to the dequeuing priority of the target message; send the pending messages to several business units according to the dequeuing priority of the target message;

[0050] A determination module, used to determine the message dequeue priority of the to-be-processed message through each message processing subunit;

[0051] In response to one of the message processing subunits preferentially calculating the message dequeue priority of the message to be processed, the message dequeue priority calculated by the message processing subunit is used as the target message dequeue priority of the message to be processed;

[0052] The processing module is used to process the message to be processed according to the message type.

[0053] In a third aspect, a message processing device is also provided, including:

[0054] one or more processors;

[0055] A storage device for storing one or more programs;

[0056] When the one or more programs are executed by the one or more processors, the one or more processors implement the message processing method as described in any one of the first aspects.

[0057] In a fourth aspect, the present application further provides a computer device, the device comprising:

[0058] A memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the message processing method as described in any one of the first aspects.

[0059] In a fifth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any message processing method described in the first aspect.

[0060] In a sixth aspect, the present application further provides a computer storage medium, the medium comprising:

[0061] A computer program is stored thereon, and when the computer program is executed by a processor, the steps of any message processing method described in the first aspect are implemented.

[0062] Compared with the prior art, the method described in the technical solution provided by the embodiment of the present application includes: receiving the pending messages of the client; sending the pending messages to the message processing subunits of several business units respectively; determining the message dequeue priority of the pending messages by each message processing subunit; in response to one of the message processing subunits preferentially calculating the message dequeue priority of the pending messages, the message dequeue priority calculated by the message processing subunit is used as the target message dequeue priority of the pending messages; sending the pending messages to the cache queue according to the target message dequeue priority; the cache queue sends the pending messages to several business units according to the target message dequeue priority; the several business units process the pending messages according to the message type. The present application solves the problems of message retention and untimely message processing in the original technology, improves the ease of use of the system, and brings a better user experience to users.

[0063] The technical solution provided in the embodiment of the present application is based on automatic claiming by each module and message processing based on a priority algorithm. It is more compatible with multi-module scenarios, enables each module to process messages more evenly, enables the cloud platform to respond more quickly to changes in virtualization layer resources, and improves the usability of the product.

[0064] The technical solution provided in the embodiment of the present application solves the problem of processing resource synchronization through websocket messages in complex scenarios with multiple modules and multiple resources. It changes the original single module processing to processing through various business modules, thereby enhancing the robustness of the system and preventing the entire business from being affected by the damage of one module.

[0065] The technical solution provided in the embodiment of the present application combines the message retention time, message type weight coefficient, and message processing coefficient to derive a priority calculation formula, which allows for more flexible message sorting and processing, solves the original problems of message retention and untimely message processing, improves the usability of the system, and provides users with a better user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0067] Figure 1 This is a structural diagram of a traditional message processing system;

[0068] Figure 2 A structural diagram of a message processing system provided in Example 1 of the present application;

[0069] Figure 3 A first flow chart of a message processing method provided in Embodiment 1 of the present application;

[0070] Figure 4 A second flow chart of the message processing method provided in Embodiment 2 of the present application;

[0071] Figure 5 A diagram showing the relationship between time and priority provided in Example 2 of the present application;

[0072] Figure 6 A specific flow chart of the message processing method provided in Example 2 of the present application;

[0073] Figure 7 A structural diagram of a message processing device provided in Embodiment 3 of the present application;

[0074] Figure 8 The exemplary system provided for Embodiment 7 of the present application can be used to implement the various embodiments described in the present application. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0076] It should be noted that, unless the context clearly requires otherwise, words such as “include”, “including” and similar words throughout the specification and claims should be interpreted as including rather than exclusive or exhaustive; that is, the meaning is “including but not limited to”.

[0077] Furthermore, in the description of the present application, unless otherwise specified, “plurality” means two or more.

[0078] like Figure 1 As shown in the figure, when the virtual machine, network, and storage resources change, messages will be pushed to the cloud platform. The message processing module of the cloud platform will store the received messages in the queue, and then regularly take out messages from the queue according to the time arrangement, and call each module to process the messages according to the message type. When this process faces a multi-module scenario, problems will arise: Scenario 1: For example, the computing module suddenly receives a lot of messages, while the network storage module receives very few messages, then most of the messages in the queue are from the computing module, and the messages from the network storage module cannot be processed in time; Scenario 2: For example, if the computing module is damaged, the messages of the computing module will remain in the queue and cannot be processed, which will cause message backlogs.

[0079] That is, in one scenario, messages are processed in reverse chronological order, that is, the first messages are processed first, and the later messages are processed later; however, this situation can easily cause message backlogs. If a certain type of message has not been processed, it will be piled up in the front, causing later messages to not be processed; in another scenario, some message resources need to respond faster, while some message resources do not have such high real-time requirements and can respond slower. In this case, the traditional message processing method is not flexible.

[0080] Embodiment 1

[0081] The present application embodiment provides a message processing method, such as Figure 2 , 3 As shown, the method is applied to a message processing system, the system includes several business units, a cache queue and a client, each business unit includes a message processing subunit, and the method includes:

[0082] Receiving the to-be-processed messages of the client through the plurality of service units, wherein the service units include a computing unit, a network unit and a storage unit;

[0083] Sending the to-be-processed messages to the message processing subunits of several business units respectively;

[0084] Determining the message dequeue priority of the to-be-processed message by each message processing subunit;

[0085] In response to one of the message processing subunits preferentially calculating the message dequeue priority of the message to be processed, the message dequeue priority calculated by the message processing subunit is used as the target message dequeue priority of the message to be processed;

[0086] Sending the to-be-processed message to the cache queue according to the dequeuing priority of the target message;

[0087] The cache queue sends the to-be-processed message to a plurality of business units according to the dequeuing priority of the target message;

[0088] The plurality of service units process the messages to be processed according to the message types.

[0089] Specifically, the message processing method designed in the present application is changed from the original unified message processing module to each business unit being able to process it, which can improve the robustness of the system. When the message processing sub-unit of any business unit is damaged, it will not affect the reception of the message. After receiving the message, the business unit will perform deduplication processing, calculate the priority, and then store it in the cache queue.

[0090] Each business unit has a scheduled task. When the task starts, messages are retrieved according to the message priority order. For each message, it is determined whether it is to be processed by this business module (unit). If so, it is removed from the cache queue and then processed. If not, the message dequeue priority is recalculated and the messages in the queue are updated. At the start of the next round of scheduled tasks, messages are retrieved according to the new priority order and processed.

[0091] Here, the business unit is applied to a cloud computing business scenario, and the business unit includes a computing unit, a network unit, and a storage unit; the computing unit is used for cloud computing of virtual machines; the network unit is used for network communication of virtual machines; and the storage unit is used for data storage of virtual machines.

[0092] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0093] The present application solves the problems of message retention and untimely message processing in the prior art, improves the usability of the system, and brings a better user experience to users.

[0094] The technical solution provided in the embodiment of the present application is based on automatic claiming by each module and message processing based on a priority algorithm. It is more compatible with multi-module scenarios, enables each module to process messages more evenly, enables the cloud platform to respond more quickly to changes in virtualization layer resources, and improves the usability of the product.

[0095] The technical solution provided in the embodiment of the present application solves the problem of processing resource synchronization through websocket messages in complex scenarios with multiple modules and multiple resources. It changes the original single module processing to processing through various business modules, thereby enhancing the robustness of the system and preventing the entire business from being affected by the damage of one module.

[0096] Embodiment 2

[0097] The present application embodiment provides a message processing method, such as Figure 4As shown, the method is applied to a message processing system, the system includes several business units, a cache queue and a client, each business unit includes a message processing subunit, and the method includes:

[0098] Step S01, receiving the to-be-processed message of the client through the plurality of business units;

[0099] The messages to be processed are sent to the message processing subunits of several business units respectively.

[0100] Specifically, the message processing subunit for processing messages is first extracted to form a common function, and other business modules (units) establish connections, receive messages, and process messages through the message processing subunit.

[0101] Step S02, performing deduplication processing on the message to be processed;

[0102] The deduplication processing of the to-be-processed message includes:

[0103] Create a message data collection;

[0104] Calculating a hash value of the message to be processed;

[0105] Determine whether the hash value of the message to be processed exists in the message data set;

[0106] If yes, delete the pending message; if no, add the hash value of the pending message to the message data set;

[0107] The deduplication processing of the to-be-processed message further includes:

[0108] Create an identifier for each message to be processed;

[0109] Determining a distributed lock according to the identifier of the message to be processed, wherein the distributed lock includes a database-based lock, a distributed application-based lock, and a distributed key-value-based lock;

[0110] When a distributed lock can be determined according to the identifier of the message to be processed, releasing the distributed lock of the message to be processed;

[0111] When the distributed lock cannot be determined according to the identifier of the message to be processed, the message corresponding to the identifier is deleted.

[0112] Specifically, a hash value of the message is calculated and the hash value is stored in the data set;

[0113] Whenever a new message is received, first calculate its hash value, and then check whether the hash value already exists in the set;

[0114] If it does not exist, it is considered to be a new message, and the corresponding processing is performed and the message hash value is added to the collection; if it already exists, it is considered to be a duplicate message and can be discarded or further processed.

[0115] When a message is received, the distributed lock is first acquired based on the unique identifier of the message; if the lock is successfully acquired, it means that the message is processed for the first time and the subsequent business logic can continue to execute; if the lock cannot be acquired, it means that the message is being processed by other modules or has been processed. At this time, you can give up processing this message or wait for a while and try to acquire the lock again.

[0116] Finally, once the message processing is complete, the lock should be released immediately so that other modules that may hold the same message can acquire the distributed lock and process the message.

[0117] Step S03: determining the message dequeue priority of the message to be processed through each message processing subunit.

[0118] Specifically, when a message is sent remotely, each module has the ability to receive the message, first deduplicate the message, then calculate the priority according to the priority calculation formula mentioned above, put it in the queue, and write the time of putting it in.

[0119] Step S03 also includes:

[0120] Step S031, obtain the message weight coefficient n, the number of times the message is processed m, and the current time t now , the time t when the message is initially placed into the cache queue start ;

[0121] Get the retention time f corresponding to the maximum priority score and the maximum priority score g;

[0122] By formula: A message dequeue priority score of the to-be-processed message is calculated.

[0123] Specifically, this application adopts a normal distribution algorithm and a custom weight method. As the time the message is placed in the cache queue increases, the priority becomes higher first, that is, it is higher than the new message just placed in, and it should be processed as soon as possible; when the message has not been processed, there may be a problem with the business module, and as time goes by, the priority becomes lower.

[0124] Wherein: n is the weight coefficient, and different values ​​are set for messages of different resources. A larger value is set for messages with high real-time requirements, and a smaller value can be set for messages with low real-time requirements; m is the number of processing times. In the present application scheme, a message can be processed by multiple business modules. If it is not a message of this module, it will continue to stay in the cache queue. The value of m is the number of times it is processed. The more times it is processed, the lower the priority is, to prevent the business module from being damaged and the message from being stuck in the queue; t now is the current time, t start is the time when the message is first put into the cache queue, f is the retention time when the priority score reaches the maximum value, and g is the maximum priority; the derivation process is initially y = -x 2 The figure is moved up g units, and if you want to move right f units, it is Figure 5 , substitute the time when the message is put into the queue into the formula, and then multiply it by the weight coefficient to get the final sorting priority score.

[0125] It is understandable that the custom message type weight coefficient is set larger for messages with high real-time requirements and more important messages, and is set smaller for messages with less real-time requirements.

[0126] As the retention time of messages increases, the priority level first increases and then decreases. This can prevent new messages from affecting messages in the queue that have not been processed yet, and can also prevent messages that have been retained in the cache queue for a long time from affecting the processing of new messages.

[0127] When a message is processed many times but has not been consumed, there may be a problem with the program. Each time a message is processed, the number of times the message is processed increases by one, and the priority decreases.

[0128] Step S032, classifying the to-be-processed messages by the four-quadrant rule, wherein the classification types include urgent and important messages, important but not urgent messages, urgent but not important messages, and neither urgent nor important messages;

[0129] Determine the message dequeue priority according to the classification type of the pending message, and process the pending message;

[0130] The step of determining the message dequeue priority of the to-be-processed message by each message processing subunit further includes:

[0131] Score and sort the messages to be processed according to their importance;

[0132] The message dequeue priority is determined according to the score of the message to be processed, and the message to be processed is processed.

[0133] Step S04, in response to one of the message processing subunits preferentially calculating the message dequeue priority of the message to be processed, the message dequeue priority calculated by the message processing subunit is used as the target message dequeue priority of the message to be processed;

[0134] Sending the to-be-processed message to the cache queue according to the dequeuing priority of the target message;

[0135] The cache queue sends the to-be-processed message to a plurality of service units according to the dequeuing priority of the target message.

[0136] Specifically, messages are pulled from the cache queue according to the priority score calculated by the calculation formula, and are sorted according to the priority, with high priority messages being processed first and low priority messages being processed later.

[0137] Step S05: the plurality of business units process the to-be-processed messages according to message types.

[0138] Specifically, different business modules can actively pull messages when processing messages and only process messages of this module. Each module can pull messages from the cache queue, and then determine whether it is processed by this module based on the message type. If not, the priority is recalculated and put into the queue; if it is processed by this module, it is put into the business module for processing.

[0139] It can be understood that the correspondence between message types and various business modules is first defined;

[0140] After receiving the message, this module determines whether it is the message to be processed by this module based on the correspondence between the message type and this module;

[0141] If it is a message to be processed by this module, it will be put into the business module for processing; if it is not a message for this module, the priority will be recalculated according to the priority calculation formula and put into the cache queue.

[0142] Step S05 also includes:

[0143] Step S051, classifying the messages to be processed;

[0144] Get the correspondence table between message types and business units;

[0145] Determine the target processing service unit corresponding to the message to be processed according to the message type of the message to be processed and the correspondence table between the message type and the service unit;

[0146] Determining whether the business unit is a target processing business unit;

[0147] If so, the message is processed by the message processing subunit of the business unit; if not, the message dequeue priority of the pending message is recalculated by each message processing subunit, and the pending message is stored in the cache queue according to the corresponding message dequeue priority calculated in priority.

[0148] Specifically, each business module has a scheduled task inside. It finds out the messages that need to be processed according to the priority sorting of the messages in the queue, and then determines whether it is the message to be processed by this module based on the message type. If it is, it is removed from the queue and then placed in the business module for processing. If not, the priority of the message is recalculated and the priority of the message in the queue is updated.

[0149] When the next scheduled task is executed, it will be sorted according to the new priority, and messages with high priority will be processed first.

[0150] After the above processing is completed, the parameters are placed in the cache queue and the key of the cached data is returned.

[0151] Step S06, Figure 5 As shown, the message dequeue priority of the to-be-processed message is set to first increase and then decrease with the retention time in the cache queue;

[0152] When the retention time of any to-be-processed message in the cache queue is greater than a first threshold, the message is deleted from the cache queue.

[0153] Specifically, in the end, if the message has not been processed in the cache queue, it will be removed after the retention time exceeds 24 hours and will not occupy queue resources.

[0154] Among them, the first threshold is about 24 hours.

[0155] Step S07, in response to the remaining message processing subunits not preferentially calculating the message dequeue priority of the to-be-processed message, the to-be-processed messages in the remaining message processing subunits and the calculated priority scores are deleted.

[0156] Specifically, the present application changes from the original single module message processing mode to a mode where each module can process messages. Each business module has a message processing subunit, which can establish a websocket message with the virtualization layer, receive a websocket message, and process a websocket message. The main steps include:

[0157] Step 1: Abstract the message processing unit into an independent functional module;

[0158] Step 2: The business module relies on this message processing subunit and has the function of message processing;

[0159] Step 3: After each module receives the message, it first deduplicates the message, then calculates the priority of the message according to the priority formula, and finally puts it into the cache queue.

[0160] Among them, the websocket message is a way of transmitting messages. Generally, the websocket message is a long connection mode. Multiple clients can connect to one server, and the server can push messages to multiple clients.

[0161] like Figure 6 As shown, the present application changes from the original single-module message processing mode to a multi-module message processing mode; the message processing adopts the method of pulling multiple modules from the queue, and determines whether it is the message type processed by this module. If so, this module processes the message; if not, the message is put back into the queue; the messages are prioritized, and a custom priority calculation formula is used to derive the priority calculation formula based on the message retention time, message type weight coefficient, and message processing coefficient, so as to sort and process the messages more flexibly; a scheme in which the message priority has a nonlinear relationship with the retention time is adopted, and the message priority first increases and then decreases with the retention time, thereby avoiding the problem of problematic messages occupying queue space all the time.

[0162] In addition, the message processing method further includes:

[0163] In response to a failure in simultaneous message processing by multiple business units, the message is processed by a single message processing module;

[0164] The message is processed by a single message processing module, including:

[0165] The message processing module receives the message to be processed from the client;

[0166] Storing the pending messages in a cache queue in chronological order;

[0167] Classifying the messages to be processed according to message types;

[0168] Sending the to-be-processed message from the cache queue to each service unit according to the message type of the to-be-processed message;

[0169] Processing the to-be-processed messages through the various business units;

[0170] The processing of the to-be-processed message by each of the business units includes:

[0171] Create an independent message queue for each business unit;

[0172] Set a priority queue for each business unit based on the importance and urgency of the messages to be processed;

[0173] Set up an error handling mechanism for each business unit and record the exceptions that occur during the message processing process;

[0174] Each business unit records detailed logs of message processing, including message content, processing time, and results;

[0175] Automatically adjust the processing capacity of each business unit based on the load of each message queue.

[0176] In this way, each business unit can focus on its core functions while communicating and collaborating efficiently and reliably through message queues.

[0177] The message processing method provided in the embodiment of the present application may be further improved and optimized without departing from the technical solution of the present application, and these improvements and optimizations should also be regarded as within the scope of protection of the present application.

[0178] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0179] The technical solution provided in the embodiment of the present application is based on automatic claiming by each module and message processing based on a priority algorithm. It is more compatible with multi-module scenarios, enables each module to process messages more evenly, enables the cloud platform to respond more quickly to changes in virtualization layer resources, and improves the usability of the product.

[0180] The technical solution provided in the embodiment of the present application solves the problem of processing resource synchronization through websocket messages in complex scenarios with multiple modules and multiple resources. It changes the original single module processing to processing through various business modules, thereby enhancing the robustness of the system and preventing the entire business from being affected by the damage of one module.

[0181] The technical solution provided in the embodiment of the present application combines the message retention time, message type weight coefficient, and message processing coefficient to derive a priority calculation formula, which allows for more flexible message sorting and processing, solves the original problems of message retention and untimely message processing, improves the usability of the system, and provides users with a better user experience.

[0182] Embodiment 3

[0183] The present application provides a message processing device, such as Figure 7 As shown, the device is applied to a message processing system, the system includes several business units, a cache queue and a client, each business unit includes a message processing subunit, and the device includes: a receiving module, a sending module, a determining module, a processing module, and a post-processing module.

[0184] In this embodiment, the receiving module is used to receive the to-be-processed messages of the client through the plurality of business units;

[0185] A sending module, used to send the pending messages to the message processing subunits of several business units respectively; send the pending messages to the cache queue according to the dequeuing priority of the target message; send the pending messages to several business units according to the dequeuing priority of the target message;

[0186] A determination module, used to determine the message dequeue priority of the to-be-processed message through each message processing subunit;

[0187] In response to one of the message processing subunits preferentially calculating the message dequeue priority of the message to be processed, the message dequeue priority calculated by the message processing subunit is used as the target message dequeue priority of the message to be processed;

[0188] The processing module is used to process the message to be processed according to the message type.

[0189] In one of the embodiments, the processing module is used to perform deduplication processing on the message to be processed;

[0190] The deduplication processing of the to-be-processed message includes:

[0191] Create a message data collection;

[0192] Calculating a hash value of the message to be processed;

[0193] Determine whether the hash value of the message to be processed exists in the message data set;

[0194] If yes, delete the pending message; if no, add the hash value of the pending message to the message data set;

[0195] The deduplication processing of the to-be-processed message further includes:

[0196] Create an identifier for each message to be processed;

[0197] Determining a distributed lock according to the identifier of the message to be processed, wherein the distributed lock includes a database-based lock, a distributed application-based lock, and a distributed key-value-based lock;

[0198] When a distributed lock can be determined according to the identifier of the message to be processed, releasing the distributed lock of the message to be processed;

[0199] When the distributed lock cannot be determined according to the identifier of the message to be processed, the message corresponding to the identifier is deleted.

[0200] In one embodiment, the determination module is used to obtain the message weight coefficient n, the number of times the message is processed m, and the current time t now , the time t when the message is initially placed into the cache queue start ;

[0201] Get the retention time f corresponding to the maximum priority score and the maximum priority score g;

[0202] By formula: A message dequeue priority score of the to-be-processed message is calculated.

[0203] In one of the embodiments, the processing module is used to classify the messages to be processed;

[0204] Get the correspondence table between message types and business units;

[0205] Determine the target processing service unit corresponding to the message to be processed according to the message type of the message to be processed and the correspondence table between the message type and the service unit;

[0206] Determining whether the business unit is a target processing business unit;

[0207] If so, the message is processed by the message processing subunit of the business unit; if not, the message dequeue priority of the pending message is recalculated by each message processing subunit, and the pending message is stored in the cache queue according to the corresponding message dequeue priority calculated in priority.

[0208] In one of the embodiments, the post-processing module is used to set the message dequeue priority of the to-be-processed message to first increase and then decrease as the retention time in the cache queue;

[0209] When the retention time of any to-be-processed message in the cache queue is greater than a first threshold, the message is deleted from the cache queue;

[0210] In response to the remaining message processing subunits not preferentially calculating the message dequeue priorities of the to-be-processed messages, the to-be-processed messages in the remaining message processing subunits and the calculated priority scores are deleted.

[0211] In one embodiment, the determination module is used to classify the to-be-processed messages according to the four-quadrant rule, wherein the classification types include urgent and important messages, important but not urgent messages, urgent but not important messages, and neither urgent nor important messages;

[0212] Determine the message dequeue priority according to the classification type of the pending message, and process the pending message;

[0213] The step of determining the message dequeue priority of the to-be-processed message by each message processing subunit further includes:

[0214] Score and sort the messages to be processed according to their importance;

[0215] The message dequeue priority is determined according to the score of the message to be processed, and the message to be processed is processed.

[0216] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0217] The technical solution provided in the embodiment of the present application is based on automatic claiming by each module and message processing based on a priority algorithm. It is more compatible with multi-module scenarios, enables each module to process messages more evenly, enables the cloud platform to respond more quickly to changes in virtualization layer resources, and improves the usability of the product.

[0218] The technical solution provided in the embodiment of the present application solves the problem of processing resource synchronization through websocket messages in complex scenarios with multiple modules and multiple resources. It changes the original single module processing to processing through various business modules, thereby enhancing the robustness of the system and preventing the entire business from being affected by the damage of one module.

[0219] The technical solution provided in the embodiment of the present application combines the message retention time, message type weight coefficient, and message processing coefficient to derive a priority calculation formula, which allows for more flexible message sorting and processing, solves the original problems of message retention and untimely message processing, improves the usability of the system, and provides users with a better user experience.

[0220] Embodiment 4

[0221] The present application also provides a message processing device, including:

[0222] one or more processors;

[0223] A storage device for storing one or more programs;

[0224] When the one or more programs are executed by the one or more processors, the one or more processors execute the message processing method described below:

[0225] Receiving the to-be-processed messages from the client through the plurality of business units;

[0226] Sending the to-be-processed messages to the message processing subunits of several business units respectively;

[0227] Determining the message dequeue priority of the to-be-processed message by each message processing subunit;

[0228] In response to one of the message processing subunits preferentially calculating the message dequeue priority of the message to be processed, the message dequeue priority calculated by the message processing subunit is used as the target message dequeue priority of the message to be processed;

[0229] Sending the to-be-processed message to the cache queue according to the dequeuing priority of the target message;

[0230] The cache queue sends the to-be-processed message to a plurality of business units according to the dequeuing priority of the target message;

[0231] The plurality of service units process the messages to be processed according to the message types.

[0232] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0233] The present application solves the problems of message retention and untimely message processing in the prior art, improves the usability of the system, and brings a better user experience to users.

[0234] The technical solution provided in the embodiment of the present application is based on automatic claiming by each module and message processing based on a priority algorithm. It is more compatible with multi-module scenarios, enables each module to process messages more evenly, enables the cloud platform to respond more quickly to changes in virtualization layer resources, and improves the usability of the product.

[0235] The technical solution provided in the embodiment of the present application solves the problem of processing resource synchronization through websocket messages in complex scenarios with multiple modules and multiple resources. It changes the original single module processing to processing through various business modules, thereby enhancing the robustness of the system and preventing the entire business from being affected by the damage of one module.

[0236] Embodiment 5

[0237] The present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following message processing method can be executed:

[0238] Receiving the to-be-processed messages from the client through the plurality of business units;

[0239] Sending the to-be-processed messages to the message processing subunits of several business units respectively;

[0240] Determining the message dequeue priority of the to-be-processed message by each message processing subunit;

[0241] In response to one of the message processing subunits preferentially calculating the message dequeue priority of the message to be processed, the message dequeue priority calculated by the message processing subunit is used as the target message dequeue priority of the message to be processed;

[0242] Sending the to-be-processed message to the cache queue according to the dequeuing priority of the target message;

[0243] The cache queue sends the to-be-processed message to a plurality of business units according to the dequeuing priority of the target message;

[0244] The plurality of service units process the messages to be processed according to the message types.

[0245] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0246] The present application solves the problems of message retention and untimely message processing in the prior art, improves the usability of the system, and brings a better user experience to users.

[0247] The technical solution provided in the embodiment of the present application is based on automatic claiming by each module and message processing based on a priority algorithm. It is more compatible with multi-module scenarios, enables each module to process messages more evenly, enables the cloud platform to respond more quickly to changes in virtualization layer resources, and improves the usability of the product.

[0248] The technical solution provided in the embodiment of the present application solves the problem of processing resource synchronization through websocket messages in complex scenarios with multiple modules and multiple resources. It changes the original single module processing to processing through various business modules, thereby enhancing the robustness of the system and preventing the entire business from being affected by the damage of one module.

[0249] Embodiment 6

[0250] The present application also provides a computer program product, including a computer program, which can implement the following message processing method when executed by a processor:

[0251] Receiving the to-be-processed messages from the client through the plurality of business units;

[0252] Sending the to-be-processed messages to the message processing subunits of several business units respectively;

[0253] Determining the message dequeue priority of the to-be-processed message by each message processing subunit;

[0254] In response to one of the message processing subunits preferentially calculating the message dequeue priority of the message to be processed, the message dequeue priority calculated by the message processing subunit is used as the target message dequeue priority of the message to be processed;

[0255] Sending the to-be-processed message to the cache queue according to the dequeuing priority of the target message;

[0256] The cache queue sends the to-be-processed message to a plurality of business units according to the dequeuing priority of the target message;

[0257] The plurality of service units process the messages to be processed according to the message types.

[0258] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0259] The technical solution provided in the embodiment of the present application solves the problem of processing resource synchronization through websocket messages in complex scenarios with multiple modules and multiple resources. It changes the original single module processing to processing through various business modules, thereby enhancing the robustness of the system and preventing the entire business from being affected by the damage of one module.

[0260] The technical solution provided in the embodiment of the present application combines the message retention time, message type weight coefficient, and message processing coefficient to derive a priority calculation formula, which allows for more flexible message sorting and processing, solves the original problems of message retention and untimely message processing, improves the usability of the system, and provides users with a better user experience.

[0261] Embodiment 7

[0262] The present application provides a computer storage medium, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0263] Receiving the to-be-processed messages from the client through the plurality of business units;

[0264] Sending the to-be-processed messages to the message processing subunits of several business units respectively;

[0265] Determining the message dequeue priority of the to-be-processed message by each message processing subunit;

[0266] In response to one of the message processing subunits preferentially calculating the message dequeue priority of the message to be processed, the message dequeue priority calculated by the message processing subunit is used as the target message dequeue priority of the message to be processed;

[0267] Sending the to-be-processed message to the cache queue according to the dequeuing priority of the target message;

[0268] The cache queue sends the to-be-processed message to a plurality of business units according to the dequeuing priority of the target message;

[0269] The plurality of service units process the messages to be processed according to the message types.

[0270] In one embodiment, before determining the message dequeue priority of the to-be-processed message by each message processing subunit, the process includes:

[0271] Deduplication processing is performed on the messages to be processed;

[0272] The deduplication processing of the to-be-processed message includes:

[0273] Create a message data collection;

[0274] Calculating a hash value of the message to be processed;

[0275] Determine whether the hash value of the message to be processed exists in the message data set;

[0276] If yes, delete the pending message; if no, add the hash value of the pending message to the message data set;

[0277] The deduplication processing of the to-be-processed message further includes:

[0278] Create an identifier for each message to be processed;

[0279] Determining a distributed lock according to the identifier of the message to be processed, wherein the distributed lock includes a database-based lock, a distributed application-based lock, and a distributed key-value-based lock;

[0280] When a distributed lock can be determined according to the identifier of the message to be processed, releasing the distributed lock of the message to be processed;

[0281] When the distributed lock cannot be determined according to the identifier of the message to be processed, the message corresponding to the identifier is deleted.

[0282] In one of the embodiments, determining the message dequeue priority of the to-be-processed message by each message processing subunit includes:

[0283] Get the message weight coefficient n, the number of times the message is processed m, and the current time t now , the time t when the message is initially placed into the cache queue start ;

[0284] Get the retention time f corresponding to the maximum priority score and the maximum priority score g;

[0285] By formula: A message dequeue priority score of the to-be-processed message is calculated.

[0286] In one embodiment, the plurality of business units process the to-be-processed messages according to the message types, including:

[0287] Classifying the messages to be processed;

[0288] Get the correspondence table between message types and business units;

[0289] Determine the target processing service unit corresponding to the message to be processed according to the message type of the message to be processed and the correspondence table between the message type and the service unit;

[0290] Determining whether the business unit is a target processing business unit;

[0291] If so, the message is processed by the message processing subunit of the business unit; if not, the message dequeue priority of the pending message is recalculated by each message processing subunit, and the pending message is stored in the cache queue according to the corresponding message dequeue priority calculated in priority.

[0292] In one embodiment, the method comprises:

[0293] Setting the message dequeue priority of the to-be-processed message to increase first and then decrease as the retention time in the cache queue;

[0294] When the retention time of any to-be-processed message in the cache queue is greater than a first threshold, the message is deleted from the cache queue.

[0295] In one embodiment, the method further comprises:

[0296] In response to the remaining message processing subunits not preferentially calculating the message dequeue priorities of the to-be-processed messages, the to-be-processed messages in the remaining message processing subunits and the calculated priority scores are deleted.

[0297] In one of the embodiments, determining the message dequeue priority of the to-be-processed message by each message processing subunit includes:

[0298] Classifying the to-be-processed messages by the four-quadrant rule, wherein the classification types include urgent and important messages, important but not urgent messages, urgent but not important messages, and neither urgent nor important messages;

[0299] Determine the message dequeue priority according to the classification type of the pending message, and process the pending message;

[0300] The step of determining the message dequeue priority of the to-be-processed message by each message processing subunit further includes:

[0301] Score and sort the messages to be processed according to their importance;

[0302] The message dequeue priority is determined according to the score of the message to be processed, and the message to be processed is processed.

[0303] The present application solves the problems of message retention and untimely message processing in the prior art, improves the usability of the system, and brings a better user experience to users.

[0304] Figure 8 An exemplary system provided for Embodiment 7 of the present application that can be used to implement various embodiments described in the present application;

[0305] like Figure 8 As shown, in some embodiments, the system can be used as any of the above-mentioned devices for message processing in each of the described embodiments. In some embodiments, the system may include one or more computer-readable media (e.g., system memory or NVM / storage device) having results and one or more processors (e.g., (one or more) processors) coupled to the one or more computer-readable media and configured to execute the results to implement the module to perform the actions described in this application.

[0306] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0307] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0308] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A message processing method, characterized in that: The method is applied to a message processing system, the system includes a plurality of business units, a cache queue and a client, each business unit includes a message processing subunit, and the method includes: Receiving the to-be-processed messages from the client through the plurality of business units; Sending the to-be-processed messages to the message processing subunits of several business units respectively; Determining the message dequeue priority of the to-be-processed message by each message processing subunit; In response to one of the message processing subunits preferentially calculating the message dequeue priority of the message to be processed, the message dequeue priority calculated by the message processing subunit is used as the target message dequeue priority of the message to be processed; Sending the to-be-processed message to the cache queue according to the dequeuing priority of the target message; The cache queue sends the to-be-processed message to a plurality of business units according to the dequeuing priority of the target message; The plurality of service units process the messages to be processed according to the message types.

2. The message processing method according to claim 1, characterized in that: Before determining the message dequeue priority of the to-be-processed message by each message processing subunit, the method includes: Deduplication processing is performed on the messages to be processed; The deduplication processing of the to-be-processed message includes: Create a message data collection; Calculating a hash value of the message to be processed; Determine whether the hash value of the message to be processed exists in the message data set; If yes, delete the pending message; if no, add the hash value of the pending message to the message data set; The deduplication processing of the to-be-processed message further includes: Create an identifier for each message to be processed; Determining a distributed lock according to the identifier of the message to be processed, wherein the distributed lock includes a database-based lock, a distributed application-based lock, and a distributed key-value-based lock; When a distributed lock can be determined according to the identifier of the message to be processed, releasing the distributed lock of the message to be processed; When the distributed lock cannot be determined according to the identifier of the message to be processed, the message corresponding to the identifier is deleted.

3. The message processing method according to claim 1, characterized in that: The determining, by each message processing subunit, the message dequeue priority of the to-be-processed message comprises: Get the message weight coefficient n, the number of times the message is processed m, and the current time t now , the time t when the message is initially placed into the cache queue start ; Get the retention time f corresponding to the maximum priority score and the maximum priority score g; By formula: A message dequeue priority score of the to-be-processed message is calculated.

4. The message processing method according to claim 1, characterized in that: The plurality of business units process the to-be-processed messages according to the message types, including: Classifying the messages to be processed; Get the correspondence table between message types and business units; Determine the target processing service unit corresponding to the message to be processed according to the message type of the message to be processed and the correspondence table between the message type and the service unit; Determining whether the business unit is a target processing business unit; If so, the message is processed by the message processing subunit of the business unit; if not, the message dequeue priority of the pending message is recalculated by each message processing subunit, and the pending message is stored in the cache queue according to the corresponding message dequeue priority calculated in priority.

5. The message processing method according to claim 1, characterized in that: The method comprises: Setting the message dequeue priority of the to-be-processed message to increase first and then decrease as the retention time in the cache queue; When the retention time of any to-be-processed message in the cache queue is greater than a first threshold, the message is deleted from the cache queue.

6. The message processing method according to claim 1, characterized in that: The method further comprises: In response to the remaining message processing subunits not preferentially calculating the message dequeue priorities of the to-be-processed messages, the to-be-processed messages in the remaining message processing subunits and the calculated priority scores are deleted.

7. The message processing method according to claim 1, characterized in that: The determining, by each message processing subunit, the message dequeue priority of the to-be-processed message comprises: Classifying the to-be-processed messages by the four-quadrant rule, wherein the classification types include urgent and important messages, important but not urgent messages, urgent but not important messages, and neither urgent nor important messages; Determine the message dequeue priority according to the classification type of the pending message, and process the pending message; The step of determining the message dequeue priority of the to-be-processed message by each message processing subunit further includes: Score and sort the messages to be processed according to their importance; The message dequeue priority is determined according to the score of the message to be processed, and the message to be processed is processed.

8. A message processing device, characterized in that: The device is applied to a message processing system, the system includes a plurality of service units, a cache queue and a client, each service unit includes a message processing subunit, and the device includes: A receiving module, used for receiving the to-be-processed messages of the client through the plurality of business units; A sending module, used to send the pending messages to the message processing subunits of several business units respectively; send the pending messages to the cache queue according to the dequeuing priority of the target message; send the pending messages to several business units according to the dequeuing priority of the target message; A determination module, used to determine the message dequeue priority of the to-be-processed message through each message processing subunit; In response to one of the message processing subunits preferentially calculating the message dequeue priority of the message to be processed, the message dequeue priority calculated by the message processing subunit is used as the target message dequeue priority of the message to be processed; The processing module is used to process the message to be processed according to the message type.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the message processing method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the message processing method according to any one of claims 1 to 7 are implemented.

Images