Micro-service message processing method, device, system and equipment

Abstract

The invention provides a micro-service message processing method, device, system and equipment, and the method comprises the steps: sending a message to a Redis database, enabling the Redis database to store the received message in a task region for storing a to-be-processed message according to a timestamp sequence, and carrying out the backup of the received message in a message backup region; when the time point of regularly capturing the messages is reached, capturing to-be-processed target messages from the task area according to the time stamp sequence of the to-be-processed messages, and distributing the target messages to corresponding user sides for processing; and if the target message is not successfully processed, sending repeated processing information indicating that the target message is not successfully processed to a Redis database, so that the Redis database recovers the target message backed up in the message backup area according to a new timestamp and stores the target message in the task area. Therefore, according to the technical scheme provided by the embodiment of the invention, the message loss can be avoided, and the message delay problem can also be avoided.

Description

Technical Field

[0001] This application relates to the field of information processing technologies, and particularly to a microservice message processing method, apparatus, system, and device. Background Art

[0002] In recent years, with the rapid development of online education systems, the technical support requirements for real-time business scenarios have been increasing day by day. In the online exam scenario, the system needs to precisely control the exam countdown and implement the function of automatic submission at the expiration, which poses strict requirements on the reliability and timeliness of delayed task processing. Systems based on the SpringCloud microservice architecture generally use the delayed queue technology to meet such requirements.

[0003] To meet the above requirements, the prior art adopts a delayed queue solution based on Redis timeout monitoring. Specifically, relying on the Redis keyspace notification mechanism (Keyspace Notifications), timeout events are triggered by setting the expiration time (TTL) of key-value pairs. When the exam starts, a key-value pair with the exam duration is set, and the submission operation is triggered by relying on the expiration event notification of Redis. However, this solution has the following core defects:

[0004] First, the event trigger delay is uncontrollable. Redis adopts a hybrid strategy combining lazy deletion and periodic deletion, and its official documentation clearly states that it does not guarantee the immediate deletion of expired keys. Actual tests show that in the case of high load, the event delay can reach the order of seconds. This poses a serious threat to the exam scenario that requires triggering accurate to the second level, and may cause candidates to obtain extra answering time or be forced to submit prematurely, damaging the fairness of the exam.

[0005] Second, the reliability of event messages is missing. The publish / subscribe (Pub / Sub) mode of Redis lacks a message persistence mechanism and a consumer confirmation (ACK) mechanism. When the subscribed client experiences a network flash break or service restart, the events in transit will be permanently lost. In a distributed exam system, this will cause some candidates' test papers to fail to be submitted on time, and it is difficult to conduct effective traceability and remediation.

[0006] Third, the server-side performance bottleneck is significant. When the system needs to handle tens of thousands of concurrent exams simultaneously, the Redis server needs to maintain a large number of key-value pairs that are about to expire. The concentrated triggering of expiration events will generate pulsed load pressure, which is extremely likely to cause the CPU usage rate to soar and the response delay to increase. Especially in a cluster environment, the cross-node event propagation mechanism further amplifies the performance loss, severely restricting the horizontal scalability of the system. Summary of the Invention

[0007] The present application provides a microservice message processing method, apparatus, system and device, which can maintain the high performance characteristics of Redis and solve the delay queue implementation scheme for event delay, loss and server bottleneck.

[0008] The technical solutions provided by the present application include:

[0009] In a first aspect, an embodiment of the present application provides a microservice message processing method, and the microservice message processing method includes:

[0010] Send a message to a specified Redis database, so that the Redis database stores the received message in a task area for storing messages to be processed in the order of time stamps, and synchronously backs up the message in a message backup area for temporarily storing the backed-up messages to be processed;

[0011] When the time point for regularly fetching messages is reached, fetch the target messages to be processed from the task area in the order of time stamps of the messages to be processed, and distribute the target messages to the corresponding user terminals for processing;

[0012] If the user terminal fails to process the target message successfully, send a repeated processing message indicating that the processing of the target message is unsuccessful to the Redis database, so that the Redis database restores the target message backed up in the message backup area to the task area and stores it in the task area according to a new time stamp.

[0013] In an embodiment of the present application, after distributing the messages to be processed to the corresponding user terminals for processing, it further includes:

[0014] If the user terminal processes the target message successfully, send a completion message indicating that the processing is successful to the Redis database, so that the Redis database deletes the target message from the message backup area and clears the message content related to the target message from the task area.

[0015] In an embodiment of the present application, before sending the repeated processing message indicating unsuccessful processing to the Redis database, it further includes:

[0016] Count the number of failures indicating that the user terminal fails to process the target message successfully;

[0017] If the number of failures corresponding to the target message reaches the failure threshold, send a removal message indicating that the retry processing has reached the failure threshold to the Redis database, so that the Redis database sends the target message in the message backup area to a dead letter area for storing messages with processing exceptions for messages to be processed;

[0018] If the number of failures corresponding to the target message does not reach the failure threshold, execute the step of sending, to the Redis database, duplicate processing information indicating that the processing of the target message is unsuccessful.

[0019] In an embodiment of the present application, the message to be processed at least includes a message body and a message index, and the message body and the message index are in an associated relationship;

[0020] Sending the message to the Redis database, so that the Redis database stores the received message in a task area for storing messages to be processed in chronological order of timestamps, and synchronously backs up the message in a message backup area for temporarily storing the backed-up messages to be processed, includes:

[0021] Sending the message to the Redis database, so that the Redis database stores the received message in a task area for storing the message body and the message index corresponding to the message to be processed in chronological order of timestamps, and synchronously backs up the message index corresponding to the message to be processed in a message backup area for temporarily storing the backed-up message indexes;

[0022] Sending, to the Redis database, duplicate processing information indicating that the processing of the target message is unsuccessful, so that the Redis database restores and stores the target message backed up in the message backup area in the task area according to a new timestamp, includes:

[0023] Sending, to the Redis database, duplicate processing information indicating that the processing of the target message is unsuccessful, so that the Redis database restores and stores the message index of the target message backed up in the message backup area in the task area according to a new timestamp, determines the message body of the target message associated with the message index according to the message index of the target message, and forms a new target message based on the determined message index and the determined target message and stores the new target message in the task area according to a new timestamp.

[0024] In an embodiment of the present application, sending, to the Redis database, removal information indicating that the retry processing has reached the failure threshold, so that the Redis database sends the target message in the message backup area to a dead letter area for storing messages with processing exceptions for messages to be processed, includes:

[0025] Sending, to the Redis database, removal information indicating that the retry processing has reached the failure threshold, so that the Redis database sends the message index corresponding to the target message in the message backup area to a dead letter area for storing message indexes corresponding to messages with processing exceptions for messages to be processed.

[0026] In a second aspect, the present application further provides a microservice message processing device, which includes:

[0027] A message sending unit, configured to send a message to a specified Redis database, so that the Redis database stores the received message in a task area for storing messages to be processed in the order of time stamps, and synchronously backs up the message in a message backup area for temporarily storing the backed-up messages to be processed; when the time point for timing message fetching is reached, trigger the message fetching unit;

[0028] The message fetching unit is configured to fetch target messages to be processed from the task area in the order of time stamps of the messages to be processed, and distribute the target messages to corresponding user terminals for processing; if the user terminal fails to process the target message successfully, trigger the message reprocessing unit;

[0029] The message reprocessing unit is configured to send reprocessing information indicating that the processing of the target message is unsuccessful to the Redis database, so that the Redis database fetches the target message backed up in the message backup area from the message backup area and stores it in the task area according to a new time stamp.

[0030] In an embodiment of the present application, it further includes:

[0031] A cleaning unit, configured to, if the user terminal successfully processes the target message, send completion information indicating successful processing to the Redis database, so that the Redis database deletes the target message from the message backup area and clears the message content related to the target message from the task area.

[0032] In an embodiment of the present application, it further includes:

[0033] A statistics unit, configured to count the number of failures indicating that the user terminal fails to process the target message successfully; if the number of failures corresponding to the target message reaches a failure threshold, trigger the unit for moving to the dead letter area, and if the number of failures corresponding to the target message does not reach the failure threshold, trigger the message reprocessing unit;

[0034] The unit for moving to the dead letter area is configured to send removal information indicating that the retry processing has reached the failure threshold to the Redis database, so that the Redis database sends the target message in the message backup area to a dead letter area for storing messages with processing exceptions for messages to be processed.

[0035] In a third aspect, an embodiment of the present application further provides a microservice system, which includes a plurality of distributed microservices and a Redis database. Each distributed microservice is provided with a message sending module and a message processor for periodically pulling Redis messages and performing distribution scheduling, and is connected to the Redis database. The Redis database is deployed with a task area, a message backup area, and a dead letter area;

[0036] The message sending module is configured to send messages to the Redis database, so that the Redis database stores the received messages in the task area for storing messages to be processed in the order of timestamps, and synchronously backs them up in the message backup area for temporarily storing the backed-up messages to be processed;

[0037] The message processor is configured to, when reaching the time point for periodically fetching messages, fetch target messages to be processed from the task area in the order of timestamps of the messages to be processed, and distribute the target messages to the corresponding client for processing; if the client fails to process the target message successfully, count the number of failures indicating that the client fails to process the target message successfully; if the number of failures corresponding to the target message reaches the failure threshold, send a removal message indicating that the retry process has reached the failure threshold to the Redis database; if the number of failures corresponding to the target message does not reach the failure threshold, send a repeated processing message indicating that the processing of the target message is unsuccessful to the Redis database;

[0038] The Redis database is configured to, when receiving the removal message, send the target message in the message backup area to the dead letter area for storing messages with processing exceptions for the messages to be processed; when receiving the repeated processing message, restore and store the backed-up target message in the message backup area in the task area according to the new timestamp.

[0039] In a fourth aspect, an electronic device includes a processor and a computer-readable storage medium, and the computer-readable storage medium stores computer-executable instructions that can be executed by the processor; the processor is configured to execute the computer-executable instructions to implement the steps of the microservice message processing method according to any one of the embodiments in the first aspect.

[0040] As can be seen from the above technical solutions, the microservice message processing method provided by this application sends a message to the Redis database, so that the Redis database stores the received message in the task area for storing messages to be processed in the order of timestamps, and synchronously backs up the message in the message backup area for temporarily storing the backed-up messages to be processed; when the time point for regularly fetching messages is reached, the target messages to be processed are fetched from the task area in the order of timestamps of the messages to be processed, and the target messages are distributed to the corresponding user terminals for processing; if the user terminal fails to process the target message successfully, a repeated processing message indicating that the processing of the target message is unsuccessful is sent to the Redis database, so that the Redis database restores the target message backed up in the message backup area according to the new timestamp and stores it in the task area. It can be seen that the technical solution provided by the embodiment of this application uses the Redis database as the storage center of messages to ensure data consistency in a distributed service environment, but no longer uses the timeout listening mechanism of Redis. Instead, the microservice application end specially designed based on the Spring Cloud system actively pulls messages from the Redis database and schedules the messages for corresponding processing. At the same time, Redis also synchronously backs up the messages in the message backup area. Applying the technical solution provided by this embodiment can avoid both message loss and message delay problems; in addition, the message timeout trigger point is also processed by the horizontally scalable microservice application end, reducing the pressure on the Redis server side. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure and used together with the specification to explain the principles of the present disclosure.

[0042] Figure 1 It is a schematic flowchart of the first microservice message processing method provided by this application;

[0043] Figure 2 It is a schematic flowchart of the second microservice message processing method provided by this application;

[0044] Figure 3 It is a flowchart example of a microservice message processing method provided by this application;

[0045] Figure 4 It is a schematic structural diagram of a microservice message processing device provided by this application;

[0046] Figure 5 It is a schematic structural diagram of an electronic device provided by this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. On the contrary, they are merely examples of apparatuses and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0048] The terms used in the present invention are for the purpose of describing particular embodiments only and are not intended to limit the present invention. The singular forms "a", "said", and "the" used in the present invention and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0049] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0050] See Figure 1 , Figure 1 A microservice message processing method provided for this application, the microservice message processing method includes the following steps:

[0051] Step 101, send a message to a specified Redis database, so that the Redis database stores the received message in the task area for storing messages to be processed in the order of timestamps, and synchronously backs up it in the message backup area for temporarily storing the backed-up messages to be processed. When the time point for regularly fetching messages is reached, execute step 102.

[0052] The microservice message processing method of this embodiment is applied to the microservice application side where distributed microservices are deployed, and each microservice application side is electrically connected to the Redis database.

[0053] In this embodiment, the user can customize the time frequency of pulling jobs, and the execution time can be accurately controlled within 1 second to avoid message delay.

[0054] The Redis database has a task area and a message backup area. When the Redis database receives a message, it will synchronously store the message in the task area and the message backup area, so that even if the message fails to be processed successfully, the message can be found in the message backup area.

[0055] Step 102: Grab the target messages to be processed from the task area in the order of the timestamps of the messages to be processed, and distribute the target messages to the corresponding client for processing; if the client fails to process the target message successfully, then execute Step 103.

[0056] This client can be understood as the application side of the registered user.

[0057] In this step, the messages to be processed are actively pulled by the application program, so that even if the network is interrupted or the application program is restarted, the Redis messages will not be lost.

[0058] The target message can be understood as the currently received message to be processed.

[0059] In this case, the message expiration trigger point is transferred to the microservice application side, that is, the client. Compared with the centralized timeout event triggered by the Redis server, it is more convenient to expand the application service level during a sharp increase in business traffic, improving scalability and efficiency.

[0060] Step 103: Send repeated processing information indicating that the processing of the target message is unsuccessful to the Redis database, so that the Redis database restores the target message already backed up in the message backup area and stores it in the task area according to the new timestamp.

[0061] After the message processing fails, there is no message in the task area, but there is a backed-up message in the message backup area. At this time, the message is restored to the task area again. In this way, the task area stores the message again, and the message is stored according to the new timestamp to wait to be pulled again next time. It can be seen that even if the message fails to be processed successfully due to network interruption or restart, the message will not be lost.

[0062] Completed Figure 1 The process shown.

[0063] It can be seen that in the technical solution provided by this application, the microservice message processing method sends a message to the Redis database, so that the Redis database stores the received message in the task area for storing messages to be processed in the order of timestamps, and synchronously backs up the message in the message backup area for temporarily storing the backed-up messages to be processed; when the time point for regularly fetching messages is reached, the target messages to be processed are fetched from the task area in the order of timestamps of the messages to be processed, and the target messages are distributed to the corresponding user terminals for processing; if the user terminal fails to process the target message successfully, a repeated processing message indicating that the processing of the target message is unsuccessful is sent to the Redis database, so that the Redis database restores and stores the target message backed up in the message backup area in the task area according to the new timestamp. It can be seen that the technical solution provided by the embodiment of this application uses the Redis database as the storage center of messages to ensure data consistency in a distributed service environment, but no longer uses the timeout listening mechanism of Redis. Instead, the microservice application end specially designed based on the Spring Cloud system actively pulls messages from the Redis database and schedules the messages for corresponding processing. At the same time, Redis also synchronously backs up the messages in the message backup area. It can be seen that applying the technical solution provided by this embodiment can avoid both message loss and message delay problems; in addition, the message timeout trigger point is also processed by the horizontally scalable microservice application end, reducing the pressure on the Redis server side.

[0064] In some embodiments, after distributing the message to be processed to the corresponding user terminal for processing, it further includes: if the user terminal successfully processes the target message, a completion message indicating successful processing is sent to the Redis database, so that the Redis database deletes the target message from the message backup area and clears the message content related to the target message from the task area.

[0065] In this embodiment, if the user terminal successfully processes the target message, a feedback indicating the target message will be sent to the Redis database, so that the Redis database knows that the target message has been successfully processed. Based on this, the Redis database deletes the target message from the backup area and simultaneously clears the relevant message content in the task area to free up space. As an embodiment, the Redis database sends the completion message to the message backup area to trigger the message backup area to delete the target message. After the deletion is completed, the message backup area sends the completion message to the task area to trigger the task area to clear the relevant message content in the task area to free up space.

[0066] In some embodiments, such as Figure 2As shown, before sending the duplicate processing information indicating unsuccessful processing to the Redis database, the following steps are further included:

[0067] Step 102: Grab the target message to be processed from the task area in the order of the timestamps of the messages to be processed, and distribute the target message to the corresponding client for processing; if the client fails to process the target message successfully, execute Step 104.

[0068] Step 104: Count the number of failures indicating that the client fails to process the target message successfully; if the number of failures corresponding to the target message reaches the failure threshold, execute Step 105, and if the number of failures corresponding to the target message does not reach the failure threshold, execute Step 103.

[0069] The number of failures can be understood as retrying to a specified number of times.

[0070] Step 105: Send removal information indicating that the retry processing has reached the failure threshold to the Redis database, so that the Redis database sends the target message in the message backup area to the dead letter area for storing messages with processing exceptions for the messages to be processed.

[0071] In this embodiment, the messages to be processed with processing exceptions in the dead letter area can be processed manually, can be deleted, or can be restored to the message backup area or the task area again. This embodiment does not limit this.

[0072] It can be seen that in the technical solution provided in this application, the dead letter area stores the messages to be processed with processing exceptions, further avoiding the problem of message loss.

[0073] As an embodiment, the message to be processed at least includes a message body and a message index, and the message body and the message index are in an associated relationship;

[0074] The implementation method of Step 101 is: send a message to the Redis database, so that the Redis database stores the received message in the task area for storing the message body and message index corresponding to the message to be processed in the order of the timestamps, and synchronously back up the message index corresponding to the message to be processed in the message backup area for temporarily storing the backed-up message indexes.

[0075] In this embodiment, the attributes of the message at least include the message body and the message index. The message body can be regarded as a carrier with a service type, a service theme, and service data, while the message index can be regarded as a unique ID that can represent the unique identity of the message. Based on this, as an embodiment, the attributes of the message can be set to carry the service module to which the message belongs, topic, id, dealy, and body. Among them, the service module to which the message belongs can be the name of a certain major service or a specific microservice application; the topic is the topic of the reference message queue and is the name of a specific type of service; the id represents the unique index ID of the message; the dealy is the time that needs to be delayed, and the time will be calculated and converted to a future moment during execution; the body is the service data carried by the message itself and can be stored in json format. Among them, the module and the topic together define a type of message, and a message is a task to be executed, which is called a Job, that is, a message.

[0076] In this embodiment, the message index is placed in the message backup area, and both the message body and the message index are placed in the task area. When the message index in the message backup area is restored to the task area, the message body associated with the message index can be determined through the message index. The message backup area only stores the message index alone, which can relatively reduce the occupancy of the Redis database.

[0077] Based on the above embodiment, the implementation method of step 103 may include: sending repeated processing information indicating that the processing of the target message is unsuccessful to the Redis database, so that the message index backed up in the message backup area is restored and stored in the task area according to a new timestamp, determining the message body of the target message associated with the message index according to the message index of the target message, and forming a new target message based on the determined message index and the determined target message and storing it in the task area according to the new timestamp.

[0078] In this embodiment, the message index stored in the message backup area. When the message index is restored to the task area, the message group body can be determined according to the message index.

[0079] In some embodiments, the sending of removal information indicating that the retry processing has reached the failure threshold to the Redis database, so that the Redis database sends the target message in the message backup area to the dead letter area for storing messages with processing exceptions for messages to be processed, includes:

[0080] Sending removal information indicating that the retry processing has reached the failure threshold to the Redis database, so that the Redis database sends the message index corresponding to the target message in the message backup area to the dead letter area for storing the message index corresponding to the processing exception for the message to be processed.

[0081] The message index is stored in the dead letter area, which means that the message index does not exist in either the message backup area or the task area. The message index is only stored in the dead letter area for manual processing, but the message index is not lost, that is, it will not be lost due to network interruption or system restart.

[0082] For easier understanding, as Figure 3 shown, a specific embodiment is given below:

[0083] In Figure 3Among them, the Spring microservices are the microservices under the Spring Cloud architecture, which are deployed in a distributed manner on each application end. These application ends are all electrically connected to the Redis database through their respective Spring microservices. In actual applications, the Producer of the application end corresponding to the Spring microservice, that is, the microservice message producer, will send a microservice message to the Redis database. The queue processor Queen Controller set in the Redis database places the received microservice message as a message to be processed in the task area dedicated to storing messages to be processed. The messages to be processed in this task area are placed in the corresponding area according to the attributes of the messages themselves. For example, the Hash value carried by the message to be processed is placed in the Hash details library, and the message index carried is placed in the Zset message index library. At the same time, a backup of the message to be processed is placed in the message backup area. An information processor is set in the application end under the Spring microservice. This information processor is set with a timing function. When it reaches the time point for timing to pull messages, it pulls the message to be processed as the target message from the area in the Redis database where the Queen Controller specifically places the messages to be processed that have reached the self-processing time point of the Redis database for distribution processing, and sends an invoke to the corresponding user end Consumer of the registered microservice, that is, the application end installed with this microservice, according to the attributes carried by the target message. The Consumer processes the message to be processed. If the processing is successful, it will feedback a return to the information processor, so that the message will be committed, the message index will be deleted from the backup area, and at the same time, the relevant message content in the task area Job Pool will be cleared to release space. All combined operation commands for Redis can be written and executed using Lua scripts to ensure the atomicity of operations. If the processing is not successful, the number of failures is counted. If the number of failures has not reached the failure threshold, the target message can be restored rollback from the message backup area to the Job Pool to wait for the next execution. If the number of failures reaches the failure threshold, that is, after reaching the specified number of errors, it will be placed in the dead letter area in the form of a dead letter from the message backup area for manual processing. In this example, the core of the queue is a processor that periodically pulls Redis messages and performs distribution scheduling, called Timer / Queue Controller. Its essence is an independent thread class designed based on Java Spring. This class of thread starts when the service starts and will obtain all registered Consumers during initialization; then it periodically pulls the jobs that have reached the processing time in the Redis task area according to the configuration and distributes them to the corresponding Consumers for processing.

[0084] In the second aspect, see Figure 4, an embodiment of the present application further provides a microservice message processing device 400, and the microservice message processing device 400 includes:

[0085] A message sending unit 401, configured to send a message to a specified Redis database, so that the Redis database stores the received message in a task area for storing messages to be processed in the order of timestamps, and synchronously backs up the message in a message backup area for temporarily storing the backed-up messages to be processed; when the time point for timing to grab messages is reached, trigger the message grabbing unit 402;

[0086] The message grabbing unit 402 is configured to grab target messages to be processed from the task area in the order of timestamps of the messages to be processed, and distribute the target messages to corresponding user terminals for processing; if the user terminal fails to process the target message successfully, trigger the message reprocessing unit 403;

[0087] The message reprocessing unit 403 is configured to send reprocessing information indicating that the processing of the target message is unsuccessful to the Redis database, so that the Redis database restores and stores the target message backed up in the message backup area in the task area according to a new timestamp.

[0088] In some embodiments, the microservice message processing device 400 further includes:

[0089] A cleaning unit, configured to, if the user terminal successfully processes the target message, send a completion message indicating successful processing to the Redis database, so that the Redis database deletes the target message from the message backup area and clears the message content related to the target message from the task area.

[0090] In some embodiments, the microservice message processing device 400 further includes:

[0091] A statistics unit, configured to count the number of failures indicating that the user terminal fails to process the target message successfully; if the number of failures corresponding to the target message reaches the failure threshold, trigger the unit for moving to the dead letter area, and if the number of failures corresponding to the target message does not reach the failure threshold, trigger the message reprocessing unit 403;

[0092] The unit for moving to the dead letter area is configured to send a removal message indicating that the retry processing has reached the failure threshold to the Redis database, so that the Redis database sends the target message in the message backup area to a dead letter area for storing messages with processing exceptions for messages to be processed.

[0093] In some embodiments, the message to be processed includes at least a message body and a message index, and the message body is associated with the message index;

[0094] The message sending unit 401 is specifically configured to:

[0095] Send a message to the Redis database, so that the Redis database stores the received messages in the task area for storing the message body and message index corresponding to the message to be processed in the order of timestamps, and synchronously backs up the message index corresponding to the message to be processed in the message backup area for temporarily storing the backed-up message indexes;

[0096] The message duplicate processing unit 403 is specifically configured to:

[0097] Send duplicate processing information indicating that the processing of the target message is unsuccessful to the Redis database, so that the Redis database restores the message indexes of the target message backed up in the message backup area to the task area according to the new timestamp, determines the message body of the target message associated with the message index according to the message index of the target message, and forms a new target message based on the determined message index and the determined target message and stores it in the task area according to the new timestamp.

[0098] In some embodiments, the unit for moving into the dead letter area is specifically configured to:

[0099] Send removal information indicating that the retry processing has reached the failure threshold to the Redis database, so that the Redis database sends the message index corresponding to the target message in the message backup area to the dead letter area for storing the message indexes corresponding to the processing exceptions for the message to be processed.

[0100] It can be seen that in the technical solution provided by this application, the microservice message processing device sends a message to the Redis database, so that the Redis database stores the received message in the task area for storing messages to be processed in the order of timestamps, and synchronously backs up the message in the message backup area for temporarily storing the backed-up messages to be processed; when the time point for timing to grab messages is reached, grab the target messages to be processed from the task area in the order of timestamps of the messages to be processed, and distribute the target messages to the corresponding client for processing; if the client fails to process the target message successfully, send a repeated processing message indicating that the processing of the target message is unsuccessful to the Redis database, so that the Redis database restores and stores the backed-up target message in the message backup area in the task area according to the new timestamp. It can be seen that the technical solution provided by the embodiment of this application uses the Redis database as the storage center of messages to ensure data consistency in a distributed service environment, but no longer uses the timeout listening mechanism of Redis. Instead, the microservice application end specifically designed based on the Spring Cloud system actively pulls messages from the Redis database and schedules the messages for corresponding processing. At the same time, Redis also synchronously backs up the messages in the message backup area. It can be seen that applying the technical solution provided by this embodiment can avoid both message loss and message delay problems; in addition, the message timeout trigger point is also processed by the horizontally scalable microservice application end, reducing the pressure on the Redis server side.

[0101] For the specific implementation process of the functions and roles of each unit in the above device, please refer to the implementation process of the corresponding steps in the above method for details, which will not be elaborated here.

[0102] In a third aspect, this application also provides a microservice system. The microservice system includes multiple distributed microservices and a Redis database. Each distributed microservice is provided with a message sending module and a message processor for timing to pull Redis messages and perform distribution scheduling, and is connected to the Redis database. The Redis database is deployed with a task area, a message backup area, and a dead letter area;

[0103] The message sending module is used to send a message to the Redis database, so that the Redis database stores the received message in the task area for storing messages to be processed in the order of timestamps, and synchronously backs up the message in the message backup area for temporarily storing the backed-up messages to be processed;

[0104] The message processor is configured to, when the time point for scheduled message fetching is reached, fetch target messages to be processed from the task area in the order of the timestamps of the messages to be processed, and distribute the target messages to the corresponding client for processing; if the client fails to process the target message successfully, count the number of failures indicating that the client fails to process the target message successfully; if the number of failures corresponding to the target message reaches the failure threshold, send a removal message indicating that the retry processing has reached the failure threshold to the Redis database, and if the number of failures corresponding to the target message does not reach the failure threshold, send a repeated processing message indicating that the processing of the target message is unsuccessful to the Redis database;

[0105] The Redis database is configured to, when receiving the removal message, send the target message in the message backup area to the dead letter area for storing messages with processing exceptions for messages to be processed; when receiving the repeated processing message, restore and store the target message already backed up in the message backup area in the task area according to the new timestamp

[0106] The implementation processes of the functions and roles of each unit in the above device are specifically described in detail in the implementation processes of the corresponding steps in the above method, and will not be elaborated here.

[0107] This application embodiment also provides an electronic device. In terms of hardware, the schematic diagram of the hardware architecture can be seen Figure 5 as shown. It includes: a machine-readable storage medium and a processor, where: the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is configured to execute the machine-executable instructions to implement the microservice message processing operations disclosed in the above examples.

[0108] The machine-readable storage medium provided in this application embodiment stores machine-executable instructions, and when the machine-executable instructions are called and executed by the processor, the machine-executable instructions cause the processor to implement the microservice message processing operations disclosed in the above examples.

[0109] Here, the machine-readable storage medium can be any electronic, magnetic, optical or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or a combination thereof.

[0110] The systems, devices, modules, or units described in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or a combination of any several of these devices.

[0111] For convenience of description, when describing the above devices, they are described as various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in one or more software and / or hardware.

[0112] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, system, or computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0113] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing device generate a device for implementing the functions specified in Figure 1 one or more flows and / or Figure 1 blocks or multiple blocks.

[0114] Moreover, these computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one or more flows or Figure 1 blocks or multiple blocks.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable apparatus to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one process or a plurality of processes and / or blocks Figure 1 one process or a plurality of processes and / or blocks Figure 1 steps for implementing the functions specified in one block or a plurality of blocks.

[0116] For the apparatus embodiments, since they basically correspond to the method embodiments, the relevant parts may be referred to the descriptions of the method embodiments for explanation. The apparatus embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or may be distributed to a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this application. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0117] The above are only the preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included within the scope of protection of this application.

Claims

1. A microservice message processing method, characterized in that, The microservice message processing method includes: Sending a message to a specified Redis database, so that the Redis database stores the received message in a task area for storing messages to be processed in chronological order of timestamps, and synchronously backs it up in a message backup area for temporarily storing the backed-up messages to be processed; When the time point for regularly fetching messages is reached, fetching target messages to be processed from the task area in chronological order of timestamps of the messages to be processed, and distributing the target messages to corresponding client ends for processing; If the client end fails to process the target message successfully, sending repeated processing information indicating that the processing of the target message is unsuccessful to the Redis database, so that the Redis database restores the target message backed up in the message backup area according to a new timestamp and stores it in the task area.

2. The microservice message processing method according to claim 1, wherein After distributing the message to be processed to the corresponding client end for processing, it further includes: If the client end processes the target message successfully, sending completion information indicating successful processing to the Redis database, so that the Redis database deletes the target message from the message backup area and clears the message content related to the target message from the task area.

3. The microservice message processing method according to claim 2, wherein, Before sending the repeated processing information indicating unsuccessful processing to the Redis database, it further includes: Counting the number of failures indicating that the client end fails to process the target message successfully; If the number of failures corresponding to the target message reaches the failure threshold, sending removal information indicating that the retry processing has reached the failure threshold to the Redis database, so that the Redis database sends the target message in the message backup area to a dead letter area for storing messages with processing exceptions for messages to be processed; If the number of failures corresponding to the target message does not reach the failure threshold, execute the step of sending repeated processing information indicating that the processing of the target message is unsuccessful to the Redis database.

4. The microservice message processing method according to claim 3, wherein The message to be processed includes at least a message body and a message index, and the message body and the message index are in an associated relationship; Sending the message to the Redis database, so that the Redis database stores the received message in a task area for storing the message body and message index corresponding to the message to be processed in chronological order of timestamps, and synchronously backing up the message index corresponding to the message to be processed in a message backup area for temporarily storing the backed-up message indexes, includes: Sending a message to the Redis database, so that the Redis database stores the received message in a task area for storing the message body and message index corresponding to the message to be processed in chronological order of timestamps, and synchronously backing up the message index corresponding to the message to be processed in a message backup area for temporarily storing the backed-up message indexes; Sending the repeated processing information indicating that the processing of the target message is unsuccessful to the Redis database, so that the Redis database restores the target message backed up in the message backup area and stores it in the task area according to a new timestamp, includes: Send duplicate processing information indicating that the processing of the target message has failed to the Redis database, so that the Redis database restores the message index of the target message backed up in the message backup area to the task area according to the new timestamp, determines the message body of the target message associated with the message index according to the message index of the target message, and forms a new target message based on the determined message index and the determined target message and stores it in the task area according to the new timestamp.

5. The microservice message processing method according to claim 4, wherein, The sending of removal information indicating that the retry processing has reached the failure threshold to the Redis database, so that the Redis database sends the target message in the message backup area to the dead letter area for storing messages with processing exceptions for the messages to be processed, includes: Send removal information indicating that the retry processing has reached the failure threshold to the Redis database, so that the Redis database sends the message index corresponding to the target message in the message backup area to the dead letter area for storing the message index corresponding to the processing exception for the message to be processed.

6. A microservice message processing device, characterized in that, The microservice message processing device includes: A message sending unit, configured to send a message to a specified Redis database, so that the Redis database stores the received message in the task area for storing messages to be processed in chronological order of timestamps, and synchronously backs it up in the message backup area for temporarily storing the backed-up messages to be processed; when the time point for timing to grab messages is reached, trigger the message grabbing unit; The message grabbing unit is configured to grab the target message to be processed from the task area in chronological order of the timestamps of the messages to be processed, and distribute the target message to the corresponding user side for processing; if the user side fails to process the target message successfully, trigger the message duplicate processing unit; The message duplicate processing unit is configured to send duplicate processing information indicating that the processing of the target message has failed to the Redis database, so that the Redis database restores and stores the backed-up target message in the message backup area in the task area according to the new timestamp.

7. The microservice message processing device according to claim 6, wherein It further includes: A cleaning unit, configured to, if the user side successfully processes the target message, send completion information indicating successful processing to the Redis database, so that the Redis database deletes the target message from the message backup area and clears the message content related to the target message from the task area.

8. The microservice message processing device according to claim 7, characterized in that, It further includes: A statistics unit, configured to count the number of failures indicating that the user side fails to process the target message successfully; If the number of failures corresponding to the target message reaches the failure threshold, trigger the move-to-dead-letter-area unit; if the number of failures corresponding to the target message does not reach the failure threshold, trigger the message duplicate processing unit; The moved-to-dead-letter-zone unit is used to send removal information indicating that the retry process has reached the failure threshold to the Redis database, so that the Redis database sends the target message in the message backup area to the dead-letter zone for storing messages with processing exceptions for the messages to be processed.

9. A microservice system, characterized in that, The microservice system includes multiple distributed microservices and a Redis database. Each distributed microservice is provided with a message sending module and a message processor for periodically pulling Redis messages and performing distribution scheduling, and is connected to the Redis database. The Redis database is deployed with a task area, a message backup area, and a dead-letter zone. The message sending module is used to send a message to the Redis database, so that the Redis database stores the received message in the task area for storing messages to be processed in the order of timestamps, and synchronously backs it up in the message backup area for temporarily storing the backed-up messages to be processed. The message processor is used to, when reaching the time point for periodically grabbing messages, grab the target message to be processed from the task area in the order of timestamps of the messages to be processed, and distribute the target message to the corresponding client for processing. If the client fails to process the target message successfully, it counts the number of failures indicating that the client fails to process the target message successfully. If the number of failures corresponding to the target message reaches the failure threshold, it sends removal information indicating that the retry process has reached the failure threshold to the Redis database. If the number of failures corresponding to the target message does not reach the failure threshold, it sends repeated processing information indicating that the processing of the target message is unsuccessful to the Redis database. The Redis database is used to, when receiving the removal information, send the target message in the message backup area to the dead-letter zone for storing messages with processing exceptions for the messages to be processed; when receiving the repeated processing information, restore and store the backed-up target message in the message backup area in the task area according to the new timestamp.

10. An electronic device, characterized in that, It includes a processor and a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions that can be executed by the processor. The processor is used to execute the computer-executable instructions to implement the steps of the microservice message processing method according to any one of claims 1 to 5.

Images