Multi-service processing and decoupling method and device for message consumption, medium and equipment
By summarizing and storing multiple service processing results to Redis during the message consumption process, determining the service to be retry and re-executed separately, the problem of re-executing the previous service when multiple service processing fails, and improving the efficiency of message consumption.
Patent Information
- Application Number
- CN202510094986.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-21
- Publication Date
- 2025-04-29
AI Technical Summary
During the message consumption process, multiple service processing corresponding to a single message has serial writing logic, which results in the need to re-execute the previously processed services if a single service processing fails, resulting in low efficiency of message consumption.
By obtaining multiple service processing results corresponding to the target message, if there is a service that fails to process, the processing results are summarized into a target result set and stored in Redis. Then, determine the business to be retry from Redis and re-execute it separately to avoid re-execution of the previously processed business.
Retrying the target services that failed to process in a single target message does not need to re-execute its previously processed services, thereby improving the consumption efficiency of the message.
Smart Images
Figure CN119902915A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of message consumption, and in particular to a method, device, medium and equipment for decoupling multi-service processing of message consumption. Background Art
[0002] Message consumption refers to the process of consumers obtaining and processing messages from a message queue. In a distributed system, producers send messages to a message queue, and consumers are responsible for reading these messages from the queue and processing them accordingly. When consumers consume messages, a single message triggers the processing of multiple associated businesses. That is, when a producer publishes a message to a message queue, there may be multiple consumers subscribing to the queue, and each consumer has its own business logic to process the message. When this single message is finally processed, it will be accompanied by the processing of multiple businesses. Only when all the associated multiple businesses are successfully executed can the corresponding message be considered to be successfully processed.
[0003] At present, multiple businesses corresponding to a single message usually adopt serial writing logic in the system, that is, multiple businesses are executed in sequence and completed one by one. The processing of each business depends on the result or status of the previous business processing. Under this business processing method, once a single business processing fails, it is necessary to re-execute the previously processed business before re-executing the failed business, resulting in low message consumption efficiency. Summary of the invention
[0004] In order to improve the efficiency of message consumption, the present application provides a method, device, medium and equipment for decoupling multi-service processing of message consumption.
[0005] In a first aspect of the present application, a method for decoupling multi-service processing of message consumption is provided, which specifically includes: Obtaining processing results of multiple target services corresponding to a target message, wherein the target message is a message consumed in a message queue with a failure retry mechanism, and the target service is a service logic or operation associated with the target message; If there is a target business that fails to be processed among the target businesses, the processing results are aggregated to obtain a corresponding target result set and stored in a preset redis; In the case where the target result set exists in the redis, the target message to which the corresponding target result set belongs is determined as a retry message, and the business to be retried is determined from each target business corresponding to the retry message, and the business to be retried is the target business that failed to be processed; The service to be retried is re-executed to obtain a corresponding retry processing result.
[0006] By adopting the above technical solution, after obtaining multiple processing results corresponding to the target message, if there is a target business that fails to be processed among the multiple target businesses processed, it means that the corresponding target message needs to be consumed again, then the corresponding multiple processing results are summarized to obtain the target result set and stored in redis. Furthermore, if the target result set is found in redis, it means that there is a target message that needs to be consumed again, then the target business that needs to be reprocessed separately is determined from the multiple target businesses corresponding to the retry message, that is, the business to be retried, and finally the business to be retried is executed separately to achieve the retry of the target business that failed to be processed in a single target message, without re-executing the previously processed business, so that the corresponding target message is consumed again, thereby improving the message consumption efficiency.
[0007] Optionally, the obtaining of processing results of multiple target services corresponding to the target message specifically includes: Obtaining initial processing results of multiple target services corresponding to the target message, wherein the initial processing result is service processing failure or service processing success; If the initial processing result is that the business processing fails, the processing result of the corresponding target business is marked with a preset first character; If the initial processing result is that the business processing is successful, the processing result of the corresponding target business is marked with a preset second character.
[0008] By adopting the above technical solution, each target business associated with the target message is processed in the corresponding processing order. If the initial processing result is a business processing failure, the processing result of the corresponding target business is marked with a preset first character, that is, the first character is used as the processing result of the corresponding target business; if the initial processing result is a business processing success, the processing result of the corresponding target business is marked with a preset second character, so as to facilitate the subsequent determination of whether to retry a single target business separately according to the processing results of each target business.
[0009] Optionally, the processing result is a preset first character or a preset second character, the processing result is the first character indicates that the target business processing fails, and the processing result is the second character indicates that the target business processing succeeds, and if there is a target business that fails to be processed among the target businesses, the processing results are aggregated to obtain a corresponding target result set and stored in a preset redis, specifically including: If there is a target business that fails to be processed among the target businesses, then based on the processing order of the target businesses, the processing results are combined to obtain a target string, and the target string is determined as a target result set corresponding to the target message. The earlier the processing order, the smaller the index position of the corresponding processing result in the target string; Determine the location identifier corresponding to the target result set, and based on the location identifier, store the target result set in a preset redis.
[0010] By adopting the above technical solution, if there are target businesses that have failed to be processed among the target businesses, it means that the corresponding target message needs to be retried, then the processing results of each target business are orderly combined, that is, according to the order of processing of each target business, the combination position of the corresponding processing results is determined. The earlier the processing order, the smaller the index position of the corresponding processing result, and finally the corresponding target result set is obtained. Furthermore, based on the positioning identifier, the target result set is stored in redis, which is not only convenient for subsequent search of the target result set from redis, but also can accurately determine the target business that needs to be reprocessed according to the position of the processing result of the first character in the target result set.
[0011] Optionally, determining the service to be retried from each target service corresponding to the retry message specifically includes: Based on the location identifier of the target result set of the retry message, searching the corresponding retry target result set from the redis; Selecting a target character in the retry target result set and determining a target index position of the target character, the target character being the first character; According to the target index position, determining a corresponding target processing sequence number in the overall processing sequence of the target service associated with the retry message, wherein the smaller the target processing sequence number, the higher the priority for processing; According to the target processing sequence number, the service to be retried is determined from the target services corresponding to the retry message.
[0012] By adopting the above technical solution, first, the corresponding retry target result set is found from redis according to the positioning identifier, and then the position of the target character in the entire retry target result set is determined according to the target index position of the target character in the retry target result set, and then its order in the overall processing sequence of all target businesses is determined, that is, the target processing sequence number. Finally, according to the target processing sequence number, the business to be retried that failed to be processed and needs to be reprocessed can be determined from all target businesses, so as to facilitate subsequent separate reprocessing.
[0013] Optionally, after re-executing the service to be retried and obtaining a corresponding retry processing result, the method further includes: The retry message is determined as a target message, and the step of obtaining processing results of multiple target services corresponding to the target message is repeatedly executed until the corresponding target message is processed.
[0014] By adopting the above technical solution, after the retry service is re-executed, the retry message is determined as the target message, and the processing results of each corresponding target service are re-acquired, that is, the processing results of this retry processing result and the processing results of the target service that has not been retried. If the first character does not exist in each processing result, it means that the retry service is successfully retried, the corresponding retry message processing is completed, and the consumption is successful. If the first character exists in each processing result, it means that the processing of the retry service fails again, then continue to process this retry service separately, so as to ensure that the target message can be consumed normally to a certain extent.
[0015] Optionally, after aggregating the processing results to obtain a corresponding target result set and storing it in a preset redis, the method further includes: Determine whether the redis service is down; If the redis service goes down, the target services corresponding to the target message are reprocessed in sequence according to the preset processing order.
[0016] By adopting the above technical solution, if the redis service goes down, it means that the redis server cannot operate normally, and even if the target result set exists in redis, it cannot be searched. When the target result set exists in redis, it is also impossible to determine the target business that failed to be processed in each target business of the corresponding target message, that is, it is impossible to perform a specified retry of the target business that failed to be processed later. In order to ensure that the target message can be successfully consumed smoothly, the corresponding target businesses are reprocessed according to the preset processing order of the currently consumed target message, thereby ensuring that the target message can be consumed normally.
[0017] Optionally, before obtaining the processing results of multiple target services corresponding to the target message, the method further includes: Get the target message and determine whether the target result set exists in the preset redis; If the target result set does not exist in the redis, determine the processing order of each target business corresponding to the target message; According to the processing order, each of the target services is processed in turn, and based on a preset exception capture mechanism, a corresponding initial processing result is obtained.
[0018] By adopting the above technical solution, after obtaining the target message to be consumed, if redis does not have the target result set, it means that multiple target businesses associated with the previously processed target message have been successfully processed, and the processed target message has been successfully consumed without retrying, which further indicates that the target message currently to be consumed (currently preparing to process the message) is being consumed for the first time. Then, each target business corresponding to the target message currently to be consumed is processed in sequence according to the preset processing order to obtain the corresponding initial processing result, thereby realizing reasonable and accurate consumption of the target message.
[0019] In a second aspect of the present application, a multi-service processing decoupling device for message consumption is provided, specifically comprising: A result acquisition module, used to obtain the processing results of multiple target services corresponding to the target message, the target message is a message consumed in a message queue with a failure retry mechanism, and the target service is the service logic or operation associated with the target message; A result storage module is used to aggregate the processing results to obtain a corresponding target result set and store it in a preset redis if there is a target business that fails to be processed among the target businesses; A business determination module, used for determining the target message to which the corresponding target result set belongs as a retry message when there is a target result set in the redis, and determining the business to be retried from each target business corresponding to the retry message, wherein the business to be retried is the target business that failed to be processed; A separate retry module is used to re-execute the business to be retried to obtain a corresponding retry processing result.
[0020] By adopting the above technical solution, the result acquisition module obtains the processing results of multiple target businesses corresponding to the target message. When there are target businesses that fail to be processed among the target businesses, the result storage module summarizes the various processing results to obtain the corresponding target result set, and stores it in the preset redis. Then, when the business determination module has the target result set in redis, it determines the retry message and determines the business to be retried from the target businesses corresponding to the retry message. Finally, the separate retry module re-executes the business to be retried to obtain the corresponding retry processing result.
[0021] In a third aspect of the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is loaded and executed by a processor, the method steps described in any one of the first aspects are performed.
[0022] In a fourth aspect of the present application, an electronic device is provided, specifically comprising: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the processor is used to load and execute the computer program stored in the memory so that the electronic device performs the method as described in any one of the first aspects.
[0023] In summary, the present application includes at least one of the following beneficial technical effects: after obtaining multiple processing results corresponding to the target message, if there is a target business that fails to be processed among the multiple target businesses being processed, it means that the corresponding target message needs to be consumed again, then the multiple corresponding processing results are summarized to obtain the target result set and store it in redis. Furthermore, if the target result set is found in redis, it means that there is a target message that needs to be consumed again, then the target business that needs to be reprocessed separately is determined from the multiple target businesses corresponding to the retry message, that is, the business to be retried, and finally the business to be retried is executed separately, so as to achieve the retry of the target business that failed to be processed in a single target message, without re-executing the previously processed business, so that the corresponding target message is re-consumed, thereby improving the message consumption efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 It is a flowchart of a multi-service processing decoupling method for message consumption provided by an embodiment of the present application; Figure 2 It is a flowchart of another method for decoupling multi-service processing of message consumption provided by an embodiment of the present application; Figure 3 It is a structural diagram of a multi-service processing decoupling device for message consumption provided by an embodiment of the present application; Figure 4 It is a structural diagram of another multi-service processing decoupling device for message consumption provided in an embodiment of the present application.
[0025] Explanation of the accompanying drawings: 11. Result acquisition module; 12. Result storage module; 13. Business determination module; 14. Individual retry module; 15. Message retry module; 16. Downtime processing module; 17. Message consumption module. DETAILED DESCRIPTION
[0026] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.
[0027] In the description of the embodiments of the present application, words such as "illustrative", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "illustrative", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "illustrative", "for example" or "for example" is intended to present related concepts in a concrete way.
[0028] In the description of the embodiments of the present application, the term "and / or" is only a kind of association relationship describing the associated objects, indicating that there may be three kinds of relationships, for example, A and / or B, which can represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" can explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.
[0029] See also Figure 1 The embodiment of the present application discloses a flowchart of a method for decoupling multi-service processing of message consumption, which can be implemented by a computer program or run on a multi-service processing decoupling device for message consumption based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application, specifically including: S101: Obtain processing results of multiple target services corresponding to a target message.
[0030] Specifically, the target message is a message consumed in a message queue with a failure retry mechanism, and the target business is the business logic or operation associated with the target message. Message Queue (MQ) refers to a communication mechanism between applications, which allows applications to communicate asynchronously by sending and receiving messages. Message queues provide a decoupled way so that different applications can handle their respective tasks independently without directly relying on each other. In an embodiment of the present application, a message queue with a failure retry mechanism is a queue mechanism that can automatically retry when message processing fails, and it is intended to ensure that the message can be successfully processed. Further, consuming messages can be understood as the process of consumers obtaining and processing messages from a message queue. Consumers refer to applications or services that receive and process messages from a message queue. Consumers usually receive messages by subscribing to specific topics or queues. Once new messages arrive in the queue, the consumer automatically receives and processes these messages.
[0031] Furthermore, since different consumers have their own corresponding business logic to handle the same target message, each target message corresponds to multiple target businesses, that is, processing a single target message may trigger the processing of multiple target businesses. The processing result is used to characterize the processing of a single target business. Exemplarily, the target message currently consumed in the message queue is order information, and the message queue has multiple consumers subscribed, and the consumers are inventory-related services, order processing services, and customer notification services, etc. Therefore, this target message triggers the processing of multiple businesses such as inventory management, order processing, and customer notification.
[0032] One achievable implementation method of obtaining the processing result is: obtaining initial processing results of multiple target services corresponding to the target message, where the initial processing result is service processing failure or service processing success; If the initial processing result is that the business processing fails, the processing result of the corresponding target business is marked with a preset first character; If the initial processing result is that the business processing is successful, the processing result of the corresponding target business is marked with a preset second character.
[0033] Before step S101, the process also includes: obtaining a target message, and determining whether a target result set exists in a preset redis; If the target result set does not exist in redis, determine the processing order of each target business corresponding to the target message; According to the processing order, each target business is processed in turn, and based on the preset exception capture mechanism, the corresponding initial processing result is obtained.
[0034] Specifically, in an embodiment of the present application, a target message sent by a message queue is received to obtain the target message. Next, it is determined whether there is a target result set in the preset redis, that is, a summary of the processing results of each target business of the processed target message. If it does not exist, it means that the multiple target businesses associated with the previously processed target message have been successfully processed, and the processed target message has been successfully consumed without retrying, thereby indicating that the target message currently to be consumed (currently preparing to carry out the message) is consumed for the first time. Further, the processing order of each target business corresponding to the acquired target message is determined, and a feasible determination method is: personnel perform external input through the terminal, and the processing order is manually set according to specific business requirements. Exemplarily, the target message is order information, and its corresponding target business includes inventory management, order processing and customer notification. If the specific order requirement is that order processing is prioritized and the confirmation and processing of the order are completed as soon as possible, then the processing order of each target business can be: order processing, inventory management and customer notification. It should be noted that the target result set includes multiple processing results of the target message of the target business that failed to be processed.
[0035] Further, in accordance with this processing order, each target business is processed in turn, and the specific processing method is: encapsulate the business logic of each target business into an independent class or module, and provide a corresponding interface for other parts to call. When processing the target business, the corresponding class or module is called through the provided interface for processing. Further, when processing a single target business, the exceptions in the target business processing process are monitored and captured based on the exception capture mechanism, wherein the exception capture (Exception Handling) mechanism is a mechanism for handling errors or abnormal situations that may occur when the program is running. In an embodiment of the present application, when the target business processing fails, the exception can be captured based on the exception capture mechanism, and then the corresponding initial processing result is determined. Exemplarily, during the processing of target business A, if an exception is captured by the exception capture mechanism, indicating that the processing of this target business A fails, then the corresponding initial processing result is a business processing failure, otherwise, if the exception is not captured, the corresponding initial processing result is a business processing success. The initial processing result is a business processing failure or a business processing success.
[0036] Furthermore, if the initial processing result is that the business processing fails, the processing result of the corresponding target business is identified by a preset first character, that is, the first character is used as the processing result of the corresponding target business; if the initial processing result is that the business processing is successful, the processing result of the corresponding target business is identified by a preset second character. In the embodiment of the present application, the first character is 0 and the second character is 1. In other embodiments, the first character may also be set to F and the second character may also be set to T.
[0037] S102: If there is a target business that fails to be processed among the target businesses, the processing results are aggregated to obtain a corresponding target result set and stored in a preset redis.
[0038] Specifically, after obtaining each processing result corresponding to the target message, determine whether there is a processing result for the first character in all processing results. If there is, it means that there is a target business that fails to be processed during the sequential processing of each target business. Then all processing results corresponding to this target message are summarized to determine the corresponding target result set. A feasible determination method is: based on the processing order of each target business, all processing results are combined to obtain a target string, and this target string is determined as the target result set corresponding to the target message, wherein the earlier the processing order of the target business, the smaller the index position of the corresponding processing result in the target string, wherein the index position refers to the position of the character in the string, usually counting from 0, and the index position of 0 represents the first character in the string. In an embodiment of the present application, the index position refers to the position of the processing result in the target string. Exemplarily, the target message corresponds to three target services, namely target service A, target service B and target service C. The processing result of target service A is 1, the processing result of target service B is 0, and the processing result of target service C is 1. The processing order of the three target services is target service B, target service A and target service C. Target service B is processed first, so the index position of its corresponding processing result is 0, which is the first character of the string. Target service B is processed second, and the index position of its corresponding processing result is 1, which is the second character of the string, and so on. The processing results of the three target services are summarized to obtain a target result set of 011.
[0039] After the target result set is determined, it is necessary to determine the location identifier corresponding to the target result set. In the embodiment of the present application, a feasible determination method is: the message ID (msgid) of the target message is used to characterize the location identifier of the corresponding target result set, wherein the message ID of the target message can be automatically generated by the message queue, which is a unique identifier of a single target message. Finally, the determined target result set is stored in the preset redis, and the location identifier corresponding to the target result set is also stored in this redis, so that the corresponding target result set can be quickly found in redis through the location identifier. Among them, redis (Remote Dictionary Server) is an open source, in-memory data structure storage system that can be used as a database, cache and message middleware. Redis supports multiple types of data structures, such as strings, hashes, lists, sets, etc. In other embodiments, if there is no target business that fails to be processed in each target business, it means that the corresponding target message is successfully processed, then the result of successful processing of the target message is returned through the message list, and the processing results of each target business are not summarized into a target result set and stored in redis.
[0040] S103: When the target result set exists in redis, the target message to which the corresponding target result set belongs is determined as a retry message, and the business to be retried is determined from each target business corresponding to the retry message.
[0041] Specifically, when each target business corresponding to a single target message is processed in the processing order, it is determined whether there is a target result set in redis. If the target result set exists in redis, it means that there are businesses that have failed to be processed in each target business of the target message corresponding to this target result set, and the corresponding target message needs to be retried. Then the target message to which the corresponding target result set belongs is determined as a retry message, and the business to be retried is determined from each target business corresponding to this retry message, that is, the target business that failed to be processed in each target business corresponding to the retry message. In an embodiment of the present application, a feasible way to determine the business to be retried is: Based on the location identifier of the target result set of the retry message, search for the corresponding retry target result set from redis; Select a target character in the retry target result set, and determine a target index position of the target character, where the target character is the first character; According to the target index position, the corresponding target processing sequence number is determined in the overall processing order of the target business associated with the retry message. The smaller the target processing sequence number, the higher the priority for processing; According to the target processing sequence number, the service to be retried is determined from the target services corresponding to the retry message.
[0042] Specifically, since each target result set is stored in redis, the location identifier of the target result set is also stored in redis. Therefore, the corresponding target result set, that is, the retry target result set, can be found from redis through the location identifier of the target result set of the retry message. Further, the target character, that is, the first character, is found from each character in the retry target result set, and the target index position of this target character in the retry target result set is determined. A feasible determination method is: determine the target index position corresponding to the target character through the preset str.index function. In other embodiments, the target index position can also be determined through the str.find function. Exemplarily, the retry target result set is 1101, the target character is 0, then the target index position of this target character is 2.
[0043] Furthermore, after the target index position is determined, the corresponding target processing sequence number is determined from the overall processing order of the target business associated with the retry message. The target processing sequence number refers to the processing order of the target business corresponding to the target character. The specific determination method is: add 1 to the target index position to obtain the target processing sequence number. The smaller the processing sequence number, the higher the priority of the corresponding target business. Exemplarily, the retry target result set is 1101, and the target index position is 2, then the target processing sequence number of the target business corresponding to 0 is 3, which is the third target business to be processed among all the target businesses that need to be processed in the retry message. Then, based on the overall processing order of the target business associated with the retry message, the third processed target business is determined as the business to be retried. Finally, according to the target processing sequence number, the business to be retried is determined from each target business corresponding to the retry message.
[0044] S104: Re-execute the service to be retried and obtain a corresponding retry processing result.
[0045] Specifically, after the business to be retried is determined, the class encapsulated by the business logic corresponding to the business to be retried is called again, and the business to be retried of the retry message is re-executed separately to obtain the corresponding retry processing result, thereby realizing the consumption retry of the retry message. There is no need to wait for the target business that has been executed before the business to be retried to be re-executed before re-executing the business to be retried, thereby improving the efficiency of message consumption.
[0046] In other embodiments, the retry message is determined as the target message, and the step of obtaining the processing results of multiple target services corresponding to the target message is repeated until the corresponding target message processing is completed. The specific process is: after the retry service is re-executed, the retry message is determined as the target message, and the processing results of each corresponding target service are re-obtained, that is, the processing results of this retry processing and the processing results of the target service that has not been retried. If the first character does not exist in each processing result, it means that the retry service to be retried is successful, the corresponding retry message processing is completed, and the consumption is successful. If the first character exists in each processing result, it means that the processing of the retry service to be retried fails again, and then the retry service to be retried is processed separately.
[0047] In other embodiments, when a target message contains a newly added target business, after the processing order of the newly added target business is set to the processing order of all target businesses corresponding to the target message, if the number of character string digits in the result set of the processing results corresponding to the target message is less than the total number of businesses of the target message, it is determined that the newly added target business has not yet been executed.
[0048] See also Figure 2 The embodiment of the present application discloses a flowchart of another method for decoupling multi-service processing of message consumption, which can be implemented by a computer program or run on a multi-service processing decoupling device for message consumption based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application, specifically including: S201: Obtain processing results of multiple target services corresponding to a target message.
[0049] S202: If there is a target business that fails to be processed among the target businesses, the processing results are aggregated to obtain a corresponding target result set and stored in a preset redis.
[0050] S203: When the target result set exists in redis, the target message to which the corresponding target result set belongs is determined as a retry message, and the business to be retried is determined from each target business corresponding to the retry message.
[0051] S204: Re-execute the service to be retried and obtain a corresponding retry processing result.
[0052] Specifically, please refer to steps S101-S104, which will not be described in detail here. In other embodiments, after a retry message is generated, the retry message will be returned to the message queue to wait for consumption or retry again, so the first number of retry messages in the message queue is counted. If the first number exceeds the preset number threshold, it means that there are many messages waiting to be consumed again, and the priority of re-consumption needs to be determined, so as to improve the consumption efficiency of each retry message. Furthermore, the historical downtime of the redis service is counted. If the historical downtime exceeds the preset threshold, it means that the redis service is prone to downtime, which makes it impossible to determine the target business that needs to be retried in a single retry message, affecting the successful consumption of the corresponding retry message. All businesses need to be reprocessed according to the business processing order corresponding to the retry message. Then, according to the second number of the target business associated with each retry message, the priority of re-consumption of the corresponding retry message is determined. The smaller the second number, the shorter the overall processing time of the associated target business, the higher the priority of re-consumption, and the higher the priority of retry. When processing the retry message according to the priority of re-consumption, once the redis service is down, the consumption efficiency of the retry message can be improved without affecting the successful consumption of the retry message. If the redis service does not go down, then according to the priority of re-consumption, the target business that failed to be processed in each retry message is retried separately.
[0053] S205: Determine whether the redis service is down; S206: If the redis service is down, the target services corresponding to the target message are reprocessed in sequence according to the preset processing order.
[0054] Specifically, determine whether the redis service is down, that is, the redis server cannot operate normally, and even if the target result set exists in redis, it cannot be searched. When the target result set exists in redis, it is also impossible to determine the target business that failed to be processed in each target business of the corresponding target message, that is, it is impossible to perform a specified retry of the subsequent failed target business. In an embodiment of the present application, a feasible way to determine whether it is down is: try to connect to the redis server through the preset redis-cli tool. If the connection fails, determine whether the redis service is down. Furthermore, at this time, in order to ensure that the target message can be successfully consumed smoothly, then according to the preset processing order of the currently consumed target message, the corresponding target businesses are reprocessed to ensure that the target message can be consumed normally.
[0055] The implementation principle of the decoupling of multi-service processing of message consumption in the embodiment of the present application is as follows: after obtaining multiple processing results corresponding to the target message, if there is a target business that has failed to be processed among the multiple target businesses being processed, it means that the corresponding target message needs to be consumed again, then the corresponding multiple processing results are summarized to obtain the target result set and store it in redis. Furthermore, if the target result set is found in redis, it means that there is a target message that needs to be consumed again, then the target business that needs to be reprocessed separately is determined from the multiple target businesses corresponding to the retry message, that is, the business to be retried, and finally the business to be retried is executed separately to achieve the retry of the target business that failed to be processed in a single target message, without re-executing the previously processed business, so that the corresponding target message is consumed again, thereby improving the message consumption efficiency.
[0056] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.
[0057] See also Figure 3 , which is a schematic diagram of the structure of the multi-service processing decoupling device for message consumption provided in an embodiment of the present application. The multi-service processing decoupling device applied to message consumption can be implemented as all or part of the device through software, hardware or a combination of both. The device includes a result acquisition module 11, a result storage module 12, a service determination module 13 and a separate retry module 14.
[0058] The result acquisition module 11 is used to obtain the processing results of multiple target services corresponding to the target message, the target message is the message consumed in the message queue with a failure retry mechanism, and the target service is the service logic or operation associated with the target message; The result storage module 12 is used to aggregate the processing results to obtain the corresponding target result set and store it in a preset redis if there is a target business that fails to be processed among the target businesses; The business determination module 13 is used to determine the target message to which the corresponding target result set belongs as a retry message when there is a target result set in redis, and determine the business to be retried from each target business corresponding to the retry message, and the business to be retried is the target business that failed to be processed; The separate retry module 14 is used to re-execute the service to be retried and obtain the corresponding retry processing result.
[0059] Optionally, the result acquisition module 11 is specifically used for: Obtaining initial processing results of multiple target services corresponding to the target message, where the initial processing result is service processing failure or service processing success; If the initial processing result is that the business processing fails, the processing result of the corresponding target business is marked with a preset first character; If the initial processing result is that the business processing is successful, the processing result of the corresponding target business is marked with a preset second character.
[0060] Optionally, the result storage module 12 is specifically used for: If there is a target business that fails to be processed among the target businesses, then based on the processing order of the target businesses, the processing results are combined to obtain a target string, and the target string is determined as the target result set corresponding to the target message. The earlier the processing order, the smaller the index position of the corresponding processing result in the target string; Determine the location identifier corresponding to the target result set, and based on the location identifier, store the target result set in the preset redis.
[0061] Optionally, the service determination module 13 is specifically configured to: Based on the location identifier of the target result set of the retry message, search for the corresponding retry target result set from redis; Select a target character in the retry target result set, and determine a target index position of the target character, where the target character is the first character; According to the target index position, the corresponding target processing sequence number is determined in the overall processing order of the target business associated with the retry message. The smaller the target processing sequence number, the higher the priority for processing; According to the target processing sequence number, the service to be retried is determined from the target services corresponding to the retry message.
[0062] Optional, such as Figure 4 As shown, the device also includes a message retry module 15, which is specifically used to: The retry message is determined as the target message, and the step of obtaining the processing results of multiple target services corresponding to the target message is repeatedly executed until the corresponding target message is processed.
[0063] Optionally, the device further includes a downtime processing module 16, which is specifically used to: Determine whether the redis service is down; If the redis service goes down, the target services corresponding to the target message will be reprocessed in sequence according to the preset processing order.
[0064] Optionally, the device further includes a message consumption module 17, which is specifically used to: Get the target message and determine whether the target result set exists in the preset redis; If the target result set does not exist in redis, determine the processing order of each target business corresponding to the target message; According to the processing order, each target business is processed in turn, and based on the preset exception capture mechanism, the corresponding initial processing result is obtained.
[0065] It should be noted that the multi-service processing decoupling device for message consumption provided in the above embodiment only uses the division of the above-mentioned functional modules as an example when executing the multi-service processing decoupling method for message consumption. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the multi-service processing decoupling device for message consumption provided in the above embodiment and the multi-service processing decoupling method for message consumption belong to the same concept. The implementation process is detailed in the method embodiment and will not be repeated here.
[0066] An embodiment of the present application further discloses a computer-readable storage medium, and the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, a multi-service processing decoupling method for message consumption in the above embodiment is adopted.
[0067] Among them, the computer program can be stored in a computer-readable medium, the computer program includes computer program code, the computer program code can be in the form of source code, object code, executable file or certain middleware, etc. The computer-readable medium includes any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer-readable medium includes but is not limited to the above-mentioned components.
[0068] Among them, through this computer-readable storage medium, a multi-business processing decoupling method for message consumption in the above embodiment is stored in a computer-readable storage medium, and is loaded and executed on a processor to facilitate the storage and application of the above method.
[0069] An embodiment of the present application further discloses an electronic device, in which a computer program is stored in a computer-readable storage medium. When the computer program is loaded and executed by a processor, the above-mentioned multi-service processing decoupling method for message consumption is adopted.
[0070] The electronic device may be a desktop computer, a laptop computer, a cloud server or other electronic device, and the electronic device includes but is not limited to a processor and a memory. For example, the electronic device may also include input and output devices, a network access device, and a bus.
[0071] Among them, the processor can adopt a central processing unit (CPU). Of course, according to actual usage, other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. can also be adopted. The general-purpose processor can adopt a microprocessor or any conventional processor, etc., and this application does not impose any restrictions on this.
[0072] Among them, the memory can be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device, or it can be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital card (SD) or a flash memory card (FC) equipped on the electronic device. Moreover, the memory can also be a combination of an internal storage unit and an external storage device of the electronic device. The memory is used to store computer programs and other programs and data required by the electronic device. The memory can also be used to temporarily store data that has been output or is to be output, and this application does not impose any restrictions on this.
[0073] Among them, through this electronic device, a multi-service processing decoupling method for message consumption in the above embodiment is stored in the memory of the electronic device, and is loaded and executed on the processor of the electronic device for easy use.
[0074] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not recorded in the present disclosure. The description and examples are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.
Claims
1. A method for decoupling multi-service processing of message consumption, characterized in that: The method comprises: Obtaining processing results of multiple target services corresponding to a target message, wherein the target message is a message consumed in a message queue with a failure retry mechanism, and the target service is a service logic or operation associated with the target message; If there is a target business that fails to be processed among the target businesses, the processing results are aggregated to obtain a corresponding target result set and stored in a preset redis; In the case where the target result set exists in the redis, the target message to which the corresponding target result set belongs is determined as a retry message, and the business to be retried is determined from each target business corresponding to the retry message, and the business to be retried is the target business that failed to be processed; The service to be retried is re-executed to obtain a corresponding retry processing result.
2. The method for decoupling multi-service processing of message consumption according to claim 1 is characterized in that: The processing results of obtaining the multiple target services corresponding to the target message specifically include: Obtaining initial processing results of multiple target services corresponding to the target message, wherein the initial processing result is service processing failure or service processing success; If the initial processing result is that the business processing fails, the processing result of the corresponding target business is marked with a preset first character; If the initial processing result is that the business processing is successful, the processing result of the corresponding target business is marked with a preset second character.
3. The method for decoupling multi-service processing of message consumption according to claim 1, characterized in that: The processing result is a preset first character or a preset second character. The processing result of the first character indicates that the target business processing fails, and the processing result of the second character indicates that the target business processing succeeds. If there is a target business that fails to be processed among the target businesses, the processing results are aggregated to obtain a corresponding target result set and stored in the preset redis, which specifically includes: If there is a target business that fails to be processed among the target businesses, then based on the processing order of the target businesses, the processing results are combined to obtain a target string, and the target string is determined as a target result set corresponding to the target message. The earlier the processing order, the smaller the index position of the corresponding processing result in the target string; Determine the location identifier corresponding to the target result set, and based on the location identifier, store the target result set in a preset redis.
4. The method for decoupling multi-service processing of message consumption according to claim 3 is characterized in that: The determining the to-be-retried service from each target service corresponding to the retry message specifically includes: Based on the location identifier of the target result set of the retry message, searching the corresponding retry target result set from the redis; Selecting a target character in the retry target result set and determining a target index position of the target character, the target character being the first character; According to the target index position, determining a corresponding target processing sequence number in the overall processing sequence of the target service associated with the retry message, wherein the smaller the target processing sequence number, the higher the priority for processing; According to the target processing sequence number, the service to be retried is determined from the target services corresponding to the retry message.
5. The method for decoupling multi-service processing of message consumption according to claim 1 is characterized in that: After re-executing the service to be retried and obtaining the corresponding retry processing result, the method further includes: The retry message is determined as a target message, and the step of obtaining processing results of multiple target services corresponding to the target message is repeatedly executed until the corresponding target message is processed.
6. The method for decoupling multi-service processing of message consumption according to claim 1, characterized in that: After the processing results are aggregated to obtain the corresponding target result set and stored in the preset redis, the method further includes: Determine whether the redis service is down; If the redis service goes down, the target services corresponding to the target message are reprocessed in sequence according to the preset processing order.
7. The method for decoupling multi-service processing of message consumption according to claim 2, characterized in that: Before obtaining the processing results of the multiple target services corresponding to the target message, the method further includes: Get the target message and determine whether the target result set exists in the preset redis; If the target result set does not exist in the redis, determine the processing order of each target business corresponding to the target message; According to the processing order, each of the target services is processed in turn, and based on a preset exception capture mechanism, a corresponding initial processing result is obtained.
8. A multi-service processing decoupling device for message consumption, characterized in that: include: A result acquisition module (11) is used to obtain processing results of multiple target services corresponding to target messages, wherein the target messages are messages consumed in a message queue with a failure retry mechanism, and the target services are business logic or operations associated with the target messages; A result storage module (12) is used to aggregate the processing results to obtain a corresponding target result set and store it in a preset redis if there is a target business that fails to be processed among the target businesses; A business determination module (13) is used to, when a target result set exists in the redis, determine the target message to which the corresponding target result set belongs as a retry message, and determine the business to be retried from each target business corresponding to the retry message, wherein the business to be retried is the target business that failed to be processed; A separate retry module (14) is used to re-execute the service to be retried to obtain a corresponding retry processing result.
9. A computer-readable storage medium having a computer program stored therein, characterized in that: When the computer program is loaded and executed by a processor, the method according to any one of claims 1 to 7 is adopted.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor loads and executes the computer program, the method according to any one of claims 1 to 7 is adopted.