Message processing method and device, electronic equipment and storage medium

Abstract

The invention provides a message processing method and device, electronic equipment and a storage medium. The method comprises the following steps: in response to a service request, executing a distributed cache operation and generating a processing message containing a service parameter; asynchronously submitting the processing message to a message queue service; if the first submission fails, triggering an instant retry mechanism to resubmit the processing message for at least one time; if the retry mechanism fails, converting the processing message into a failure record and persistently storing the failure record in a database; and scanning the failure record in the database through a timing compensation service, regenerating a processing message and submitting the processing message to a message queue service. According to the method and the device, the requirement of manual intervention is completely eliminated, the high performance of the system is maintained while the reliability of the message is ensured through cooperation of all links, and the problems of response delay, insufficient reliability and the like existing in a traditional manual processing mode are perfectly solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of message processing, and in particular to a message processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the wide application of distributed systems, message queues as key components bear the important role of business decoupling and asynchronous communication. In business scenarios such as order transactions and logistics tracking, ensuring the reliable delivery of messages directly affects the integrity and consistency of business data.

[0003] In the prior art, when message delivery fails, the processing mode of manual intervention is usually adopted: after the operation and maintenance personnel find the failed message delivery through the monitoring system, the message content is manually extracted from the log, the request is reconstructed and submitted again.

[0004] However, the current mode needs manual continuous monitoring of system status and manual intervention and processing after finding the problem, which cannot process the failed message delivery in real time and automatically, increases the operation and maintenance cost, and is difficult to ensure the timeliness and reliability of message processing. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide a message processing method, device, electronic equipment and storage medium to solve the problem that the prior art needs manual continuous monitoring of system status and cannot process the failed message delivery in real time and automatically. The specific technical solutions are as follows:

[0006] In a first aspect, the present application provides a message processing method, comprising:

[0007] In response to a business request, performing a distributed cache operation and generating a processing message containing business parameters;

[0008] Asynchronously submitting the processing message to a message queue service;

[0009] If the first submission fails, triggering an instant retry mechanism to perform at least one re-submission on the processing message;

[0010] If the retry mechanism fails, converting the processing message into a failure record and persistently storing it to a database;

[0011] Scanning the failure record in the database through a timing compensation service, regenerating a processing message and submitting it to the message queue service.

[0012] In one possible implementation, the retry mechanism comprises the following steps:

[0013] Detecting a protocol error code returned by the message queue service, and generating an analysis result containing an error type and a retryable flag based on the protocol error code;

[0014] creating an original copy of the processing message in memory when the retryable flag is true;

[0015] obtaining retry parameters from a predefined policy library according to the error type;

[0016] performing a synchronous retry submission operation using the original copy and the retry parameters.

[0017] In one possible implementation, the retry parameters include an initial retry interval, a backoff coefficient, and a retry number threshold, and the performing a synchronous retry submission operation using the original copy and the retry parameters includes:

[0018] reading the original copy to perform a first retry submission using the initial retry interval as a current interval;

[0019] recomputing the current interval according to the backoff coefficient if the retry submission fails and the retry number threshold is not reached;

[0020] continuing to perform the retry submission using the recomputed current interval until the submission is successful or the retry number threshold is reached.

[0021] In one possible implementation, the method further includes:

[0022] obtaining a statistical result by counting success rates and service response delays of all retry submission operations within a preset time window;

[0023] dynamically adjusting retry parameters in the predefined policy library according to the statistical result.

[0024] In one possible implementation, the persisting the processing message converted into a failure record to a database includes:

[0025] parsing business data and metadata in the processing message;

[0026] constructing a failure record including a business data field, a metadata field, and a status field according to the business data and the metadata, wherein the business data field is used to store core business parameters extracted from the processing message, the metadata field is used to store message creation time, failure cause, and retry number, and the status field is used to mark the failure record as a to-be-processed state;

[0027] atomically writing the failure record to a target data table of the database through a database transaction.

[0028] In one possible implementation, the scanning the failure record in the database through a timing compensation service, regenerating a processing message, and submitting the processing message to a message queue service includes:

[0029] querying, at a preset period, the database for a failure record in a pending state;

[0030] for each failure record queried, parsing a business data field and a metadata field in the failure record;

[0031] reconstructing, according to the business data field and the metadata field, a processing message conforming to a message queue service protocol;

[0032] submitting the reconstructed processing message to the message queue service;

[0033] updating, when the submission is successful, a state field of the failure record to a processed state.

[0034] In one possible implementation, the method further includes:

[0035] monitoring, in real time, a health state indicator of the message queue service;

[0036] when an abnormal health state is monitored, persistently storing a newly received business request to the database;

[0037] when a recovered health state is monitored, processing the business requests in the database through a timing compensation service.

[0038] In a second aspect, the present application provides a message processing apparatus, comprising:

[0039] an execution module configured to execute a distributed cache operation and generate a processing message containing business parameters in response to a business request;

[0040] a submission module configured to asynchronously submit the processing message to a message queue service;

[0041] a retry module configured to trigger an instant retry mechanism to perform at least one re-submission of the processing message if the first submission fails;

[0042] a storage module configured to convert the processing message into a failure record and persistently store the failure record to a database if the retry mechanism fails;

[0043] a re-submission module configured to scan the database for a failure record through a timing compensation service, re-generate a processing message, and submit the processing message to the message queue service.

[0044] In one possible implementation, the retry module is specifically configured to:

[0045] detect a protocol error code returned by the message queue service, and generate an analysis result containing an error type and a retryable flag based on the protocol error code;

[0046] creating an original copy of the processing message in memory when the retryable flag is true;

[0047] obtaining retry parameters from a predefined policy library according to the error type;

[0048] performing a synchronous retry commit operation using the original copy and the retry parameters.

[0049] In one possible implementation, the retry parameters include an initial retry interval, a backoff coefficient, and a retry number threshold, and the retry module is further configured to:

[0050] using the initial retry interval as a current interval to read the original copy to perform a first retry commit;

[0051] if the retry commit fails and the retry number threshold is not reached, recalculating the current interval according to the backoff coefficient;

[0052] continuing to perform the retry commit using the recalculated current interval until the commit is successful or the retry number threshold is reached.

[0053] In one possible implementation, the apparatus further includes an adjustment module configured to:

[0054] statistically obtaining a success rate and a service response delay of all retry commit operations in a preset time window to obtain a statistical result;

[0055] dynamically adjusting retry parameters in the predefined policy library according to the statistical result.

[0056] In one possible implementation, the storage module is specifically configured to:

[0057] parsing service data and metadata in the processing message;

[0058] constructing a failure record including a service data field, a metadata field, and a status field according to the service data and the metadata, wherein the service data field is used to store core service parameters extracted from the processing message, the metadata field is used to store message creation time, failure cause, and retry number, and the status field is used to mark the failure record as a to-be-processed state;

[0059] atomically writing the failure record into a target data table of the database through a database transaction.

[0060] In one possible implementation, the retry commit module is specifically configured to:

[0061] querying, at a preset period, the failure record in the database in the to-be-processed state;

[0062] For each failure record found, parse the business data fields and metadata fields in the failure record;

[0063] According to the business data fields and the metadata fields, reconstruct a processing message conforming to a message queue service protocol;

[0064] Submit the reconstructed processing message to a message queue service;

[0065] When the submission is successful, update a status field of the failure record to a processed state.

[0066] In one possible implementation, the apparatus further comprises a monitoring module for:

[0067] Monitoring health state indicators of the message queue service in real time;

[0068] When a health state exception is monitored, persistently store newly received business requests to the database;

[0069] When the health state is recovered, process the business requests in the database through a timing compensation service.

[0070] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus;

[0071] The memory is configured to store a computer program;

[0072] The processor is configured to execute the program stored in the memory, and implement the method steps of any one of the first aspect.

[0073] In a fourth aspect, a computer readable storage medium is provided, characterized in that the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method steps of any one of the first aspect.

[0074] In a fifth aspect, a computer program product containing instructions is provided, which, when executed on a computer, causes the computer to perform the message processing method of any one of the above aspects.

[0075] Embodiments of the present application have the following beneficial effects:

[0076] The embodiment of the application provides a message processing method and device, electronic equipment and storage medium. In the application, first, distributed cache processing is performed on a service request, and a standardized message is generated, so that temporary storage and rapid access of data are ensured; then, the message is submitted to a message queue service in an asynchronous mode, so that high throughput of the system is maintained. When initial submission fails, an instant retry mechanism is automatically triggered to resend the message, so that rapid recovery from failure is realized. If the retry mechanism still fails, the message is converted into a structured record and stored in a database, so that data loss is avoided. Finally, a timing compensation service is used to automatically reprocess the failed record, so that complete closed-loop management is formed. Through the application, the need for manual intervention is completely eliminated, and through cooperation of various links, high performance of the system is maintained while message reliability is ensured, so that problems such as response delay and insufficient reliability in the traditional manual processing mode are perfectly solved.

[0077] Of course, implementing any product or method of the application does not necessarily need to achieve all the advantages described above at the same time. BRIEF DESCRIPTION OF DRAWINGS

[0078] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and serve to explain the principles of the application, together with the description.

[0079] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the field, other drawings can also be obtained based on these drawings without any creative work.

[0080] One or more embodiments are exemplarily illustrated by pictures in the drawings corresponding to the embodiments, and the exemplarily illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, unless otherwise specified. The drawings in the drawings do not constitute a proportional limitation.

[0081] Figure 1 A flowchart of a message processing method provided by the embodiment of the application;

[0082] Figure 2 A flowchart of another message processing method provided by the embodiment of the application;

[0083] Figure 3 A flowchart of still another message processing method provided by the embodiment of the application;

[0084] Figure 4 A processing flowchart of a message processing method applied to brand marketing activities provided by the embodiment of the application;

[0085] Figure 5 A structural schematic diagram of a message processing device provided by an embodiment of the present application is shown in the figure.

[0086] Figure 6 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0087] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0088] The following disclosure provides many different embodiments, or examples, for implementing different structures of the present application. For the purpose of simplification, the components and arrangements of specific examples are described in the following. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeatedly refer to numbers and / or letters in different examples. Such repetition is for the purpose of simplification and clarity, and does not indicate the relationship between the various embodiments and / or arrangements discussed.

[0089] Figure 1 A flowchart of a message processing method provided by an embodiment of the present application is shown in the figure. The method can be applied to one or more electronic devices such as smart phones, notebook computers, desktop computers, portable computers, servers, etc. In addition, the execution subject of the method can be hardware or software. When the execution subject is hardware, the execution subject can be one or more of the above-mentioned electronic devices. For example, a single electronic device can execute the method, or multiple electronic devices can cooperate with each other to execute the method. When the execution subject is software, the method can be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made herein.

[0090] As shown in the figure, the method specifically includes: Figure 1

[0091] S101, in response to a service request, performing a distributed cache operation and generating a processing message containing service parameters.

[0092] The distributed cache operation refers to an atomic operation of high-speed read and write of service data by using a memory database (such as Redis) under a distributed system architecture, including but not limited to data insertion, update and query.

[0093] ​In the embodiments of the present application, after the system receives a service request, firstly, an atomic operation (such as inventory deduction) is performed in the distributed cache, and then the operation result is combined with the request parameters to encapsulate a structured "processing message" (containing service ID, operation type, timestamp and other key parameters). This design not only ensures the real-time performance of the service operation, but also provides a complete data carrier for subsequent asynchronous processing.

[0094] S102, the processing message is asynchronously submitted to the message queue service.

[0095] Asynchronous submission refers to a non-blocking message transmission mode, which is different from the traditional synchronous response mode.

[0096] In the embodiments of the present application, the system delivers the generated "processing message" to the message queue service (such as Kafka / RabbitMQ) in an asynchronous manner, and the main thread is not blocked during this process, so it can continue to process other requests. This design significantly improves the throughput and response speed of the system, and realizes the decoupling of the business processing flow.

[0097] S103, if the first submission fails, triggering an instant retry mechanism to perform at least one re-submission on the processing message.

[0098] The instant retry mechanism refers to the automatic retry process initiated by the system immediately (usually within a millisecond delay) after detecting the message submission failure, rather than relying on manual intervention.

[0099] In the embodiments of the present application, when the first submission fails, the system will automatically trigger the retry process according to the error type (such as network timeout, service unavailable, etc.), maintain a message copy in memory, and perform at least one re-submission according to a predefined strategy (such as exponential backoff algorithm). This mechanism effectively deals with transient failures such as network jitter, greatly improving the success rate of message delivery.

[0100] S104, if the retry mechanism fails, the processing message is converted into a failure record and stored in the database.

[0101] The failure record refers to the persistent storage object obtained by standardizing the processing message according to the database table structure, which contains business data and metadata.

[0102] In the embodiment of the present application, S104 can specifically include the following steps: parsing the service data and the metadata in the processing message; constructing a failure record containing a service data field, a metadata field and a state field according to the service data and the metadata, wherein the service data field is used to store the core service parameters extracted from the processing message, the metadata field is used to store the message creation time, the failure reason and the retry count, and the state field is used to mark the failure record as a to-be-processed state; and atomically writing the failure record into the target data table of the database through a database transaction.

[0103] The service data refers to the actual service information (such as order ID, transaction amount and core parameters) carried in the processing message; the metadata refers to auxiliary information (such as timestamp and operation type) describing the characteristics of the message; and the database transaction refers to a database operation ensuring atomicity, consistency, isolation and durability.

[0104] In the scheme, the processing message is first parsed to separate the service data and the metadata; then the service data is mapped to the main field of the database table, the metadata is converted into an auxiliary field, and a state marking field is added; and finally the structured record is written into the database through a database transaction. The scheme separates and stores the service data and the metadata in the processing message in a structured manner, not only ensuring the complete preservation of the core service parameters, but also recording the key context information in the message processing process; with the aid of the atomic writing feature of the database transaction, the risk of data loss is fundamentally eliminated; at the same time, through the explicit state marking field, a clear processing identifier is provided for the subsequent time compensation service, realizing the degradation storage. Compared with the manual processing mode, the scheme has achieved significant improvement in data integrity, processing reliability and system automation, and provides effective protection for the reliable delivery of messages in high-concurrency scenarios.

[0105] S105, scanning the failure record in the database through a time compensation service, regenerating a processing message and submitting it to a message queue service.

[0106] The time compensation service refers to an automated processing service periodically executed according to a preset time interval.

[0107] In the embodiment of the present application, S105 can specifically include the following steps: querying the failure record in the database with a state of to-be-processed state according to a preset period; parsing the service data field and the metadata field in the failure record for each failure record; reconstructing a processing message conforming to the message queue service protocol according to the service data field and the metadata field; submitting the reconstructed processing message to the message queue service; and updating the state field of the failure record to a processed state when the submission is successful.

[0108] The preset period refers to a time interval of a timing task execution configured in advance according to a service requirement; and the to-be-processed state is a specific mark value indicating that a failure record is waiting for compensation processing.

[0109] In this scheme, the system first automatically scans the database according to the configured time interval, filters out the failure records in the state of "to-be-processed", then performs deep analysis on each record, recovers the original service parameters from the service data field and acquires the necessary context information from the metadata field, then reassembles the data obtained by analysis into a processing message meeting the format requirements according to the protocol specification of the message queue service, after the reconstruction is completed, the system automatically submits the message to the message queue service for delivery, when the message queue service confirms the successful reception, immediately updates the state of the corresponding failure record to "processed", and completes the closed-loop management. Through the automatic periodic scanning, data reconstruction and state updating mechanism, this scheme realizes the reliable recovery and final delivery of the persistent failure message, and through the accurate management of the state field, avoids the problems of repeated processing or missed processing, and significantly improves the timeliness, accuracy and automation degree of message processing compared with the manual processing mode, and provides reliable message final consistency guarantee for the distributed system.

[0110] In the embodiment of the application, first, the business request is subjected to distributed cache processing and a standardized message is generated, ensuring temporary storage and fast access of data; then the message is submitted to the message queue service in an asynchronous manner, maintaining the high throughput of the system. When the first submission fails, the system automatically triggers the instant retry mechanism to resend the message, realizing the rapid recovery of faults. If the retry mechanism still fails, the message is converted into a structured record and stored persistently in the database, ensuring that the data is not lost. Finally, the failure record is automatically reprocessed by the timing compensation service, forming a complete closed-loop management. Through the application, not only the need for manual intervention is completely eliminated, but also the high performance of the system is maintained while ensuring the reliability of the message through the cooperation of each link, perfectly solving the problems of response delay and insufficient reliability existing in the traditional manual processing mode.

[0111] Referring to Figure 2 Another embodiment flowchart of a message processing method provided by the embodiment of the application is provided. The flowchart Figure 2 The flowchart shown in the above Figure 1 The flowchart shown in the above Figure 2 The flowchart shown in the above

[0112] S201, detecting a protocol error code returned by the message queue service, and generating an analysis result containing an error type and a retryable flag based on the protocol error code.

[0113] Protocol error code, refers to the standard error code returned by the message queue service when the message delivery fails, each code corresponds to a specific error type (such as 404 indicates that the queue does not exist, 503 indicates that the service is unavailable, etc.).

[0114] In the embodiments of the application, by analyzing these protocol error codes, a structured analysis result is generated, which contains two key information: 1) "error type" is used to distinguish whether it is a network failure, service exception or business error, etc.; 2) "retry flag" is a Boolean value, indicating whether the error is suitable for immediate retry (such as network jitter can be retried, while business parameter error should not be retried). This step provides an intelligent decision basis for subsequent processing.

[0115] S202, when the retry flag is true, create an original copy of the processing message in memory.

[0116] The original copy refers to a data copy created in memory that is completely consistent with the initial processing message, including message headers and message bodies and all other components.

[0117] In the embodiments of the application, after determining that the retry flag is true, an independent space is allocated in memory immediately, and the message content is completely copied by deep copy method to ensure that the message data used by the retry remains consistent even if the original message is modified or released during the subsequent retry process. This mechanism effectively solves the data inconsistency problem caused by message tampering in the traditional scheme.

[0118] S203, according to the error type, obtain retry parameters from a pre-defined policy library.

[0119] The pre-defined policy library is a configurable policy management system that stores processing strategies corresponding to various errors, and each strategy contains multiple "retry parameters".

[0120] Retry parameters refer to a set of configuration parameters obtained from the pre-defined policy library for accurately controlling message retry behavior.

[0121] In the embodiments of the application, according to the error type identified in S201, the matching retry strategy is queried from the policy library to obtain specific parameter configurations, such as: 1) "initial retry interval" determines the waiting time for the first retry; 2) "backoff coefficient" controls the time interval growth law of subsequent retries; 3) "maximum retry number" limits the total number of retry attempts. This strategy management makes the system able to take differentiated recovery measures for different types of errors.

[0122] In another embodiment, the method can further include the steps of: statistically obtaining a statistical result of success rates and service response delays of all retry submission operations within a preset time window; and dynamically adjusting retry parameters in the predefined policy library according to the statistical result.

[0123] The preset time window refers to a specific time period set by the system for performance statistics; the success rate refers to the proportion of successful submission in the retry operation; and the service response delay reflects the processing time consumption of the message queue service.

[0124] In this scheme, the system continuously monitors the retry operation result, and statistically obtains the overall success rate and average response delay within a set time range; based on these performance indicators, the retry parameter configuration in the predefined policy library is dynamically optimized, for example, the initial retry interval is appropriately extended when the success rate decreases, or the backoff coefficient is adjusted when the service response delay increases. This adaptive adjustment mechanism can intelligently adapt the retry strategy to the actual operating conditions of the message queue service by monitoring the system state in real time and feeding back the optimization, not only achieving the goal of automation, but also significantly improving the environmental adaptability and overall efficiency of the retry mechanism, and achieving a better balance in system throughput and message reliability compared with the fixed parameter scheme.

[0125] S204, performing a synchronous retry submission operation using the original copy and the retry parameters.

[0126] The synchronous retry submission operation refers to the system blocking and waiting until obtaining a clear retry result (success or failure).

[0127] In the embodiments of the present application, the retry parameters include: initial retry interval, backoff coefficient, and retry number threshold, and correspondingly, S204 can specifically include the following steps: using the initial retry interval as the current interval, reading the original copy to perform the first retry submission; if the retry submission fails and the retry number threshold is not reached, recalculating the current interval according to the backoff coefficient; using the recalculated current interval to continue performing the retry submission until the submission is successful or the retry number threshold is reached.

[0128] The initial retry interval refers to the waiting time (usually in milliseconds) before the first retry operation, which is dynamically set according to the error type, for example, 200 ms for network type errors and 1 s for service unavailable type errors; the backoff coefficient is a multiplier factor (usually a floating point number between 1.5 and 2.0) that determines the growth rule of the retry interval, and when using the exponential backoff algorithm, the Nth retry interval = initial interval x (backoff coefficient)^(N-1); the retry number threshold represents the maximum upper limit of the allowed retry number (typical value is 3-5 times).

[0129] In the scheme, the initial retry interval is first used as the reference waiting time, the original message copy saved in the memory is read to perform the first retry submission; if the first submission fails and the retry number does not reach the upper limit, the current interval is recalculated based on the backoff coefficient; then the updated interval time is used to continue the retry process, and the process is repeated until the message is successfully submitted or the maximum retry number limit is reached. The scheme realizes optimization through three safeguard mechanisms: the initial interval ensures the timeliness of the first retry, the backoff coefficient realizes intelligent adjustment of the interval time, and the number threshold controls the consumption of retry resources. The three work together to meet the functional requirements of "message retransmission" and achieve the best balance between retry efficiency and system stability. Compared with the fixed interval retry scheme, the message delivery success rate in a complex network environment is significantly improved, and the waste of system resources caused by excessive retries is effectively avoided.

[0130] Figure 2 As shown in the flow, first, the fine analysis based on the protocol error code can accurately identify recoverable transient faults and avoid invalid retries; second, the creation of the original copy ensures the integrity and consistency of the business data in the retry process; third, the parameterized configuration provided by the strategy library supports the implementation of differentiated optimal retry strategies for different error types; and finally, the dynamic backoff retry execution process effectively controls the system resource consumption while ensuring reliable message delivery. Thus, the problems of low efficiency and insufficient reliability in manual processing are solved.

[0131] Referring to Figure 3 An embodiment flowchart of another message processing method provided by the embodiments of the present application is shown in FIG. 4. As shown in the figure, the flow can include the following steps: Figure 3

[0132] S301, the health state indicators of the message queue service are monitored in real time.

[0133] The health state indicators are a set of multi-dimensional monitoring data used to comprehensively evaluate the running state of the message queue service, specifically including: 1) basic performance indicators (such as request response time, throughput); 2) error state indicators (such as error code distribution, failed request proportion); 3) resource usage indicators (such as connection pool usage, memory usage).

[0134] In the embodiments of the present application, a lightweight monitoring agent is deployed to collect the above-mentioned indicator data at a fixed sampling frequency (such as every second), and a sliding window algorithm is used for real-time analysis. When any core indicator exceeds the preset threshold for multiple times (such as 3 times) in a row, a state change event is triggered. This fine-grained monitoring mechanism can accurately identify various abnormal situations such as service degradation and network partition, providing reliable decision basis for subsequent processing.

[0135] ​S302. When an abnormal health status is detected, the newly received service requests are persistently stored in the database.

[0136] An abnormal health status refers to a service unavailability status determined by the system based on a comprehensive evaluation algorithm.

[0137] In this embodiment, the specific processing flow includes: 1) immediately activating the degradation mode and stopping the sending of new requests to the abnormal message queue service; 2) performing data verification and format conversion on the received business requests; 3) atomically writing the request data to a dedicated degradation storage table through database transactions; and 4) recording detailed exception context information (such as exception start time, scope of impact, etc.). The data persistence adopts an optimized strategy of "first memory buffering, then batch writing," which ensures data reliability while avoiding the impact of high-frequency I / O operations on system performance. This mechanism ensures that all business requests are properly preserved during service anomalies, completely preventing data loss.

[0138] S303. When the health status is detected to have recovered, the business requests in the database are processed through the timed compensation service.

[0139] A healthy status recovery requires the simultaneous fulfillment of three conditions: 1) core indicators remain stable within the normal threshold range; 2) service handshake verification is successfully completed; and 3) test message delivery verification is passed.

[0140] In this embodiment, the compensation process is as follows: 1) Gradually increase the trigger frequency of the timed compensation service; 2) Use a batch processing strategy to control the compensation traffic; 3) Perform integrity verification on each record and then reconstruct the message; 4) Handle compensation failures through an exponential backoff mechanism. This design ensures timely processing of backlogged requests while avoiding secondary impacts on the newly restored service during the compensation process.

[0141] Figure 3 The process described first reduces anomaly detection time from minutes in traditional solutions to seconds through multi-dimensional health monitoring; second, intelligent degradation processing reduces the risk of data loss during anomalies to near zero; and finally, controllable compensation and recovery ensure a smooth transition of the system to normal operation. This entire solution, through fully automated monitoring, degradation, and recovery mechanisms, achieves the goal of "fault-free" high availability for message processing services, perfectly supporting the stability requirements of high-concurrency scenarios such as e-commerce promotions and financial transactions.

[0142] Optionally, this application embodiment also provides a processing flowchart for a message processing method applied to brand marketing activities. In this scenario, the merchant splits the purchased membership card benefits into multiple additional seating slots. Users participate in the activity by watching advertisements, and the system needs to ensure the reliability and data consistency of the additional seating process. Figure 4 As shown, the specific steps are as follows.

[0143] 1、Request pre-processing stage

[0144] When a user initiates a request through the "seat" interface, the system first performs an atomic pre-processing operation in Redis: deducting the current active seat inventory; recording the user's participation status (marked as "has participated"); updating the user's active participation frequency; when the number of seats reaches the upper limit, marking as "has been divided" status.

[0145] 2、Asynchronous recording stage

[0146] After successful preprocessing, the system asynchronously sends a seat record message through a message queue (such as RabbitMQ), containing the following key information: user ID and participation time; seat activity ID and corresponding member card information; pre-processing result status.

[0147] 3、Message guarantee mechanism

[0148] Instant retry: when the first message sending fails, the system automatically triggers an instant retransmission;

[0149] Degradation processing: if the retransmission still fails, the complete message data is stored persistently in the MySQL database, including: business data: user ID, activity ID, member card details, etc.; metadata: creation time, failure reason, retry count, etc.; processing status: marked as "to be compensated";

[0150] Timed compensation: scan the MySQL "to be compensated" records through a timed task, and resend them to the message queue.

[0151] The present application realizes significant technical improvement in the brand seat business scenario by constructing a multi-level guaranteed message processing mechanism: Redis preprocessing ensures the atomicity of operations under high concurrency, message queue realizes business decoupling and traffic peak clipping, instant retry deals with transient faults, degradation storage guarantees data reliability, and timed compensation realizes eventual consistency. This ladder-type processing architecture not only completely eliminates the need for manual intervention, but also effectively maintains the system's high throughput capacity while ensuring message processing reliability, perfectly solving the contradiction between message loss risk and system performance bottleneck in traditional solutions.

[0152] Based on the same technical concept, the embodiments of the present application also provide a message processing device, as shown in Figure 5 The device comprises:

[0153] The execution module 51 is configured to respond to a business request, perform a distributed cache operation, and generate a processing message containing business parameters;

[0154] The submission module 52 is configured to asynchronously submit the processing message to a message queue service;

[0155] a retry module 53, configured to trigger an instant retry mechanism to perform at least one re-submission of the processing message if the first submission fails;

[0156] a storage module 54, configured to convert the processing message into a failure record and store the failure record into a database if the retry mechanism fails;

[0157] a re-submission module 55, configured to scan the failure record in the database by a timing compensation service, regenerate the processing message, and submit the processing message to the message queue service.

[0158] In a possible implementation, the retry module is specifically configured to:

[0159] detect a protocol error code returned by the message queue service, and generate an analysis result containing an error type and a retryable flag based on the protocol error code;

[0160] create an original copy of the processing message in the memory when the retryable flag is true;

[0161] obtain a retry parameter from a predefined policy library according to the error type;

[0162] perform a synchronous retry submission operation using the original copy and the retry parameter.

[0163] In a possible implementation, the retry parameter includes an initial retry interval, a back-off coefficient, and a retry number threshold, and the retry module is further configured to:

[0164] use the initial retry interval as a current interval, read the original copy, and perform a first retry submission;

[0165] if the retry submission fails and the retry number threshold is not reached, recalculate the current interval according to the back-off coefficient;

[0166] continue to perform the retry submission using the recalculated current interval until the submission is successful or the retry number threshold is reached.

[0167] In a possible implementation, the apparatus further includes an adjustment module configured to:

[0168] obtain a statistical result by counting success rates and service response delays of all retry submission operations in a preset time window;

[0169] dynamically adjust the retry parameter in the predefined policy library according to the statistical result.

[0170] In a possible implementation, the storage module is specifically configured to:

[0171] parsing service data and metadata in the processing message;

[0172] constructing a failure record containing a service data field, a metadata field and a status field according to the service data and the metadata, wherein the service data field is used to store core service parameters extracted from the processing message, the metadata field is used to store message creation time, failure reason and retry count, and the status field is used to mark the failure record as a to-be-processed state;

[0173] atomically writing the failure record into a target data table of the database through a database transaction.

[0174] In a possible implementation, the resubmission module is specifically configured to:

[0175] querying the failure record in the database in a to-be-processed state according to a preset period;

[0176] parsing the service data field and the metadata field in the failure record for each failure record queried;

[0177] reconstructing a processing message conforming to a message queue service protocol according to the service data field and the metadata field;

[0178] submitting the reconstructed processing message to the message queue service;

[0179] updating the status field of the failure record to a processed state when the submission is successful.

[0180] In a possible implementation, the apparatus further includes a monitoring module configured to:

[0181] monitoring a health state indicator of the message queue service in real time;

[0182] persistently storing a newly received service request to the database when an abnormal health state is monitored;

[0183] processing the service request in the database through a timing compensation service when a recovered health state is monitored.

[0184] Based on the same technical concept, the embodiments of the present application further provide an electronic device, as shown in the accompanying drawings, comprising a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112 and the memory 113 complete mutual communication through the communication bus 114, Figure 6

[0185] The memory 113 is used to store a computer program.

[0186] ​The processor 111, when executing the program stored in the memory 113, implements the following steps:

[0187] In response to the service request, a distributed cache operation is performed and a processing message containing service parameters is generated;

[0188] The processing message is asynchronously submitted to a message queue service;

[0189] If the first submission fails, an instant retry mechanism is triggered to re-submit the processing message at least once;

[0190] If the retry mechanism fails, the processing message is converted into a failure record and stored in a database;

[0191] The database is scanned by a timing compensation service to regenerate the processing message and submit it to the message queue service.

[0192] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0193] The communication interface is used for communication between the above electronic device and other devices.

[0194] The memory can include a Random Access Memory (RAM) and can also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0195] The processor described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0196] In another embodiment provided in the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps of any of the message processing methods.

[0197] In another embodiment provided in the present application, a computer program product containing instructions is provided, and when the computer program product is executed on a computer, the computer program product causes the computer to execute any of the message processing methods in the above embodiments.

[0198] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0199] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus a general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in the various embodiments or some parts of the embodiments.

[0200] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0201] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A message processing method, characterized in that, The method includes: In response to business requests, perform distributed caching operations and generate processing messages containing business parameters; The processing message is submitted asynchronously to the message queue service; If the first submission fails, the immediate retry mechanism is triggered to resubmit the processing message at least once; If the retry mechanism fails, the processing message will be converted into a failure record and persistently stored in the database; The system scans the database for failure records using a timed compensation service, regenerates the processing message, and submits it to the message queue service.

2. The method according to claim 1, characterized in that, The retry mechanism includes the following steps: Detect the protocol error code returned by the message queue service, and generate an analysis result containing the error type and retry flag based on the protocol error code; When the retry flag is true, an original copy of the processed message is created in memory; Retrieve retry parameters from the predefined policy library based on the error type; Perform a synchronous retry commit operation using the original copy and the retry parameters.

3. The method according to claim 2, characterized in that, The retry parameters include: initial retry interval, backoff coefficient, and retry count threshold. The step of performing a synchronous retry commit operation using the original copy and the retry parameters includes: Using the initial retry interval as the current interval, read the original copy and perform the first retry commit; If the retry submission fails and the retry count threshold has not been reached, the current interval is recalculated based on the backoff coefficient. Continue retrying the commit using the recalculated current interval until the commit succeeds or the retry threshold is reached.

4. The method according to claim 2, characterized in that, The method further includes: The success rate and service response latency of all retry submissions within a preset time window are statistically analyzed to obtain the results. The retry parameters in the predefined strategy library are dynamically adjusted based on the statistical results.

5. The method according to claim 1, characterized in that, The step of converting the processing message into a failure record and persistently storing it in the database includes: Parse the business data and metadata in the processing message; Based on the business data and the metadata, a failure record is constructed that includes a business data field, a metadata field, and a status field. The business data field is used to store the core business parameters extracted from the processed message, the metadata field is used to store the message creation time, the reason for failure, and the number of retries, and the status field is used to mark the failure record as pending processing. The failure record is atomically written to the target data table of the database through a database transaction.

6. The method according to claim 5, characterized in that, The step of scanning the failure records in the database through a timed compensation service, regenerating the processing message, and submitting it to the message queue service includes: Query the database for failure records that are in a pending state according to a preset period; For each failed record retrieved, parse the business data fields and metadata fields in the failed record; Based on the business data fields and the metadata fields, reconstruct the processing message that conforms to the message queue service protocol; Submit the refactoring processing message to the message queue service; When the submission is successful, update the status field of the failed record to the processed status.

7. The method according to claim 1, characterized in that, The method further includes: Real-time monitoring of the health status indicators of the message queue service; When an abnormal health status is detected, newly received service requests will be persistently stored in the database. When the health status is detected to have recovered, the business requests in the database are processed through a timed compensation service.

8. A message processing device, characterized in that, The device includes: The execution module is used to respond to business requests, perform distributed caching operations, and generate processing messages containing business parameters. The submission module is used to asynchronously submit the processed message to the message queue service; The retry module is used to trigger an immediate retry mechanism to resubmit the processing message at least once if the first submission fails. The storage module is used to convert the processing message into a failure record and persistently store it in the database if the retry mechanism fails. The resubmission module is used to scan the failure records in the database through the timed compensation service, regenerate the processing message, and submit it to the message queue service.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the message processing method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the message processing method according to any one of claims 1-7.

Images