Kafka message dyeing method based on agent implementation and storage medium

Abstract

The application relates to a Kafka message dyeing method based on an Agent and a storage medium, which comprises the following steps: in the production end, a production interceptor is used to intercept and dye a message to be sent, so that a target message carrying a dyeing mark is obtained; and the target message is sent to the Kafka service end; in the consumption end, a consumption interceptor is used to intercept the target message obtained from the Kafka service end, it is judged whether the target message contains the dyeing mark, if the dyeing mark exists, a second consumption interceptor is used to perform a dyeing removal treatment on the dyeing mark, so that a target message after dyeing removal is obtained, and the target message after dyeing removal is sent to the Kafka service end. The application can completely separate the dyeing logic from the business code, ensures the message transmission efficiency, ensures the safety and transparency of the dyeing message through a dyeing verification and dyeing removal mechanism, and realizes precise routing.

Description

Technical Field

[0001] This application relates to the field of distributed message processing technology, and in particular to a Kafka message coloring method and storage medium based on Agent. Background Technology

[0002] With the widespread adoption of microservice architectures and distributed systems, message brokers have become core components for system decoupling and asynchronous communication. Apache Kafka, as a high-throughput, highly available distributed messaging system, plays a crucial role in big data processing, real-time computing, and system integration. However, as system scale and business complexity increase, Kafka faces numerous challenges in message management and operational debugging.

[0003] In existing technologies, to achieve differentiated message processing and end-to-end tracing, the common approach is to create additional topics or consumer groups. For example, creating separate Kafka topics for canary or test messages isolates different message types into different topics. While this approach is simple to implement, it leads to a surge in the number of topics, increasing the management complexity and resource consumption of the Kafka cluster. Another approach is to create separate consumer groups for different types of consumers, processing the same message in different consumer groups. While this avoids an increase in the number of topics, it introduces complex message consumption control logic, and improper handling can lead to message backlog or even message loss.

[0004] In addition, existing technologies include message content-based filtering schemes, which require implementing filtering logic on each consumer side. This results in highly intrusive business code and necessitates modifying and restarting all related services when filtering conditions change, leading to relatively low processing efficiency. Summary of the Invention

[0005] This application provides a Kafka message coloring method and storage medium based on Agent, aiming to solve the problems of strong intrusiveness and low processing efficiency of traditional business code.

[0006] Firstly, embodiments of this application provide a Kafka message coloring method based on an Agent. The Kafka message coloring system includes a Kafka server, a producer Agent, and a consumer Agent. The producer Agent has a production interceptor, and the consumer Agent has a consumption interceptor. The method includes: On the production side, the production interceptor intercepts and colors the messages to be sent to obtain the target message carrying the coloring identifier; and then sends the target message to the Kafka server. On the consumer side, the consumer interceptor is used to intercept the target message obtained from the Kafka server, and to determine whether the target message contains the coloring identifier. If it does, the second consumer interceptor is used to decolorize the coloring identifier to obtain the decolorized target message, and the decolorized target message is sent to the Kafka server.

[0007] A further technical solution is that the consumer interceptor includes a first consumer interceptor and a second consumer interceptor, and the staining and destaining treatment based on the staining mark includes: The dyed mark is destained using the first consumer interceptor to obtain the staining result, wherein the staining result includes a successful staining result and a failed staining result. If the dyeing result is a successful dyeing result, then determine whether the dyeing mark meets the preset destaining conditions; If the dyed mark meets the preset destaining conditions, the second consumer interceptor is used to destain the dyed mark.

[0008] A further technical solution is that the process of using the second consumer interceptor to destain the dyed identifier includes the following verification logic: Integrity verification: Check whether the data structure of the color identifier is complete and whether there are any missing fields or format errors; Permission verification: Compare the permission information in the coloring identifier with the current consumer's preset access policy to confirm that the current consumer has the permission to process the coloring message; Validity verification: Obtain the timestamp or validity period field from the color identifier and determine whether the current time is within the valid lifespan of the color identifier.

[0009] A further technical solution is that the preset destaining conditions include preset destaining timing and preset destaining scenarios, and the determination of whether the dyed mark meets the preset destaining conditions includes: Determine whether the dyeing mark meets the preset destaining time; If the dyeing mark meets the preset destaining time, then determine whether the dyeing mark meets the preset destaining scenario; If the dyeing identifier matches the preset destaining scenario, then the dyeing identifier is determined to meet the preset destaining conditions.

[0010] A further technical solution is that the production interceptor includes a first production interceptor and a second production interceptor. The first production interceptor is used to intercept and color the messages to be sent, and the second production interceptor is used to monitor the traffic indicators, processing status and system performance data of the colored messages in real time, obtain the collected data, and report the collected data to a preset visualization interface.

[0011] A further technical solution is that, after obtaining the staining result, the method further includes: If the staining result is a staining failure, a preset exception handling operation is triggered. The preset exception handling operation includes one of the following: generating an alarm message and intercepting the staining message, redirecting the target message to a dead-letter queue, or downgrading the target message to a normal message and continuing to perform regular business processing.

[0012] A further technical solution is that the process of using the production interceptor to intercept and color-process the message to be sent, thereby obtaining the target message carrying the color-coded identifier, includes: The production interceptor is used to intercept messages to be sent; The intercepted message to be sent is parsed to obtain message attributes; The message attributes are matched with a preset coloring strategy to obtain the matching result; If the matching result is a successful match, the message to be sent is colored based on the preset coloring strategy to obtain the target message carrying the coloring identifier.

[0013] A further technical solution is that the coloring identifier includes coloring type, coloring version, business identifier, coloring ID, and coloring timestamp.

[0014] A further technical solution is that the step of coloring the message to be sent based on the preset coloring strategy includes: Based on the preset coloring strategy, the preset coloring identifier is injected into the message header of the message to be sent.

[0015] Secondly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the above-described method.

[0016] This application provides a Kafka message coloring method based on an Agent. The Kafka message coloring system includes a Kafka server, a producer Agent, and a consumer Agent. The producer Agent has a producer interceptor, and the consumer Agent has a consumer interceptor. The method includes: at the producer end, using the producer interceptor to intercept and color messages to be sent, obtaining target messages carrying coloring identifiers; and sending the target messages to the Kafka server; at the consumer end, using the consumer interceptor to intercept the target messages obtained from the Kafka server, determining whether the target messages contain the coloring identifiers, and if so, using the second consumer interceptor to decolorize the coloring identifiers, obtaining decolorized target messages, and sending the decolorized target messages to the Kafka server.

[0017] This application deploys Agent interception components on both the production and consumption ends, automatically injecting and extracting coloring information at the system's underlying entry point for message sending and receiving. This completely separates the coloring logic from the business code, ensuring efficient message transmission. Furthermore, the coloring and decolorization mechanisms provided by this invention ensure the security and transparency of colored messages, achieving precise routing. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0021] Figure 1 A flowchart illustrating the first embodiment of an Agent-based Kafka message coloring method provided in this application; Figure 2 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0024] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0025] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0026] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0027] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0028] To address the aforementioned issues, this embodiment provides a Kafka message coloring method based on an Agent. This method is applied to a Kafka message coloring system, which mainly comprises four components: a Kafka producer Agent, a Kafka server, a Kafka consumer Agent, and a coloring control center. The producer Agent has a production interceptor, and the consumer Agent has a consumption interceptor. The method in this embodiment mainly includes two stages: a producer processing flow and a consumer processing flow.

[0029] Example 1: Step S100: On the production side, the production interceptor is used to intercept and color the message to be sent, so as to obtain the target message carrying the coloring mark; and the target message is sent to the Kafka server.

[0030] Specifically, the producer agent is embedded in the message producer application as a plugin or dependency package, achieving non-intrusive deployment through the interceptor interface provided by the Kafka client. This means that the producer application does not need to modify its original business logic code; it only needs to add the interceptor class name to the configuration file to enable the coloring feature. When the producer application is ready to send a message, the producer interceptor automatically intercepts the message. After interception, the producer interceptor determines whether the message needs coloring based on a preset coloring strategy. If coloring is required, a coloring identifier is generated and injected into the message, thus obtaining the target message carrying the coloring identifier. Finally, the producer interceptor sends the target message to the Kafka server.

[0031] Step S200: On the consumer side, the consumer interceptor is used to intercept the target message obtained from the Kafka server. It is determined whether the target message contains a coloring identifier. If it does, the coloring identifier is checked and decolored to obtain the decolored target message. The decolored target message is then sent to the Kafka server.

[0032] Specifically, the consumer agent is also embedded in the message consumer application in a non-intrusive manner. When the consumer application pulls messages from the Kafka server, the consumer interceptor first intercepts the target message. The consumer interceptor checks whether the target message carries a coloring identifier. If it does not carry a coloring identifier, it means the message is a normal message, and the consumer interceptor directly passes it to the consumer application for regular business processing. If it carries a coloring identifier, it means the message is a colored message and requires special handling.

[0033] The consumer interceptor first verifies the coloring identifier, checking its legality, integrity, and validity to prevent illegal or expired colored messages from entering the business system. The specific logic of this verification process will be described in detail in subsequent embodiments. If the verification passes, the consumer interceptor further determines whether the coloring identifier needs to be decolored. Decolorization removes the coloring identifier, restoring the message to its normal state. The timing and scenarios for decolorization can be flexibly configured according to business needs. After decolorization, the decolored target message is obtained, and the consumer interceptor passes it to the consumer application for subsequent business processing.

[0034] Through the above approach, this embodiment achieves non-intrusive message coloring for Kafka messages. The agents on both the producer and consumer sides intervene in the message flow process via an interceptor mechanism, without requiring modification to the Kafka server code or the business code of the producers and consumers, significantly reducing system transformation costs and operational complexity. Simultaneously, by injecting, validating, and removing coloring markers, precise identification, routing, and control of specific messages are achieved, meeting the needs of various business scenarios such as canary releases and end-to-end tracing.

[0035] Example 2: This embodiment refines the internal architecture and processing logic of the consumer agent based on embodiment 1. Specifically, the consumer interceptor includes a first consumer interceptor and a second consumer interceptor. The specific process of staining and destaining based on the staining identifier is achieved through the collaborative work of these two interceptors.

[0036] The consumer-side agent is logically designed as a dual-interceptor architecture. The first consumer interceptor is mainly responsible for the verification logic related to security and compliance, i.e., "infection detection"; while the second consumer interceptor is mainly responsible for the processing logic related to data cleaning and restoration, i.e., "infection removal".

[0037] Since message verification typically involves computationally intensive or I / O-intensive operations such as permission checks and signature verification, while message desmearing mainly modifies the metadata in the message header, their performance characteristics differ. Therefore, separation allows for independent performance optimization or caching strategies for the first interceptor. Secondly, this separation builds defense depth; if message verification fails, the message will be directly intercepted or redirected without loading and executing subsequent desmearing logic, thus saving system resources and improving the efficiency of handling abnormal messages.

[0038] The specific processing procedure includes the following steps: Step S201: Use the first consumer interceptor to perform destaining treatment on the dyed mark to obtain the dyeing result, wherein the dyeing result includes dyeing success result and dyeing failure result.

[0039] Specifically, when the consumer agent intercepts a target message and identifies that it contains a coloring identifier, it first triggers the first consumer interceptor. This interceptor reads the coloring identifier field from the message header and calls preset validation rules for verification. For example, it verifies whether the coloring identifier's format conforms to a preset JSON structure, or verifies whether the identifier is in a blacklist. After verification, it outputs a clear coloring result status.

[0040] Step S202: If the staining result is a successful staining result, then determine whether the staining mark meets the preset destaining conditions.

[0041] Specifically, the system only proceeds to the decolorization condition determination stage after the first consumer interceptor confirms the message's legitimacy and validity. This design ensures that only safe-to-be-colored messages are further processed. The preset decolorization conditions are a set of business rules used to determine whether the coloring identifier needs to be removed. For example, the system can configure rules to stipulate that decolorization is only required when the message is about to be persisted to a database, while the coloring identifier can be retained to maintain tracing capabilities if the message is only used for in-memory computation or forwarded to downstream systems. The determination process involves matching the current message's context information (such as target storage type, downstream system identifier, etc.) with the preset rules.

[0042] Step S203: If the dyed mark meets the preset destaining conditions, the dyed mark is destaining using the second consumer interceptor.

[0043] Specifically, when the destaining condition is successfully matched, the second consumer interceptor is activated. This interceptor performs the specific "destaining" action, which involves removing or clearing specific fields related to the staining identifier (such as X-Stain-Type, X-Stain-ID, etc.) from the message header of the target message. After this step, the target message has been restored to a standard, unstained message state before being passed to the business logic code of the business consumer. This processing method is completely transparent to the business code; the business consumer does not need to be aware of the existence of the staining identifier, thus achieving complete decoupling between the business logic and the underlying message control mechanism.

[0044] The dual-interceptor architecture described above achieves physical isolation and logical decoupling between the staining and destaining logic.

[0045] Example 3: This embodiment, based on Embodiment 2, provides a detailed explanation of the specific verification logic for the staining process performed by the first consumer interceptor. Specifically, the process of using the second consumer interceptor to destain the stained identifier includes the following verification logic: integrity verification, authorization verification, validity period verification, and signature verification.

[0046] First, for integrity verification, the first consumer interceptor checks whether the data structure of the coloring identifier is complete and whether there are any missing fields or incorrect formats. Specifically, since coloring identifiers are usually stored in the message header in JSON format, integrity verification first attempts to parse this JSON string. If parsing fails, the integrity verification is directly deemed to have failed. If parsing succeeds, it further checks whether the required fields exist. For example, it checks whether core fields such as coloring type, coloring ID, and coloring timestamp are empty or contain invalid values. In practical applications, the specific manifestations of missing fields may include: a missing coloring type field making it impossible to identify the message purpose, a missing coloring ID field making it impossible to perform link tracing, or a coloring timestamp field format that does not conform to the ISO 8601 standard, leading to time parsing errors. Through integrity verification, incomplete coloring messages caused by network transmission errors or production program abnormalities can be effectively filtered out, preventing subsequent processing stages from throwing exceptions due to data format issues, thereby ensuring system stability.

[0047] Secondly, for permission verification, the first consumer interceptor compares the permission information in the coloring identifier with the current consumer's preset access policy to confirm that the current consumer has the permission to process the colored message. Specifically, permission verification aims to achieve fine-grained access control. The coloring identifier may contain a business identifier or a tenant identifier, while the preset access policy defines the range of messages that the current consumer is authorized to process. For example, in a multi-tenant system, a consumer instance of tenant A should not process colored messages from tenant B. The first consumer interceptor extracts the tenant ID from the coloring identifier and matches it against the locally configured tenant whitelist. If the match is successful, it means that the current consumer has the processing permission; if the match fails, it means that the message may have been misrouted or there is a risk of unauthorized access, and the coloring result will be marked as failed. This mechanism effectively prevents message leakage or misprocessing across tenants, enhancing system security.

[0048] Secondly, for validity period verification, the first consumer interceptor obtains the timestamp or validity period field from the coloring identifier to determine whether the current time is within the valid lifecycle of the coloring identifier. Specifically, coloring messages often have time-sensitive characteristics. For example, in a canary release scenario, a coloring message may only be valid within the release window, or in a limited-time promotion scenario, the message's validity period is limited to the activity period. The first consumer interceptor reads the coloring timestamp in the coloring identifier and calculates the expiration time point by combining it with a preset valid duration (e.g., 1 hour), or directly reads the explicitly specified expiration time field in the coloring identifier. Then, it compares the expiration time point with the current system time. Considering that different nodes in a distributed system may have clock deviations, this embodiment preferably introduces a clock deviation tolerance mechanism, for example, allowing a time error of plus or minus 5 seconds. If the current time has exceeded the expiration time point (after deducting the tolerance error), the coloring message is determined to have expired, and the verification fails. Validity period verification can avoid the interference of expired messages to the business system and ensure the timeliness of business logic.

[0049] Finally, for signature verification, the first consumer interceptor uses a preset key to verify the signature of the stained identifier to confirm the legitimacy of the message source and prevent tampering. Specifically, signature verification is the verification step with the highest security requirements. When generating the stained identifier at the production end, the system uses a private key or shared key to perform a hash operation on the key fields of the stained identifier (such as the stained ID and timestamp), generating a digital signature and attaching it to the identifier. At the consumption end, the first consumer interceptor uses the corresponding public key or shared key, employs the same hash algorithm (such as HMAC-SHA256), recalculates the signature, and compares it with the carried signature. If the signatures do not match, it indicates that the message has been tampered with during transmission or has an illegal source.

[0050] It is important to note that the signature verification is an optional step, and its execution is determined by the security level configuration command. This is because signature verification involves complex cryptographic operations, which consume CPU resources and increase message processing latency. For internal trusted network environments or scenarios with extremely high performance requirements, the signature verification function can be disabled by configuring the security level to "low," relying only on integrity, authorization, and validity period verification. Conversely, for cross-network transmission or highly sensitive business scenarios, signature verification can be enabled by configuring the security level to "high." This flexible configuration mechanism allows the present invention to achieve a balance between security and performance, adapting to different business needs.

[0051] It should be understood that the verification logic of the above four dimensions does not exist in isolation, but can be used in combination. In a preferred embodiment, the first consumer interceptor executes the verification in the order of integrity verification, permission verification, validity period verification, and signature verification. If a preceding verification fails, the subsequent verification is immediately terminated and a verification failure result is returned, thereby avoiding unnecessary computational overhead and improving verification efficiency. This "short-circuit verification" mechanism is an important contribution of this embodiment to system performance optimization.

[0052] Example 4: This embodiment, based on Embodiment 2, provides a detailed explanation of the specific logic for determining whether a dyed mark meets preset destaining conditions. Specifically, the preset destaining conditions include preset destaining timing and preset destaining scenarios. The process of determining whether a dyed mark meets the preset destaining conditions is a sequential judgment logic based on timing and scenario.

[0053] Specifically, the first step is to determine whether the coloring flag meets the preset decolorization timing. The preset decolorization timing defines the time window or trigger node where the decolorization operation is allowed to execute. In a preferred embodiment, the preset decolorization timing includes, but is not limited to: before message consumption confirmation, after message consumption confirmation, before business logic execution, and after business logic execution. For example, for business scenarios with high reliability requirements, the system can configure the preset decolorization timing as "after message consumption confirmation," meaning that the decolorization operation will only be triggered after the consumer successfully processes the message and commits the consumption offset, thereby ensuring that the coloring flag still exists when the business processing fails and causes message retry, facilitating problem tracking and traffic control. Conversely, if configured as "before business logic execution," the decolorization operation will be executed before the message is delivered to the business code. This approach is suitable for scenarios with extremely high requirements for business transparency, ensuring that the business code is completely unaware of the existence of the coloring flag. It should be understood that the selection of the preset decolorization timing depends on the specific business requirements and system architecture, and this embodiment does not impose any restrictions on this.

[0054] If the coloring identifier meets the preset decolorization timing, the system further determines whether the coloring identifier meets the preset decolorization scenario. The preset decolorization scenario defines the business context in which the decolorization operation is allowed to be executed. The technical consideration behind this design is that not all messages that meet the timing need to be decolorized; in some scenarios, retaining the coloring identifier is crucial for subsequent business processes. In a preferred embodiment, the preset decolorization scenario includes, but is not limited to: persistent storage scenario, forwarding to downstream system scenario, and cross-cluster synchronization scenario. For example, when a message is about to be persistently stored in the database, in order to avoid the coloring identifier polluting the business data table structure or occupying storage space, the system determines that the current scenario is a "persistent storage scenario," which meets the decolorization condition, and performs the decolorization operation. However, when the message needs to be forwarded to the downstream system for further processing, if decolorization is performed at this time, the downstream system will not be able to recognize the coloring attribute of the message, resulting in a break in end-to-end tracing or loss of grayscale traffic. Therefore, the system determines that the current scenario is a "forwarding to downstream system scenario," which does not meet the decolorization condition, and retains the coloring identifier. Through this scenario-based fine-grained control, this embodiment achieves dynamic management of the coloring identifier's lifecycle.

[0055] If the dyed identifier matches the preset de-dyeing scenario, it is ultimately determined that the dyed identifier meets the preset de-dyeing conditions, and the system will trigger the second consumer interceptor to perform de-dyeing processing. This serial judgment logic (timing matches -> scenario matches -> de-dyeing is executed) constructs a robust defense mechanism, effectively preventing over-dyeing or missed de-dyeing. Over-dyeing may lead to the loss of critical link information, affecting the observability of the system; while missed de-dyeing may lead to business data pollution or system compatibility issues. This embodiment ensures that the de-dyeing operation is executed at the correct time and in the correct scenario through a combination of dual condition verification, demonstrating the high availability and refined management capabilities of the solution.

[0056] To more clearly illustrate the technical effects of this embodiment, the following description uses a specific scenario of e-commerce order processing as an example: In this scenario, an order message carrying a grayscale coloring identifier is intercepted by the consumer agent. The system's preset decolorization timing is "after business logic execution," and the preset decolorization scenario is "persistent storage scenario." When the first consumer interceptor successfully verifies the coloring, the system first determines the timing. Since the business logic has not yet been completed, the timing condition of "after business logic execution" is not met, so decolorization is not performed temporarily. At this time, the coloring identifier is retained for grayscale routing during business processing. When the business logic is completed and the order data is ready to be written to the database, the system determines the timing again, and this time the condition of "after business logic execution" is met. Next, the system determines the scenario; the current operation is a database write, which belongs to the "persistent storage scenario," meeting the decolorization condition. Therefore, the second consumer interceptor is triggered, removing the coloring identifier from the message header to ensure that the data written to the database is clean business data. Through the above process, this embodiment perfectly balances the contradiction between business transparency and link observability.

[0057] Example 5: This embodiment, based on Embodiment 1, refines the internal architecture and functions of the production agent. Specifically, the production interceptor includes a first production interceptor and a second production interceptor. The first production interceptor is used to intercept and color the messages to be sent, while the second production interceptor is used to monitor the traffic indicators, processing status, and system performance data of the colorized messages in real time, obtain collected data, and report the collected data to a preset visualization interface to display the grayscale traffic ratio, link tracing path, and abnormal alarm information.

[0058] The production agent also employs a dual-interceptor architecture. The first production interceptor, located at the front of the interceptor chain, is primarily responsible for the core coloring logic, namely performing key operations such as message interception, policy matching, and identifier injection. The second production interceptor, located at the back of the interceptor chain, is mainly responsible for bypass monitoring functions.

[0059] Specifically, the traffic metrics monitored in real time by the second production interceptor include, but are not limited to: total number of messages sent, number of colored messages, percentage of colored messages (i.e., grayscale traffic ratio), and message sending rate (TPS). Processing status metrics include, but are not limited to: number of successful coloring attempts, number of failed coloring attempts, policy matching hit rate, and coloring pass rate. System performance data includes, but is not limited to: interceptor processing time, network transmission latency, memory usage, and CPU utilization. After being collected by the second production interceptor, this data is encapsulated according to a preset data format. In a preferred embodiment, the data format is JSON, including fields such as timestamp, metric name, metric value, and tag, to facilitate subsequent storage and analysis.

[0060] The second production interceptor collects data and reports it to a pre-defined visualization interface. This interface can be a standalone dashboard system or integrated into the coloring control center. On the visualization interface, operations personnel can intuitively view a chart showing the proportion of grayscale traffic. This chart displays the changing trend of the proportion of colored messages in the total traffic in the form of a line chart or pie chart, helping operations personnel to assess the impact of grayscale releases in real time. Simultaneously, the visualization interface also displays the link tracing path, connecting the message flow process at the production end, Kafka server, and consumer end through the coloring ID, forming a complete call chain diagram, facilitating the rapid location of performance bottlenecks or faulty nodes. Furthermore, when coloring failure, verification failure, or system performance anomalies occur, the visualization interface automatically pops up anomaly alarm information to remind operations personnel to handle the situation promptly.

[0061] This embodiment employs a dual-interceptor architecture, decoupling the message coloring function from the monitoring function. The first production interceptor focuses on the core coloring business logic, ensuring the efficiency and reliability of message sending; the second production interceptor focuses on the collection and reporting of monitoring data, ensuring the observability of the system. Each performs its own function without affecting the other, avoiding performance bottlenecks or code maintenance difficulties caused by an overly complex single interceptor. Secondly, the bypass monitoring mechanism of the second production interceptor is completely transparent to the business code. Business developers do not need to embed any monitoring points in the code to obtain detailed runtime data, greatly reducing the system's development and maintenance costs. Finally, through real-time display in a visual interface, operations personnel can fully grasp the operational status of message coloring, providing strong data support and decision-making basis for scenarios such as canary releases and full-link tracing.

[0062] Example 6: This embodiment, based on Embodiment 2, provides a detailed explanation of the handling logic after a dyeing failure. Specifically, after obtaining the dyeing result, if the result is a dyeing failure, a preset exception handling operation is triggered. It should be understood that a dyeing failure means that the dyeing message may have security risks, incorrect data format, or be expired, etc. If it is directly allowed to enter the business system, it may seriously affect the system stability or data consistency. Therefore, this embodiment constructs a multimodal exception handling mechanism, providing different handling strategies according to the severity of the exception and the business fault tolerance requirements.

[0063] In a preferred embodiment, the pre-defined exception handling operations include generating an alarm message and intercepting the tampered message. This approach is suitable for scenarios with severe security threats. For example, when the first consumer interceptor discovers that the digital signature of the tampered identifier has been altered during the signature verification stage, or when it discovers during the authorization verification stage that the current consumer does not have the authority to process the message, the system determines that the message is a malicious attack or unauthorized access. At this time, the system immediately generates a high-priority alarm message and sends it to the operation and maintenance monitoring platform or security response center, while directly intercepting the message to prevent it from being delivered to the business consumer. The interception operation can be implemented by throwing an exception, returning an error code, or marking the message as "unconsumable". This "circuit breaker" mechanism can effectively block the spread of risks and protect the security of core business systems.

[0064] As an alternative implementation, the pre-defined exception handling operation includes redirecting the target message to a dead-letter queue. This approach is suitable for data problem scenarios requiring manual intervention. A dead-letter queue is a special message queue specifically designed to store messages that cannot be consumed normally. When a colored message fails to be colored due to data format errors, missing fields, or version incompatibility, the system does not discard the message directly but forwards it to a pre-defined dead-letter queue. The advantage of this design is that it avoids abnormal messages blocking the processing flow of normal messages while preserving the original data of the message, facilitating subsequent troubleshooting, data repair, or resubmission by maintenance personnel. For example, during e-commerce promotional events, if the coloring identifier format of a certain type of promotional message experiences compatibility issues due to system upgrades, the dead-letter queue mechanism can ensure that these messages are not lost and can be consumed after the problem is fixed.

[0065] As another optional implementation, the preset exception handling operation includes downgrading the target message to a normal message and continuing regular business processing. This approach is suitable for non-critical coloring business scenarios. In some business scenarios, the coloring identifier is only used for auxiliary functions (such as link tracing or non-core canary routing), and even if the coloring identifier fails to validate, the business data of the message itself remains valid. In this case, to ensure high availability and continuity of business, the system can choose to ignore the coloring identifier and downgrade the message to a normal message. Specifically, after the first consumer interceptor fails to validate, it clears the coloring identifier field in the message header, marks the message as a normal message, and then passes it to the business consumer for regular processing. This "downgrade" strategy reflects the system's flexible design, ensuring the operation of core business processes even when the coloring function is unavailable.

[0066] Example 7: This embodiment, based on Embodiment 1, provides a detailed explanation of the specific implementation process of using a production interceptor to intercept and color-code messages to be sent on the production side, thereby obtaining target messages carrying color identifiers. Specifically, this process includes four key steps: message interception, message parsing, policy matching, and color-coding.

[0067] First, the producer interceptor intercepts messages to be sent. Specifically, the producer agent implements the `ProducerInterceptor` interface provided by the Kafka client to intervene in the message sending process before the message is serialized and partitioned. When the producer application calls the `send()` method, the message is first captured by the producer interceptor. This interception process is transparent; the producer application does not need to be aware of the interceptor's existence, nor does it need to modify any business logic. The interceptor can obtain the message's context information, including but not limited to the topic name, partition information, and timestamp.

[0068] Secondly, the intercepted message to be sent is parsed to obtain message attributes.

[0069] Parsing Kafka messages involves extracting the message headers, key, and body. For the message body, depending on the configured parsing rules, both text (JSON, XML, etc.) and binary formats are supported. Specifically, the production interceptor delves into the internal structure of the Kafka message to extract key message attributes. These attributes primarily include: the message key, typically used for message partitioning or business identification, such as an order ID or user ID; the message value, i.e., the message body content, which may contain specific business data, such as JSON-formatted order details; and the message header, a collection of key-value pairs used to store metadata. When parsing the message body, the interceptor performs deserialization processing based on preset content types (such as JSON, XML, Avro, etc.) to extract deeper business fields. For example, for an order message, the parsing process will extract specific fields such as "order amount," "user level," and "product category," which will serve as input parameters for subsequent strategy matching. It should be understood that the message parsing process supports multiple data formats and can be extended to support new formats by configuring parser plugins, thereby adapting to different business scenarios.

[0070] Next, the message attributes are matched against a preset coloring strategy to obtain the matching result. Specifically, the preset coloring strategy is a set of rules pre-configured in the coloring control center and synchronized to the production agent. Each coloring strategy defines the matching conditions and the corresponding coloring action. The matching process compares the parsed message attributes with the conditions in the strategy. In a preferred embodiment, strategy matching supports multiple matching rules: exact matching, such as determining whether the "user level" field is equal to "VIP"; regular expression matching, such as determining whether the "order ID" field conforms to a specific encoding rule; range matching, such as determining whether the "order amount" field is greater than 1000 yuan; and combined matching, i.e., logical AND, OR, and NOT combinations of multiple conditions. The matching process is executed according to the priority order of the strategies. Once a strategy matches successfully, subsequent matching stops and a successful matching result is returned; if all strategies fail to match, a failed matching result is returned. This strategy-driven design pattern achieves a complete separation between business rules and code logic. When business requirements change, such as needing to adjust the filtering conditions for grayscale traffic, operations and maintenance personnel only need to modify the policy configuration in the coloring control center. Without restarting the producer application, the policy will take effect in real time, which greatly improves the system's flexibility and operational efficiency.

[0071] Finally, if the matching result is a successful match, the message to be sent is colored according to the preset coloring strategy to obtain the target message carrying the coloring identifier. Specifically, when the match is successful, the production interceptor generates a coloring identifier according to the coloring rules defined in the successful matching strategy. The specific content of the coloring identifier may include coloring type (such as GRAY_RELEASE), coloring version, business identifier, and other information. Subsequently, the interceptor injects the coloring identifier into the message. The preferred injection method is to write it into the message header, so as not to destroy the business data structure of the message body, nor to affect the storage and forwarding logic of the Kafka server. The message after coloring is the target message, which carries a specific coloring identifier and can be identified and differentiated at the subsequent consumer end. If the matching result is a failed match, the interceptor does not modify the message in any way and directly sends it to the Kafka server as a normal message.

[0072] Through the steps described above, this embodiment automates and intelligently processes message coloring on the production side. The complete data flow from interception to target message generation not only ensures the accuracy of the coloring process but also, through the separation of strategy and code, endows the system with extremely high flexibility and scalability, enabling it to quickly respond to complex and ever-changing business needs.

[0073] Example 8: This embodiment, based on the above embodiments, provides a detailed description of the specific data structure of the coloring identifier. Specifically, the coloring identifier includes fields such as coloring type, coloring version, business identifier, coloring ID, and coloring timestamp.

[0074] The stain type (X-Stain-Type) field can be GRAY_RELEASE, TRACING, TEST, etc.

[0075] The X-Stain-Version field identifies the version number of the staining policy or the application service. This field is particularly important in canary release scenarios. For example, the staining version can be set to "v2.3.0", indicating that the message should be processed by the service instance with version number v2.3.0. When the system undergoes a rolling upgrade, service instances of different versions may coexist. The consumer agent compares the staining version with the current service version to decide whether to receive the message. This mechanism ensures precise isolation of canary traffic and avoids system failures caused by version incompatibility.

[0076] The X-Stain-Business field is used to identify specific business scenarios or objects. This field provides fine-grained control over business aspects. For example, the business identifier can be set to "INTELLIGENT_DISCOUNT" to indicate that the message involves smart discount services; or it can be set to "TENANT_A" to indicate that the message belongs to tenant A. In multi-tenant systems or complex business systems, the business identifier can help the consumer agent implement more granular permission verification and routing decisions, such as allowing only consumers of a specific tenant to process colored messages with that tenant's identifier.

[0077] The X-Stain-ID field is a globally unique identifier for the stain. This field typically uses a UUID (Universally Unique Identifier) ​​or an ID generated by the Snowflake algorithm. The X-Stain-ID remains unchanged throughout the entire message flow, serving as a crucial anchor point for end-to-end tracing. When a message is stained at the production end, the system generates a unique stain ID and injects it into the message header. Subsequently, regardless of how many microservices or Kafka topics the message passes through, this stain ID is passed through. By collecting the stain IDs, consumer agents or monitoring systems can connect logs and monitoring data scattered across different service nodes to form a complete call chain diagram, thereby quickly locating performance bottlenecks or faulty nodes.

[0078] The staining timestamp (X-Stain-Timestamp) field records the specific time when the staining operation occurred. This field typically uses the ISO 8601 standard UTC time format, such as "2023-10-27T10:00:00Z". The staining timestamp plays a crucial role in the staining verification process. The consumer agent reads the staining timestamp and combines it with a preset validity period (e.g., 1 hour) to calculate the expiration time of the staining identifier. If the current system time exceeds this expiration time, the staining message is determined to be invalid, thereby triggering the exception handling logic described in Example 6. This timestamp-based validity verification mechanism effectively prevents expired messages from interfering with the business system and ensures the timeliness of business logic.

[0079] In some embodiments, custom metadata (X-Stain-Metadata) may also be included, which may be extended information stored in JSON format.

[0080] Example 9: This embodiment, based on Embodiments 7 and 8, provides a detailed description of the specific implementation of the coloring process. Specifically, based on a preset coloring strategy, a preset coloring identifier is injected into the message header of the message to be sent.

[0081] It should be understood that the Kafka message structure mainly consists of three parts: message key, message value, and message header. The message body typically carries core business data and is serialized using formats such as JSON, Avro, and Protobuf. Injecting coloring identifiers by modifying the message body—for example, forcibly inserting coloring identifier fields into JSON-formatted business data—would lead to a series of drawbacks: First, it would disrupt the original structure of the business data, requiring producers and consumers to synchronously modify their serialization and deserialization logic, making it highly intrusive; second, if the business data uses a binary format (such as Protobuf), modifying the message body would require redefining the schema, resulting in high upgrade costs; finally, the modified message body would increase in size, increasing network bandwidth and storage space consumption.

[0082] In contrast, this embodiment employs a message header injection approach, which offers significant technical advantages. The message header is a metadata storage area natively supported by Kafka, stored as key-value pairs and existing independently of the message body. When performing staining processing, the production interceptor calls the Header API provided by the Kafka client to write the staining identifier (typically serialized as a JSON string or byte array) defined in Embodiment 8 into the message header. In a preferred embodiment, the production interceptor creates a Header item named "X-Stain-Metadata" and stores the JSON string of the staining identifier as its value.

[0083] Specifically, the injection process includes the following steps: First, the producer interceptor constructs a coloring identifier object based on the successfully matched coloring strategy, and populates fields such as coloring type, coloring version, business identifier, coloring ID, and coloring timestamp. Second, the coloring identifier object is serialized into a byte array. Finally, the serialized byte array is added to the message header by calling the `add()` interface of the `headers()` method of `Kafka ProducerRecord`. This process is completely transparent to the business code; the producer application does not need to be aware of the existence of the coloring identifier when sending messages, nor does it need to modify any business logic code.

[0084] On the consumer side, the consumer interceptor also uses the Header API to quickly extract the staining identifier based on a preset key name (such as "X-Stain-Metadata") without parsing the message body. This "non-intrusive" design achieves complete decoupling between the staining control logic and the business processing logic. The business system can evolve independently, and the staining strategy can be flexibly configured without affecting each other. Furthermore, since the parsing efficiency of the message header is much higher than the deserialization efficiency of the message body, the solution in this embodiment minimizes the impact on message transmission performance while ensuring the staining function, achieving a perfect balance between transparent transmission and efficient parsing.

[0085] To more clearly illustrate the application effect of the technical solution of this invention in actual business, this embodiment takes the canary release scenario of an e-commerce order system as an example for detailed explanation. An e-commerce platform plans to upgrade its order processing system, adding a smart discount calculation function. To avoid the system risks that may arise from a full release, it decides to use the agent-based Kafka message coloring method provided by this invention to achieve canary verification of the new version service.

[0086] In this scenario, the system environment includes a Kafka cluster, an order service (producer), an order processing service (consumer), and a new version of the smart discount service. The specific implementation process of this embodiment mainly includes the following stages: The first phase involves configuring the coloring strategy. Operations personnel create a canary release strategy in the coloring control center. Specifically, the configured matching rule is: order messages with an order amount greater than 1000 yuan and a user level of VIP. This rule aims to guide high-value core user traffic to the new version service for verification. Simultaneously, the coloring identifier is configured, for example, setting the coloring type to GRAY_RELEASE, the coloring version to v2.3.0, and the business identifier to INTELLIGENT_DISCOUNT. After the strategy is configured, the coloring control center distributes it to the production agent and consumer agent.

[0087] The second stage involves message coloring on the production side. When the order service generates a new order message, the production agent uses a production interceptor to intercept the message to be sent. The interceptor parses the intercepted message, extracting message attributes such as order amount and user level. Then, it matches these attributes against a preset coloring strategy. If the order amount is 1500 yuan and the user level is VIP, the match is successful. Based on this strategy, the production interceptor generates a coloring identifier containing coloring type, coloring version, business identifier, coloring ID, and coloring timestamp. Next, the production interceptor injects this coloring identifier into the message header of the message to be sent, obtaining the target message carrying the coloring identifier. Because the coloring identifier is stored in the message header, the message body maintains the original business data structure, ensuring the compatibility of order data within the Kafka cluster. Finally, the target message is sent to the Kafka server.

[0088] The third stage involves message routing and processing on the consumer side. The consumer agent uses a consumer interceptor to retrieve target messages from the Kafka server and intercepts them. The interceptor first determines whether the target message contains a coloring identifier. For target messages carrying grayscale identifiers, the consumer interceptor verifies the coloring identifier, including integrity verification, permission verification, and expiration verification, ensuring the message is legitimate and not expired. After successful verification, the consumer agent routes the message to a service instance that has deployed the new version of the smart discount function for processing, based on the coloring version v2.3.0 in the coloring identifier. For ordinary order messages without coloring identifiers, they are routed to the existing stable version service instance. This differentiated routing based on coloring identifiers achieves logical isolation between grayscale traffic and normal traffic, without the need to create additional Kafka topics or consumer groups.

[0089] The fourth phase involves monitoring and adjustments. On the production side, the second production interceptor monitors the traffic metrics and processing status of colored messages in real time and reports the collected data to a pre-defined visualization interface. Operations personnel can observe metrics such as the proportion of gray-scale traffic, processing success rate, and response time through this visualization interface. Initially, the system is configured to route only 5% of VIP user order traffic to the new version service. After a period of observation, if the new version service runs stably and business metrics are normal, operations personnel can adjust the coloring strategy to gradually expand the gray-scale range to 20%, 50%, and eventually 100%. If an anomaly is detected, the strategy can be immediately adjusted to switch traffic back to the stable version, achieving a rapid rollback.

[0090] Compared to existing technologies, this embodiment offers significant technical advantages. Existing technologies typically require creating separate Kafka topics for canary messages, which not only increases the complexity of topic management but also leads to data redundancy and increased storage costs. This embodiment, however, utilizes message coloring technology to achieve fine-grained traffic segmentation on existing topics without creating additional topics. Calculations show that this solution saves approximately 70% of the topic management configuration workload and avoids performance degradation in the Kafka cluster caused by a surge in the number of topics. Furthermore, since the coloring logic is encapsulated in the Agent, business code does not need to be aware of the specific details of canary releases, greatly reducing system modification costs and operational complexity, achieving truly non-intrusive canary releases.

[0091] Based on the above embodiments, the production agent may include a policy acquisition module, a message parsing module, a policy matching module, an identifier addition module, and a message sending module.

[0092] The system comprises the following modules: a policy acquisition module, which periodically retrieves the latest coloring policies from the coloring control center and receives real-time notifications when policies change; a message parsing module, which parses intercepted Kafka messages and extracts message headers, message keys, and message body content; a policy matching module, which determines whether a message needs coloring and how to color it based on the rules in the coloring policy; an identifier addition module, which adds coloring identifiers to specific fields in the message headers of messages that need coloring; and a message sending module, which sends the processed messages to the Kafka cluster.

[0093] The consumer agent can include an identifier extraction module, a staining verification module, a routing decision module, a destaining module, and a monitoring and reporting module. Specifically, the identifier extraction module extracts staining identifier information from the message header; the staining verification module verifies the legality and validity of the staining identifier; the routing decision module determines the message processing method based on the staining identifier; the destaining module removes the staining identifier after message consumption; and the monitoring and reporting module collects the processing status and performance metrics of stained messages and reports them to the staining control center.

[0094] The coloring control center serves as the system's management hub, providing coloring policy management, monitoring, and system configuration functions. The coloring control center can include a policy management module, a policy distribution module, a monitoring and storage module, and a visual interface. Specifically, the policy management module provides CRUD (Create, Read, Update, Delete) and version management functions for policies; the policy distribution module pushes coloring policies to the production and consumer agents; the monitoring and storage module collects and stores monitoring data for coloring messages; and the visual interface provides a web-based management interface for users to easily configure policies and view system status.

[0095] Corresponding to the above agent-based Kafka message coloring method, this application also provides an agent-based RabbitMQ message coloring device. This agent-based RabbitMQ message coloring device includes a unit for executing the above agent-based Kafka message coloring method, and can be configured in terminals such as desktop computers, tablets, and laptops.

[0096] like Figure 2 As shown in the figure, this application provides a computer device including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114. Memory 113 is used to store computer programs; In one embodiment of this application, when the processor 111 executes the program stored in the memory 113, it implements the agent-based Kafka message coloring method provided in any of the foregoing method embodiments, including: On the production side, the production interceptor intercepts and colors the messages to be sent to obtain the target message carrying the coloring identifier; and then sends the target message to the Kafka server. On the consumer side, the consumer interceptor is used to intercept the target message obtained from the Kafka server, and to determine whether the target message contains the coloring identifier. If it does, the second consumer interceptor is used to decolorize the coloring identifier to obtain the decolorized target message, and the decolorized target message is sent to the Kafka server.

[0097] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0098] Therefore, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, wherein when the computer program is executed by a processor, it implements the steps of the agent-based Kafka message coloring method provided in any of the foregoing method embodiments, including: On the production side, the production interceptor intercepts and colors the messages to be sent to obtain the target message carrying the coloring identifier; and then sends the target message to the Kafka server. On the consumer side, the consumer interceptor is used to intercept the target message obtained from the Kafka server, and to determine whether the target message contains the coloring identifier. If it does, the second consumer interceptor is used to decolorize the coloring identifier to obtain the decolorized target message, and the decolorized target message is sent to the Kafka server.

[0099] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk, or any other physical storage medium capable of storing program code. The computer-readable storage medium can be non-volatile or volatile.

[0100] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0102] The steps in the methods of this application embodiment can be adjusted, merged, or deleted according to actual needs. The units in the apparatus of this application embodiment can be merged, divided, or deleted according to actual needs. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0103] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0104] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0105] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Since these modifications and variations fall within the scope of the claims and their equivalents, this application also intends to include these modifications and variations.

[0106] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A Kafka message coloring method based on Agent implementation, characterized in that, The Kafka message coloring system includes a Kafka server, a producer agent, and a consumer agent. The producer agent has a production interceptor, and the consumer agent has a consumption interceptor. The method includes: On the production side, the production interceptor intercepts and colors the messages to be sent to obtain the target message carrying the coloring identifier; and then sends the target message to the Kafka server. On the consumer side, the consumer interceptor is used to intercept the target message obtained from the Kafka server, and to determine whether the target message contains the coloring identifier. If it does, the second consumer interceptor is used to decolorize the coloring identifier to obtain the decolorized target message, and the decolorized target message is sent to the Kafka server.

2. The method according to claim 1, characterized in that, The consumer interceptor includes a first consumer interceptor and a second consumer interceptor. The staining and destaining processes based on the staining identifier include: The dyed mark is destained using the first consumer interceptor to obtain the staining result, wherein the staining result includes a successful staining result and a failed staining result. If the dyeing result is a successful dyeing result, then determine whether the dyeing mark meets the preset destaining conditions; If the dyed mark meets the preset destaining conditions, the second consumer interceptor is used to destain the dyed mark.

3. The method according to claim 2, characterized in that, The process of destaining the dyed identifier using the second consumer interceptor includes the following verification logic: Integrity verification: Check whether the data structure of the color identifier is complete and whether there are any missing fields or format errors; Permission verification: Compare the permission information in the coloring identifier with the current consumer's preset access policy to confirm that the current consumer has the permission to process the coloring message; Validity verification: Obtain the timestamp or validity period field from the color identifier and determine whether the current time is within the valid lifespan of the color identifier.

4. The method according to claim 2, characterized in that, The preset destaining conditions include preset destaining timing and preset destaining scenarios. Determining whether the dyed marker meets the preset destaining conditions includes: Determine whether the dyeing mark meets the preset destaining time; If the dyeing mark meets the preset destaining time, then determine whether the dyeing mark meets the preset destaining scenario; If the dyeing identifier matches the preset destaining scenario, then the dyeing identifier is determined to meet the preset destaining conditions.

5. The method according to claim 1, characterized in that, The production interceptor includes a first production interceptor and a second production interceptor. The first production interceptor is used to intercept and color the messages to be sent, and the second production interceptor is used to monitor the traffic indicators, processing status and system performance data of the colored messages in real time, obtain the collected data, and report the collected data to a preset visualization interface.

6. The method according to claim 2, characterized in that, After obtaining the staining results, the method further includes: If the staining result is a staining failure, a preset exception handling operation is triggered. The preset exception handling operation includes one of the following: generating an alarm message and intercepting the staining message, redirecting the target message to a dead-letter queue, or downgrading the target message to a normal message and continuing to perform regular business processing.

7. The method according to claim 1, characterized in that, The process of intercepting and coloring the message to be sent using the production interceptor to obtain the target message carrying the coloring identifier includes: The production interceptor is used to intercept messages to be sent; The intercepted message to be sent is parsed to obtain message attributes; The message attributes are matched with a preset coloring strategy to obtain the matching result; If the matching result is a successful match, the message to be sent is colored based on the preset coloring strategy to obtain the target message carrying the coloring identifier.

8. The method according to claim 1, characterized in that, The coloring identifier includes coloring type, coloring version, business identifier, coloring ID, and coloring timestamp.

9. The method according to claim 1, characterized in that, The step of coloring the message to be sent based on the preset coloring strategy includes: Based on the preset coloring strategy, the preset coloring identifier is injected into the message header of the message to be sent.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, can implement the method as described in any one of claims 1-9.

Images