A message middleware testing method and system based on multi-client simulation

Abstract

The application discloses a message middleware testing method and system based on multi-client simulation, and relates to the technical field of software testing. The method comprises the following steps: extracting client behavior from real interaction records of a message middleware, analyzing message structure and field attribute information, and constructing a field description list; based on semantic information and context relationship in a message protocol specification, designing a self-limiting mechanism, a consistency constraint mechanism, a structure rationality guarantee mechanism and an interaction timing change mechanism, and combining the field description list to generate a variation data packet conforming to semantic and format specifications; simulating multiple logically independent clients in a single test process, sending the variation data packet to the message middleware according to the arranged interaction timing, and monitoring the running state of the message middleware and business layer exceptions in the message interaction process.

Description

Technical Field

[0001] This application relates to the field of network communication technology, and in particular to a message middleware testing method and system based on multi-client simulation. Background Technology

[0002] Message middleware, as a key hub in modern distributed system architecture, handles the asynchronous communication needs between applications, completing message exchange through mechanisms such as receiving, persisting, and forwarding. Given its role in transporting critical business data, the correctness of its functionality, message consistency, and system reliability are particularly crucial.

[0003] The core value of message middleware lies in supporting multiple communication modes, including but not limited to one-to-one queue forwarding and many-to-many publish-subscribe modes. In the publish-subscribe mode, messages need to be accurately forwarded from the sender to all subscribed receivers, involving complex message interaction mechanisms such as message deduplication, reception confirmation, consumption order guarantee, and exception queue handling.

[0004] Existing fuzzing techniques for message middleware typically have the following limitations:

[0005] 1. Limitations of Single-Client Testing Perspective: Existing fuzzing tools primarily employ a single-client connection testing mode, detecting protocol layer anomalies by sending illegal communication protocol frames. This approach focuses on robustness testing of protocol parsing and struggles to cover all scenarios of the core functional logic of message middleware—end-to-end processing such as cross-client message routing and forwarding, and state synchronization.

[0006] 2. Insufficient coverage of message semantics and business logic: The single-client test mode cannot simulate real multi-terminal interaction scenarios and is difficult to capture defects in message interaction, such as message deduplication, reception confirmation, consumption order guarantee, and exception queue handling.

[0007] 3. Limited Monitoring and Judgment Dimensions: Traditional fuzz testing methods rely on process signals and operating system-level anomaly capture, but their ability to detect business-layer anomalies in message middleware (such as message loss, message duplication, delivery order violations, and transaction consistency violations) is limited. These situations are often subtle and difficult to reproduce, and relying solely on crash event detection will miss the scope of investigation.

[0008] The limitations of existing technologies in the above aspects make it difficult to fully discover many hidden logical problems in message middleware, which poses a challenge to the reliability and trustworthiness of distributed systems. Summary of the Invention

[0009] This application provides a message middleware testing method and system based on multi-client simulation, aiming to solve the problems of limited perspective in single-client testing, insufficient coverage of message semantics and business logic, and single monitoring and judgment dimensions in the existing technology.

[0010] Firstly, a message middleware testing method based on multi-client simulation is provided, including:

[0011] Extract client behavior from the actual interaction records of the message middleware, including the behavior of message producers and message consumers; parse the interaction messages according to message type and time sequence to obtain the message structure and attribute information of each field, and construct a field description list based on the attribute information;

[0012] Based on the semantic information and contextual relationships in the message protocol specification, at least four key mechanisms for guiding mutation strategies are designed, including a self-restriction mechanism, a consistency constraint mechanism, a structural rationality guarantee mechanism, and an interaction timing change mechanism; combined with the field description list, a mutation data packet conforming to semantic and format specifications is generated.

[0013] Simulate multiple logically independent clients within a single test process, send the mutated data packets to the message middleware according to the orchestrated interaction sequence, and monitor the running status of the message middleware and business layer anomalies during the message interaction process.

[0014] In the above scheme, optionally, the attribute information of the field includes field name, field value, type, label, length, and semantic constraints; the semantic constraints include the logical scope of the field, the dependency relationship between fields, and the position requirements of the field in the interaction sequence.

[0015] In the above scheme, optionally, the self-limiting mechanism is used to ensure that the size of the mutated data packet does not exceed the legal range specified by the message protocol; the consistency constraint mechanism is used to maintain the predefined logical relationship between fields and avoid generating mutated data that violates field dependencies; the structural rationality guarantee mechanism is used to guide the correct parsing of the data packet structure based on key fields and ensure the structural integrity of the mutated data packet; the interaction timing change mechanism is used to adjust the message sending order between multiple clients to cover the internal processing logic of the message middleware under different timings.

[0016] Optionally, in the above scheme, generating a variant data packet that conforms to semantic and format specifications further includes: performing a variant operation on a single field or a combination of multiple fields based on the field description list, wherein the variant operation includes at least one of replacement, insertion, deletion, boundary value testing, and random fuzzing.

[0017] Optionally, in the above scheme, the monitoring of business layer anomalies during message interaction includes at least one of the following: detecting message loss, message duplication, message order disorder, transaction consistency violation, and reception confirmation anomaly.

[0018] Optionally, before extracting client behavior from real interaction records, the above solution also includes: injecting monitoring code into the message middleware through instrumentation to capture runtime error or crash information.

[0019] Optionally, the above scheme may further include: recording the mutation configuration, message interaction log, middleware running status and system snapshot when the exception is triggered, in order to support defect reproduction and root cause analysis.

[0020] Secondly, a message middleware testing system based on multi-client simulation is provided, including:

[0021] The interactive message parsing module is used to extract client behavior from the actual interaction records of the message middleware, parse the message structure and attribute information of each field according to the message type and timing, and construct a list of field descriptions.

[0022] The mutation data generation module is used to design self-restriction mechanisms, consistency constraint mechanisms, structural rationality guarantee mechanisms, and interaction timing change mechanisms based on the semantic information and contextual relationships in the message protocol specification, and generate mutation data packets that conform to semantic and format specifications in combination with the field description list;

[0023] The multi-client simulation execution module is used to simulate multiple logically independent clients within a single test process and send the mutated data packets to the message middleware according to the orchestrated interaction sequence.

[0024] The anomaly monitoring module is used to monitor the running status of the message middleware and business layer anomalies during message interaction.

[0025] Optionally, in the above scheme, the anomaly monitoring module is also used to record the mutation configuration, message interaction log, middleware running status and system snapshot when an anomaly is triggered.

[0026] In the above scheme, optionally, the self-limitation mechanism, consistency constraint mechanism, structural rationality guarantee mechanism, and interaction timing change mechanism in the mutation data generation module are used to ensure that the data packet size is legal, the field logic is consistent, the structure is correct, and the timing scenarios of multiple clients are covered.

[0027] Compared with the prior art, this application has at least the following beneficial effects:

[0028] Based on further analysis and research of existing technical problems, this application recognizes the fundamental shortcomings of existing fuzz testing methods, which are limited to a single client perspective, cannot cover message semantics and multi-terminal interaction logic, and rely solely on operating system-level anomaly capture to find hidden defects in the business layer. By simulating multiple logically independent clients within a single test process and combining self-limitation mechanisms, consistency constraint mechanisms, structural rationality guarantee mechanisms, and interaction timing change mechanisms to generate mutated data packets that conform to semantic specifications, this application achieves the technical effect of covering end-to-end processing scenarios such as cross-client message routing and forwarding and state synchronization, effectively capturing business logic defects at the interaction layer such as message deduplication, reception confirmation, and consumption order guarantee, and realizing real-time monitoring of hidden business layer anomalies such as message loss, duplication, and disordered order. Attached Figure Description

[0029] Figure 1 This application provides an illustration of an application environment for a message middleware testing method based on multi-client simulation, as shown in one embodiment of the present application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0031] In the description of this application: unless otherwise stated, "multiple" means two or more. Expressions such as "including", "comprising", and "having" also mean "not limited to" (certain units, components, materials, steps, etc.).

[0032] This application aims to provide a message middleware testing method and system based on multi-client simulation, further improving the coverage of existing technologies in testing core message middleware functions such as message forwarding semantics, concurrent interaction state management, and reception acknowledgment mechanisms. By simultaneously simulating multiple logically independent client simulations within a single test process, it accurately orchestrates complete operations with message interaction relationships and introduces multi-dimensional variations. Furthermore, it deploys a real-time verification mechanism at the semantic level of message interaction, thereby easily improving the test coverage and defect discovery effectiveness of message middleware.

[0033] In one embodiment, a message middleware testing method based on multi-client simulation is provided, including steps S1-S3.

[0034] Step S1: Extract client behavior from the actual interaction records of the message middleware, including the behavior of message producers and message consumers; parse the interaction messages according to message type and time sequence to obtain the message structure and attribute information of each field, and construct a field description list based on the attribute information.

[0035] In one embodiment, the attribute information of the field includes the field name, field value, type, label, length, and semantic constraints; the semantic constraints include the logical scope of the field, the dependencies between fields, and the positional requirements of the field in the interaction sequence.

[0036] In one embodiment, before extracting client behavior from real interaction records, the method further includes: injecting monitoring code into the message middleware via instrumentation to capture runtime error or crash information.

[0037] The following is combined with Figure 1 The method provided in the embodiments of the present invention will be described in detail. Figure 1 A flowchart of this embodiment is shown, in which ① to ⑦ respectively represent the main execution steps of the method.

[0038] Interactive message parsing phase:

[0039] First, extract client behavior (corresponding to) from the actual interactions and interaction records of the message middleware. Figure 1 (①) Including but not limited to message producers and message consumers. Based on different message types and time sequences in the interaction, messages are precisely classified, and information such as packet headers, data load, and message flow is extracted.

[0040] Based on the response of the message middleware during interaction and the message protocol specification, accurately parse the message structure and the attribute information of each field (field name and field value) (corresponding to...). Figure 1 (② and ③ in the text).

[0041] By parsing the obtained project attributes, corresponding processing logic was customized for specific field types, converting message data represented by hexadecimal byte sequences into a list containing field descriptions (corresponding to...). Figure 1 (④ in the text)

[0042] Step S2: Based on the semantic information and contextual relationships in the message protocol specification, design at least four key mechanisms to guide the mutation strategy, including a self-restriction mechanism, a consistency constraint mechanism, a structural rationality guarantee mechanism, and an interaction timing change mechanism; combine the field description list to generate a mutation data packet that conforms to the semantic and format specifications.

[0043] Figure 1 In the message data packet generation stage:

[0044] A deep understanding of the semantic information and contextual relationships in the message protocol specification led to the design of four key mechanisms to guide the mutation strategy (corresponding to...). Figure 1 (See points ⑤ and ⑥). Four key mechanisms associate semantic information, combined with field description lists, to jointly guide the generation of field-aware, legitimate mutation data packets. Mutation strategies: involving different levels and types of mutations, providing a flexible data generation mechanism.

[0045] In one embodiment, the self-limiting mechanism is used to ensure that the size of the mutated data packet does not exceed the legal range specified by the message protocol; the consistency constraint mechanism is used to maintain the predefined logical relationship between fields and avoid generating mutated data that violates field dependencies; the structural rationality guarantee mechanism is used to guide the correct parsing of the data packet structure based on key fields and ensure the structural integrity of the mutated data packet; the interaction timing change mechanism is used to adjust the message sending order between multiple clients to cover the internal processing logic of the message middleware under different timings.

[0046] Among them, there are four key mechanisms (self-restriction mechanism, consistency constraint mechanism, structural rationality guarantee mechanism, and interactive timing change mechanism).

[0047] Self-limiting mechanism: Ensures that the size of data packets during the mutation process is always within the legal range, avoiding invalid or non-compliant mutations.

[0048] Consistency constraint mechanism: By constraining the relationships between fields, it ensures logical consistency between fields, thereby avoiding unreasonable variation results.

[0049] Structural rationality guarantee mechanism: The parsing process is guided by key fields to ensure that the modified data packets can correctly reflect the structural changes of the device and avoid parsing errors.

[0050] Interaction timing change mechanism: Based on the timing differences of multi-client interactions, ensure that the test scenarios sufficiently cover more of the message middleware's internal logic.

[0051] In one embodiment, generating a variant data packet that conforms to semantic and format specifications includes performing a variant operation on a single field or a combination of multiple fields based on a list of field descriptions, wherein the variant operation includes at least one of replacement, insertion, deletion, boundary value testing, and random fuzzing.

[0052] Step S3: Simulate multiple logically independent clients within a single test process, send the mutated data packets to the message middleware according to the orchestrated interaction sequence, and monitor the running status of the message middleware and business layer anomalies during the message interaction process.

[0053] In one embodiment, monitoring business layer anomalies during message interaction includes detecting at least one of the following: message loss, message duplication, message order disorder, transaction consistency violation, and reception confirmation anomaly.

[0054] In one embodiment, the method further includes: recording the mutation configuration, message interaction log, middleware running status, and system snapshot when the exception is triggered, in order to support defect reproduction and root cause analysis.

[0055] In this embodiment, Figure 1 Meanwhile, by instrumenting the program, it is possible to monitor for potential errors or crashes (corresponding to...). Figure 1 (⑦ in the text).

[0056] Key points of this application:

[0057] To address the temporal structure of message middleware interactions with multiple clients, this application proposes a message middleware testing method based on multi-client simulation. This method dynamically extracts the type, label, length, and semantic constraints (such as logical range and temporal relationships) of each field, focusing on the temporal structure of interactions between multiple clients and the message middleware. By converting the original interaction records into a structured list of field descriptions, it significantly enhances message middleware responses in multi-client scenarios, solving the test coverage problem of traditional methods that ignore message middleware processing across multiple clients. This provides high-fidelity input for mutation strategies.

[0058] This application proposes a mutation strategy based on a combination of semantics and format specifications, which significantly improves the success rate of mutated data passing the host input validation mechanism. Traditional methods typically focus only on the type and size of fields, ignoring the semantic information of the fields, which makes the generated mutated data easily rejected by the host input validation mechanism.

[0059] This application combines four mechanisms—self-restriction, consistency constraints, structural rationality, and interactive timing changes—with field structure to ensure that mutated data simultaneously meets the USB protocol format specification and semantic logic. This collaborative strategy significantly increases the probability of test cases passing input validation, thereby triggering the exception handling logic of the kernel driver.

[0060] Compared with existing technologies, the message middleware fuzzing method and system based on multi-client simulation provided in this application mainly achieve the following performance improvements:

[0061] 1. Significantly improves test coverage and defect detection capabilities:

[0062] By orchestrating multi-client collaboration scenarios, testing can cover core functionalities of message middleware, such as end-to-end message flow, routing and distribution, and acknowledgment mechanisms. Consistency verification at the message semantic layer can fully capture hidden defects such as message loss, duplication, and out-of-order delivery. The effectiveness of the testing and the defect detection rate are significantly improved compared to traditional tools.

[0063] 2. Significantly reduce the proportion of invalid test cases:

[0064] The mutation strategy, which combines protocol specification awareness with message semantic constraints, ensures that most mutated data can pass the input validation of the message middleware. Compared to simple mutation methods, the test cases generated in this application are more efficient and effectively optimize computational resource consumption.

[0065] 3. Achieve fully automated testing processes and low-cost operation and maintenance:

[0066] The entire process, from multi-client simulation creation, scenario orchestration, mutation execution to anomaly detection, is fully automated. It automatically and efficiently performs message protocol and semantic analysis, significantly reducing the engineering costs of test design. It supports rapid adaptation and expansion for different message middleware.

[0067] 4. Generate high-quality, reproducible test cases:

[0068] Each anomaly detection fully records the mutated configuration that triggered the anomaly, message interaction records, middleware logs, and system state snapshots. Engineers can use this detailed contextual information to quickly perform root cause analysis and defect repair, significantly reducing debugging costs.

[0069] In one embodiment, a message middleware testing system based on multi-client simulation is provided, comprising:

[0070] The interactive message parsing module is used to extract client behavior from the actual interaction records of the message middleware, parse the message structure and attribute information of each field according to the message type and timing, and construct a list of field descriptions.

[0071] The mutation data generation module is used to design self-restriction mechanisms, consistency constraint mechanisms, structural rationality guarantee mechanisms, and interaction timing change mechanisms based on the semantic information and contextual relationships in the message protocol specification, and generate mutation data packets that conform to semantic and format specifications in combination with the field description list;

[0072] The multi-client simulation execution module is used to simulate multiple logically independent clients within a single test process and send the mutated data packets to the message middleware according to the orchestrated interaction sequence.

[0073] The anomaly monitoring module is used to monitor the running status of the message middleware and business layer anomalies during message interaction.

[0074] The specific implementation details of each module can be found in the above description of the limitations of a message middleware testing method based on multi-client simulation, and will not be repeated here.

[0075] In one embodiment, the anomaly monitoring module is also used to record the mutation configuration, message interaction log, middleware running status, and system snapshot when an anomaly is triggered.

[0076] In one embodiment, the self-restriction mechanism, consistency constraint mechanism, structural rationality guarantee mechanism, and interaction timing change mechanism in the mutation data generation module are respectively used to ensure that the data packet size is legal, the field logic is consistent, the structure is correct, and the timing scenarios of multiple clients are covered.

[0077] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A message middleware testing method based on multi-client simulation, characterized in that, include: Extract client behavior from the actual interaction records of the message middleware, including the behavior of message producers and message consumers; The interactive messages are parsed according to the message type and timing to obtain the message structure and attribute information of each field, and a field description list is constructed based on the attribute information. Based on the semantic information and contextual relationships in the message protocol specification, at least four key mechanisms for guiding mutation strategies are designed, including a self-restriction mechanism, a consistency constraint mechanism, a structural rationality guarantee mechanism, and an interaction timing change mechanism; combined with the field description list, a mutation data packet conforming to semantic and format specifications is generated. Simulate multiple logically independent clients within a single test process, send the mutated data packets to the message middleware according to the orchestrated interaction sequence, and monitor the running status of the message middleware and business layer anomalies during the message interaction process.

2. The method according to claim 1, characterized in that, The attribute information of the field includes the field name, field value, type, label, length, and semantic constraints; the semantic constraints include the logical scope of the field, the dependencies between fields, and the positional requirements of the field in the interaction sequence.

3. The method according to claim 1, characterized in that, The self-limiting mechanism ensures that the size of the mutated data packet does not exceed the legal range specified by the message protocol; the consistency constraint mechanism maintains the predefined logical relationships between fields to avoid generating mutated data that violates field dependencies; the structural rationality guarantee mechanism guides the correct parsing of the data packet structure based on key fields to ensure the structural integrity of the mutated data packet; and the interaction timing change mechanism adjusts the message sending order among multiple clients to cover the internal processing logic of the message middleware under different timings.

4. The method according to claim 1, characterized in that, The generation of the variant data packet conforming to semantic and format specifications further includes: performing a variant operation on a single field or a combination of multiple fields based on the field description list, wherein the variant operation includes at least one of replacement, insertion, deletion, boundary value testing, and random fuzzing.

5. The method according to claim 1, characterized in that, The business layer anomalies in the monitoring message interaction process include at least one of the following: message loss, message duplication, message order disorder, transaction consistency violation, and reception confirmation anomaly.

6. The method according to claim 1, characterized in that, Before extracting client behavior from real interaction records, the process also includes: inserting monitoring code into the message middleware through instrumentation to capture runtime errors or crash information.

7. The method according to claim 1, characterized in that, The method also includes recording the mutation configuration, message interaction logs, middleware running status, and system snapshots when an exception is triggered, in order to support defect reproduction and root cause analysis.

8. A message middleware testing system based on multi-client simulation, characterized in that, include: The interactive message parsing module is used to extract client behavior from the actual interaction records of the message middleware, parse the message structure and attribute information of each field according to the message type and timing, and construct a list of field descriptions. The mutation data generation module is used to design self-restriction mechanisms, consistency constraint mechanisms, structural rationality guarantee mechanisms, and interaction timing change mechanisms based on the semantic information and contextual relationships in the message protocol specification, and generate mutation data packets that conform to semantic and format specifications in combination with the field description list; The multi-client simulation execution module is used to simulate multiple logically independent clients within a single test process and send the mutated data packets to the message middleware according to the orchestrated interaction sequence. The anomaly monitoring module is used to monitor the running status of the message middleware and business layer anomalies during message interaction.

9. The system according to claim 8, characterized in that, The anomaly monitoring module is also used to record the mutation configuration, message interaction logs, middleware running status, and system snapshots when an anomaly is triggered.

10. The system according to claim 8, characterized in that, In the mutation data generation module, the self-limitation mechanism, consistency constraint mechanism, structural rationality guarantee mechanism, and interaction timing change mechanism are respectively used to ensure that the data packet size is legal, the field logic is consistent, the structure is correct, and the timing scenarios of multiple clients are covered.

Images