Test method based on multi-modal context protocol and three-stage verification mechanism

Abstract

The application relates to a test method based on a multi-modal context protocol and a three-stage verification mechanism, and the test method comprises the following steps: capturing multi-modal data related to a user operation sequence, system state data and external events in real time; performing multi-modal fusion on the multi-modal data by an MCP engine to construct a dynamic context graph; performing three-stage verification based on the multi-modal data and the context graph; dynamically generating test cases, intelligently adjusting a test strategy; and analyzing and generating repair suggestions. The application can improve test case generation efficiency and coverage, reduce manual intervention, realize cross-application and cross-system test case migration and execution, enhance defect detection rate, reduce false positive rate, provide an intelligent and automated test process, reduce the test threshold, and is especially suitable for intelligent test processes of complex scenes such as financial transactions, industrial control and Internet of Things devices.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of software testing, and particularly relates to a test method based on a multi-modal context protocol and a three-stage verification mechanism. BACKGROUND

[0002] In the field of software testing, the traditional test scheme has many limitations. For example, the test method relying on the preset rule library cannot capture the user operation sequence and system state change in real time, resulting in insufficient test case coverage. At the same time, the AI test scheme of generating test cases through a single model is limited by the model cognitive boundary, and the defect detection rate is low. In addition, the interface status code, database transaction, log time sequence and other verification links are independently executed, and cannot form causal relationship, resulting in increased defect positioning time. SUMMARY

[0003] In order to solve the problems in the prior art, the present application provides a test method based on a multi-modal context protocol and a three-stage verification mechanism, comprising the following steps:

[0004] Step S1: Real-time capture of multi-modal data related to user operation sequence, system state data and external events through sensor nodes deployed in the test environment;

[0005] Step S2: Real-time transmission of multi-modal data to the MCP engine, and multi-modal fusion of the MCP engine on the multi-modal data to construct a dynamic context graph;

[0006] Step S3: Three-stage verification based on multi-modal data and the context graph;

[0007] Step S4: Dynamic generation of test cases and intelligent adjustment of test strategies by the self-evolution test framework based on the context graph and the three-stage verification results;

[0008] Step S5: Analysis of the test results of the self-evolution test framework and generation of repair suggestions.

[0009] In step S1, the deployed sensor nodes include API monitoring probes, database log collectors and user operation recording plug-ins, and the multi-modal data captured by the API monitoring probes, the database log collectors and the user operation recording plug-ins and the capture methods are as follows:

[0010] API monitoring probe: through the eBPF-based bypass monitoring technology, the API request / response data packets related to API call in and out of the test environment are captured in the kernel layer; and by integrating the OpenTelemetry standard protocol, the mainstream APM tool is compatible, the structured data is obtained through application layer burying, and distributed cross-service link tracking is realized;

[0011] Database log collector: Different capture methods are used for different database systems: for MySQL / MariaDB database systems, binary log parsing is performed to capture complete data change event logs through ROW mode; for Oracle database systems, the LogMiner tool is used to extract redo logs, and the V$LOGMNR_CONTENTS view is used to parse SQL_REDO and SQL_UNDO operation logs; for MongoDB database systems, the oplog.rs collection is listened to to capture replica set change event logs;

[0012] User operation recording plug-in: JavaSript code is injected into the browser to capture native component interaction events.

[0013] Among them, the API monitoring probe captures API request / response data packets in the kernel layer, and also performs the following operations:

[0014] SSL / TLS traffic is decrypted through dynamic certificate replacement technology to obtain plaintext data corresponding to the complete structure of the API request / response data packet, including request header, parameter body, status code, and response timestamp;

[0015] Based on regular expressions and machine learning models, sensitive fields are dynamically identified, and real-time desensitization is achieved using AES-256 encryption or hash replacement technology;

[0016] White list / black list filtering is achieved through a configurable rule engine, and only API request / response data packets related to API calls in the test scenario are retained.

[0017] Among them, the database log collector uses a two-phase commit protocol and a unique incremental sequence number binding mechanism when capturing logs, and only captures logs with a unique incremental sequence number greater than or equal to the transaction commit bit.

[0018] Among them, when the user operation recording plug-in captures native component interaction events by injecting JavaSript code into the browser, it includes:

[0019] By overwriting the prototype in the JavaSript code, the native API is overwritten to intercept XHR / Fetch requests for request / response full-link data capture;

[0020] By performing event proxying and deep listening in the JavaSript code, DOM event listening is achieved to track asynchronous operation chain data;

[0021] By using the MutationObserver API in the JavaSript code to listen to the structure changes of the DOM, dynamic content loading capture is achieved.

[0022] In the step S3, based on the multi-modal data and the context graph, three-stage verification is performed, including:

[0023] Converting the user operation involved in the native component interaction event of the user into a standardized operation sequence;

[0024] Generating a device feature fingerprint through Device Fingerprinting technology and binding the operation sequence.

[0025] In the step S3, based on the multi-modal data and the context graph, three-stage verification is performed, including:

[0026] In the step S31, interface layer verification: performing state code verification and content compliance verification on the plaintext data of the API request / response data packet captured by the API monitoring probe;

[0027] In the step S32, data layer verification: comparing the logs captured by the database log collector and the API calls captured by the API monitoring probe to verify transaction atomicity;

[0028] In the step S33, log layer verification: performing timestamp alignment on the system logs in the multi-modal data to detect abnormal operation sequences.

[0029] In the step S2, when the multi-modal data is transmitted to the MCP engine in real time, each sensor node first converts the original multi-modal data into a unified format defined by the MCP protocol, and then transmits the data to the MCP engine in real time through the Kafka message queue.

[0030] In the step S1, the sensor node dynamically adjusts the capture frequency and strategy of the multi-modal data according to the test scenario.

[0031] In the step S2, when the MCP engine performs multi-modal fusion on the multi-modal data to construct a dynamic context graph, it also generates a multi-dimensional context feature vector combining device fingerprint information, which is used to support the construction and causal reasoning of the dynamic context graph.

[0032] The application integrates multi-modal context protocol (MCP), dynamic anomaly injection and multi-model collaborative verification mechanism, which can improve the test case generation efficiency and coverage, reduce manual intervention, realize cross-application and cross-system test case migration and execution, enhance defect detection rate, reduce false positive rate, provide intelligent and automated test process, and reduce test threshold. It is especially suitable for intelligent test process of complex scenarios such as financial transactions, industrial control and Internet of Things devices. DETAILED DESCRIPTION

[0033] In order to have a further understanding of the technical solutions and beneficial effects of the present application, the technical solutions of the present application and the beneficial effects generated thereby are described in detail below.

[0034] In order to improve the test case generation efficiency and coverage, reduce manual intervention, realize cross-application and cross-system test case migration and execution, enhance defect detection rate, reduce false positive rate, and provide intelligent and automated test process, reduce the test threshold, the present application realizes the context dynamic transmission of the test scene and the cross-system test case migration by constructing a multi-modal context protocol (MCP).

[0035] The test method based on the multi-modal context protocol and the three-stage verification mechanism provided by the present application comprises the following steps S1-S4.

[0036] Step S1: Real-time capture of multi-modal data related to user operation sequences (such as order submission processes in financial transactions), system state data (such as server load, memory occupancy), and external events (such as network delay fluctuations) through sensor nodes deployed in the test environment.

[0037] The sensor nodes include API monitoring probes, database log collectors, and user operation recording plugins, and the multi-modal data captured by each of the API monitoring probes, database log collectors, and user operation recording plugins and the capture method are described as follows.

[0038] 1. API monitoring probe

[0039] (1) Adopt bypass listening technology based on eBPF (extended Berkeley Packet Filter) to capture API request / response data packets related to API calls in and out of the test environment at the kernel layer, avoiding performance loss to the business system.

[0040] The bypass listening technology based on eBPF supports HTTP / HTTPS protocol analysis, decrypts SSL / TLS traffic through dynamic certificate replacement technology (MITM mode), and extracts clear data corresponding to the complete structure of API request / response data packets in real time.

[0041] Decryption of SSL / TLS traffic is a kind of encryption / decryption security measure, and API request / response data packets are the contents that need to be encrypted / decrypted, and the decrypted plaintext data directly corresponds to the complete structure of API request / response, including: request header (such as User-Agent, Content-Type), parameter body (such as JSON format business parameter), status code (such as HTTP 200 indicating success), response timestamp and other metadata. Therefore, the application integrates the SSL / TLS decryption capability to the non-intrusive monitoring probe, breaking through the limitation that the traditional API monitoring cannot process encrypted traffic.

[0042] (2) Integrate OpenTelemetry standard protocol, compatible with mainstream APM tools (such as Jaeger, Zipkin), obtain structured data through application layer burying, and realize distributed cross-service link tracking.

[0043] APM (Application Performance Management) tools such as Jaeger and Zipkin are mainly used for distributed link tracking. It mainly realizes cross-service link tracking and performance analysis through standardized data collection and transmission mechanism. APM and API request / response data packet capture are complementary to each other. APM obtains structured data (such as request path, parameter, response state) through application layer burying, realizes link tracking, performance index collection and error diagnosis, and APM depends on code or framework integration. API request / response data packet capture obtains HTTP header, load content and other information through bypassing traffic mirroring (such as tcpdump), which belongs to non-intrusive monitoring. APM and API request / response data packet capture are both for the purpose of improving system observability, realizing end-to-end fault diagnosis, performance record collection and other goals.

[0044] Finally, the API monitoring probe of the application simultaneously supports intelligent filtering and sensitive data desensitization and adaptive sampling strategy.

[0045] Intelligent filtering and sensitive data desensitization: based on regular expression and machine learning model (such as TF-IDF keyword extraction) to dynamically identify sensitive fields (such as user password, ID card number), and realize real-time desensitization by using AES-256 encryption or hash replacement technology. Through the configuration rule engine (such as Drools), white list / black list filtering is realized, and only API request / response data packets related to test scenarios (such as only capturing POST requests of payment interface) are reserved.

[0046] Adaptive sampling strategy: dynamically adjust the sampling rate according to system load (e.g. automatically reduce the sampling frequency to 10% when CPU usage > 80%), balance monitoring granularity and performance overhead. Support differentiated monitoring based on business priority (e.g. full sampling for core transaction interfaces, 1% sampling for auxiliary query interfaces).

[0047] At the same time, the sampling strategy can also be adjusted based on the linkage with the MCP engine: the MCP engine dynamically adjusts the sampling priority of the API monitoring probe according to the context graph of the current test scenario (e.g. user operation sequence, interface call chain). For example, when a high-risk operation (e.g. large amount transfer) is detected, the sampling rate of the related API is automatically increased. The sampling data is injected into the MCP engine in real time to update the event timing and data flow dependency in the context graph. If the sampling data is missing, causing the context graph to be incomplete, the MCP engine will trigger the sampling strategy compensation mechanism.

[0048] 2. Database log collector

[0049] Some systems, such as financial transaction systems, use different databases simultaneously, so the present application uses different capture methods for different database systems:

[0050] For MySQL / MariaDB database systems: parse binary logs (binlogs), capture complete data change event logs (INSERT / UPDATE / DELETE) through ROW mode, support GTID (Global Transaction Identifier) to realize breakpoint resume;

[0051] For Oracle database systems: extract redo logs through the LogMiner tool, and parse SQL_REDO and SQL_UNDO operation logs by combining the V$LOGMNR_CONTENTS view;

[0052] For MongoDB database systems: listen to the oplog.rs collection, capture replica set change event logs, and support BSON format parsing and field-level difference comparison.

[0053] The database log collector of the present application converts three kinds of log formats through an adaptation layer, converts them into a unified change event stream (including operation type, target data, timestamp), and injects them into the MCP engine to build a dynamic context graph (details below).

[0054] The application binds a two-phase commit protocol with a unique incremental sequence number LSN (Log Sequence Number) of a database log mechanism, only captures logs with a unique incremental sequence number greater than or equal to a transaction commit bit, and ensures that the captured logs are strictly consistent with the transaction commit state.

[0055] The two-phase commit protocol (2PC) is a distributed transaction coordination protocol used to ensure that operations across multiple nodes are either all successfully committed or all rolled back, thereby guaranteeing the data consistency of a distributed system. The two phases are a preparation phase and a commit phase. In the application, the application scenario is when an operation involves multiple databases (such as a MySQL order database + an Oracle account database), and 2PC ensures that: all nodes are committed → the transaction takes effect; any node fails → all are rolled back.

[0056] The unique incremental sequence number LSN is a sequence number assigned by the database to each log to mark the write order.

[0057] The binding of 2PC and LSN means that: (1) state synchronization mechanism: after a transaction is committed through 2PC, the database writes a commit record to the log and assigns an LSN, which strictly corresponds to the persistence state of the transaction; (2) collector behavior: the log collector only captures logs with an LSN greater than or equal to the transaction commit bit, avoiding reading uncommitted intermediate states (such as temporary data before rollback).

[0058] Therefore, compared with the existing method of directly reading the latest log, there is a disadvantage that it may contain uncommitted transactions and additional transaction state verification is required. The application triggers collection through 2P2 commit events, ensures that the LSN corresponds to the committed state, and eliminates the collection of uncommitted or rolled back transactions (such as partially successful orders in financial transactions), thereby naturally avoiding the risk of inconsistent states. Through LSN breakpoint resuming, the defect of repeated collection is also avoided. This scheme is particularly suitable for scenarios such as finance and the Internet of Things, which require high consistency across systems.

[0059] For the log data captured by the database log collector, the application uses the Checkpoint mechanism to periodically persist the offset, avoiding log duplication or loss due to network interruption; for continuously repeated log events (such as heartbeat queries), RLE (Run-Length Encoding) compression is used to reduce network bandwidth occupancy. Using the Protobuf serialization protocol instead of JSON reduces the data volume by more than 30%.

[0060] 3、User Behavior Recorder

[0061] Support mobile terminal Hybrid App monitoring, through injecting JavaSript code in the browser, capturing native component interaction events through WebViewJavaScript Bridge, including intercepting XHR / Fetch requests, implementing DOM (Document Object Model, same below) event listening and DOM structure change listening, the implementation mechanism is as follows:

[0062] By prototype overriding and proxy mode in JavaSript code, overriding native API to intercept XHR / Fetch request to capture request / response full-link data, thereby capturing network request and response;

[0063] By event proxying and deep listening in JavaSript code to implement DOM event listening, realizing asynchronous operation chain data tracking, thereby recording user interaction behaviors such as clicking, inputting, etc.

[0064] By using MutationObserver API in JavaSript code to listen to the structure change of DOM (such as node insertion / deletion, attribute change, text content modification, subtree change, SPA routing jump, asynchronous rendering), dynamic content loading capture is realized.

[0065] Therefore, the user operation recording plug-in of the present application realizes front-end operation full-link monitoring through cross-technology cooperation.

[0066] Taking the financial transaction test process as an application scene example, one capture example is as follows: (1) User operation: DOM listening captures clicking "transfer" button → records element path button#transfer; (2) Request interception: intercepts POST / transfer request → desensitizes field amount= ; (3) DOM verification: MutationObserver detects successful pop-up rendering → generates operation chain: [clicking transfer → calling API → displaying result]; (4) Three-order verification (detailed below): data layer checks database balance change → log layer analyzes operation time sequence legality.

[0067] In the present application, after the user operation recording plug-in captures the native component interaction event, the following (1)-(3) processing is also performed.

[0068] (1) Operation serialization and association with context

[0069] The user's native component interaction event is converted into a standardized operation sequence, recording operation type, target element, input value, timestamp, etc. Device Fingerprinting technology is used to generate device feature fingerprints (including browser version, screen resolution, User-Agent, operating system type, etc.), which are bound to the operation sequence.

[0070] (2) Asynchronous operation synchronization processing

[0071] Promise chain tracking is used for AJAX requests to ensure that the user operation sequence of the operation sequence is consistent with the actual execution sequence.

[0072] AJAX requests are asynchronous, and multiple requests may be completed out of order due to network delays and different server response speeds. For example, if a user clicks a button three times to trigger three AJAX requests, the third request may return a result first, causing the interface display order to be inconsistent with the user operation order. The.then() method of Promise will be executed in the order of writing, and the operation in the next.then() will only be triggered after the previous Promise is completed (resolve), ensuring that the execution sequence is consistent with the code writing order.

[0073] The user operation sequence refers to the physical order of multiple user operations (such as the order of clicking buttons, inputting, etc.). For example, the user clicks "Save" first and then "Submit", and the corresponding AJAX requests for these two operations need to be executed in this order.

[0074] Actual execution sequence: In an asynchronous environment, without constraints, the "Submit" request may be completed first, and the "Save" request may be completed later, causing logical errors (such as submitting without saving). Through the Promise chain, the processing logic of the "Save" request is ensured to be completely executed before the processing logic of the "Submit" request, ensuring that the execution result is consistent with the user operation sequence.

[0075] (3) Real-time playback of operation sequences through WebSocket long connections, supporting operation migration across domain test environments.

[0076] Step S2: Real-time transmission of multi-modal data to the MCP engine, which performs correlation analysis and multi-modal context fusion based on a time window (e.g., 5 seconds) to construct a dynamic context graph and generate causal chains. At the same time, multi-dimensional context feature vectors are generated based on device fingerprint information to dynamically support the construction of the context graph and causal reasoning.

[0077] Specifically, each sensor node first converts the original multi-modal data into a unified format defined by the MCP protocol, and transmits it to the MCP engine in real time through the Kafka message queue. Schema Registry is used to manage data version compatibility and avoid parsing failures caused by field changes.

[0078] The context feature vector is a structured representation carrier formed after multi-modal data fusion. Its core role is to quantify the environmental dependence and behavior patterns of user operations through multi-dimensional features, supporting the construction of dynamic context graph and causal reasoning. The context feature vector plays a core role in the "multi-modal data structured mapping and causal relationship quantization" in the present application. It supports the construction of dynamic context graph, causal reasoning, anomaly detection and business decision-making by vectorizing the environmental dependence, system event logic consistency and device features of user operations. It runs through key links such as multi-modal fusion, causal chain construction and anomaly detection, and is the technical cornerstone of the MCP engine to realize "operation-event-environment" full-link correlation analysis.

[0079] The method for the MCP engine to perform correlation analysis and multi-modal fusion on multi-modal data involves the following: obtaining the user's click "submit order" operation from the multi-modal data, aligning the operation with the / order / submit request of the specific business endpoint corresponding to the API call monitored by the API monitoring probe through transaction ID and time sequence, and associating the timestamp with the "database deduction record" and "order status change" events in the database log to form a causal chain of "user clicks'submit order' -> interface call -> database deduction -> payment response". Obtain the transaction ID from the multi-modal data, and associate the database log deduction record with the response data of the payment interface call through the transaction ID to form a complete operation track.

[0080] In the present application, the "API request / response data packet", "API call" and " / order / submit request" constitute a technical hierarchical relationship from abstract to specific, and their association can be expanded through the logical chain of "data flow - operation behavior - business scenario":

[0081] API Invocation: refers to the operation behavior of an application interacting with a backend service or third-party system through a programming interface (API). For example, the front end calls the order submission interface and payment interface of the back end. It is the behavior subject of monitoring, focusing on "who calls what, when, and the result of the call".

[0082] API Request / Response Payload: refers to the specific data content transmitted during API calling, including request parameters (such as user ID, product ID, amount), response results (such as success / failure status code, returned order number), header information (such as authentication Token, content type), etc. It is the data carrier of monitoring, focusing on "what specific data is called, and whether the data is compliant".

[0083] / order / submit request: is a specific API endpoint (Endpoint) or operation path, which belongs to the specific implementation of the business scenario level. For example, in an e-commerce system, " / order / submit" may correspond to the business operation of "submitting an order", and its request data packet may contain a list of goods, a shipping address, a payment method, etc., and the response data packet may contain an order ID, a payment link or error information.

[0084] "API Request / Response Payload" is the data content of API calling, "API calling" is the behavior process, and " / order / submit request" is a specific business endpoint. The three together constitute the complete logic of API monitoring: by monitoring the API calling behavior and its data packet content, implementing business scenario-based monitoring rules for specific endpoints (such as " / order / submit"), ultimately realizing the guarantee of system stability, data compliance and business logic correctness.

[0085] In the present application, the transaction ID is a globally unique identifier in a distributed transaction or cross-system operation, used to bind a group of logically related operations (such as deduction, payment, status update), ensuring that these operations are traceable and associated in multiple systems / modules. For example, in an e-commerce payment scenario, after a user submits an order, the system generates a transaction ID (such as tx_id=TX001), which accompanies the entire payment process: from triggering a payment request on the front end, to deducting money in the back-end database, to the response callback of the payment interface (such as Alipay / WeChat). When the system performs a deduction operation, the database log records the operation, including the transaction ID, operation time, deduction amount, user account, etc. There may be multiple deduction records in the same database log, but each record corresponds to a different transaction ID; and the same transaction ID may be associated with multiple database log entries (such as deduction, status update) and payment interface responses.

[0086] After building the dynamic context graph, the MCP engine also performs the following operations:

[0087] 1. Dynamically adjust sensor granularity according to test scenario: enable full sampling for high-risk scenarios (such as large-value transfer), and use sampling mode for low-risk scenarios (such as page browsing);

[0088] 2. Generate MCP protocol files based on the context graph, including test scenario constraint rules (such as "order submission must trigger payment interface call"), data flow dependency relationships (such as "user ID must be consistent in database session"), and exception injection thresholds (such as "trigger retry mechanism when network delay exceeds 500ms");

[0089] 3. Store optimization of multi-modal data and context graph:

[0090] Use a time series database (such as InfluxDB) to store data captured by API monitoring probes and data log collectors, both of which are associated through the "timestamp + tag" mechanism of the time series database, supporting core functions such as full-link tracking, exception diagnosis, and performance optimization. This design is particularly critical in scenarios that require strict sequential dependencies and real-time analysis (such as payment systems and order processing);

[0091] Use a document database (such as MongoDB) to store the context graph and the operation sequences captured by the user operation recording plug-in.

[0092] Step S3: Three-stage verification based on multi-modal data and context graph, including interface layer verification, data layer verification, and log layer verification, which are described in detail as follows.

[0093] 1. Interface layer verification

[0094] As mentioned above, the interface return value of the API monitoring probe is the clear data of the API request / response data packet it captures, including request header, parameter body, status code, response timestamp, etc. Interface layer verification first checks the status code (such as HTTP 200 / 404 / 500), and at the same time detects the compliance of the response content (such as JSON field missing, data type error) through semantic analysis.

[0095] The role of interface return value in the construction of context graph and subsequent verification process is reflected in the following three levels of technical association:

[0096] Causal chain construction: In the context graph causal chain construction process, the transaction ID (such as tx_id=TX001) in the response content is matched with the transaction ID in the database log and user operation record, forming a causal chain of "user clicks'submit order' -> interface call -> database deduction -> payment response". Therefore, the "order status change" node in the graph can be associated with the payment_status field returned by the interface through the transaction ID, and then in the verification phase, it needs to verify whether the status change is consistent with the interface response;

[0097] Multi-dimensional feature fusion: The device fingerprint information in the response content (such as browser version, operating system type) is fused with the device dimension feature vector in the graph to form the environmental dependence representation of user operation. Then in the verification phase, if the response content shows that the user submits an order using an old version of the browser, and the device fingerprint feature vector in the graph can be associated, then the compatibility verification logic is triggered;

[0098] Abnormal diagnosis and decision support: The semantic analysis results of the response content (such as error code, abnormal field) are associated with the business rule nodes in the graph to support abnormal detection and decision. Therefore, if the interface returns a 404 error, the graph can locate the corresponding user operation and system log through the transaction ID to diagnose whether it is caused by routing error or resource non-existence; if the amount field in the response content is inconsistent with the order amount, the graph can trigger the amount verification rule to determine that it is a business logic exception.

[0099] 2、Data layer verification: Compare the logs captured by the database log collector and the API calls captured by the API monitoring probe to verify transaction atomicity (such as "the deduction operation must be accompanied by order status update").

[0100] As described above, the database log collector captures event logs in real time, recording atomic operations at the database level (such as UPDATE account SET balance=..., INSERT INTO order_status...). Such logs contain transaction ID (such as tx_id=TX001), operation timestamp, affected row number, SQL statement, etc. metadata, which are the physical carriers of transaction atomicity.

[0101] The API calls captured by the API monitoring probe can form the calling sequence, parameters, response status and transaction ID of the API interface. For example, in the "submit order" process, the interface calling sequence may include: calling the / order / submit interface (carrying order information); calling the / payment / charge interface (executing deduction); calling the / order / update_status interface (updating the order status to "paid").

[0102] These interface calls are associated through transaction ID (such as TX001) to form a logical operation sequence.

[0103] The core of transaction consistency check is to verify the logical consistency of database operation logs and interface calling sequence, which is implemented through the following mechanisms:

[0104] Transaction ID concatenation: The charge record in the database operation log (e.g., balance update of tx_id=TX001) must be associated with the charge interface call in the interface call sequence (e.g., / payment / charge?tx_id=TX001), ensuring that the "charge operation" belongs to the same transaction at the database level and the interface level.

[0105] Timestamp alignment: The charge timestamp in the database operation log (e.g., 2025-09-04 14:30:05) must be later than the timestamp of the "submit order" request in the interface call sequence (e.g., 2025-09-04 14:30:00) and earlier than the timestamp of the order status update (e.g., 2025-09-04 14:30:10), forming a "request-charge-status update" time sequence chain.

[0106] Business rule verification: By comparing the order status change in the database operation log (e.g., status from "to be paid" to "paid") with the status update request in the interface call sequence (e.g., / order / update_status?status=paid), the transaction atomicity is verified (e.g., "charge operation must be accompanied by order status update").

[0107] In the data layer verification process, if the database operation log shows that the charge is successful but the interface call sequence lacks the order status update request, the graph can mark the transaction as "inconsistent state exception", triggering an alarm or rollback mechanism.

[0108] In the data layer verification process, data lineage tracking is supported, and knowledge graph technology is used to extract core subgraphs from the context graph. The core subgraph is a "parameter-transmission-storage" full-link view formed by associating interface parameters, transmission status, and storage fields through transaction ID and timestamp. Its acquisition details rely on the cooperation of API monitoring, database log collection, knowledge graph construction, and other technical modules. Identifying data tampering risks is achieved through rule engine verification, anomaly detection algorithms, and risk response mechanisms. This design realizes end-to-end tracing from user operations to database changes, ensuring that data is traceable and verifiable from the interface to storage. It is the core technical means to realize data security and compliance in this invention.

[0109] In addition, for payment scenarios, the context graph can analyze user operation environment combined with device fingerprint information. If abnormal device characteristics (e.g., high-frequency operation + device fingerprint change) are detected, suspicious transactions can be blocked or require secondary verification.

[0110] 3. Log layer verification: Timestamp alignment on system logs in multi-modal data to detect abnormal operation sequences. Discover implicit associations between log events (e.g., "disk IO peaks always accompany cache invalidation events") through the Apriori algorithm.

[0111] In this invention, system logs focus on global event tracking, database logs focus on data consistency, and database operation logs are refined to specific SQL operations.

[0112] In this invention, the Bayesian network integrates the results of the above three layers of verification to achieve accurate positioning of defect probability distribution.

[0113] Step S4: The self-evolution test framework dynamically generates test cases and intelligently adjusts test strategies based on the context graph and the three-stage verification results. Specifically, it includes steps S41-S43.

[0114] Step S41, abnormal injection strategy generation: Based on reinforcement learning models (such as DQN algorithm), dynamically adjust abnormal injection parameters according to historical test data, break through the fixed threshold limit of traditional testing. For example, after 10 consecutive tests without finding defects, automatically increase the network packet loss rate to 15% to break through the regular test boundary.

[0115] In this process, a cold start processing mechanism and a failure detection and recovery mechanism are used.

[0116] Cold start processing: Use transfer learning technology to import pre-trained parameters from historical models in similar test scenarios, reducing the amount of training data required for cold start. In the early stage of model training, load the artificially defined expert rule library (such as "trigger retry mechanism when network packet loss rate > 10%") as a fallback strategy for decision-making.

[0117] Failure detection and recovery: Deploy model behavior monitoring probes to detect abnormal decision patterns (such as selecting low-yield actions for 10 consecutive times) in real time. When model failure is detected, trigger the three-stage verification co-processor to perform root cause analysis, and perform recovery operations (such as rolling back to the last stable version of model parameters) based on the analysis results.

[0118] Step S42, knowledge graph enhancement: Inject defect positioning results (such as "specific parameter combination causes core module crash") into the domain knowledge graph, and infer potential associated defects (such as "similar parameter boundary values may cause memory leaks") through graph neural networks (GNN).

[0119] Step S43, test case self-migration: when a cross-system scenario is detected (such as "order service calls payment gateway"), the MCP engine automatically encapsulates the current test context (user session ID, authentication token, etc.) as a migration package, and passes it to the downstream system through a standardized interface (such as a RESTful API), realizing seamless reuse of test cases.

[0120] In this process, the interface parameter adaptation principle is as follows:

[0121] The migration package contains the source system interface parameter template (such as {"order_id": "string", "amount": "number"} ) and parameter mapping rules (such as source.order_id → target.tx_id) ;

[0122] After receiving the migration package, the target system loads the target interface definition through the interface protocol parser, and dynamically adapts the interface parameters according to the parameter mapping rules;

[0123] For missing mandatory fields, extract the default value from the context metadata of the migration package (such as extracting user_agent from device fingerprint).

[0124] Step S5: analyze the test results of the self-evolution test framework and generate repair suggestions.

[0125] Defect localization: the third-order verification coprocessor integrates the verification results of each layer, calculates the defect probability distribution through Bayesian network, and locates the most likely fault point (such as "payment callback interface timeout causes order status not to be updated").

[0126] Repair suggestion generation: the self-evolution framework queries the historical defect library (containing 5000+ repair schemes), and generates targeted repair suggestions (such as "add Redis lock to prevent concurrent deduction") combined with the current context characteristics (such as programming language, framework version).

[0127] Test report generation: automatically generate an interactive report containing test coverage heat map, defect trend analysis, and repair priority sorting, and support filtering key information through natural language query (such as "show high-risk defects in the past 24 hours").

[0128] The present application can achieve the following technical effects:

[0129] 1. Efficiency improvement: through the automatic context modeling of the MCP engine, the time for manually writing test cases is reduced, and the test case generation speed is improved by 50%: through the three-order verification mechanism, the multi-dimensional coverage of interface, data, and log is realized, the test blind area is reduced, and the regression test coverage rate is increased from 75% to 92%;

[0130] 2. Precision enhancement: Reduce the missed detection rate by multi-model collaborative verification (such as interface status code verification + data bloodline tracking + log time sequence analysis), so that the defect detection rate is improved by 40%; Bayesian network integrates multi-source verification results to filter false defect alarms, reducing false alarm rate by 25%;

[0131] 3. Intelligent expansion: Reduce the test threshold by filtering test reports through natural language queries (such as "display high-risk defects in the past 24 hours"), support non-technical personnel to participate in the test process; Through MCP engine automatically encapsulates test context (such as user session ID, authentication token), realizes seamless reuse across systems, so that the test case self-migration efficiency is improved by 60%;

[0132] 4. System load optimization: Reduce 30% performance overhead by dynamically adjusting the sampling rate of API monitoring probes according to system load (such as automatically reducing frequency when CPU usage > 80%); Reduce network bandwidth occupancy by 50% through RLE compression and Protobuf serialization protocol optimization for log transmission.

[0133] Although the present application has been described by the above preferred embodiments, it is not intended to limit the scope of protection of the present application, and any person skilled in the art can make various changes and modifications to the above embodiments without departing from the spirit and scope of the present application. Therefore, the protection scope of the present application is defined by the claims.

Claims

1. A test method based on multi-modal context protocol and three-stage verification mechanism, characterized in that, Includes the following steps: Step S1: Capture multimodal data related to user operation sequences, system status data, and external events in real time by deploying sensor nodes in the test environment. The deployed sensor nodes include API monitoring probes, database log collectors, and user operation recording plugins. Step S2: Transmit multimodal data to the MCP engine in real time. The MCP engine performs multimodal fusion on the multimodal data to construct a dynamic context graph. Step S3: Perform three-stage validation based on multimodal data and context graph, including: Step S31, Interface Layer Verification: Perform status code verification and content compliance verification on the plaintext data of API request / response packets captured by the API monitoring probe; Step S32, Data Layer Verification: Compare the logs captured by the database log collector with the API calls captured by the API monitoring probe to verify the atomicity of the transaction; Step S33, Log layer verification: Timestamp alignment is performed on the system logs in the multimodal data to detect abnormal operation sequences; Step S4: The self-evolving testing framework dynamically generates test cases and intelligently adjusts testing strategies based on the context graph and the three-phase verification results. Step S5: Analyze the test results of the self-evolution test framework and generate repair suggestions.

2. The test method based on multi-modal context protocol and three-stage verification mechanism as claimed in claim 1, wherein, In step S1, the multimodal data captured by the API monitoring probe, the database log collector, and the user operation recording plugin, and the capture methods are as follows: API monitoring probe: It captures raw network packets related to API calls—API request / response data packets—entering and leaving the test environment at the kernel level through eBPF-based bypass listening technology; and, by integrating the OpenTelemetry standard protocol, it is compatible with mainstream APM tools and obtains structured data through application layer instrumentation to achieve distributed cross-service link tracing. Database log collector: Different capture methods are used for different database systems: For MySQL / MariaDB database systems, binary log parsing is performed to capture complete data change event logs in ROW mode; for Oracle database systems, redo logs are extracted using the LogMiner tool, and SQL_REDO and SQL_UNDO operation logs are parsed using the V$LOGMNR_CONTENTS view; for MongoDB database systems, replica set change event logs are captured by listening to the oplog.rs collection. User interaction recording plugin: Captures native component interaction events by injecting JavaScript code into the browser.

3. The testing method based on multimodal context protocol and three-phase verification mechanism as described in claim 2, characterized in that, After capturing API request / response packets at the kernel level, the API monitoring probe also performs the following operations: Decrypt SSL / TLS traffic using dynamic certificate replacement technology to obtain plaintext data corresponding to the complete structure of API request / response data packets, including request headers, parameter bodies, status codes, and response timestamps; Based on regular expressions and machine learning models, sensitive fields are dynamically identified, and AES-256 encryption or hash replacement technology is used to achieve real-time desensitization. Through the configuration of the rule engine, whitelist / blacklist filtering is implemented to retain only API request / response packets related to the test scenario.

4. The test method based on multi-modal context protocol and three-stage verification mechanism as claimed in claim 2, wherein, When capturing logs, the database log collector uses a two-phase commit protocol and a unique incremental sequence number binding mechanism to capture only logs with a unique incremental sequence number greater than or equal to the transaction commit bit.

5. The test method based on multi-modal context protocol and three phase verification mechanism as claimed in claim 2, wherein, The user operation recording plug-in captures native component interaction events by injecting JavaScript code into the browser, including: By overwriting the prototype in JavaScript code, the native API is overwritten to intercept XHR / Fetch requests to capture request / response full-link data. By event proxying and deep listening in JavaScript code, DOM event listening is achieved to track asynchronous operation chain data. By using the MutationObserver API in JavaScript code to listen to DOM structure changes, dynamic content loading capture is achieved.

6. The test method based on multi-modal context protocol and three phase verification mechanism as claimed in claim 2, wherein, After the user operation recording plug-in captures native component interaction events by injecting JavaScript code into the browser, it also performs the following operations: Convert the user's native component interaction events into standardized operation sequences. Generate device feature fingerprints using Device Fingerprinting technology and bind them to the operation sequences.

7. The test method based on multi-modal context protocol and three phase verification mechanism as claimed in claim 1 wherein: In step S2, when transmitting real-time multi-modal data to the MCP engine, each sensor node first converts the original multi-modal data into a unified format defined by the MCP protocol and transmits it to the MCP engine in real time through the Kafka message queue.

8. The test method based on multi-modal context protocol and three phase verification mechanism as claimed in claim 1 wherein: In step S1, the sensor node dynamically adjusts the capture frequency and strategy of multi-modal data according to the test scenario.

9. The test method based on multi-modal context protocol and three phase verification mechanism as claimed in claim 1 wherein: In step S2, when the MCP engine performs multi-modal fusion on the multi-modal data to construct a dynamic context graph, it also generates a multi-dimensional context feature vector based on device fingerprint information to support the construction and causal reasoning of the dynamic context graph.