Api mirroring test method and device based on full-link traffic tracking

Abstract

This application discloses an API mirror testing method and apparatus based on end-to-end traffic tracing, relating to the field of software testing technology. The method includes: using a production fact dataset tagged with TraceID, which can completely cover the entire call path; simultaneously, the source code of the software under test, as the authoritative definition of software behavior, together with the end-to-end production fact dataset, serves as input to a large language model, providing the large language model with dual constraints of real business logic and actual operating state, thereby effectively suppressing the generation of illusory content by the large language model; furthermore, by performing business logic verification on the benchmark test program generated by the large language model in an isolated sandbox, and establishing only the verified benchmark test program as a trusted business benchmark, the authenticity and correctness of the verification rules are ensured; and then, independent verification test cases are used for admission verification, thereby improving the accuracy of API mirror testing.

Description

Technical Field

[0001] This application relates to the field of software testing technology, and in particular to an API mirror testing method and apparatus based on end-to-end traffic tracing. Background Technology

[0002] API (Application Programming Interface) testing is a health check of the APIs after a software update to ensure that each API in the updated software system can correctly receive requests, process logic, and return expected results, and that it is stable, secure, and efficient under various conditions.

[0003] Currently, API testing often combines shadow testing with LLM (Large Language Model). This involves using an LLM-driven approach to generate rules and test cases needed during testing, and then using these rules and test cases to verify the consistency of output between the old and new systems. However, testing requires collecting production environment traffic, replaying it in a mirror environment, and comparing output differences. This is problematic because collecting production environment traffic cannot comprehensively cover all data points in the testing process. This makes it easy for rules and test cases generated by LLM based on production environment traffic to deviate from actual production environment traffic, making it difficult to suppress the illusion created by LLM and introducing uncertainty into the rules and test cases generated by LLM. Therefore, the API testing approach combining LLM and shadow testing cannot meet the requirements of accurate testing.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide an API mirror testing method and apparatus based on end-to-end traffic tracing, which aims to solve the technical problem that the API testing method using LLM combined with shadow testing cannot meet the requirements of accurate testing.

[0006] Obtain the source code of the software under test and the production fact dataset marked with TraceID, collected through end-to-end traffic analysis; The large language model is invoked to generate a benchmark test program based on the source code and the production fact dataset. The benchmark test program is then validated for business logic in an isolated sandbox. The validated benchmark test program is determined as a trusted business benchmark. Based on the trusted business benchmark, the large language model is invoked to generate verification rules, and independent verification cases are used to perform admission verification on the verification rules, retaining the target verification rules that pass the verification. In a preset mirror environment, the API requests used when collecting the production fact dataset are replayed for the benchmark instance and the test instance to obtain the benchmark response and the test response; the benchmark instance is the application instance corresponding to the currently running software, and the test instance is the application instance built and running based on the source code in the mirror environment; Based on the target verification rules, the benchmark response and the tested response are compared, and the API test results are output.

[0007] In one embodiment, the steps of generating verification rules based on the trusted business benchmark by invoking the large language model, performing admission verification on the verification rules using independent verification test cases, and retaining the target verification rules that pass the verification include: The trusted business benchmark and multi-dimensional business semantic information related to the software under test are used as the basis for business semantics to drive the large language model to generate verification rules for determining whether the response difference meets business expectations. Based on the benchmark testing program, a first response that conforms to the trusted business benchmark, a second response that violates the trusted business benchmark, and a preset expected response are constructed in the isolation sandbox. For the first response, the second response, and the expected response, the verification rules are executed respectively to obtain the corresponding difference determination results; When all the difference determination results are completely consistent with the expected results corresponding to the trusted business benchmark, the verification rule is determined to have passed the independent admission verification, and the target verification rule that has passed the verification is retained.

[0008] In one embodiment, after the step of executing the verification rule for the first response and the second response respectively to obtain the corresponding difference determination results, the method further includes: When the difference determination result is not completely consistent with the expected result, the inconsistent abnormal determination result is extracted from the difference determination result; Extract the abnormal expected results corresponding to the abnormal determination results from the expected results; The anomaly determination result and the expected anomaly result are fed back to the large language model. The verification rule is optimized based on the large language model, and the independent admission verification is re-executed on the optimized verification rule until the verification passes, thereby obtaining the target verification rule.

[0009] In one embodiment, after the steps of generating verification rules by invoking the large language model based on the trusted business benchmark, performing admission verification on the verification rules using independent verification test cases, and retaining the target verification rules that pass the verification, the method further includes: The target verification rules are abstracted into Skills, and a binding relationship is established between the Skills and the generated benchmark test program and the production fact dataset. When the comparison results based on the target verification rules indicate a difference, a structured work order containing TraceID full-link context information, difference fields, and state transition comparisons is generated, and the structured work order is fed back to the test engineer. Receive manual feedback from the test engineer regarding the differences in the structural engineering document; Based on the human feedback, the large language model is invoked to optimize the benchmark test program and the verification rules, and independent admission verification is performed. The validated rules that pass the validation will be used as the new version of the Skill, and the new version of the Skill will be used as the new target validation rules for the next API test.

[0010] In one embodiment, prior to the step of invoking the large language model and generating a benchmark program based on the source code and the production fact dataset, the method further includes: Obtain the business requirements description and API interface specifications of the software under test; The step of calling the large language model and generating a benchmark program based on the source code and the production fact dataset includes: The large language model is invoked to perform business analysis on the source code, the business requirement description, and the API interface specification to obtain the business benchmark of the software under test. Based on the business benchmark, a benchmark test program is generated to verify the business behavior of the instance under test.

[0011] In one embodiment, the step of invoking the large language model to perform business analysis on the source code, the business requirement description, and the API interface specification to obtain the business benchmark of the software under test includes: The large language model is invoked to perform a differential comparison of the source code, business requirement description, and API interface specification of the software under test and the currently running software to determine the added, modified, and / or deleted functions of the software under test. Based on each function, generate corresponding business behavior descriptions; Obtain historical data for testing the currently running software, including historical trusted business benchmarks, bound Skill rule bases, and historical human feedback records; The business behavior description is correlated and compared with the historical data to generate the business benchmark of the software under test.

[0012] In one embodiment, the mirror environment has the same hardware configuration, network topology configuration, and middleware version as the actual production environment. The instance under test is deployed in the mirror environment, and an independent shadow database of the instance under test is configured. The step of replaying the API requests used when collecting the production fact dataset in a preset mirror environment for both the benchmark instance and the instance under test, and obtaining the benchmark response and the response under test; wherein the benchmark instance is the application instance corresponding to the currently running software, and the instance under test is an application instance built and running based on the source code in the mirror environment, includes: In a pre-defined mirror environment, the instance under test is used to process the full-link API requests captured from the production environment based on the shadow database, and the test response returned by the instance under test and the baseline response returned by the baseline instance are collected.

[0013] Furthermore, to achieve the above objectives, this application also proposes an API mirroring testing device based on end-to-end traffic tracing, the API mirroring testing device based on end-to-end traffic tracing comprising: The acquisition module is used to acquire the source code of the software under test and the production fact dataset marked with TraceID through end-to-end traffic collection; The first verification module is used to call the large language model, generate a benchmark test program based on the source code and the production fact dataset, perform business logic verification on the benchmark test program in an isolation sandbox, and determine the verified benchmark test program as a trusted business benchmark. The second verification module is used to generate verification rules based on the trusted business benchmark by calling the large language model, and to perform admission verification on the verification rules using independent verification cases, and retain the target verification rules that pass the verification. The replay module is used to replay the API requests used when collecting the production fact dataset in a preset mirror environment for the benchmark instance and the instance under test, and to obtain the benchmark response and the response under test; the benchmark instance is the application instance corresponding to the currently running software, and the instance under test is an application instance built and running based on the source code in the mirror environment; The output module is used to compare the benchmark response and the tested response based on the target verification rules and output the API test results.

[0014] In addition, to achieve the above objectives, this application also proposes an API mirroring test device based on end-to-end traffic tracing. The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the API mirroring test method based on end-to-end traffic tracing as described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the API mirroring test method based on end-to-end traffic tracing as described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: Using a production fact dataset tagged with TraceID ensures complete coverage of the entire call path, avoiding data bias caused by incomplete traffic sampling in traditional shadow testing. Simultaneously, the source code of the software under test, serving as the authoritative definition of software behavior, along with the full-link production fact dataset, acts as input to the large language model. This provides the large language model with dual constraints of real business logic and actual runtime state, effectively suppressing the generation of misleading content in the benchmark program that contradicts the actual behavior of the software under test. Furthermore, by performing business logic verification on the benchmark program generated by the large language model in an isolated sandbox, and establishing only verified benchmark programs as trusted business benchmarks, it ensures that verification rules are generated or established on a real and correct business semantic basis. Independent verification cases then perform admission verification, using dual filtering to ensure that the target verification rules used for response comparison have high reliability and low uncertainty, thereby improving the accuracy of API mirror testing. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an embodiment of the API mirroring testing method based on end-to-end traffic tracing in this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the API mirroring testing method based on end-to-end traffic tracing in this application; Figure 3 This is a flowchart illustrating Embodiment 3 of the API mirroring testing method based on end-to-end traffic tracing in this application; Figure 4 This is a schematic diagram of the module structure of the API mirroring testing device based on end-to-end traffic tracing in an embodiment of this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the API mirroring test method based on end-to-end traffic tracing in the embodiments of this application.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The main solution in this application embodiment is: It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions. The following uses an API tester as an example to illustrate this embodiment and the subsequent embodiments.

[0024] Based on this, the embodiments of this application provide an API mirroring testing method based on end-to-end traffic tracing, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the API mirroring testing method based on end-to-end traffic tracing in this application.

[0025] In this embodiment, the API mirroring test method based on end-to-end traffic tracing includes steps S10 to S50: Step S10: Obtain the source code of the software under test and the production fact dataset marked with TraceID collected through end-to-end traffic; It should be noted that the software under test is an upgraded or updated version of the currently running software. The source code is a collection of source files for the new or modified version of the software program to be verified. The TraceID tag is used in distributed systems to assign a globally unique identifier to each user request entering the system, and this identifier is passed in all subsequent cross-service calls following that request. The production fact dataset is a collection of real user requests tagged with TraceIDs and their corresponding complete responses, obtained after full-link traffic collection.

[0026] Understandably, by acquiring production fact datasets marked with TraceIDs from end-to-end traffic collection, it is possible to accurately and completely reproduce the actual business scenarios and call chains of online users.

[0027] In its specific implementation, the steps of the end-to-end traffic collection include: assigning a globally unique TraceID to each business request in the production environment; associating all call chain data throughout the entire lifecycle of the business request with the TraceID; collecting all input and output response data during the interaction process of the business request across systems, middleware, and subroutines; synchronously collecting snapshots of runtime parameters associated with call timestamps, system configuration parameters, and service identifiers; tracking and recording the state transition process of key fields related to business verification in the database during the execution of the business request, and generating a state transition diagram corresponding to the TraceID; and structurally encapsulating all data associated with the same TraceID to form a complete production fact dataset marked with TraceID. Step S20: Invoke the large language model, generate a benchmark test program based on the source code and the production fact dataset, perform business logic verification on the benchmark test program in an isolated sandbox, and determine the verified benchmark test program as a trusted business benchmark. It should be noted that the benchmark programs are executed in an isolated sandbox to quickly verify the functional correctness of the instance under test at the micro level. The isolated sandbox is a lightweight runtime environment for securely executing unit tests, featuring process isolation, file system snapshots, and network namespace isolation. A trusted business benchmark, within the isolated sandbox, is an authoritative set of judgments established regarding the expected behavior of the software in a specific business scenario when the execution results of the instance under test on a set of benchmark programs generated by an LLM all meet expectations.

[0028] Understandably, by generating benchmark test programs through a large language model and executing them in an isolated sandbox to drive the test instance, and when the execution result is consistent with the expected result, the behavior path is established as a trusted business benchmark. This trusted business benchmark is then used as an independent, empirically verified business behavior anchor point, specifically to guide the generation of subsequent verification rules. This logically decouples functional correctness verification (completed by unit tests) from difference acceptability determination (completed by verification rules), fundamentally avoiding the self-proof defect caused by the shared origin and mutual dependence of verification rules and test cases.

[0029] Understandably, since the trusted business benchmark is an objective fact obtained by actually executing the test instance and verifying its output, rather than being subjectively generated by the large language model, the verification rules generated based on this trusted business benchmark have a reliable business semantic foundation. This provides a solid premise for subsequent independent access verification through heterogeneous verification test cases, improves the objectivity of the entire API image testing system, reduces the impact of subjectivity on test results, and thus improves the accuracy of API image testing using the large language model.

[0030] In the specific implementation, production requests from the production fact dataset are input one by one into a benchmark program in an isolated sandbox, and the program output results are obtained. After removing the differences between the output results and the production response output due to non-business-related fields caused by differences in timestamps, random numbers, and server hardware, the consistency of the remaining key business fields is verified. When the consistency of the key business fields corresponding to all production request inputs reaches a preset standard, the consistency is determined to meet the requirements, and the benchmark program is determined as the trusted business benchmark for the current running software version. When the consistency does not meet the preset standard, the inconsistent business fields and corresponding link contexts are extracted, fed back to the large language model to optimize the benchmark program, and the consistency verification is re-executed until the preset standard is met.

[0031] Step S30: Based on the trusted business benchmark, call the large language model to generate verification rules, and use independent verification cases to perform admission verification on the verification rules, retaining the target verification rules that pass the verification. It's important to note that a validation rule is a logical expression used to intelligently determine whether the difference between the baseline response and the tested response of two APIs falls within the business acceptable range. A validation test case is a set of test samples used to independently verify the validation rule; its construction logic, data source, or coverage path must be independent of the benchmark test program used to generate the trusted business baseline. Admission verification uses heterogeneous validation test cases to verify whether the validation rule can accurately distinguish between compliant and non-compliant responses.

[0032] Understandably, the large language model driven by the trusted business benchmark generates verification rules for judging response differences. This gives the rules themselves clear business semantics, rather than simple field matching. Furthermore, by introducing verification test cases (including positive test cases, negative test cases, and boundary test cases) that are different from the benchmark test program, the verification rules are independently admitted for verification, forming a closed-loop self-verification mechanism. This mechanism identifies and filters out unreliable rules generated by the illusion or logical errors of the large language model's self-verification, ensuring that the final target verification rules have a high degree of discriminative accuracy.

[0033] Step S40: In a preset mirror environment, replay the API requests used when collecting the production fact dataset for the benchmark instance and the test instance to obtain the benchmark response and the test response; the benchmark instance is the application instance corresponding to the currently running software, and the test instance is the application instance built and running based on the source code in the mirror environment; It should be noted that the mirror environment is a high-fidelity simulation environment built for performing API mirror testing. The baseline instance is a software system instance currently running stably in the production environment, whose business logic has been verified to be correct, used as a reference standard for API behavior comparison. The instance under test is an executable software service instance compiled, deployed, and started in an isolated environment based on the source code of the software under test. API request replay is a technical process that accurately reproduces real user API calls recorded in real-time or stored historically in the production environment in the mirror environment and synchronously sends them to the baseline instance and the instance under test. The baseline response is the complete HTTP response returned by the baseline instance after receiving the exact same input request during the API request replay process. The instance under test response is the complete HTTP response returned by the instance under test after receiving the exact same input request during the API request replay process.

[0034] Understandably, by using a stable benchmark instance running in a production environment as the gold standard, and dynamically building the test instance based on the latest source code of the software under test, it can be ensured that both sides of the test comparison are in a real and executable state. This avoids the information gaps caused by relying solely on static code analysis or document contracts, and improves the accuracy of subsequent comparison and detection.

[0035] Understandably, replaying real production traffic in a highly simulated mirror environment can most realistically reproduce the actual usage scenarios and complex interaction paths of online users. Furthermore, by synchronously sending identical requests to the benchmark instance and the instance under test, it can be ensured that the external stimuli conditions under both instances are completely consistent. Any observed differences in response can be uniquely attributed to changes in the software under test itself, thereby completely eliminating the influence of external factors on the testing of the software under test.

[0036] Understandably, by constructing tests with the same input source and dual instances in parallel, the most realistic and comprehensive assessment of the impact of software changes is achieved, avoiding online incidents caused by incomplete scenario coverage, thus enhancing the quality assurance capabilities before software release.

[0037] It should be noted that the mirror environment has the same hardware configuration, network topology configuration, and middleware version as the actual production environment. The instance under test is deployed in the mirror environment, and an independent shadow database of the instance under test is configured.

[0038] It should be noted that the independent shadow database is a database instance that is configured specifically for the instance under test, with the same structure as the production database but with isolated data.

[0039] Understandably, by requiring the mirror environment to be consistent with the production environment in terms of hardware configuration, network topology, and middleware version, and by configuring an independent shadow database for the instance under test, a high-fidelity, zero-risk test sandbox is built, effectively eliminating the problem of tests passing but deployment failing due to differences between the test environment and the production environment.

[0040] Optionally, step S40 further includes: In a pre-defined mirror environment, the instance under test is used to process the full-link API requests captured from the production environment based on the shadow database, and the test response returned by the instance under test and the baseline response returned by the baseline instance are collected.

[0041] It should be noted that end-to-end API requests are real user API calls captured in real time or in batches by traffic recording components deployed on production gateways, service meshes, or proxy layers.

[0042] In practical implementation, the end-to-end API requests captured from the production environment are essentially traffic corresponding to a complete business transaction. This traffic uses a globally unique tracking ID as the association key. The content of the traffic includes not only the original API call request and response, but also: the identifiers of the calling and called systems, complete input parameters and output results, the timestamp when the call occurred, a snapshot of the relevant system configuration parameters, and records of data state changes triggered by the call (e.g., a record in the database changing from 'initial' to 'processing'). Through the tracking ID, all the information contained in the traffic can be linked together to form an end-to-end, traceable, and complete logical closed loop.

[0043] Understandably, in a mirrored environment, the baseline instance and the instance under test simultaneously process the exact same API requests captured from the production environment, and the responses of the two are compared. This achieves the most realistic and comprehensive assessment of the impact of software changes. Furthermore, since the inputs are completely consistent, any observed differences in response can be uniquely attributed to the logical changes in the software under test itself, rather than external environmental disturbances, thus ensuring the accuracy of defect localization.

[0044] Step S50: Based on the target verification rules, compare the benchmark response with the tested response and output the API test results.

[0045] It should be noted that the API test results are structured conclusions output after intelligently comparing paired benchmark responses and tested responses based on target validation rules.

[0046] Understandably, by using target verification rules with independent access verification and business semantics to compare the baseline response and the response under test, it is possible to intelligently distinguish between harmless technical differences (such as timestamps and random IDs) and harmful business logic defects (such as calculation errors and data loss). By replacing the crude method of byte-level differences with intelligent comparison based on business intent, the test results can directly reflect business value.

[0047] This embodiment provides an API mirror testing method based on end-to-end traffic tracing. It uses a production fact dataset tagged with TraceID, which can completely cover the entire call path, avoiding data bias caused by incomplete traffic sampling in traditional shadow testing. Simultaneously, the source code of the software under test, as the authoritative definition of software behavior, along with the end-to-end production fact dataset, serves as input to the large language model. This provides the large language model with dual constraints of real business logic and actual operating state, effectively suppressing the generation of illusory content by the large language model that does not match the actual behavior of the software under test when generating benchmark programs. Furthermore, by performing business logic verification on the benchmark programs generated by the large language model in an isolated sandbox, and establishing only verified benchmark programs as trusted business benchmarks, it ensures that the verification rules are generated or established on a real and correct business semantic basis. Then, independent verification cases are used for admission verification. This double filtering ensures that the target verification rules used for response comparison have high reliability and low uncertainty, thereby improving the accuracy of API mirror testing.

[0048] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Step S30 also includes steps S01 to S04: Step S01: Using the trusted business benchmark and the multi-dimensional business semantic information related to the software under test as the basis for business semantics, drive the large language model to generate verification rules for determining whether the response difference meets business expectations. Step S02: Based on the benchmark test program, construct a first response that conforms to the trusted service benchmark and a second response that violates the trusted service benchmark in the isolation sandbox; Step S03: For the first response and the second response, execute the verification rules respectively to obtain the corresponding difference determination results; Step S04: When all the difference determination results are completely consistent with the expected results corresponding to the trusted business benchmark, determine that the verification rule has passed the independent admission verification and retain the target verification rule that has passed the verification.

[0049] It should be noted that the response difference pattern is a response structure or numerical change pattern that repeatedly appears during API version evolution, obtained through statistical analysis or clustering of historical traffic in the production environment. The business semantic basis is a set of contextual information used to guide the large language model in generating validation rules with business understanding capabilities. The first response is a simulated API response constructed in an isolated sandbox based on a benchmark program and fully conforming to trusted business benchmarks. The second response is a simulated API response constructed in an isolated sandbox based on the same benchmark program but intentionally introducing errors that violate trusted business benchmarks. The difference determination result is a binary or multivariate determination conclusion output by the validation rules after analyzing the input response (first response or second response).

[0050] Understandably, using trusted business benchmarks and real-world response discrepancies in the production environment as the basis for business semantics drives the large language model to generate validation rules. This allows the rules to not only know what is correct but also which changes are permissible by the business, thus possessing stronger real-world adaptability. Based on the original benchmark test program, a first response that conforms to the benchmark and a second response that violates the benchmark are proactively constructed in an isolated sandbox as validation samples independent of the rule generation process. This completely severs the direct circular dependency between the validation rules and their generation basis, effectively avoiding the logical flaws in existing technologies where rules and unit tests are generated from the same source and validate themselves.

[0051] Understandably, a verification rule is only allowed to be used as a target verification rule when its judgment results on these independently constructed positive / negative samples are completely consistent with the trusted business benchmark. This ensures that the rule's discriminative ability has undergone unbiased and rigorous empirical testing, providing standard verification rules for subsequent API testing. This improves the accuracy and reliability of difference judgment in API mirror testing and reduces false positives and false negatives caused by AI illusions or logical biases.

[0052] In its implementation, a trusted business benchmark verified through unit testing and locked versions is used as the sole input for generating LLM difference rules. This includes difference scenarios based on production traffic snapshots. Rule generation without a benchmark is prohibited, ensuring that all rule judgment logic fully aligns with verified business expectations, thus reducing the risk of LLM illusions from the outset. Based on the trusted business benchmark, LLM automatically generates directly executable difference rule code, achieving difference judgment at the business semantic level. The core logic of the rules strictly adheres to the pre-anchored business benchmark, accurately distinguishing between dynamic differences that meet business expectations and substantial defects that violate the benchmark. The difference rule is a dedicated rule for version iteration scenarios, used to intelligently determine traffic that represents expected structural changes due to functional changes among the output differences of the same set of production traffic under different software versions.

[0053] In practical implementation, it is also necessary to use a pre-anchored business benchmark test program that is not generated from the same source as the rules, and to perform full automated verification of the generated difference rules in a sandbox environment. The core verification logic is: simulate the production output and test output of different business scenarios, feed them to the difference rules to be verified, and compare whether the output results of the rules are completely consistent with the expected results of the business benchmark.

[0054] Furthermore, after step S03, the API mirroring test method based on end-to-end traffic tracing also includes: When the difference determination result is not completely consistent with the expected result, the inconsistent abnormal determination result is extracted from the difference determination result; Extract the abnormal expected results corresponding to the abnormal determination results from the expected results; The anomaly determination result and the expected anomaly result are fed back to the large language model. The verification rule is optimized based on the large language model, and the independent admission verification is re-executed on the optimized verification rule until the verification passes, thereby obtaining the target verification rule.

[0055] It should be noted that the expected result is the correct judgment conclusion that should be made for the same verification response, as predetermined by the trusted business benchmark. The anomaly judgment result is the inconsistency identified during the comparison of the difference judgment result and the expected result. The anomaly expected result is the correct judgment conclusion stipulated by the trusted business benchmark, corresponding to the aforementioned anomaly judgment result.

[0056] It should be noted that optimizing the validation rules involves the large language model adjusting its understanding of business semantics or rule expression based on feedback error cases, generating a new version of validation rules that is logically more accurate and comprehensive.

[0057] Understandably, when the independent admission verification fails, the verification rules are not simply discarded. Instead, specific abnormal judgment results and their corresponding abnormal expected results are actively extracted from the inconsistency between the difference judgment results and the expected results, forming a precise error feedback signal. This allows the large language model to be placed in a closed-loop learning environment, enabling it to make targeted corrections based on real error judgment cases, rather than relying on vague general prompts, thereby improving the directionality and effectiveness of rule optimization.

[0058] Understandably, by feeding back abnormal sample pairs to the large language model to optimize the validation rules and re-executing the complete independent admission validation on the new rules, the API tester is able to continuously learn and improve itself, and even if the initially generated rules have defects, it can approach the optimal solution through iteration.

[0059] Understandably, because the entire optimization process always takes the trusted business benchmark as the sole source of truth and is rigorously tested through independently constructed verification test cases, the judgment criteria of the API tester remain objective and consistent even in multiple iterations. This effectively prevents the model from generating new illusions or logical drifts during the correction process, ensuring that the final target verification rules are not only highly accurate but also have a traceable and verifiable generation process. This greatly enhances the credibility and usability of the large language model in key quality assurance stages.

[0060] In practical implementation, if the access verification fails to execute test cases, the API tester automatically extracts the failure scenario, error cause, and deviation from the business baseline, feeds it back to the LLM to locate rule logic vulnerabilities and optimize them, and re-executes the full access verification. If it fails for three consecutive iterations, a work order is automatically generated, triggering manual intervention. Subsequent rule iterations and optimizations must be based on the updated and verified business baseline and benchmark test program, and undergo regression verification through all historical test cases to ensure that the new rules do not break the compatibility of the original business scenarios. The rules are permanently bound to the corresponding versions of the business baseline and benchmark test program, achieving full lifecycle traceability.

[0061] Furthermore, after step S30, the API mirroring test method based on end-to-end traffic tracing also includes: The target verification rules are abstracted into Skills, and a binding relationship is established between the Skills and the generated benchmark test program. When the comparison results based on the target verification rules indicate a difference, a structured work order containing the difference context information is generated, and the structured work order is fed back to the test engineer. Receive manual feedback from the test engineer regarding the differences in the structural engineering document; Based on the human feedback, the large language model is invoked to optimize the benchmark test program and the verification rules, and independent admission verification is performed. The validated rules that pass the validation will be used as the new version of the Skill, and the new version of the Skill will be used as the new target validation rules for the next API test.

[0062] It's important to note that a Skill is a reusable knowledge unit resulting from the structured abstraction and asset-based encapsulation of target validation rules. A binding relationship is the explicit association established within the system between a Skill and the benchmark test program it depends on or validates. Difference context information is the complete technical context automatically collected and associated by the system when differences are discovered during API response comparison. A structured work order is a standard work order automatically generated based on difference context information and conforming to the defect management system format. Manual feedback is the conclusive opinion filled in by the test engineer in the work order system after assessing the differences in the structured work order. The new version of the Skill is a Skill instance with a higher version number generated after optimizing validation rules based on manual feedback and passing independent access validation. It inherits the metadata of the old version and records the reasons for changes and the basis for optimization, forming a traceable knowledge evolution chain.

[0063] Understandably, by abstracting the target verification rules into Skills and establishing their binding relationship with the benchmark test program, the structured accumulation and asset management of test knowledge are realized, so that the business verification logic is no longer scattered in scripts or models, but becomes a traceable, reusable, and evolvable enterprise-level asset.

[0064] Understandably, when a difference is detected, a structured work order containing complete context is automatically generated and pushed to the test engineer to reduce the cognitive load and communication cost of manual judgment. After receiving human feedback, the human feedback is used as a high-quality supervisory signal to drive the large language model to simultaneously optimize the benchmark test program and verification rules, forming a human-machine collaborative enhancement loop of machine execution, human judgment and machine learning.

[0065] Understandably, by re-executing independent access checks on the optimized rules and using the successful rules as the new version of the Skill in the next round of testing, a continuously evolving and increasingly accurate intelligent API tester is built. This not only avoids misjudgments caused by a lack of business common sense in purely automated solutions, but also avoids the inefficiency and scalability of purely manual testing. Thus, an organic integration of automation and intelligence is achieved.

[0066] In practical implementation, the difference verification rules are abstracted into reusable, version-manageable, and cross-scenario-combinable Skills. A strong binding mechanism between the rules and the benchmark test program is established, an organizational-level test knowledge base is built, and the test rules are assetized and accumulated.

[0067] Based on the first and second embodiments of this application, the same or similar content as the above embodiments in the third embodiment of this application can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 Before step S20, the API mirroring test method based on end-to-end traffic tracing further includes step S1: Step S1: Obtain the business requirements description and API interface specifications of the software under test; It should be noted that business requirements descriptions are written in natural language or semi-structured form, outlining the business objectives, user scenarios, functional scope, and non-functional constraints that the software should achieve. API interface specifications are technical contract documents that define the service interfaces exposed by the software.

[0068] Understandably, before generating benchmark test programs, in addition to relying on source code, business requirement descriptions and API interface specifications are also introduced as context inputs. This allows the large language model to perform cross-validation and semantic fusion from three dimensions: business intent, interface contract, and implementation logic. This overcomes the fundamental limitation of coverage testing, which only focuses on what was done and does not care whether it was done correctly.

[0069] Optionally, step S20 further includes step S2: Step S2: Call the large language model to perform business analysis on the source code, the business requirement description, and the API interface specification to obtain the business benchmark of the software under test, and generate a benchmark test program to verify the business behavior of the test instance based on the business benchmark.

[0070] It should be noted that business analysis is a process in which the large language model cross-references and semantically fuses the business requirement description, API interface specifications, and source code. The business baseline is an authoritative set of descriptions of the expected output behavior of the software under test under given input conditions, obtained through the aforementioned business analysis.

[0071] Understandably, the business benchmarks obtained through this analysis are no longer isolated technical assertions, but semantic verification standards that carry real business rules. This enables benchmark testing programs to be business correctness oriented, accurately cover core value paths, and thus improve the effectiveness of unit testing.

[0072] Understandably, by integrating requirements and standards, the testing blind spots of syntactically correct but business-incorrect tests have been significantly reduced. Compared to writing test cases manually, it avoids insufficient coverage caused by misunderstandings or omissions in requirements, thus realizing a leap from code-level testing to business-level verification. This provides key pre-testing capabilities for building a truly business-value-centric intelligent API image testing system, improving the accuracy and efficiency of software quality assurance.

[0073] In its implementation, LLM simultaneously analyzes three types of input sources to accurately identify the core expectations of business logic and version changes. This analysis precedes rule generation, anchoring an unshakeable, rigid business benchmark. This replaces the traditional manual rule-making approach, specifically: business logic descriptions and acceptance criteria in requirements and system design documents; code change differences (accurately identifying newly added / modified business logic branches and expected behaviors); and mirror test system specifications (including environment configuration and API interface input / output standards). Based on this anchored business benchmark, LLM independently generates benchmark test programs aligned with business expectations, covering three core scenarios. These serve as the sole rigid benchmark for subsequent verification of the tested program, generation and verification of difference rules, and are completely decoupled from rule generation. Positive example test cases simulate normal API returns that meet business expectations, verifying the positive correctness of the business logic; negative example test cases simulate abnormal API returns that do not meet business expectations, verifying the business logic's exception handling capabilities; and boundary test cases simulate boundary scenarios such as null values, abnormal formats, and exceeding thresholds, verifying the boundary compatibility of the business logic.

[0074] Furthermore, step S2 also includes: The large language model is invoked to perform a differential comparison of the source code, business requirement description, and API interface specification of the software under test and the currently running software to determine the added, modified, and / or deleted functions of the software under test. Based on each function, generate corresponding business behavior descriptions; Obtain historical data for testing the currently running software, including historical trusted business benchmarks, bound Skill rule bases, and historical human feedback records; The business behavior description is correlated and compared with the historical data to generate the business benchmark of the software under test.

[0075] It should be noted that differential comparison is a process in which a large language model performs structural alignment and semantic-level difference identification on the source code, business requirement descriptions, and API interface specifications of two software programs. Historical data refers to structured knowledge assets accumulated during previous tests of the currently running software. Association comparison involves semantically matching and contextualizing newly generated business behavior descriptions with relevant entries in historical data to determine whether the new behavior is consistent with, conflicts with, or expands upon existing knowledge, thereby deciding how to construct a new business benchmark.

[0076] Understandably, by calling a large language model to perform cross-version, multi-dimensional comparisons of the source code, business requirements, and API specifications of the software under test and the currently running software, it is possible to accurately identify additions, modifications, or deletions at the functional level, rather than remaining at the superficial differences in lines of code or files, thus providing a clear scope of changes for testing focus.

[0077] Understandably, by generating corresponding business behavior descriptions based on the identified functional changes, and further comparing and correlating these descriptions with historical trusted business benchmarks, Skill rule bases, and manual feedback records, intelligent reuse and incremental evolution of historical test assets are achieved.

[0078] In practical implementation, in software version update (iteration) scenarios (e.g., upgrading from V1.0 to V1.1), the operations that need to be performed by the large language model are as follows: Incremental benchmark construction: Based on the trusted business benchmarks verified in version V1.0, combined with the requirement change documents, code differences, and interface specification updates of version V1.1, a business benchmark adapted to V1.1 is generated; the source code, business requirement descriptions, and API interface specifications of V1.1 and V1.0 are structurally aligned and semantically differentiated to accurately locate the new, modified, and / or deleted functions in version V1.1; then, historical data from testing version V1.0 is obtained, and the newly generated V1.1 business behavior descriptions are correlated and compared with this historical data to intelligently reuse existing assets and ensure compatibility of changes, and the newly generated V1.1 business benchmarks are used to guide the generation of benchmark testing programs and difference rules. The historical data includes historical trusted business benchmarks, bound Skill rule bases, and historical human feedback records.

[0079] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the API mirroring testing method based on end-to-end traffic tracing in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0080] This application also provides an API mirroring testing device based on end-to-end traffic tracing; please refer to [reference needed]. Figure 4 The API mirroring testing device based on end-to-end traffic tracing includes: The acquisition module 10 is used to acquire the source code of the software under test and the production fact dataset marked with TraceID through end-to-end traffic collection; The first verification module 20 is used to call the large language model, generate a benchmark test program based on the source code and the production fact dataset, perform business logic verification on the benchmark test program in an isolation sandbox, and determine the verified benchmark test program as a trusted business benchmark. The second verification module 30 is used to generate verification rules based on the trusted business benchmark by calling the large language model, and to perform admission verification on the verification rules using independent verification cases, and retain the target verification rules that pass the verification. The replay module 40 is used to replay the API requests used when collecting the production fact dataset in a preset mirror environment for the benchmark instance and the test instance, and obtain the benchmark response and the test response; the benchmark instance is the application instance corresponding to the currently running software, and the test instance is the application instance built and running in the mirror environment based on the source code; The output module 50 is used to compare the benchmark response and the tested response based on the target verification rules and output the API test results.

[0081] The API mirroring testing device based on end-to-end traffic tracing provided in this application, employing the API mirroring testing method based on end-to-end traffic tracing in the above embodiments, can solve the technical problem that the API testing method combining LLM and shadow testing cannot meet the requirements of accurate testing. Compared with the prior art, the beneficial effects of the API mirroring testing device based on end-to-end traffic tracing provided in this application are the same as those of the API mirroring testing method based on end-to-end traffic tracing provided in the above embodiments, and other technical features in the API mirroring testing device based on end-to-end traffic tracing are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0082] This application provides an API mirroring test device based on end-to-end traffic tracing. The API mirroring test device based on end-to-end traffic tracing includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the API mirroring test method based on end-to-end traffic tracing in the above embodiment 1.

[0083] The following is for reference. Figure 5 This document illustrates a structural diagram of an API mirroring test device suitable for implementing embodiments of this application based on end-to-end traffic tracing. The API mirroring test device based on end-to-end traffic tracing in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The API mirroring test device based on end-to-end traffic tracing shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0084] like Figure 5 As shown, the API mirroring test device based on end-to-end traffic tracing may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the API mirroring test device based on end-to-end traffic tracing. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the API mirroring test equipment based on end-to-end traffic tracing to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows an API mirroring test equipment based on end-to-end traffic tracing with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented alternatively.

[0085] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0086] The API mirroring testing device based on end-to-end traffic tracing provided in this application, employing the API mirroring testing method based on end-to-end traffic tracing in the above embodiments, can solve the technical problem that the API testing method combining LLM and shadow testing cannot meet the requirements of accurate testing. Compared with the prior art, the beneficial effects of the API mirroring testing device based on end-to-end traffic tracing provided in this application are the same as those of the API mirroring testing method based on end-to-end traffic tracing provided in the above embodiments, and other technical features in this API mirroring testing device based on end-to-end traffic tracing are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0087] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0088] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0089] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the API mirroring test method based on end-to-end traffic tracing in the above embodiments.

[0090] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0091] The aforementioned computer-readable storage medium may be included in an API image testing device based on end-to-end traffic tracing; or it may exist independently and not be assembled into an API image testing device based on end-to-end traffic tracing.

[0092] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by an API mirror testing device based on end-to-end traffic tracing, the API mirror testing device based on end-to-end traffic tracing performs the following: acquires the source code of the software under test and a production fact dataset marked with TraceID collected through end-to-end traffic tracing; invokes a large language model to generate a benchmark test program based on the source code and the production fact dataset; performs business logic verification on the benchmark test program in an isolated sandbox; determines the verified benchmark test program as a trusted business benchmark; based on the trusted business benchmark, invokes the large language model to generate verification rules, and uses independent verification test cases to perform admission verification on the verification rules, retaining the target verification rules that pass the verification; in a preset mirror environment, replays the API requests used when collecting the production fact dataset for the benchmark instance and the instance under test, obtaining the benchmark response and the response under test; the benchmark instance is the application instance corresponding to the currently running software, and the instance under test is an application instance built and running based on the source code in the mirror environment; and compares the benchmark response and the response under test based on the target verification rules, outputting the API test results.

[0093] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0094] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0095] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0096] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described API mirroring test method based on end-to-end traffic tracing. This solves the technical problem that API testing methods combining LLM and shadow testing cannot meet the requirements for accurate testing. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the API mirroring test method based on end-to-end traffic tracing provided in the above embodiments, and will not be repeated here.

[0097] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the API mirroring test method based on end-to-end traffic tracing as described above.

[0098] The computer program product provided in this application can solve the technical problem that the API testing method combining LLM and shadow testing cannot meet the requirements of accurate testing. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the API mirror testing method based on end-to-end traffic tracing provided in the above embodiments, and will not be repeated here.

[0099] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. An API mirror testing method based on end-to-end traffic tracing, characterized in that, The method includes: Obtain the source code of the software under test and the production fact dataset marked with TraceID, collected through end-to-end traffic analysis; The large language model is invoked to generate a benchmark test program based on the source code and the production fact dataset. The benchmark test program is then validated for business logic in an isolated sandbox. The benchmark test program that passes the validation is determined as a trusted business benchmark. Based on the trusted business benchmark, the large language model is invoked to generate verification rules, and independent verification cases are used to perform admission verification on the verification rules, retaining the target verification rules that pass the verification. In a preset mirror environment, the API requests used when collecting the production fact dataset are replayed for the benchmark instance and the test instance to obtain the benchmark response and the test response; the benchmark instance is the application instance corresponding to the currently running software, and the test instance is the application instance built and running based on the source code in the mirror environment; Based on the target verification rules, the benchmark response and the tested response are compared, and the API test results are output.

2. The API mirror testing method based on end-to-end traffic tracing as described in claim 1, characterized in that, The steps of generating verification rules based on the trusted business benchmark, calling the large language model, performing admission verification on the verification rules using independent verification test cases, and retaining the target verification rules that pass the verification include: The trusted business benchmark and multi-dimensional business semantic information related to the software under test are used as the basis for business semantics to drive the large language model to generate verification rules for determining whether the response difference meets business expectations. Based on the benchmark testing program, a first response that conforms to the trusted business benchmark, a second response that violates the trusted business benchmark, and a preset expected response are constructed in the isolation sandbox. For the first response, the second response, and the expected response, the verification rules are executed respectively to obtain the corresponding difference determination results; When all the difference determination results are completely consistent with the expected results corresponding to the trusted business benchmark, the verification rule is determined to have passed the independent admission verification, and the target verification rule that has passed the verification is retained.

3. The API mirror testing method based on end-to-end traffic tracing as described in claim 2, characterized in that, After the step of executing the verification rule for the first response and the second response respectively to obtain the corresponding difference determination results, the method further includes: When the difference determination result is not completely consistent with the expected result, the inconsistent abnormal determination result is extracted from the difference determination result; Extract the abnormal expected results corresponding to the abnormal determination results from the expected results; The anomaly determination result and the expected anomaly result are fed back to the large language model. The verification rule is optimized based on the large language model, and the independent admission verification is re-executed on the optimized verification rule until the verification passes, thereby obtaining the target verification rule.

4. The API mirror testing method based on end-to-end traffic tracing as described in claim 1, characterized in that, After the steps of generating verification rules by calling the large language model based on the trusted business benchmark, performing admission verification on the verification rules using independent verification test cases, and retaining the target verification rules that pass the verification, the method further includes: The target verification rules are abstracted into Skills, and a binding relationship is established between the Skills and the generated benchmark test program and the production fact dataset. When the comparison results based on the target verification rules indicate a difference, a structured work order containing TraceID full-link context information, difference fields, and state transition comparisons is generated, and the structured work order is fed back to the test engineer. Receive manual feedback from the test engineer regarding the differences in the structural engineering document; Based on the human feedback, the large language model is invoked to optimize the benchmark test program and the verification rules, and independent admission verification is performed. The validated rules that pass the validation will be used as the new version of the Skill, and the new version of the Skill will be used as the new target validation rules for the next API test.

5. The API mirror testing method based on end-to-end traffic tracing as described in claim 1, characterized in that, Before the step of calling the large language model and generating a benchmark program based on the source code and the production fact dataset, the method further includes: Obtain the business requirements description and API interface specifications of the software under test; The step of calling the large language model and generating a benchmark program based on the source code and the production fact dataset includes: The large language model is invoked to perform business analysis on the source code, the business requirement description, and the API interface specification to obtain the business benchmark of the software under test. Based on the business benchmark, a benchmark test program is generated to verify the business behavior of the instance under test.

6. The API mirroring testing method based on end-to-end traffic tracing as described in claim 5, characterized in that, The step of calling the large language model to perform business analysis on the source code, the business requirement description, and the API interface specification to obtain the business benchmark of the software under test includes: The large language model is invoked to perform a differential comparison of the source code, business requirement description, and API interface specification of the software under test and the currently running software to determine the added, modified, and / or deleted functions of the software under test. Based on each function, generate corresponding business behavior descriptions; Obtain historical data for testing the currently running software, including historical trusted business benchmarks, bound Skill rule bases, and historical human feedback records; The business behavior description is correlated and compared with the historical data to generate the business benchmark of the software under test.

7. The API mirror testing method based on end-to-end traffic tracing as described in claim 1, characterized in that, The mirror environment has the same hardware configuration, network topology configuration, and middleware version as the actual production environment. The instance under test is deployed in the mirror environment, and an independent shadow database for the instance under test is configured. The step of replaying the API requests used when collecting the production fact dataset in a preset mirror environment for both the benchmark instance and the instance under test, and obtaining the benchmark response and the response under test; wherein the benchmark instance is the application instance corresponding to the currently running software, and the instance under test is an application instance built and running based on the source code in the mirror environment, includes: In a pre-defined mirror environment, the instance under test is used to process the full-link API requests captured from the production environment based on the shadow database, and the test response returned by the instance under test and the baseline response returned by the baseline instance are collected.

8. An API mirroring testing device based on end-to-end traffic tracing, characterized in that, The device includes: The acquisition module is used to acquire the source code of the software under test and the production fact dataset marked with TraceID through end-to-end traffic collection; The first verification module is used to call the large language model, generate a benchmark test program based on the source code and the production fact dataset, perform business logic verification on the benchmark test program in an isolation sandbox, and determine the verified benchmark test program as a trusted business benchmark. The second verification module is used to generate verification rules based on the trusted business benchmark by calling the large language model, and to perform admission verification on the verification rules using independent verification cases, and retain the target verification rules that pass the verification. The replay module is used to replay the API requests used when collecting the production fact dataset in a preset mirror environment for the benchmark instance and the instance under test, and to obtain the benchmark response and the response under test; the benchmark instance is the application instance corresponding to the currently running software, and the instance under test is an application instance built and running based on the source code in the mirror environment; The output module is used to compare the benchmark response and the tested response based on the target verification rules and output the API test results.

9. An API mirroring test device based on end-to-end traffic tracing, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the API mirroring test method based on end-to-end traffic tracing as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the API mirroring test method based on end-to-end traffic tracing as described in any one of claims 1 to 7.

Images