Test case verification method, apparatus, device, and medium

Abstract

This invention relates to the field of automated testing technology, and discloses a test case verification method, apparatus, device, and medium. It is applied to a verification system deployed between the target under test and the testing system via an interface. The method includes: receiving identification information of the currently executed test case and execution information of the current test step sent by the testing system, as well as a status signal fed back by the target under test in response to the testing system; obtaining the standard test case corresponding to the identification information; performing consistency verification between the execution information of the current test step and the corresponding test step in the standard test case; if the verification passes, generating a verification signal based on the status signal and sending it to the testing system, so that the testing system generates test results based on the verification signal; receiving the test results fed back by the testing system, and generating a verification report based on the comparison between the test results and the expected results corresponding to the current test step in the standard test case. This verifies the accuracy of the test cases, thereby greatly improving testing efficiency.

Description

Technical Field

[0001] This invention relates to the field of automated testing technology, specifically to test case verification methods, apparatus, equipment, and media. Background Technology

[0002] The development and application of automated test cases in the field of electronic function and performance testing usually involves designing test cases based on system and software requirements, verifying the system and software requirements of each Electronic Control Unit (ECU) one by one, and then determining whether the system and software functions meet the design requirements based on the automated test results.

[0003] Current technology requires testers to write test cases into scripts, debug them, and then run the automated test cases to obtain test results. In practical applications, the number of test cases is often in the thousands or more, which is enormous. Debugging and verifying each test case individually is time-consuming and makes it difficult to efficiently and automatically verify the logical correctness, procedural accuracy, and robustness to abnormal input signals of these automated test cases. If test cases are defective and applied directly without verification, it may not only lead to unreliable test results but also mask real product problems and even introduce additional debugging costs, resulting in high test case development costs and low development efficiency.

[0004] Therefore, there is an urgent need for a test case verification scheme that can efficiently and automatically verify automated test cases during test execution, thereby ensuring the quality of the test cases themselves, further improving the development efficiency of test cases, and reducing development costs. Summary of the Invention

[0005] This invention provides a test case verification method, apparatus, device, and medium to address the problems mentioned in the above-mentioned technical background, such as the large number of test cases leading to long test case debugging cycles, neglecting the verification of the quality of the test cases themselves, resulting in high development costs and seriously affecting the efficiency of test case development.

[0006] In a first aspect, the present invention provides a test case verification method applied to a verification system, wherein the verification system is deployed between the target under test and the test system through an interface, and the method includes: Receive identification information of the currently executing test case and execution information of the current test step sent by the test system, as well as status signals fed back by the target under test in response to the test system; Obtain the standard test cases corresponding to the identification information. The standard test cases contain at least one test step and the expected results of the corresponding steps. Perform a consistency check between the execution information of the current test step and the corresponding test steps in the standard test cases; If the verification passes, a verification signal is generated based on the status signal and sent to the test system so that the test system can generate test results based on the verification signal. Receive test results from the testing system and generate a verification report based on the comparison between the test results and the expected results corresponding to the current test step in the standard test case.

[0007] This invention adds an intermediate verification system between the target under test and the test system. On the one hand, it verifies the consistency between the test system's test steps and standard test cases. On the other hand, it achieves comprehensive verification of the effectiveness and accuracy of the automated test system itself by controllably generating verification signals and comparing the test system's output results with standard expectations. At the same time, through an automated consistency verification mechanism, it can batch and automatically identify problems such as step deviations, test case mismatches, and parameter inconsistencies during the execution of test cases. This eliminates the need for testers to manually debug each test case throughout the entire process, significantly shortening the test case development and debugging cycle, reducing labor costs, and significantly improving the efficiency of test case development.

[0008] In one optional implementation, the execution information of the current test step is checked for consistency with the corresponding test steps in the standard test cases, including: Verify whether the step identifier of the current test step matches the step identifier that should be executed in the standard test case; And / or, verify whether the execution order of the current test steps is consistent with the order of steps that should be executed in the standard test case; And / or, verify whether the signal identifiers of the signals involved in the current test step match the signal identifiers specified in the standard test case; And / or, verify whether the signal values ​​of the signals involved in the current test step are within the valid range of the corresponding signals specified in the standard test case.

[0009] This invention constructs a multi-dimensional, comprehensive, and configurable test step consistency verification mechanism, which fully covers the four core elements of test step execution: identity identification, execution process, test object, and verification benchmark. Subsequent test processes are only executed after verification is passed, which can intercept non-compliant test executions at the source and completely avoid invalid tests. At the same time, it maximizes adaptability to the needs of different test scenarios, balancing control precision and test execution efficiency. This further ensures test effectiveness, reduces development costs, and improves the credibility of test results.

[0010] In one optional implementation, generating a verification signal based on a state signal includes: The status signal is processed according to a preset verification strategy to generate a verification signal; wherein the preset verification strategy includes at least one of the following operation methods: Keep the state signal unchanged; Change the signal value of at least one of the status signals to an unexpected value that does not conform to the valid range of the corresponding signal specified in the standard test case; Change the signal value of at least one of the state signals to a non-theoretical value that is outside the theoretical effective range of the signal; Filter the status signals to select the parts of the signals that are related to the specified simulation node and / or the specified function; Add at least one interference signal that is unrelated to the current test scenario to the status signal. The interference signal includes signals whose signal identifier is not in the preset valid set, and / or signals whose signal identifier is in the preset valid set but whose signal data format does not match the preset format.

[0011] This invention constructs a multi-mode, configurable, and fully covered verification signal generation mechanism. Through multiple types of combinable signal processing operations, it fully covers the verification needs of the entire automated testing system, thereby supporting the full-dimensional verification of the automated testing system from basic functions and anomaly identification capabilities to robustness. At the same time, it significantly reduces test development costs and fundamentally ensures the credibility of the testing system.

[0012] In one optional implementation, a verification report is generated based on a comparison of the test results with the expected results corresponding to the current test step in the standard test case, including: If the test result is consistent with the expected result corresponding to the current test step in the standard test case, then the current test step is determined to have passed the verification, and the step identifier and the result of the step verification of the current test step are recorded to generate a verification report for the currently executed test case. If the test result is inconsistent with the expected result corresponding to the current test step in the standard test case, the current test step is determined to have failed. The step identifier, the step verification failure result, and the reason for the verification failure of the current test step are recorded to generate a verification report for the currently executed test case. At the same time, the verification of the currently executed test case is terminated.

[0013] This invention precisely compares the test results output by the testing system with the expected results preset in the standard test cases. Using the source benchmark of the requirement design as a standard, it directly determines whether the execution results of the automated testing system meet the design requirements. For the first time, it realizes a step-by-step closed-loop verification of the effectiveness of the automated testing system itself. This not only ensures the rigor and accuracy of the verification results, but also fundamentally guarantees the credibility of the entire automated testing system.

[0014] In an optional implementation, after determining that the current test step has passed verification, the test case verification method further includes: Check if the current test step is the last test step of the currently executed test case; If the current test step is the last test step of the currently executed test case, then the process of recording the step identifier and the step verification pass result of the current test step is executed to generate a verification report for the currently executed test case. If the current test step is not the last test step of the currently executed test case, then the next test step is determined according to the execution order of the test steps, the next test step is updated to the current test step, and the process of verifying the consistency of the execution information of the current test step with the corresponding test steps in the standard test case is repeated.

[0015] This invention uses single-step verification as the sole prerequisite for executing subsequent steps, strictly follows the execution order preset by standard test cases, and ensures that all test cases are tested without omissions, skips, or incorrect order through a final step detection mechanism. This design avoids invalid verification caused by incorrect step timing or missed requirements from the root of the execution process, greatly ensuring that the verification process aligns with the system / software requirements design.

[0016] In one optional implementation, the test case verification method further includes: The verification of the currently executing test case shall be terminated when all test steps of the currently executing test case have been completed, or when the consistency verification between the execution information of the current test step and the corresponding test step in the standard test case fails, or when an external verification termination instruction is received.

[0017] This invention, by designing multiple parallel-triggered termination conditions, fully covers all termination scenarios throughout the entire lifecycle of test case verification. These scenarios include standardized termination after all steps are completed in normal execution scenarios, timely interception of anomalies in execution deviation scenarios, and flexible manual intervention in scenarios with sudden demands. This completely solves the pain point of the crude termination logic in existing technologies, constructs a complete closed loop of verification process, and balances the integrity of the verification process with the flexibility of execution control.

[0018] In one optional implementation, obtaining the standard test cases corresponding to the identification information includes: Search the test case library of the verification system for the standard test case corresponding to the identification information; If the search for standard test cases fails, the verification process for the currently executing test cases is terminated, and the reason for termination is recorded.

[0019] This invention sets the standard test case search and verification as the first execution step in the verification process. If the search fails, all subsequent processes are terminated directly, eliminating the need to execute meaningless test steps. This can completely intercept invalid verification without a benchmark from the source, thereby significantly shortening the invalid execution cycle, reducing test resource consumption, further reducing the cost of test case development and debugging, and greatly improving the efficiency of test development.

[0020] Secondly, the present invention provides a test case verification device applied to a verification system, wherein the verification system is deployed between the target under test and the test system via an interface, and the device includes: The receiving module is used to receive identification information of the currently executed test case and execution information of the current test step sent by the test system, as well as status signals fed back by the target under test in response to the test system; The acquisition module is used to acquire the standard test cases corresponding to the identification information, wherein the standard test cases include at least one test step and the expected results of the corresponding steps; The verification module is used to verify the consistency between the execution information of the current test step and the corresponding test step in the standard test case; The generation module is used to generate a verification signal based on the status signal and send it to the test system if the verification passes, so that the test system generates a test result based on the verification signal. The comparison module is used to receive the test results fed back by the test system, and generate a verification report based on the comparison between the test results and the expected results corresponding to the current test step in the standard test case.

[0021] The test case verification device of this invention adds an intermediate verification system between the target under test and the test system. On the one hand, it verifies the consistency between the test steps of the test system and the standard test cases. On the other hand, it achieves comprehensive verification of the effectiveness and accuracy of the automated test system itself by controllably generating verification signals and comparing the output results of the test system with the standard expectations. At the same time, through the automated consistency verification mechanism, it can batch and automatically check for problems such as step deviations, test case mismatches, and parameter inconsistencies in the execution process of test cases. This eliminates the need for testers to manually debug each test case throughout the entire process, significantly shortens the development and debugging cycle of test cases, reduces labor costs, and significantly improves the development and delivery efficiency of test cases.

[0022] Thirdly, the present invention provides an electronic device, which includes a controller, a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the test case verification method of the first aspect or any corresponding embodiment described above.

[0023] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the test case verification method of the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0024] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the test case verification system. Figure 2 This is a schematic diagram of the first type of test case verification method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the second process of the test case verification method according to an embodiment of the present invention; Figure 4 This is a flowchart illustrating the workflow of the test case verification system; Figure 5 This is a structural block diagram of a test case verification device according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] In practical applications, the traditional testing architecture consists of an automated testing system (also called a test system) and the object under test, such as an ECU, to test the ECU's functionality. However, this type of architecture suffers from a large number of automated test cases, resulting in long debugging cycles for the test tools. Furthermore, existing technologies lack the ability to verify the execution accuracy, anomaly detection capabilities, and robustness of the test system itself, posing a core risk of undetected logical defects in the test system and completely distorted test results. Therefore, this embodiment provides a new architecture: a test case verification system (also called a verification system) is bridged between the automated testing system and the ECU under test via a network interface. This system verifies the test system and its executed automated test tools, thereby improving the efficiency of test case development and reducing development costs.

[0028] In this embodiment, the test architecture is described using the ECU under test as an example. Figure 1This is a schematic diagram of the test case verification system. As shown in the diagram, the structure includes: the ECU under test, the test case verification system, and the automated testing system. The interaction process includes: 1. The ECU sends a status signal to the test case verification system during testing; 2. The test case verification system modifies and forwards the ECU status signal to the automated testing system; 3. The automated testing system sends the current test step and current test result of the current automated test case to the test case verification system; 4. The test case verification system determines whether the test steps of the automated testing system are consistent with the current test case steps, modifies the test steps according to the test system's test requirements, and executes them, thereby achieving the purpose of modifying the test ECU.

[0029] It should be noted that, Figure 1 The ECU under test (DUT) in this context refers to the actual electronic control unit hardware or a simulation environment running its software. Its function is to receive external input signals, such as sensor signals and CAN (Controller Area Network) messages, then run its internal control software and output corresponding control commands or status information. The automated test system sends stimulus signals to the DUT according to predetermined automated test cases (script files or configuration files stored in the automated test system, defining specific test steps, input conditions, execution order, and expected results; it is the "instruction manual" guiding how the automated test system works). Simultaneously, it receives feedback signals from the DUT and compares them with the expected results in the test cases, outputting a "pass" or "fail" conclusion. A test case verification system is a software program that runs on a personal computer (PC) and bridges the ECU and automated testing system via a network interface. It requires no dedicated hardware and receives ECU status signals and test steps from the automated testing system (such as test case numbers and current test step information). It modifies and forwards ECU status signals according to test requirements (e.g., changing normal signals to abnormal values ​​to test whether the automated testing system can correctly report errors; or adding irrelevant diagnostic signals to verify the testing system's anti-interference capability), while simultaneously verifying the consistency of the test steps. Its internal test case library (storing standardized, verified test cases containing correct test step numbers, correct signal value ranges, and accurate expected results) is used to manage verification rules, modification strategies, and the test cases themselves.

[0030] In one specific embodiment, Figure 1The interactive process includes: the ECU under test sends its status signals (such as sensor data and internal status variables) to the middle-layer test case verification system via a CAN bus or Ethernet interface; the automated test system also sends test steps (such as reading a signal, triggering a function, or verifying an output) to the middle-layer test case verification system via a network interface. Upon receiving this information, the middle-layer test case verification system first performs a consistency check on the test steps to determine if they conform to preset rules and requirements. If the check passes, the middle-layer test case verification system modifies the ECU status signals according to the test strategy (e.g., forcing a sensor value to a fault value or simulating an abnormal operating condition) and forwards the modified signals to the automated test system. The automated test system executes the test steps based on the modified signals and feeds the test results back to the middle layer. The middle-layer test case verification system then compares the test results with the expected results to ultimately determine whether the automated test system's functionality meets the design requirements. This embodiment achieves comprehensive testing and verification of the automated test system through pure software deployment, without the need for additional hardware, significantly improving testing efficiency and reducing development costs.

[0031] According to the verification system provided above, which is deployed between the target under test and the test system via an interface, a test case verification method embodiment is correspondingly provided. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0032] This embodiment provides a test case verification method. Figure 2 This is a schematic diagram of the first type of test case verification method according to an embodiment of the present invention, as shown below. Figure 2 As shown, the process includes the following steps: Step S201: Receive the identification information of the currently executed test case and the execution information of the current test step sent by the test system, as well as the status signal fed back by the target under test in response to the test system.

[0033] In this embodiment, the identification information of a test case refers to the unique identification information used to distinguish different test cases. It is the sole retrieval basis for the verification system to locate the corresponding standard test case, including but not limited to at least one of the following: unique test case number, test case version number, functional module code to which the test case belongs, and test case requirement traceability number. The implementation method is as follows: at the initial moment of starting the execution of a single test case, the automated testing system synchronously pushes the identification information of the test case to the verification system through a preset software communication protocol. The verification system extracts and parses the identification information from the received communication message through preset message parsing rules for subsequent matching and searching of standard test cases.

[0034] To further explain, in this embodiment, the execution information of the current test step refers to the full execution process data related to the currently executing test step, generated in real time by the automated testing system during the execution of the current test case. It is the core comparison object for the verification system to perform step consistency verification, including but not limited to at least one of the following: the step identifier of the current test step, the step execution sequence number, the signal identifier involved in the step, the preset signal verification range of the step, the step execution timestamp, and the expected result corresponding to the step. The implementation method is as follows: when the automated testing system switches to the synchronous execution of the current test step, it pushes the execution information of the step to the verification system in real time through a preset communication protocol; after the verification system parses and extracts the execution information, it compares it with the corresponding step information in the standard test case in multiple dimensions to complete the consistency verification.

[0035] To further explain, in this embodiment, the status signal fed back by the target under test in response to the test system refers to the real-time operating status raw data generated and output by the target under test (such as the ECU under test) after receiving the test command issued by the automated test system and executing corresponding actions according to its own operating logic. This data is the basic data source for the verification system to generate verification signals, including but not limited to at least one of the following: sensor acquisition signals, internal operating status variables, fault level signals, function execution feedback messages, input / output interface signals, and bus communication messages of the target under test. The implementation method is as follows: the target under test synchronously sends the real-time status signal to the verification system through a preset communication link (such as Ethernet); after receiving the raw status signal, the verification system can process it according to the preset verification strategy, generate a verification signal, and forward it to the automated test system to realize multi-dimensional verification of the automated test system.

[0036] Step S202: Obtain the standard test case corresponding to the identification information. The standard test case includes at least one test step and the expected result of the corresponding step.

[0037] In existing technologies, the execution test cases and verification benchmarks of automated testing systems use the same set of data. Testers can arbitrarily modify the execution test cases, and there are no independent, fixed third-party standard test cases as verification benchmarks. This only enables functional performance testing of the target under test and cannot verify the consistency between the execution process of the automated testing system itself and the original requirements. In this embodiment, by pre-building an independent standard test case library in the bridging intermediate verification system and accurately matching and retrieving standard test cases based on unique identification information, the system achieves, for the first time, independent control of third-party verification benchmarks separate from the automated testing system. This fundamentally solves the core pain points of existing technologies, such as missing benchmarks, black-box verification processes, and unreliable test results, and provides a core foundation for the full-dimensional closed-loop verification of the automated testing system itself.

[0038] To further clarify, the specific method for obtaining standard test cases in this embodiment is not limited in detail and can be adaptively adjusted according to actual needs. For example, benchmark retrieval can only be completed when a standard test case corresponding to the identification information is found; if there is no matching result or the matching result is not unique, it is determined that benchmark retrieval has failed.

[0039] Step S203: Perform a consistency check between the execution information of the current test step and the corresponding test steps in the standard test cases.

[0040] In this embodiment, consistency verification refers to the process by which the verification system uses standard test steps as the sole benchmark and, according to preset configurable verification rules, performs a compliance comparison of all information of the currently executed steps of the automated test system item by item to determine whether the actual execution actions of the automated test system fully conform to the original requirements design. The verification results are only divided into two categories: verification passed and verification failed, with no intermediate states, to ensure that the judgment logic is rigorous and unambiguous. It should be noted that the specific content of the preset configurable verification rules in this embodiment can be adaptively adjusted based on actual needs. For example, the verification content may include at least one of the following: the unique step identifier of the current test step, the step execution sequence number, the target signal identifier involved in the step, the preset effective verification range of the signal for the step, the step execution timing requirements, and the action execution specifications corresponding to the step.

[0041] Step S204: If the verification passes, a verification signal is generated based on the status signal and sent to the test system so that the test system generates a test result based on the verification signal.

[0042] In this embodiment, the received status signal from the target under test refers to the original, unmodified real-time operating status data generated and output by the target under test (such as an on-board ECU or industrial control unit) after receiving the test command issued by the automated test system and executing the corresponding action according to its own operating logic. It is the only basic data source for the verification system to generate verification signals. Its specific content can be found in the previous text and will not be repeated here.

[0043] To further explain, in this embodiment, the verification signal refers to the target signal generated by the verification system after performing corresponding processing operations on the original state signal based on the original state signal and according to the preset verification strategy bound to the current test step. This target signal is the sole input basis for the automated test system to execute the current test step and generate test results. Note that the specific content of the preset verification strategy is not limited and can be adaptively determined according to actual needs. For example, the preset verification strategy may include: pass-through processing without modification of the original signal (i.e., keeping all content, format, and timing of the state signal completely unchanged, directly generating a verification signal to verify the automated test system's basic identification and judgment capabilities for normal operating conditions), or targeted modification processing of unexpected values ​​of the original signal (i.e., modifying the signal value of at least one target signal in the state signal to an unexpected value that does not conform to the effective range specified in the standard test case, generating a verification signal to verify the automated test system's ability to identify abnormal operating conditions); this is only an example.

[0044] To further explain, in this embodiment, the verification system encapsulates the generated verification signal into a message according to a preset communication protocol (including but not limited to CAN bus protocol, Ethernet TCP / IP protocol) that can be recognized by the automated testing system, and sends it to the automated testing system in real time. After receiving the verification signal, the automated testing system uses it as a feedback signal of the target under test, and parses and judges the verification signal according to the execution logic of the current test step, and finally generates and outputs the test result of the current test step.

[0045] It should be noted that in this embodiment, if the verification fails, the verification termination process will be triggered directly, and the user will not have permission to perform this step.

[0046] Step S205: Receive the test results fed back by the test system, and generate a verification report based on the comparison between the test results and the expected results corresponding to the current test step in the standard test case.

[0047] In this embodiment, the specific content of the test results can be adaptively adjusted according to actual test requirements, such as including at least one of the following: unique identifier information of the corresponding test case, unique step identifier of the current test step, step execution completion timestamp, final judgment result of the test system for the current step (pass / fail), actual value of the verification signal parsed by the test system, judgment basis description, and abnormal alarm information.

[0048] To further explain, in this embodiment, the expected result refers to the compliance judgment benchmark that is uniquely bound to the current test step and pre-embedded in the standard test case. It is strongly correlated with the verification strategy adopted in this verification signal generation and is the only benchmark for the verification system to determine whether the output result of the automated test system meets the design requirements. The process involves comparing the test results fed back by the automated test system item by item to determine whether the output result of the automated test system is completely consistent with the design expectation. The comparison result can be divided into the judgment result of comparison consistency and comparison inconsistency, and then the judgment result is used as the content of the generated verification report.

[0049] The test case verification method provided in this embodiment adds an intermediate verification system between the target under test and the test system. On the one hand, it verifies the consistency between the test steps of the test system and the standard test cases. On the other hand, it achieves comprehensive verification of the effectiveness and accuracy of the automated test system itself by controllably generating verification signals and comparing the output results of the test system with the standard expectations. At the same time, through the automated consistency verification mechanism, it can batch and automatically check for problems such as step deviations, test case mismatches, and parameter inconsistencies in the execution process of test cases. This eliminates the need for testers to manually debug each test case throughout the entire process, significantly shortens the development and debugging cycle of test cases, reduces labor costs, and significantly improves the development efficiency of test cases.

[0050] This embodiment provides a test case verification method. Figure 3 This is a schematic diagram of a second type of test case verification method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps: Step S301: Receive identification information of the currently executing test case and execution information of the current test step from the test system, as well as status signals fed back by the target under test in response to the test system. For details, please refer to [link to relevant documentation]. Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0051] Step S302: Obtain the standard test case corresponding to the identification information. The standard test case includes at least one test step and the expected result of the corresponding step.

[0052] In this embodiment, obtaining the standard test cases corresponding to the identification information in step S302 above includes: Step a1: Search for the standard test case corresponding to the identification information in the test case library of the verification system.

[0053] In this embodiment, a standardized test case library is pre-built in the verification system, and a fast retrieval index is established. The unique identifier information of the received test cases is used as the retrieval keywords to accurately match and retrieve the corresponding standard test cases. Note that if a unique matching standard test case is found, the retrieval is completed, and it is loaded into the runtime memory as the verification benchmark; if no matching result is found or the matching result is not unique, the search is determined to have failed, the verification termination process is triggered, and the reason is recorded.

[0054] Step a2: If the search for standard test cases fails, the verification process for the currently executed test cases is terminated, and the reason for termination is recorded.

[0055] In this embodiment of the invention, by setting the standard test case search and verification as the first execution step in the verification process, the failure of the search will directly terminate all subsequent processes, eliminating the need to execute meaningless test steps. This can completely intercept invalid verification without a benchmark from the source, thereby significantly shortening the invalid execution cycle, reducing test resource consumption, further reducing the cost of test case development and debugging, and greatly improving the efficiency of test development.

[0056] Step S303: Perform a consistency check between the execution information of the current test step and the corresponding test step in the standard test case.

[0057] Specifically, step S303 includes: Step S3031: Verify whether the step identifier of the current test step matches the step identifier that should be executed in the standard test case; and / or, verify whether the execution order of the current test step is consistent with the execution order of the current test step in the standard test case; and / or, verify whether the signal identifier of the signal involved in the current test step matches the signal identifier specified in the standard test case; and / or, verify whether the signal value of the signal involved in the current test step is within the effective range of the corresponding signal specified in the standard test case.

[0058] In this embodiment, the verification content includes at least one of the following: step identifier, step execution order, signal identifier, and signal value range. Specifically, step identifier verification involves comparing the unique ID of the current step (e.g., "STEP_VOLT_001") with the standard step ID to avoid mismatched or incorrect execution. Step execution order verification involves comparing the execution order number of the current step (e.g., "3") with the execution order number in the standard test case to determine if there are skipped steps, missed steps, or reversed order. Signal identifier verification involves comparing the target signal identifier involved in the current step (e.g., "BATTERY_VOLT") with the signal identifier specified in the standard step to prevent signal mismatch and deviation of the test object from requirements. Signal value range verification involves comparing the preset effective voltage range of the current step (e.g., 2.2V~3.65V) with the range specified in the standard step to avoid misjudgment or omission of test results due to a loosened or tightened verification range.

[0059] In one specific embodiment, if all selected dimensions match, the verification is deemed successful, and the subsequent verification signal generation process is allowed; if any dimension does not match, the verification is deemed unsuccessful, the current use case verification is immediately terminated, and the unsuccessful dimension, deviation content, and step identifier are recorded in a structured manner.

[0060] This invention constructs a multi-dimensional, comprehensive, and configurable test step consistency verification mechanism, which fully covers the four core elements of test step execution: identity identification, execution process, test object, and verification benchmark. Subsequent test processes are only executed after verification is passed, which can intercept non-compliant test executions at the source and completely avoid invalid tests. At the same time, it maximizes adaptability to the needs of different test scenarios, taking into account both control precision and test execution efficiency. This further ensures test effectiveness, reduces development costs, and improves the credibility of test results.

[0061] In step S304, if the verification passes, a verification signal is generated based on the status signal and sent to the test system so that the test system can generate test results based on the verification signal.

[0062] In this embodiment, the step S304 above, which generates a verification signal based on a state signal, includes: Step b1: Process the status signal according to the preset verification strategy to generate a verification signal; wherein the preset verification strategy includes at least one of the following operation methods: Keep the state signal unchanged; Change the signal value of at least one of the status signals to an unexpected value that does not conform to the valid range of the corresponding signal specified in the standard test case; Change the signal value of at least one of the state signals to a non-theoretical value that is outside the theoretical effective range of the signal; Filter the status signals to select the parts of the signals that are related to the specified simulation node and / or the specified function; Add at least one interference signal that is unrelated to the current test scenario to the status signal. The interference signal includes signals whose signal identifier is not in the preset valid set, and / or signals whose signal identifier is in the preset valid set but whose signal data format does not match the preset format.

[0063] It should be noted that keeping the status signal unchanged in this embodiment aims to verify the basic recognition capability of the test system for normal operating conditions and to verify whether it can output the correct test pass result for signals that meet the requirements; for example, keeping the original voltage signal of 3.2V and the message format completely unchanged, a verification signal is directly generated to test the recognition capability of the automated test system for normal operating conditions.

[0064] To further explain, in this embodiment, the signal value of at least one of the status signals is changed to an unexpected value that does not conform to the effective range of the corresponding signal specified in the standard test case. This is intended to verify the anomaly identification capability of the test system and check whether it can output the correct test failure result for signals that do not meet the requirements. For example, the voltage signal value is changed to 2.5V (which is an unexpected value that does not conform to the standard effective range of 2.8V~3.4V) to generate a verification signal for testing the anomaly identification capability of the system.

[0065] To further explain, in this embodiment, changing the signal value of at least one of the status signals to a non-theoretical value that exceeds the theoretical effective range of the signal is intended to verify the boundary processing capability of the test system, that is, whether problems such as program crash, logical confusion, misjudgment or missed judgment occur under abnormal operating conditions of illegal values; for example, changing the voltage signal value to 2.0V (a non-theoretical value that exceeds the theoretical effective range of 2.2V~3.65V) generates a verification signal for testing the boundary processing capability of the system.

[0066] To further explain, in this embodiment, some signals related to a specified simulation node and / or a specified function are filtered from the status signals. This is intended to verify the robustness of the test system, that is, whether problems such as program crashes, logical confusion, misjudgments, and omissions occur under abnormal operating conditions where signals are missing. For example, some signals related only to the battery management simulation node are filtered from the original message, and other irrelevant signals are removed to generate a verification signal, which is used to test the robustness of the system in signal missing scenarios.

[0067] To further explain, in this embodiment, at least one interference signal unrelated to the current test scenario is added to the status signal. This is intended to verify the anti-interference capability of the test system, i.e., whether problems such as program crashes, logical confusion, misjudgments, or missed judgments occur under abnormal operating conditions of communication interference. For example, an interference signal with signal ID 0x0B (not within the preset valid set 0x00~0x0A) is added to the original message, or an interference signal with ID 0x05 but whose DLC (Data Length Code) does not conform to the preset format is added to generate a verification signal for testing the anti-interference capability of the system.

[0068] In this embodiment of the invention, a multi-mode, configurable, and fully covered verification signal generation mechanism is constructed. That is, through multiple types of combinable signal processing operations, the verification requirements of the automated testing system in all scenarios are fully covered. This supports the full-dimensional verification of the automated testing system from basic functions and anomaly identification capabilities to robustness, while significantly reducing test development costs and ensuring the reliability of the testing system from the root.

[0069] Step S305: Receive the test results fed back by the test system, and generate a verification report based on the comparison between the test results and the expected results corresponding to the current test step in the standard test case.

[0070] In this embodiment, step S305 includes: Step S3051: Receive the test results fed back by the test system.

[0071] In this embodiment, the relevant content of the test results is as described above and will not be repeated here.

[0072] Step S3052: If the test result is consistent with the expected result corresponding to the current test step in the standard test case, then the current test step is determined to have passed verification, and the step identifier and the result of the step verification of the current test step are recorded to generate a verification report for the currently executed test case.

[0073] In this embodiment, after determining that the current test step has passed verification in step S3052, the test case verification method of this embodiment further includes: Step c1: Check whether the current test step is the last test step of the currently executed test case.

[0074] In this embodiment, since a test case contains multiple test steps, this step aims to accurately determine whether the verification of all steps of the entire test case has been completed after the current test step is verified. This determines whether to trigger the verification report generation process or switch to the next test step in a preset order to continue execution, thereby greatly ensuring the balance between the integrity of test case verification and execution efficiency.

[0075] Step c2: If the current test step is the last test step of the currently executed test case, then execute the process of recording the step identifier and the step verification pass result of the current test step to generate a verification report for the currently executed test case.

[0076] Step c3: If the current test step is not the last test step of the currently executed test case, then determine the next test step according to the execution order of the test steps, update the next test step to the current test step, and repeat the process of verifying the consistency between the execution information of the current test step and the corresponding test steps in the standard test case.

[0077] In this embodiment of the invention, by using single-step verification as the sole prerequisite for the execution of subsequent steps, the process strictly follows the execution order preset by the standard test cases. At the same time, the final step detection mechanism ensures that all test cases are tested without omissions, skips, or incorrect order. This design avoids invalid verification caused by incorrect step timing or missed requirements from the root of the execution process, greatly ensuring that the verification process conforms to the system / software requirements design.

[0078] Step S3053: If the test result is inconsistent with the expected result corresponding to the current test step in the standard test case, the current test step is determined to have failed verification. The step identifier, the step verification failure result and the verification failure reason of the current test step are recorded to generate a verification report for the currently executed test case. At the same time, the verification of the currently executed test case is terminated.

[0079] In this embodiment, when it is clear that the verification of the current test step has failed, the entire process of consistency verification, verification signal generation, and result comparison of the subsequent steps of the current test case is immediately stopped, and no further instructions are sent to the test system, which can avoid the waste of computing power and time resources.

[0080] In this embodiment of the invention, the test results output by the test system are precisely compared with the expected results preset in the standard test cases. Using the source benchmark of the requirement design as the standard, the execution results of the automated test system are directly determined to meet the design requirements. For the first time, a step-level closed-loop verification of the effectiveness of the automated test system itself is realized. This not only ensures the rigor and accuracy of the verification results, but also ensures the credibility of the entire automated test system from the root.

[0081] In practical applications, even if a test step fails a consistency check or its execution logic deviates completely from the standard test case, traditional solutions will still force the remaining steps to run, resulting in a serious waste of testing computing power and time resources. Alternatively, there are drawbacks such as poor controllability of the automated testing process after startup, inability to flexibly mitigate losses in unexpected scenarios, and inconvenience in starting and stopping during the debugging phase. Based on this, this embodiment also designs a termination method for test case verification for different testing needs. Therefore, the test case verification method in this embodiment further includes: terminating the verification of the currently executing test case when all test steps of the currently executing test case have been completed, or when the consistency check between the execution information of the current test step and the corresponding test step in the standard test case fails, or when an externally input verification termination command is received.

[0082] It's worth noting that for large-scale test case development, debugging, and batch verification scenarios involving over 6,000 test cases, terminating the test upon detecting anomalies (such as failed consistency checks) significantly reduces the time spent on ineffective execution and lowers test resource consumption. This addresses the core industry pain points of long development and debugging cycles, high costs, and low efficiency from the execution stage. Furthermore, using externally input verification termination commands as termination triggers provides testers with a flexible manual intervention point. In unexpected scenarios such as abnormal test environments, ECU malfunctions, incorrect verification strategy configurations, or the need for temporary start / stop of test case debugging, testers can issue termination commands at any time to mitigate losses, helping to avoid unnecessary resource consumption and time wastage. Simultaneously, during the test case development and debugging phase, the verification process can be flexibly started and stopped, significantly improving debugging efficiency and further enhancing the solution's practicality, controllability, and scenario adaptability.

[0083] In this embodiment of the invention, by designing multiple parallel-triggered termination conditions, all termination scenarios of the entire lifecycle of test case verification are fully covered. These include standardized closing after all steps are completed in normal execution scenarios, timely interception of anomalies in execution deviation scenarios, and flexible manual intervention in sudden demand scenarios. This completely solves the pain point of the crude termination logic in existing technologies, constructs a complete closed loop of verification process, and takes into account both the integrity of the verification process and the flexibility of execution control.

[0084] In one specific embodiment, based on Figure 1 This paper proposes a test case verification system and its corresponding workflow, including the following specific steps: Step 1: Receive the test case number executed by the automated testing system; the automated testing system actively transmits this information via software network signals.

[0085] Step 2: Check if the test case number exists in the test case library. If the number exists, continue running; otherwise, exit the test and record the test case number and the reason for exit in the test report. The test case steps will describe the signal content, including the signal value and the signal value range. Compare whether the signal value meets the expectations of the test case steps.

[0086] Step 3: Receive the test steps and expected results executed by the automated testing system, identify and compare whether the test steps and expected results in the test cases are consistent. If they are consistent, continue running; otherwise, exit the test and record the test case number and the reason for exit in the test report.

[0087] Step 4: After receiving the ECU status signal, you can choose to perform the following operations: ① The current ECU status signal is forwarded completely and consistently to the automated test system to test whether the automated test cases can correctly determine the current ECU status and output the result of the current test step passing the test.

[0088] ② Modify the current ECU status signal to an unexpected result and forward it to the automated test system to test whether the automated test case can correctly determine the current ECU status and output the result that the current test step failed.

[0089] ③ Modify the current ECU status signal to a theoretically non-existent value and forward it to the automated testing system to test the robustness of the automated test cases. For example, the normal value range of a lithium iron phosphate battery system is defined as 2.2V~3.65V, and theoretically non-existent values ​​are values ​​less than 2.2V and greater than 3.65V; or, in the system design, the signal Fault_level can only be 0, 1, 2, or 3, and theoretically non-existent values ​​are values ​​other than 0, 1, 2, or 3.

[0090] ④ Select a portion of the forwarded values ​​of the current ECU status signal to automate the test system and test the robustness of the automated test cases. For example, the automated test system contains various ECU simulation nodes required by the ECU under test, and different functional messages of various simulation nodes. Various simulation nodes and functions can be used as filtering conditions.

[0091] ⑤ Forward the current ECU status signal completely and add irrelevant diagnostic signals to the automated test system to test the robustness of the automated test cases. For example, since the signal ID range required by the ECU under test in its normal operating environment is 0~10, adding signals with IDs other than 0~10 will result in irrelevant diagnostic signals; or, in the CAN network, if the ID is in the range of 0~10, but the DLC does not match the required message of the ECU under test, it can also be used as an irrelevant diagnostic signal.

[0092] Step 5: Receive the test results from the automated test system executing the automated test cases, and determine whether the automated test cases' judgment results on the ECU status signals meet the test expectations in Step 4. If they meet the expectations, continue testing; otherwise, record the corresponding test step number and the reason for exiting the test.

[0093] Step 6: Repeat steps 3, 4, and 5 until the current test case ends.

[0094] Step 7: Repeat steps 1, 2, 3, 4, 5, and 6 until the testing of all current automated test cases is completed.

[0095] Note that in this embodiment, the boundary conditions for test termination include not only the complete execution of all steps of the currently executing test case, but also inconsistencies between the test step results and the expected results during execution and / or manual intervention to stop the test.

[0096] Step 8: Output the test report of the test cases to verify the system.

[0097] In summary, this embodiment achieves the technical effect of automated test cases by adding the operation of modifying ECU status signals to the ECU and automated testing system; it realizes multi-dimensional automated test cases by selectively changing ECU status signals; and it improves the debugging efficiency of automated test cases by adding a test case verification system to the testing system.

[0098] In one specific embodiment, Figure 4 This is a flowchart illustrating the workflow of the test case verification system. As shown in the diagram, the process includes the following steps: Step S1, ECU status.

[0099] In this embodiment, the ECU status is the original data source for the entire process.

[0100] Step S2: The test case verification system modifies and forwards the ECU status to the automated test system.

[0101] In this embodiment, this step is a core intermediate control unit in the entire process. It is a third-party verification carrier independent of the ECU under test and the automated testing system, and its core functions are twofold: 1. Controllable signal processing: Receives the original status signal from the ECU and can flexibly process the signal according to verification requirements (such as keeping the original value unchanged, modifying it to an abnormal / boundary value, injecting interference signals, filtering some signals, etc.) to simulate full-scenario test conditions such as normal, abnormal, boundary, and communication interference. 2. Signal forwarding: The processed ECU status signal is forwarded to the downstream automated test system, replacing the link in the traditional direct connection architecture where the ECU directly sends signals to the test system. This achieves complete control over the input to the test system and provides controllable test conditions for subsequent verification of the validity of test cases.

[0102] Step S3: Receive the test results of each test case from the automated testing system.

[0103] In this embodiment, the test case verification system receives the test results in this step, thereby providing a core comparison object for subsequent test case validity verification. That is, after the automated test system receives the ECU status signal forwarded by the verification system, it will parse the signal and determine its compliance according to the preset automated test cases, and output the test results of the corresponding test cases, such as the final determination of whether the test passes or fails, the signal parsing value, the determination basis, and the abnormal alarm.

[0104] Step S4: Determine whether the current test automation test case design is normal based on the expected results of the test cases.

[0105] In this embodiment, this step is the core judgment step of the entire verification process. Because the test case verification system pre-stores standard expected results bound to the test conditions (strongly corresponding to the ECU signal processing strategy), for example, when the verification system modifies the ECU signal to a fault abnormal value, the corresponding expected result is "the automated test case should determine that the test fails." This step completes the validity judgment of the test case design by accurately comparing the actual output result of the automated test system with the preset standard expected result. If the results are consistent, the test case design is considered to be normal and can accurately identify the corresponding working conditions. If the results are inconsistent, the test case is determined to have design flaws (such as missed exceptions, false positives, signal recognition errors, logic vulnerabilities, etc.).

[0106] Step S5: Output the test automation test case report.

[0107] In this embodiment, this step is the final output loop of the entire process. Specifically, the test case verification system generates a standardized and traceable test case verification report based on the verification data from the entire process. The report includes: basic information of the test cases, a complete record of the ECU signal processing process, test results from the automated testing system, a conclusion on the validity of the test case design, defect details and root causes, verification time, etc., providing accurate data support for test case optimization, defect repair, and compliance archiving, and is compatible with the relevant requirements of the ECU functional safety standard for the testing system.

[0108] This embodiment also provides a test case verification device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" refers to a combination of software and / or hardware that performs a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementations, or combinations of software and hardware, are also possible and contemplated.

[0109] This embodiment provides a test case verification device applied to a verification system. The verification system is deployed between the target under test and the test system via an interface, such as... Figure 5 As shown, the device includes: The receiving module 501 is used to receive the identification information of the currently executed test case and the execution information of the current test step sent by the test system, as well as the status signals fed back by the target under test in response to the test system.

[0110] The acquisition module 502 is used to acquire the standard test cases corresponding to the identification information. The standard test cases contain at least one test step and the expected results of the corresponding steps.

[0111] The verification module 503 is used to verify the consistency between the execution information of the current test step and the corresponding test steps in the standard test cases.

[0112] The generation module 504 is used to generate a verification signal based on the status signal and send it to the test system if the verification passes, so that the test system can generate test results based on the verification signal.

[0113] The comparison module 505 is used to receive the test results fed back by the test system and generate a verification report based on the comparison between the test results and the expected results corresponding to the current test steps in the standard test cases.

[0114] In some alternative implementations, the acquisition module 502 includes: The first acquisition submodule is used to search for the standard test cases corresponding to the identification information from the test case library of the verification system.

[0115] The second acquisition submodule is used to terminate the verification process of the currently executed test case and record the reason for termination if the search for standard test cases fails.

[0116] In some optional implementations, the verification module 503 includes: a verification submodule, used to verify whether the step identifier of the current test step matches the step identifier that should be executed in the standard test case; and / or, to verify whether the execution order of the current test step is consistent with the order of the steps that should be executed in the standard test case; and / or, to verify whether the signal identifier of the signal involved in the current test step matches the signal identifier specified in the standard test case; and / or, to verify whether the signal value of the signal involved in the current test step is within the effective range of the corresponding signal specified in the standard test case.

[0117] In some optional implementations, the generation module 504 includes: a generation submodule, used to process the status signal according to a preset verification strategy to generate a verification signal; wherein the preset verification strategy includes at least one of the following operation modes: Keep the state signal unchanged; Change the signal value of at least one of the status signals to an unexpected value that does not conform to the valid range of the corresponding signal specified in the standard test case; Change the signal value of at least one of the state signals to a non-theoretical value that is outside the theoretical effective range of the signal; Filter the status signals to select the parts of the signals that are related to the specified simulation node and / or the specified function; Add at least one interference signal that is unrelated to the current test scenario to the status signal. The interference signal includes signals whose signal identifier is not in the preset valid set, and / or signals whose signal identifier is in the preset valid set but whose signal data format does not match the preset format.

[0118] In some alternative implementations, the comparison module 505 includes: The first comparison submodule is used to receive the test results fed back by the test system.

[0119] The second comparison submodule is used to determine that the current test step has passed verification if the test result is consistent with the expected result corresponding to the current test step in the standard test case, and to record the step identifier and the result of the step verification of the current test step in order to generate a verification report for the currently executed test case.

[0120] The third comparison submodule is used to determine that the current test step has failed if the test result is inconsistent with the expected result corresponding to the current test step in the standard test case. It records the step identifier, the step verification failure result and the reason for the verification failure of the current test step, so as to generate a verification report for the currently executed test case; at the same time, it terminates the verification of the currently executed test case.

[0121] In some optional implementations, the second alignment submodule includes: The first comparison unit is used to detect whether the current test step is the last test step of the currently executed test case.

[0122] The second comparison unit is used to execute a process that records the step identifier and the verification result of the current test step if the current test step is the last test step of the currently executed test case, so as to generate a verification report for the currently executed test case.

[0123] The third comparison unit is used to determine the next test step according to the execution order of the test steps if the current test step is not the last test step of the currently executed test case, update the next test step to the current test step, and repeat the process of verifying the consistency between the execution information of the current test step and the corresponding test steps in the standard test case.

[0124] In some optional implementations, the apparatus further includes a termination module, configured to terminate the verification of the currently executed test case when all test steps of the currently executed test case have been completed, or when the consistency verification between the execution information of the current test step and the corresponding test step in the standard test case fails, or when an externally input verification termination instruction is received.

[0125] The test case verification device provided in this embodiment of the invention can execute the test case verification method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.

[0126] The test case verification device in this embodiment of the invention adds an intermediate verification system between the target under test and the test system. On the one hand, it verifies the consistency between the test steps of the test system and the standard test cases. On the other hand, it achieves comprehensive verification of the effectiveness and accuracy of the automated test system itself by controllably generating verification signals and comparing the output results of the test system with the standard expectations. At the same time, through the automated consistency verification mechanism, it can batch and automatically check for problems such as step deviations, test case mismatches, and parameter inconsistencies in the execution process of test cases. This eliminates the need for testers to manually debug each test case throughout the entire process, significantly shortens the development and debugging cycle of test cases, reduces labor costs, and significantly improves the development and delivery efficiency of test cases.

[0127] In this embodiment, Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. See below for details. Figure 6This diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device includes a controller, which may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from memory 608 into a random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the electronic device. The processor 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0128] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0129] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a memory 608, or installed from a ROM 602. When the computer program is executed by the processor 601, it performs the functions defined in the test case verification method of the embodiments of the present invention.

[0130] Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0131] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the test case verification method shown in the above embodiments is implemented.

[0132] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A test case verification method, characterized in that, Applied to a verification system, which is deployed between the target under test and the test system via an interface, the method includes: Receive identification information of the currently executed test case and execution information of the current test step sent by the test system, as well as status signals fed back by the target under test in response to the test system; Obtain the standard test case corresponding to the identification information, wherein the standard test case includes at least one test step and the expected result of the corresponding step; Perform a consistency check between the execution information of the current test step and the corresponding test step in the standard test case; If the verification passes, a verification signal is generated based on the status signal and sent to the test system, so that the test system generates a test result based on the verification signal. The system receives the test results from the test system and generates a verification report based on the comparison between the test results and the expected results corresponding to the current test step in the standard test case.

2. The test case verification method according to claim 1, characterized in that, The step of verifying the consistency between the execution information of the current test step and the corresponding test step in the standard test case includes: Verify whether the step identifier of the current test step matches the step identifier that should be executed in the standard test case; And / or, verify whether the execution order of the current test steps is consistent with the order of steps that should be executed in the standard test case; And / or, verify whether the signal identifiers of the signals involved in the current test step match the signal identifiers specified in the standard test case; And / or, verify whether the signal value of the signal involved in the current test step is within the valid range of the corresponding signal specified in the standard test case.

3. The test case verification method according to claim 1, characterized in that, The generation of the verification signal based on the state signal includes: The status signal is processed according to a preset verification strategy to generate a verification signal; wherein the preset verification strategy includes at least one of the following operation modes: Keep the state signal unchanged; Change the signal value of at least one of the status signals to an unexpected value that does not conform to the valid range of the corresponding signal specified in the standard test case; Change the signal value of at least one of the state signals to a non-theoretical value that is outside the theoretical effective range of the signal; Filter the signals from the status signals to select the portions of the signals that are related to the specified simulation node and / or the specified function; At least one interference signal unrelated to the current test scenario is added to the status signal. The interference signal includes signals whose signal identifiers are not in a preset valid set, and / or signals whose signal identifiers are in a preset valid set but whose signal data format does not match the preset format.

4. The test case verification method according to claim 1, characterized in that, Based on the comparison between the test results and the expected results corresponding to the current test step in the standard test case, a verification report is generated, including: If the test result is consistent with the expected result corresponding to the current test step in the standard test case, then the current test step is determined to have passed verification, and the step identifier and the result of the step verification of the current test step are recorded to generate a verification report for the currently executed test case. If the test result is inconsistent with the expected result corresponding to the current test step in the standard test case, the current test step is determined to have failed verification. The step identifier, the step verification failure result, and the reason for the verification failure of the current test step are recorded to generate a verification report for the currently executed test case. At the same time, the verification of the currently executed test case is terminated.

5. The test case verification method according to claim 4, characterized in that, After determining that the current test step has passed verification, the method further includes: Check whether the current test step is the last test step of the currently executed test case; If the current test step is the last test step of the currently executed test case, then the process of recording the step identifier and the step verification pass result of the current test step is executed to generate a verification report for the currently executed test case. If the current test step is not the last test step of the currently executed test case, then the next test step is determined according to the execution order of the test steps, the next test step is updated to the current test step, and the process of verifying the consistency between the execution information of the current test step and the corresponding test step in the standard test case is repeated.

6. The test case verification method according to any one of claims 1 to 5, characterized in that, The method further includes: The verification of the currently executing test case shall be terminated when all test steps of the currently executing test case have been completed, or when the consistency verification between the execution information of the current test step and the corresponding test step in the standard test case fails, or when an external verification termination instruction is received.

7. The test case verification method according to claim 1, characterized in that, The standard test cases for obtaining the identification information include: Search the test case library of the verification system for the standard test case corresponding to the identification information; If the search for the standard test case fails, the verification process for the currently executed test case is terminated, and the reason for termination is recorded.

8. A test case verification device, characterized in that, The device is used in a verification system, which is deployed between the target under test and the test system via an interface. The device includes: The receiving module is used to receive identification information of the currently executed test case and execution information of the current test step sent by the test system, as well as status signals fed back by the target under test in response to the test system; The acquisition module is used to acquire the standard test cases corresponding to the identification information, wherein the standard test cases include at least one test step and the expected results of the corresponding steps; The verification module is used to verify the consistency between the execution information of the current test step and the corresponding test step in the standard test case; The generation module is used to generate a verification signal based on the status signal and send it to the test system if the verification passes, so that the test system generates a test result based on the verification signal. The comparison module is used to receive the test results fed back by the test system, and generate a verification report based on the comparison between the test results and the expected results corresponding to the current test step in the standard test case.

9. An electronic device, characterized in that, The electronic device includes a controller, which includes a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the test case verification method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the test case verification method according to any one of claims 1 to 7.

Images