A code testing method, apparatus, device, and storage medium

By generating an executable file from the test case list, running the file, and performing data verification, the problems of high resource consumption and uncontrollable results in existing chip testing methods are solved, achieving efficient and reliable code testing.

CN123068271APending Publication Date: 2026-11-10厦门国科安芯科技有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611138770.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-29
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

In existing chip code testing methods, ATE equipment consumes a lot of testing resources, self-developed automated testing equipment has problems with neglecting certain steps, test results are uncontrollable, and manual testing by testers is unreliable in terms of quality and efficiency.

Method used

A code testing method is provided, which obtains a test case list, schedules test cases to generate executable files, runs the files to obtain actual and expected output data, determines the verification results based on the data verification mechanism, and generates test records, including numerical comparisons and execution time records of the macro encapsulation layer, the verification execution layer, and the verification strategy layer.

Benefits of technology

It enables reliable code testing, improves test quality and efficiency, reduces resource consumption, and ensures the controllability and accuracy of test results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123068271A_ABST
    Figure CN123068271A_ABST
Patent Text Reader

Abstract

This disclosure provides an exemplary embodiment of a code testing method, apparatus, device, and storage medium. The method includes: obtaining a test case list, scheduling test cases in the test case list, and generating an executable file; running the executable file, obtaining actual output data and expected output data of a target checkpoint, and determining a verification result between the actual output data and the expected output data based on a preset data verification mechanism; obtaining execution data, and generating a test record based on the execution data and the verification result, wherein the execution data includes at least the execution time. This invention effectively realizes code testing through test case scheduling, execution, data verification, and test record generation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of code testing technology, and in particular to a code testing method, apparatus, device, and storage medium. Background Technology

[0002] Currently, chip code testing methods mainly include ATE equipment testing, self-developed automated testing equipment testing, and manual testing by testers. However, in existing technologies, ATE equipment code testing requires a lot of resources, self-developed automated testing equipment code testing may have some oversights, leading to uncontrollable test results, and manual testing by testers is unreliable in terms of both test quality and efficiency. Summary of the Invention

[0003] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a code testing method, apparatus, device, and storage medium.

[0004] This exemplary embodiment provides a code testing method, the method comprising: Obtain a list of test cases, schedule the test cases in the list, and generate an executable file; Run the executable file to obtain the actual output data and expected output data of the target checkpoint, and determine the verification result of the actual output data and the expected output data based on the preset data verification mechanism; Obtain execution data, and generate test records based on the execution data and the verification results. The execution data includes at least the execution time.

[0005] In some exemplary embodiments, the data verification mechanism includes a macro encapsulation layer, a verification execution layer, and a verification strategy layer; The method for determining the verification result between the actual output data and the expected output data based on a preset data verification mechanism includes: Macro definitions are obtained based on the macro encapsulation layer, and the actual output data and expected output data of the target checkpoint are transmitted to the verification execution layer together. Based on the verification execution layer, the actual output data and the expected output data are numerically compared, and the numerical comparison results are temporarily stored in a preset verification buffer. The numerical comparison results include consistency and inconsistency. The module type of the module under test is obtained. Based on the verification strategy layer, the verification strategy corresponding to the module type is matched. The verification strategy is used as the judgment benchmark for the numerical comparison. The verification strategy includes register precise comparison, tolerance range comparison, timing tolerance comparison, cyclic redundancy check and frame format check, count value tolerance and interrupt trigger check, readback comparison and error correction code check, transmission length and destination address data check, and reset timeout check. After performing the numerical comparison at each target checkpoint of the executable file, the result summarization function provided by the verification execution layer is called to summarize and determine the numerical comparison results in the verification buffer. If all the numerical comparison results are consistent, the executable file is deemed to have passed the verification. If any of the numerical comparison results are inconsistent, the executable file verification is deemed to have failed. The result of the summary determination is used as the verification result between the actual output data and the expected output data.

[0006] In some exemplary embodiments, obtaining the actual output data and expected output data of the target checkpoint includes: Based on a preset control script, a low-level communication with the debugging interface is established to control the execution of the executable file. The execution control includes starting, pausing, and terminating. The execution control is paused at a preset target checkpoint, and the actual output data is obtained based on the type of the target checkpoint. The target checkpoint includes function return checkpoint, register checkpoint, and communication data checkpoint. If the target checkpoint is the function return checkpoint, then obtain the status code returned after the current function under test is executed, as the actual output data. The status code includes a correct code and an error code. If the target checkpoint is the register checkpoint, then based on the underlying communication, the current value of the corresponding peripheral register in the target hardware platform is obtained as the actual output data; If the target checkpoint is the communication data checkpoint, then the data payload received by the communication bus is read as the actual output data; Based on the type of the target checkpoint and the test case identifier corresponding to the executable file, the expected output data of the target checkpoint is obtained.

[0007] In some exemplary embodiments, obtaining a test case list, scheduling test cases from the test case list, and generating an executable file includes: Obtain the user-input command for the name of the module to be tested, find the corresponding module file path based on the name of the module to be tested, and obtain the list of test cases under the module file path; If the name of the module to be tested does not exist or is entered incorrectly, an error message will be sent to the user. Obtain the order of test cases in the test case list, and schedule each test case sequentially based on the order to form a test execution loop; If all test cases in the test case list have been scheduled, the test execution loop ends. In each test execution loop, based on the currently scheduled test cases, the parameter information of the target project is dynamically allocated, and the currently scheduled test cases are compiled to generate executable files.

[0008] In some exemplary embodiments, scheduling test cases in the test case list includes: During the scheduling of test cases in the test case list, it is dynamically checked whether the description information of the currently scheduled test case in the test case list is consistent with the actual name of the currently scheduled test case; If they match, continue executing the current test case. If there is a discrepancy, the scheduling is terminated and an error report is returned to the user. The error report includes a prompt message indicating that the description information is inconsistent with the actual name.

[0009] In some exemplary embodiments, running the executable file includes: Download the executable file to the target hardware platform; The executable file is run based on the target hardware platform.

[0010] In some exemplary embodiments, the step of obtaining execution data and generating test records based on the execution data and the verification results, wherein the execution data includes at least the execution time, including: Based on a preset communication script, a debugging communication link is established with the target hardware platform, the original result data corresponding to the verification result is extracted from the target hardware platform, and the original result data is transmitted back to the main control script environment; Based on the master control script in the master control script environment, obtain the expected execution results of the test cases; The original result data and the expected execution result are compared. If the original result data indicates that the verification is passed and the expected execution result is also passed, or if the original result data indicates that the verification is failed and the expected execution result is also failed, then the execution is determined to be consistent with the expectation; otherwise, it is determined to be inconsistent. Based on the comparison of the results, the final judgment result of the test case is generated; The execution time is the length of time from the start of the test case to the generation of the final judgment result. The execution data is obtained based on the original result data, the final judgment result, and the execution time; Based on the preset test record template, the execution data and the verification results are filled into the corresponding positions to generate test records.

[0011] An exemplary embodiment of this disclosure also provides a code testing apparatus, the apparatus comprising: The first generation module is used to obtain a test case list, schedule the test cases in the test case list, and generate an executable file. The determination module is used to run the executable file, obtain the actual output data and expected output data of the target checkpoint, and determine the verification result of the actual output data and the expected output data based on a preset data verification mechanism. The second generation module is used to acquire execution data and generate test records based on the execution data and the verification results. The execution data includes at least the execution time.

[0012] In some exemplary embodiments, the first generation module is specifically used for: Obtain the user-input command for the name of the module to be tested, find the corresponding module file path based on the name of the module to be tested, and obtain the list of test cases under the module file path; If the name of the module to be tested does not exist or is entered incorrectly, an error message will be sent to the user. Obtain the order of test cases in the test case list, and schedule each test case sequentially based on the order to form a test execution loop; If all test cases in the test case list have been scheduled, the test execution loop ends. In each test execution loop, based on the currently scheduled test cases, the parameter information of the target project is dynamically allocated, and the currently scheduled test cases are compiled to generate executable files.

[0013] In some exemplary embodiments, the first generation module specifically includes: During the scheduling of test cases in the test case list, it is dynamically checked whether the description information of the currently scheduled test case in the test case list is consistent with the actual name of the currently scheduled test case; If they match, continue executing the current test case. If there is a discrepancy, the scheduling is terminated and an error report is returned to the user. The error report includes a prompt message indicating that the description information is inconsistent with the actual name.

[0014] In some exemplary embodiments, the determining module is specifically used for: Download the executable file to the target hardware platform; The executable file is run based on the target hardware platform.

[0015] In some exemplary embodiments, the determining module is specifically used for: Based on a preset control script, a low-level communication with the debugging interface is established to control the execution of the executable file. The execution control includes starting, pausing, and terminating. The execution control is paused at a preset target checkpoint, and the actual output data is obtained based on the type of the target checkpoint. The target checkpoint includes function return checkpoint, register checkpoint, and communication data checkpoint. If the target checkpoint is the function return checkpoint, then obtain the status code returned after the current function under test is executed, as the actual output data. The status code includes a correct code and an error code. If the target checkpoint is the register checkpoint, then based on the underlying communication, the current value of the corresponding peripheral register in the target hardware platform is obtained as the actual output data; If the target checkpoint is the communication data checkpoint, then the data payload received by the communication bus is read as the actual output data; Based on the type of the target checkpoint and the test case identifier corresponding to the executable file, the expected output data of the target checkpoint is obtained.

[0016] In some exemplary embodiments, the data verification mechanism includes a macro encapsulation layer, a verification execution layer, and a verification strategy layer; The determining module specifically includes: Macro definitions are obtained based on the macro encapsulation layer, and the actual output data and expected output data of the target checkpoint are transmitted to the verification execution layer together. Based on the verification execution layer, the actual output data and the expected output data are numerically compared, and the numerical comparison results are temporarily stored in a preset verification buffer. The numerical comparison results include consistency and inconsistency. The module type of the module under test is obtained. Based on the verification strategy layer, the verification strategy corresponding to the module type is matched. The verification strategy is used as the judgment benchmark for the numerical comparison. The verification strategy includes register precise comparison, tolerance range comparison, timing tolerance comparison, cyclic redundancy check and frame format check, count value tolerance and interrupt trigger check, readback comparison and error correction code check, transmission length and destination address data check, and reset timeout check. After performing the numerical comparison at each target checkpoint of the executable file, the result summarization function provided by the verification execution layer is called to summarize and determine the numerical comparison results in the verification buffer. If all the numerical comparison results are consistent, the executable file is deemed to have passed the verification. If any of the numerical comparison results are inconsistent, the executable file verification is deemed to have failed. The result of the summary determination is used as the verification result between the actual output data and the expected output data.

[0017] In some exemplary embodiments, the second generation module specifically includes: Based on a preset communication script, a debugging communication link is established with the target hardware platform, the original result data corresponding to the verification result is extracted from the target hardware platform, and the original result data is transmitted back to the main control script environment; Based on the master control script in the master control script environment, obtain the expected execution results of the test cases; The original result data and the expected execution result are compared. If the original result data indicates that the verification is passed and the expected execution result is also passed, or if the original result data indicates that the verification is failed and the expected execution result is also failed, then the execution is determined to be consistent with the expectation; otherwise, it is determined to be inconsistent. Based on the comparison of the results, the final judgment result of the test case is generated; The execution time is the length of time from the start of the test case to the generation of the final judgment result. The execution data is obtained based on the original result data, the final judgment result, and the execution time; Based on the preset test record template, the execution data and the verification results are filled into the corresponding positions to generate test records.

[0018] An exemplary embodiment of this disclosure also provides an electronic device, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the code testing method as provided in the exemplary embodiment of this disclosure.

[0019] This disclosure also provides a computer-readable storage medium storing a computer program for executing the code testing method provided in this disclosure.

[0020] The technical solution provided by the exemplary embodiments of this disclosure has the following advantages compared with the prior art: The code testing method provided in this exemplary embodiment obtains a test case list, schedules the test cases in the test case list, and generates an executable file; runs the executable file, obtains the actual output data and expected output data of the target checkpoint, and determines the verification result of the actual output data and the expected output data based on a preset data verification mechanism; obtains execution data, and generates a test record based on the execution data and the verification result, wherein the execution data includes at least the execution time; the present invention effectively realizes code testing through test case scheduling, execution, data verification, and test record generation. Attached Figure Description

[0021] The above and other features, advantages, and aspects of the exemplary embodiments of this disclosure will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0022] Figure 1 A flowchart illustrating a code testing method provided as an exemplary embodiment of this disclosure; Figure 2 A schematic diagram of a code testing apparatus provided for an exemplary embodiment of this disclosure; Figure 3 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present disclosure. Detailed Implementation

[0023] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While some exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be construed as limited to the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the accompanying drawings and exemplary embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0024] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0025] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one exemplary embodiment" means "at least one exemplary embodiment"; the term "another exemplary embodiment" means "at least one additional exemplary embodiment"; the term "some exemplary embodiments" means "at least some exemplary embodiments". Definitions of other terms will be given in the description below.

[0026] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0027] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0028] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0029] To address the aforementioned issues, this disclosure provides a code testing method, which will be described below with reference to specific exemplary embodiments.

[0030] Figure 1 This is a flowchart illustrating a code testing method provided as an exemplary embodiment of the present disclosure. The method can be executed by a code testing device, which can be implemented in software and / or hardware and is generally integrated into an electronic device.

[0031] An exemplary embodiment of this disclosure provides a code testing method, including: Step S101: Obtain the test case list, schedule the test cases in the test case list, and generate an executable file.

[0032] In some exemplary embodiments, obtaining a test case list, scheduling test cases from the test case list, and generating an executable file includes: Obtain the user-input command for the name of the module to be tested, find the corresponding module file path based on the name of the module to be tested, and obtain the list of test cases under the module file path; If the name of the module to be tested does not exist or is entered incorrectly, an error message will be sent to the user. Obtain the order of test cases in the test case list, and schedule each test case sequentially based on the order to form a test execution loop; If all test cases in the test case list have been scheduled, the test execution loop ends. In each test execution loop, based on the currently scheduled test cases, the parameter information of the target project is dynamically allocated, and the currently scheduled test cases are compiled to generate executable files.

[0033] In some exemplary embodiments, scheduling the test cases in the test case list includes: During the scheduling of test cases in the test case list, it is dynamically checked whether the description information of the currently scheduled test case in the test case list is consistent with the actual name of the currently scheduled test case; If they match, continue executing the current test case. If there is a discrepancy, the scheduling is terminated and an error report is returned to the user. The error report includes a prompt message indicating that the description information is inconsistent with the actual name.

[0034] In some exemplary embodiments, the user enters the specified name of the module to be tested in the command-line window interface, generates the command for the name of the module to be tested, and starts the code testing process.

[0035] In some exemplary embodiments, after obtaining the user's input instruction for the name of the module to be tested, a judgment mechanism is first triggered. If there is no test case corresponding to the name of the module to be tested (the name of the module to be tested does not exist) or the user inputs an incorrect name of the module to be tested, the user is prompted in the command line window interface. If it is determined that the user inputs the correct module name instruction or that the corresponding name of the module to be tested exists, the corresponding module file path is found according to the input name of the module to be tested, and then the test case list of the module is searched and read.

[0036] In some exemplary embodiments, the test case list is used to summarize all test cases of the module under test, and the test cases are used to perform code testing on the module under test; the test case list can be configured and dynamically adjusted by the user, and the user can choose the combination of test cases to be executed according to the actual testing needs, and customize the sequence of test case execution.

[0037] In some exemplary embodiments, the test case list includes information such as the sorting order of the test cases, the name of each test case, and the expected execution result.

[0038] In some exemplary embodiments, after obtaining the information in the test case list, a loop mechanism is executed to schedule and execute all operation steps associated with each test case in sequence based on the order of the test cases in the test case list, such as environment initialization and interface calls.

[0039] In some exemplary embodiments, the loop mechanism ends after all test cases in the test case list have been executed.

[0040] In some exemplary embodiments, the parameter information of the target project refers to a series of compilation and linking environment variables that need to be configured when generating an executable file based on test cases, such as macro definitions, compilation parameters, memory address mapping, debugging interfaces, file paths, etc.

[0041] In some exemplary embodiments, test cases exist in the form of source code (such as MCAL test programs written in C language). However, the target hardware platform (such as a microcontroller based on the RISC-V architecture) that performs subsequent code testing cannot recognize the source code. Therefore, the test cases are compiled into a sequence of instructions executable by the target hardware platform for code testing.

[0042] Step S102: Run the executable file to obtain the actual output data and expected output data of the target checkpoint, and determine the verification result of the actual output data and the expected output data based on the preset data verification mechanism.

[0043] In some exemplary embodiments, running the executable file includes: Download the executable file to the target hardware platform; The executable file is run based on the target hardware platform.

[0044] In some exemplary embodiments, obtaining the actual output data and expected output data of the target checkpoint includes: Based on a preset control script, a low-level communication with the debugging interface is established to control the execution of the executable file. The execution control includes starting, pausing, and terminating. The execution control is paused at a preset target checkpoint, and the actual output data is obtained based on the type of the target checkpoint. The target checkpoint includes function return checkpoint, register checkpoint, and communication data checkpoint. If the target checkpoint is the function return checkpoint, then obtain the status code returned after the current function under test is executed, as the actual output data. The status code includes a correct code and an error code. If the target checkpoint is the register checkpoint, then based on the underlying communication, the current value of the corresponding peripheral register in the target hardware platform is obtained as the actual output data; If the target checkpoint is the communication data checkpoint, then the data payload received by the communication bus is read as the actual output data; Based on the type of the target checkpoint and the test case identifier corresponding to the executable file, the expected output data of the target checkpoint is obtained.

[0045] In some exemplary embodiments, the data verification mechanism includes a macro encapsulation layer, a verification execution layer, and a verification strategy layer; The method for determining the verification result between the actual output data and the expected output data based on a preset data verification mechanism includes: Macro definitions are obtained based on the macro encapsulation layer, and the actual output data and expected output data of the target checkpoint are transmitted to the verification execution layer together. Based on the verification execution layer, the actual output data and the expected output data are numerically compared, and the numerical comparison results are temporarily stored in a preset verification buffer. The numerical comparison results include consistency and inconsistency. The module type of the module under test is obtained. Based on the verification strategy layer, the verification strategy corresponding to the module type is matched. The verification strategy is used as the judgment benchmark for the numerical comparison. The verification strategy includes register precise comparison, tolerance range comparison, timing tolerance comparison, cyclic redundancy check and frame format check, count value tolerance and interrupt trigger check, readback comparison and error correction code check, transmission length and destination address data check, and reset timeout check. After performing the numerical comparison at each target checkpoint of the executable file, the result summarization function provided by the verification execution layer is called to summarize and determine the numerical comparison results in the verification buffer. If all the numerical comparison results are consistent, the executable file is deemed to have passed the verification. If any of the numerical comparison results are inconsistent, the executable file verification is deemed to have failed. The result of the summary determination is used as the verification result between the actual output data and the expected output data.

[0046] In some exemplary embodiments, the target checkpoint works as follows: each function call or data interaction in the test case will output status or data. By monitoring the function return value or the correctness of the interaction data, the current software and hardware status is monitored after the function is called to determine whether it is consistent with the configuration expectation. If there is a module under test that needs to check register values, the value of the peripheral register is read and compared with the expected value.

[0047] In some exemplary embodiments, the control script pauses the execution of the program corresponding to the code use case at a preset target checkpoint and obtains the corresponding actual output data according to the type of the target checkpoint.

[0048] In some exemplary embodiments, a function return checkpoint refers to a target checkpoint where a status code is returned after the function is executed; a register checkpoint refers to a target checkpoint where the current value of the corresponding peripheral register in the target hardware platform is read; and a communication data checkpoint refers to a target checkpoint where data interaction is performed.

[0049] In some exemplary embodiments, the status code returned after the current function under test is executed is an inherent return value, the meaning of which is predetermined by the function.

[0050] In some exemplary embodiments, the communication data checkpoint is used in a test scenario where an external device cooperates with a communication module to send data, and determines whether the communication function is normal by comparing the actual received data with the expected received data.

[0051] In some exemplary embodiments, the data payload refers to the effective data content that actually needs to be transmitted after removing protocol overhead during data transmission.

[0052] In some exemplary embodiments, the expected output data is data pre-written during the test case design phase, such as expected function return values, expected register configuration values, and expected received data.

[0053] In some exemplary embodiments, the macro encapsulation layer is used to provide test case writers with a simplified calling interface, which encapsulates the verification logic into standardized macro definitions. When performing verification on the code, the preset macro definitions are called to trigger the corresponding verification strategy and verification operation, thereby reducing the complexity of writing test cases.

[0054] In some exemplary embodiments, the verification buffer is a storage area pre-allocated in the memory of the target hardware platform specifically for testing, used to temporarily store the numerical comparison results obtained by comparing the actual output data and the expected output data; the verification buffer predefines the verification result for each macro definition call.

[0055] In some exemplary embodiments, the verification execution layer is used to implement specific comparison logic, temporarily store the verification results in the verification buffer, and after all test cases have been executed, the results are summarized through the result summary function, and the overall test result of the corresponding test cases is reported as to whether it has passed. The final result does not need to be output by other peripheral devices. The memory information of the verification buffer can be automatically read through the debugging interface using a preset debugging script for subsequent operations.

[0056] In some exemplary embodiments, the module under test is developed in a standardized manner, and the corresponding functions include input parameter checks, software status and hardware status judgments, and return the execution result of the function. Therefore, by comparing the numerical values, that is, comparing the actual function return value with the expected function return value, the correctness of the function execution can be determined.

[0057] In some exemplary embodiments, the verification strategy layer employs differentiated verification strategies for different modules, thereby ensuring the accuracy and reliability of the verification structure.

[0058] In some exemplary embodiments, the module type of the module under test is obtained, and based on the verification strategy layer, the verification strategy corresponding to the module type is matched, and the verification strategy is used as the judgment criterion for the numerical comparison. The module types of the module under test (MCAL module) include Dio, Port, ADC, PWM, ICU, CAN, SPI, I2C, GPT, Timer, Flash, DMA, and WDT.

[0059] In some exemplary embodiments, differentiated verification strategies are used for different modules under test, as follows: If the module under test is a Dio / Port, the verification strategy uses precise register comparison, that is, checking whether the configuration registers such as GPIO direction, level, pull-up / pull-down are written with the correct values; If the module under test is an ADC, the verification strategy adopts tolerance range comparison, that is, the analog signal has quantization noise and circuit deviation, and the tolerance verification is adopted by adding and subtracting the expected value and N least significant bits (LSB). If the module under test is PWM / ICU, the verification strategy adopts timing tolerance comparison, that is, if there is clock jitter in the frequency or duty cycle, the verification is performed by adding or subtracting the expected value and the tolerance percentage. If the module under test is CAN / SPI / I2C, the verification strategy adopts cyclic redundancy check (CRC) and frame format check. That is, the communication data adopts cyclic redundancy check, and the frame ID, DLC and data field are verified at the same time. If the module under test is a GPT / Timer, the verification strategy adopts count value tolerance and interrupt trigger verification, that is, the timer count value is allowed to deviate by 1 count unit (tick), and at the same time, it verifies whether the interrupt is triggered on time. If the module under test is Flash, the verification strategy adopts readback comparison and error correction code verification, that is, readback comparison after writing, and error correction code verification in combination with RISC-V hardware. If the module under test is DMA, the verification strategy adopts the data verification of transmission length and destination address, that is, to verify the number of data bytes and the content of the destination address after the module under test has completed the transmission; If the module under test is a WDT, the verification strategy adopts reset timeout verification, that is, the watchdog reset is determined by the machine cause (mcause) register in the RISC-V hardware.

[0060] As an example, after generating the executable file, the pre-defined control script sends standardized instructions to the debugging tool. These standardized instructions control the debugging tool to download the compiled ELF format executable file to the Flash memory of the target hardware platform through a debugging interface (such as a JTAG debugger). After the download is complete, the control script sends a boot instruction to the target hardware platform through the JTAG debugger, enabling the executable file to run on the target hardware platform. The target hardware platform is a microcontroller based on the open-source Reduced Instruction Set Architecture (RISC-V), and the executable file can run directly without relying on other operating systems.

[0061] As an example, if the target checkpoint is a function return checkpoint, then obtain the status code 0 returned after the current function under test (the program corresponding to the test case) is executed. 0 indicates that the function was executed correctly, and this correct code is used as the actual output data of the target checkpoint. If the returned status code is -1 or a non-zero value, it indicates that the returned status code is an error code, and this error code is used as the actual output data of the target checkpoint.

[0062] As an example, for CAN / SPI / I2C modules, the external device sends data set in the test case, and the data is compared to see if they are completely consistent. If they are completely consistent, a correct code is returned; if they are not completely consistent, an error code is returned. For Dio / Port modules, the value of the peripheral register is checked to see if it is completely consistent with the expected setting value. If it is completely consistent, a correct code is returned; if it is not completely consistent, an error code is returned.

[0063] Step S103: Obtain execution data, and generate test records based on the execution data and the verification results. The execution data includes at least the execution time.

[0064] In some exemplary embodiments, the step of obtaining execution data and generating test records based on the execution data and the verification results, wherein the execution data includes at least the execution time, including: Based on a preset communication script, a debugging communication link is established with the target hardware platform, the original result data corresponding to the verification result is extracted from the target hardware platform, and the original result data is transmitted back to the main control script environment; Based on the master control script in the master control script environment, obtain the expected execution results of the test cases; The original result data and the expected execution result are compared. If the original result data indicates that the verification is passed and the expected execution result is also passed, or if the original result data indicates that the verification is failed and the expected execution result is also failed, then the execution is determined to be consistent with the expectation; otherwise, it is determined to be inconsistent. Based on the comparison of the results, the final judgment result of the test case is generated; The execution time is the length of time from the start of the test case to the generation of the final judgment result. The execution data is obtained based on the original result data, the final judgment result, and the execution time; Based on the preset test record template, the execution data and the verification results are filled into the corresponding positions to generate test records.

[0065] In some exemplary embodiments, the original result data refers to the judgment results of the comparison of each value temporarily stored in the verification buffer by the verification execution layer, as well as the result of the summary judgment by the result summary function.

[0066] In some exemplary embodiments, a debug communication link refers to a connection established between a communication script and a target hardware platform via a debug tool for data transmission and instruction exchange.

[0067] In some exemplary embodiments, the expected execution result refers to the result that the tester pre-writes based on the test objective during the test case design phase, which is expected to be obtained after execution, including expected pass and expected fail.

[0068] In some exemplary embodiments, the expected execution result and the expected output data are different. The expected output data is analyzed at the micro level for a single target checkpoint, while the expected execution result is analyzed at the macro level for the entire test case.

[0069] In some exemplary embodiments, execution data refers to the set of data required to generate test records, including raw result data, execution time, and the final judgment result of test cases generated based on result comparison.

[0070] In some exemplary embodiments, the master control script environment is a platform that runs the master control script to perform code test scheduling, compilation, record generation, etc. Unlike the target hardware platform, the master control script is used to schedule and control the code test scripts.

[0071] In some exemplary embodiments, the main control script automatically compares the acquired raw result data with the expected execution result, generates a clear judgment conclusion based on the established pass / fail criteria, and feeds back the conclusion to the command line window interface in real time; for emphasis, correct and incorrect results will be output in different colored fonts.

[0072] To implement the exemplary embodiments described above, this disclosure also proposes a code testing apparatus.

[0073] Figure 2This is a schematic diagram of a code testing apparatus provided as an exemplary embodiment of the present disclosure. The apparatus 200 can be implemented by software and / or hardware and is generally integrated into an electronic device. For example... Figure 2 As shown, the device 200 includes: a first generation module 201, a determination module 202, and a second generation module 203, wherein, The first generation module 201 is used to obtain a test case list, schedule the test cases in the test case list, and generate an executable file. The determination module 202 is used to run the executable file, obtain the actual output data and expected output data of the target checkpoint, and determine the verification result of the actual output data and the expected output data based on a preset data verification mechanism. The second generation module 203 is used to acquire execution data and generate test records based on the execution data and the verification results. The execution data includes at least the execution time.

[0074] In some exemplary embodiments, the first generation module 201 is specifically used for: Obtain the user-input command for the name of the module to be tested, find the corresponding module file path based on the name of the module to be tested, and obtain the list of test cases under the module file path; If the name of the module to be tested does not exist or is entered incorrectly, an error message will be sent to the user. Obtain the order of test cases in the test case list, and schedule each test case sequentially based on the order to form a test execution loop; If all test cases in the test case list have been scheduled, the test execution loop ends. In each test execution loop, based on the currently scheduled test cases, the parameter information of the target project is dynamically allocated, and the currently scheduled test cases are compiled to generate executable files.

[0075] In some exemplary embodiments, the first generation module 201 specifically includes: During the scheduling of test cases in the test case list, it is dynamically checked whether the description information of the currently scheduled test case in the test case list is consistent with the actual name of the currently scheduled test case; If they match, continue executing the current test case. If there is a discrepancy, the scheduling is terminated and an error report is returned to the user. The error report includes a prompt message indicating that the description information is inconsistent with the actual name.

[0076] In some exemplary embodiments, the determining module 202 is specifically used for: Download the executable file to the target hardware platform; The executable file is run based on the target hardware platform.

[0077] In some exemplary embodiments, the determining module 202 is specifically used for: Based on a preset control script, a low-level communication with the debugging interface is established to control the execution of the executable file. The execution control includes starting, pausing, and terminating. The execution control is paused at a preset target checkpoint, and the actual output data is obtained based on the type of the target checkpoint. The target checkpoint includes function return checkpoint, register checkpoint, and communication data checkpoint. If the target checkpoint is the function return checkpoint, then obtain the status code returned after the current function under test is executed, as the actual output data. The status code includes a correct code and an error code. If the target checkpoint is the register checkpoint, then based on the underlying communication, the current value of the corresponding peripheral register in the target hardware platform is obtained as the actual output data; If the target checkpoint is the communication data checkpoint, then the data payload received by the communication bus is read as the actual output data; Based on the type of the target checkpoint and the test case identifier corresponding to the executable file, the expected output data of the target checkpoint is obtained.

[0078] In some exemplary embodiments, the data verification mechanism includes a macro encapsulation layer, a verification execution layer, and a verification strategy layer; The determining module 202 specifically includes: Macro definitions are obtained based on the macro encapsulation layer, and the actual output data and expected output data of the target checkpoint are transmitted to the verification execution layer together. Based on the verification execution layer, the actual output data and the expected output data are numerically compared, and the numerical comparison results are temporarily stored in a preset verification buffer. The numerical comparison results include consistency and inconsistency. The module type of the module under test is obtained. Based on the verification strategy layer, the verification strategy corresponding to the module type is matched. The verification strategy is used as the judgment benchmark for the numerical comparison. The verification strategy includes register precise comparison, tolerance range comparison, timing tolerance comparison, cyclic redundancy check and frame format check, count value tolerance and interrupt trigger check, readback comparison and error correction code check, transmission length and destination address data check, and reset timeout check. After performing the numerical comparison at each target checkpoint of the executable file, the result summarization function provided by the verification execution layer is called to summarize and determine the numerical comparison results in the verification buffer. If all the numerical comparison results are consistent, the executable file is deemed to have passed the verification. If any of the numerical comparison results are inconsistent, the executable file verification is deemed to have failed. The result of the summary determination is used as the verification result between the actual output data and the expected output data.

[0079] In some exemplary embodiments, the second generation module 203 specifically includes: Based on a preset communication script, a debugging communication link is established with the target hardware platform, the original result data corresponding to the verification result is extracted from the target hardware platform, and the original result data is transmitted back to the main control script environment; Based on the master control script in the master control script environment, obtain the expected execution results of the test cases; The original result data and the expected execution result are compared. If the original result data indicates that the verification is passed and the expected execution result is also passed, or if the original result data indicates that the verification is failed and the expected execution result is also failed, then the execution is determined to be consistent with the expectation; otherwise, it is determined to be inconsistent. Based on the comparison of the results, the final judgment result of the test case is generated; The execution time is the length of time from the start of the test case to the generation of the final judgment result. The execution data is obtained based on the original result data, the final judgment result, and the execution time; Based on the preset test record template, the execution data and the verification results are filled into the corresponding positions to generate test records.

[0080] The code testing apparatus provided in the exemplary embodiments of this disclosure can execute the code testing method provided in any exemplary embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0081] To implement the exemplary embodiments described above, this disclosure also proposes a computer program product, including a computer program / instructions that, when executed by a processor, implement the code testing method in the exemplary embodiments described above.

[0082] Figure 3 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this disclosure.

[0083] The following is a detailed reference. Figure 3The diagram illustrates a structural schematic suitable for implementing the electronic device 300 in the exemplary embodiments of this disclosure. The electronic device 300 in the exemplary embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the exemplary embodiments disclosed herein.

[0084] like Figure 3 As shown, the electronic device 300 may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a memory 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processor 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0085] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0086] In particular, according to exemplary embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, exemplary embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such exemplary embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from memory 308, or installed from ROM 302. When the computer program is executed by processor 301, it performs the functions defined in the code testing method of exemplary embodiments of the present disclosure.

[0087] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0088] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0089] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0090] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the aforementioned code testing method.

[0091] Electronic devices can be programmed with computer program code in one or more programming languages ​​or combinations thereof to perform the operations of this disclosure. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0092] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various exemplary embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0093] The units described in the exemplary embodiments of this disclosure can be implemented in software or in hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0094] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0095] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0096] The above description is merely a preferred exemplary embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0097] Furthermore, although the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. Multitasking and parallel processing may be advantageous in certain environments. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual exemplary embodiments may also be implemented in combination in a single exemplary embodiment. Conversely, various features described in the context of a single exemplary embodiment may also be implemented individually or in any suitable sub-combination in multiple exemplary embodiments.

[0098] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A code testing method, characterized in that, include: Obtain a list of test cases, schedule the test cases in the list, and generate an executable file; Run the executable file to obtain the actual output data and expected output data of the target checkpoint, and determine the verification result of the actual output data and the expected output data based on the preset data verification mechanism; Obtain execution data, and generate test records based on the execution data and the verification results. The execution data includes at least the execution time.

2. The method according to claim 1, characterized in that, The data verification mechanism includes a macro encapsulation layer, a verification execution layer, and a verification strategy layer; The method for determining the verification result between the actual output data and the expected output data based on a preset data verification mechanism includes: Macro definitions are obtained based on the macro encapsulation layer, and the actual output data and expected output data of the target checkpoint are transmitted to the verification execution layer together. Based on the verification execution layer, the actual output data and the expected output data are numerically compared, and the numerical comparison results are temporarily stored in a preset verification buffer. The numerical comparison results include consistency and inconsistency. The module type of the module under test is obtained. Based on the verification strategy layer, the verification strategy corresponding to the module type is matched. The verification strategy is used as the judgment benchmark for the numerical comparison. The verification strategy includes register precise comparison, tolerance range comparison, timing tolerance comparison, cyclic redundancy check and frame format check, count value tolerance and interrupt trigger check, readback comparison and error correction code check, transmission length and destination address data check, and reset timeout check. After performing the numerical comparison at each target checkpoint of the executable file, the result summarization function provided by the verification execution layer is called to summarize and determine the numerical comparison results in the verification buffer. If all the numerical comparison results are consistent, the executable file is deemed to have passed the verification. If any of the numerical comparison results are inconsistent, the executable file verification is deemed to have failed. The result of the summary determination is used as the verification result between the actual output data and the expected output data.

3. The method according to claim 1, characterized in that, The acquisition of the actual output data and expected output data of the target checkpoint includes: Based on a preset control script, a low-level communication with the debugging interface is established to control the execution of the executable file. The execution control includes starting, pausing, and terminating. The execution control is paused at a preset target checkpoint, and the actual output data is obtained based on the type of the target checkpoint. The target checkpoint includes function return checkpoint, register checkpoint, and communication data checkpoint. If the target checkpoint is the function return checkpoint, then obtain the status code returned after the current function under test is executed, as the actual output data. The status code includes a correct code and an error code. If the target checkpoint is the register checkpoint, then based on the underlying communication, the current value of the corresponding peripheral register in the target hardware platform is obtained as the actual output data; If the target checkpoint is the communication data checkpoint, then the data payload received by the communication bus is read as the actual output data; Based on the type of the target checkpoint and the test case identifier corresponding to the executable file, the expected output data of the target checkpoint is obtained.

4. The method according to claim 1, characterized in that, The steps of obtaining the test case list, scheduling the test cases in the test case list, and generating an executable file include: Obtain the user-input command for the name of the module to be tested, find the corresponding module file path based on the name of the module to be tested, and obtain the list of test cases under the module file path; If the name of the module to be tested does not exist or is entered incorrectly, an error message will be sent to the user. Obtain the order of test cases in the test case list, and schedule each test case sequentially based on the order to form a test execution loop; If all test cases in the test case list have been scheduled, the test execution loop ends. In each test execution loop, based on the currently scheduled test cases, the parameter information of the target project is dynamically allocated, and the currently scheduled test cases are compiled to generate executable files.

5. The method according to claim 1, characterized in that, Scheduling the test cases in the test case list includes: During the scheduling of test cases in the test case list, it is dynamically checked whether the description information of the currently scheduled test case in the test case list is consistent with the actual name of the currently scheduled test case; If they match, continue executing the current test case. If there is a discrepancy, the scheduling is terminated and an error report is returned to the user. The error report includes a prompt message indicating that the description information is inconsistent with the actual name.

6. The method according to claim 1, characterized in that, Running the executable file includes: Download the executable file to the target hardware platform; The executable file is run based on the target hardware platform.

7. The method according to claim 1, characterized in that, The process involves acquiring execution data, generating test records based on the execution data and the verification results, wherein the execution data includes at least the execution time, and includes: Based on a preset communication script, a debugging communication link is established with the target hardware platform, the original result data corresponding to the verification result is extracted from the target hardware platform, and the original result data is transmitted back to the main control script environment; Based on the master control script in the master control script environment, obtain the expected execution results of the test cases; The original result data and the expected execution result are compared. If the original result data indicates that the verification is passed and the expected execution result is also passed, or if the original result data indicates that the verification is failed and the expected execution result is also failed, then the execution is determined to be consistent with the expectation; otherwise, it is determined to be inconsistent. Based on the comparison of the results, the final judgment result of the test case is generated; The execution time is the length of time from the start of the test case to the generation of the final judgment result. The execution data is obtained based on the original result data, the final judgment result, and the execution time; Based on the preset test record template, the execution data and the verification results are filled into the corresponding positions to generate test records.

8. A code testing apparatus, the apparatus comprising: The first generation module is used to obtain a test case list, schedule the test cases in the test case list, and generate an executable file. The determination module is used to run the executable file, obtain the actual output data and expected output data of the target checkpoint, and determine the verification result of the actual output data and the expected output data based on a preset data verification mechanism. The second generation module is used to acquire execution data and generate test records based on the execution data and the verification results. The execution data includes at least the execution time.

9. An electronic device, characterized in that, include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the steps of the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program / instruction thereon, which, when executed by a processor, implements the steps of the method described in any one of claims 1-7.