Test case determination method and device and electronic equipment

Abstract

The invention discloses a test case determination method and device and electronic equipment. The method comprises the steps that a to-be-verified target code unit is determined; the target code unit represents a code unit corresponding to the to-be-verified function point in the algorithm model; obtaining a coverage rate database, wherein the coverage rate database represents the coverage degree of each test case in the test case library on a preset granularity code unit in the algorithm model; and querying in a coverage rate database based on the target code unit to obtain a test case covering the target code unit as a target test case.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically to a method, apparatus and electronic device for determining test cases. Background Technology

[0002] In the verification process of functions applied to data processing (such as video encoding and decoding), verifiers need to select effective test cases for the specific function to be tested in order to complete the verification of the function to be tested. Currently, verifiers usually select potentially relevant test cases based on the function to be tested. This test case selection is inefficient and inaccurate, and cannot meet the actual verification needs. Summary of the Invention

[0003] In view of the above, this application provides the following technical solution:

[0004] A method for determining test cases, comprising:

[0005] Identify the target code unit to be verified; the target code unit represents the code unit in the algorithm model that corresponds to the functional point to be verified.

[0006] Obtain a coverage database, which represents the degree to which each test case in the test case library covers the preset granularity code units in the algorithm model;

[0007] Based on the target code unit, a query is performed in the coverage database to obtain the test cases that cover the target code unit as the target test cases.

[0008] Optionally, the process of creating the coverage database includes:

[0009] Run all test cases in the test case library to execute the algorithm model, and automatically insert coverage detection code through compiler options. Record the number of times each code unit is executed during the runtime of each test case, and generate a coverage data file.

[0010] Parse the coverage data file to obtain the coverage data of each test case for each code unit;

[0011] Based on the coverage data, the coverage database is constructed.

[0012] Optionally, constructing the coverage database based on the coverage data includes:

[0013] Based on the attribute information of each test case in the test case library, a parent table is constructed;

[0014] Based on the coverage data of each code unit for each test case, a sub-table is constructed; the sub-table establishes a mapping relationship with the parent table through association identifiers to represent the correspondence between each test case and the covered code unit;

[0015] The coverage database is constructed based on the parent table and the child table.

[0016] Optionally, the step of querying the coverage database based on the target code unit to obtain test cases covering the target code unit as target test cases includes:

[0017] Search the sub-table for records that cover the target code unit, and obtain the association identifier corresponding to the record;

[0018] Based on the association identifier, query the parent table for matching test cases;

[0019] The retrieved test cases are identified as target test cases covering the target code unit and output.

[0020] Optionally, it also includes:

[0021] In response to the addition of a test case to the test case library, the newly added test case is run to execute the algorithm model;

[0022] Obtain the new coverage data and update the coverage database.

[0023] Optionally, it also includes:

[0024] During the algorithm model verification phase, all test cases are run and the coverage database is established.

[0025] Before the hardware design verification phase begins, the coverage database is queried based on the target code unit corresponding to the function point to be verified, and test cases covering the target code unit are pre-selected.

[0026] The pre-selected test cases are provided to the hardware verification environment to obtain hardware verification results.

[0027] Optionally, it also includes:

[0028] After obtaining the hardware verification results, analyze the difference between the hardware verification results and the expected results of the algorithm model;

[0029] Based on the differences, identify the target code units in the algorithm model that have deviations;

[0030] Based on the identified target code unit, query the coverage database for test cases that cover the target code unit as regression verification test cases;

[0031] The regression verification test cases are then re-provided to the hardware verification environment to verify the correctness of the optimized algorithm model.

[0032] Optionally, it also includes:

[0033] Receive multiple target code units corresponding to multiple functional points to be verified;

[0034] The test cases covering each target code unit are queried in the coverage database to obtain a set of candidate test cases corresponding to each target code unit.

[0035] Each candidate test case set is processed to obtain a target test case set that meets the coverage requirements of multiple functional points.

[0036] Output the target test case set, wherein the target test case set is used to verify collaborative scenarios of multiple functional points.

[0037] A test case determination device, comprising:

[0038] A determining unit is used to determine the target code unit to be verified; the target code unit represents the code unit in the algorithm model that corresponds to the function point to be verified.

[0039] The acquisition unit is used to acquire a coverage database, wherein the coverage database represents the degree of coverage of each test case in the test case library to the preset granularity code units in the algorithm model;

[0040] The query unit is used to query the coverage database based on the target code unit to obtain the test cases that cover the target code unit as target test cases.

[0041] An electronic device, comprising:

[0042] A memory for storing computer programs and the data generated by the execution of said computer programs;

[0043] A processor for executing the computer program to achieve:

[0044] Identify the target code unit to be verified; the target code unit represents the code unit in the algorithm model that corresponds to the functional point to be verified.

[0045] Obtain a coverage database, which represents the degree to which each test case in the test case library covers the preset granularity code units in the algorithm model;

[0046] Based on the target code unit, a query is performed in the coverage database to obtain the test cases that cover the target code unit as the target test cases. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0048] Figure 1 A flowchart illustrating a test case determination method provided in an embodiment of this application;

[0049] Figure 2 A flowchart for obtaining algorithm model coverage is provided as an embodiment of this application;

[0050] Figure 3 A flowchart illustrating the creation of a coverage database provided in this application embodiment;

[0051] Figure 4 This is a schematic diagram of a test case query provided in an embodiment of this application;

[0052] Figure 5 A flowchart illustrating the algorithm model verification and hardware design verification process provided in this application embodiment;

[0053] Figure 6 This is a schematic diagram of a test case determination device provided in an embodiment of this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] The terms "first" and "second," etc., used in this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units may include steps or units not listed, but may include steps or units not listed.

[0056] This application provides a test case determination method applicable to chip verification, such as in advanced integration verification processes for hardware designs like video processing unit decoders. The method identifies target code units (e.g., lines of code, code blocks, or code functions) in the algorithm model corresponding to the functional point to be verified, and obtains a pre-established coverage database. This database characterizes the degree of coverage of each test case in the test case library to a pre-defined granularity code unit in the algorithm model. Then, based on the target code unit, a query is performed in the coverage database to directly obtain test cases covering that target code unit as target test cases. By establishing a reverse retrieval mechanism between code units and test cases, the functional point to be verified is mapped to the target code unit in the algorithm model. Utilizing the pre-established coverage database, it achieves direct reverse lookup of test cases covering the code logic, solving the technical problems of traditional verification methods where internal code execution is invisible and test case selection relies on external port data or human experience. This improves verification efficiency and the accuracy of test case selection.

[0057] See Figure 1 The diagram illustrates a flowchart of a test case determination method provided in an embodiment of this application. The method may include the following steps:

[0058] S101. Identify the target code unit to be verified.

[0059] The target code unit represents the code unit corresponding to the function point to be verified in the algorithm model. Its granularity can be set according to the verification requirements, including code line, code block, code function, etc. For example, in the embodiment of this application, line code level granularity is adopted, that is, the target code unit represents the code line.

[0060] For example, in the high-level synthesis (HLS) verification of the decoder hardware of the video processing unit (VPU), the first step is to identify the function point to be verified and map it to the corresponding code unit (e.g., line of code) in the algorithm model. The function point to be verified can be described as "the current image patch uses the motion vector of the left image patch as its predicted motion vector, and the left image patch is in a valid state." This function point corresponds to a specific code unit in the motion vector prediction module of the algorithm model code, for example, line 138 in its code file. This line of code implements the logic of determining the validity of the left image patch and reading its motion vector; therefore, line 138 is identified as the target code unit.

[0061] This step transforms abstract functional point verification requirements into specific code unit retrieval requirements, achieving a precise correlation between verification requirements and code logic, and solving the problem of disconnect between test case selection and actual functional point coverage in traditional verification.

[0062] S102, Obtain the coverage database.

[0063] The coverage database characterizes the degree to which each test case in the test case library covers pre-defined granularity code units in the algorithm model. These pre-defined granularity code units can be code snippets of different granularities, such as lines of code, code blocks, or functions; this embodiment primarily uses lines of code as an example. The coverage database establishes a mapping relationship between test cases and code units. By recording the number of times each test case executes each code unit during runtime, it quantifies the coverage of the algorithm model code by the test cases. For example, in a video decoder verification scenario, the coverage database can record whether each test video stream executes line 138 of the code file during runtime, and the number of times it is executed. The pre-built coverage database provides structured storage of the test case and code unit coverage relationship, eliminating the need to rerun all test cases each time test cases are selected, significantly improving retrieval efficiency.

[0064] For example, verifiers can directly access a pre-generated algorithm model (such as Cmodel) code coverage database through the test terminal. This database is constructed from JSON-formatted coverage data generated after running all the compressed bitstream files in the test case library. It contains attribute information of all bitstream test cases, line code information of each code file, and the hit count value of each bitstream test case for each line of code.

[0065] The algorithm model in this application embodiment can be implemented in C / C++, or it can include models implemented in other high-level languages ​​commonly used in the art. In specific embodiments, a C model (such as the Cmodel model) implemented in C language is used as an example.

[0066] S103. Based on the target code unit, query the coverage database to obtain the test cases that cover the target code unit as the target test cases.

[0067] Based on the target code unit determined in step S101, a query is performed in the coverage database obtained in step S102 to filter out test cases that cover the target code unit as target test cases. Specifically, the query process uses the target code unit (such as line 138 of the mv_prediction.cpp file) as the search condition, searches for records with an execution count greater than zero in the sub-tables of the coverage database, obtains the corresponding association identifiers (such as test case IDs) for these records, and then queries the parent table through the association identifiers to obtain complete test case information, finally outputting all test cases covering the target code unit. For example, when a verifier needs to test the function point "using motion vectors as predicted motion vectors when the left image block is valid", by querying the coverage database for test cases that have executed line 138 of mv_prediction.cpp, test video streams such as "4K_basketball_001.h264" and "1080p_racing_005.h264" that can trigger this function point can be directly obtained. This step enables precise reverse retrieval from code logic to test cases, transforming the traditional screening process that relies on human experience or port data into an automated data-driven query, significantly improving the efficiency and accuracy of test case screening.

[0068] This application provides a test case determination method. First, the target code unit to be verified is determined, and then a coverage database is obtained. Based on the target code unit, a query is performed in the coverage database to obtain the test cases that cover the target code unit as the target test cases. By establishing a reverse detection mechanism between code units and test cases, the limitation of relying on external port signals to select test cases in traditional verification is overcome, achieving accurate and rapid screening of internal code logic coverage test cases. Furthermore, the granularity of the target code unit can be flexibly set to adapt to different levels of verification needs, exhibiting strong versatility and applicability to most algorithm model verification scenarios.

[0069] An embodiment of this application also provides a method for creating a coverage database. The creation process includes: running all test cases in the test case library to execute the algorithm model, automatically inserting coverage detection code through compiler options, recording the number of times each test case is executed on each code unit, and generating a coverage data file; parsing the coverage data file to obtain the coverage level data of each test case on each code unit; and constructing a coverage database based on the coverage level data.

[0070] Specifically, when creating the coverage database, step S201 is executed first: compile-time instrumentation is performed on the algorithm model code. Compiler instrumentation refers to the automatic insertion of detection instructions to record code execution status into the target code during the compilation phase through compiler options, thus eliminating the need for manual modification of the source code and exhibiting a zero-intrusion characteristic. This process is achieved through the GCC compiler's `-fprofile-arcs` and `-ftest-coverage` options. The `-fprofile-arcs` option inserts counting code on each arc of the program's control flow graph, recording the execution count of each path during program execution; the `-ftest-coverage` option generates a code structure information file, recording static structure information such as line numbers, functions, and basic blocks of the source code. After compilation, a `.gcno` file containing the code structure information is generated.

[0071] Subsequently, step S202 executes all test cases in the test case library, with each test case independently executing the instrumented algorithm model. During test case execution, the compiler-inserted detection instructions automatically record the execution count of each code unit. A code unit refers to the smallest measurable code fragment in the algorithm model, which can be a line of code, a code block, or a function, etc., and its execution count quantifies the degree of coverage of the algorithm model code by the test case. To differentiate the coverage data generated by different test cases, an independent .gcda file can be generated for each test case at runtime, specified via environment variables or command-line arguments.

[0072] After execution, each test case generates a corresponding .gcda coverage data file, which contains the number of times each code unit was executed during the test case's runtime. After all test cases have run, the generated coverage data file needs to be parsed. Coverage data refers to structured information obtained after parsing that clearly reflects the coverage of each code unit by each test case, typically including key fields such as code file path, line number, and execution count. For example, the gcovr tool (an open-source code coverage report generation tool) can be used for parsing. gcovr can read the .gcda and .gcno files generated by the compiler and parse them into a standardized JSON format coverage report. The resulting JSON report contains the coverage data of each test case for each code unit. Figure 2 As shown, it illustrates the process of obtaining algorithm model coverage, such as... Figure 2As shown, the left side contains all the prepared test stream files (such as video stream files), and the right side contains the algorithm model (such as cmodel) code, which is the algorithm model to be verified. Verification personnel first compile and adapt the algorithm model to support coverage collection using the gcovr tool; then they run all streams, i.e., run each test video stream one by one, generating a corresponding JSON format line-level coverage report after each stream is run; finally, they output a JSON coverage file for all streams for subsequent library construction.

[0073] Finally, a coverage database is constructed based on the parsed coverage data. A coverage database is a database system that organizes and stores coverage data according to a specific structure; its function is to establish a bidirectional mapping relationship between test cases and code units. The coverage data in the JSON report is organized and stored according to the designed database structure, establishing the correspondence between test cases and code units.

[0074] In this embodiment, zero-intrusive coverage acquisition is achieved through automatic instrumentation via compiler options, eliminating the need for manual modification of the algorithm model code. This solves the problems of traditional solutions requiring manual insertion of breakpoints and high code maintenance costs, significantly reducing the workload of verification personnel.

[0075] In some embodiments of this application, a coverage database is constructed based on coverage data, including:

[0076] S301. Construct a parent table based on the attribute information of each test case in the test case library.

[0077] S302. Based on the coverage data of each code unit for each test case, construct a sub-table.

[0078] S303. Construct a coverage database based on parent and child tables. The child tables establish a mapping relationship with the parent table through association identifiers to represent the correspondence between each test case and the covered code unit.

[0079] In step S301, the parent table is the main table of the coverage database, storing the unique identifier and basic attribute information of each test case, with test cases as the core dimension. The attribute information of the test cases includes, but is not limited to, unique test case ID, test case file name, bitstream type, bitstream resolution, bitrate, and other information characterizing the test case's features. In step S302, the sub-table is the subordinate table of the coverage database, split by code file, with code unit as the core dimension. Each code file corresponds to a sub-table, storing information about all code units in that file and the coverage level of each test case. The association identifier is a common field between the parent and sub-tables, such as a test case ID, which enables a one-to-one mapping between test cases in the parent table and code unit coverage data in the sub-table. The coverage database generated in step S303 can be a relational database, consisting of a parent table and multiple sub-tables forming a recursive structure through the association identifier. It supports SQL queries, incremental updates, and other operations, and can be implemented using a lightweight database engine, adaptable to localized deployment and use.

[0080] See Figure 3 This illustrates a flowchart of a coverage database creation method provided in an embodiment of this application. Figure 3 In the table, the rightmost entries `test_1_stream`, `test_2_stream`, ..., `test_n_stream` represent individual test cases in the test case library (e.g., compressed bitstream files for VPU decoder verification in this embodiment). These are the core data dimensions of the coverage database's parent table. Each test case corresponds to an independent record in the parent table, containing unique attribute information such as test case ID, filename, and bitstream parameters. `code_file_0.cp`, `code_file_1.cp`, etc., represent the code files of the algorithm model (e.g., C / C++ code files for Cmodel in this embodiment). Each code file corresponds to a sub-table in the coverage database. The sub-table uses code units (line code, code block, etc.) as its core dimension, containing data such as code unit identifier, execution count (hit count), and coverage status. Figure 3 The "hit count" represents the coverage of the corresponding test case to the code unit in the corresponding code file. It is the core business data that connects the parent table and the child table, enabling precise association between test cases and code units.

[0081] The specific processing flow for building a coverage database based on coverage data is as follows: First, execute step S301 to extract unique attribute information of test cases as the dimension, assign association identifiers (test case IDs), and build a parent table with the identifier as the primary key; then execute step S302 to split and build sub-tables according to the code files of the algorithm model. The sub-tables use code unit identifiers as the core field and association identifiers as foreign keys, and enter the coverage data of the corresponding test cases for each code unit; then, establish a strong association mapping between the parent table and all sub-tables through the association identifiers; finally, execute step S303 to integrate the parent table, sub-tables, and association relationships, solidify the data based on a lightweight relational database engine, and form a complete coverage database that supports bidirectional retrieval.

[0082] In this embodiment, a two-tiered structure of parent and child tables enables refined and efficient storage of test case and code unit coverage relationships, solving the problems of disorganized and difficult-to-retrieve coverage data in traditional methods. The bidirectional fast retrieval feature adapts to different verification needs, allowing verification of code coverage through test cases and filtering of test cases by code unit, thus improving the database's usability.

[0083] Furthermore, in this embodiment of the application, the target test cases are obtained by querying the coverage database based on the target code unit, including:

[0084] S401. Search the sub-table for the record that covers the target code unit and obtain the association identifier corresponding to the record.

[0085] S402, Test cases that query for matching results in the parent table based on the association identifier.

[0086] S403. Identify the queried test cases as target test cases covering the target code unit and output them.

[0087] First, step S402 searches the sub-table for records covering the target code unit, obtaining the corresponding association identifiers. The sub-table search uses the target code unit as the retrieval condition, searching for records in the corresponding code file sub-table with an execution count greater than zero. For example, when the target code unit is line 138 of the file mv_prediction.cpp, the search in the coverage_mv_prediction sub-table for records where line_number=138 and exec_count>0, extracting the association identifiers (i.e., test case IDs) from these records. Second, step S403 searches the parent table for matching test cases based on the obtained association identifiers. The parent table search uses the obtained set of test case IDs as the retrieval condition, querying the test_cases parent table for corresponding complete test case information, including test case file name, resolution, scenario type, and other attribute information. Finally, step S404 identifies the queried test cases as target test cases covering the target code unit and outputs them. The target test case output organizes and displays the parent table query results, presenting all test cases hitting the target code unit and their attribute information in list form for direct use by verification personnel.

[0088] See Figure 4 This illustration shows a test case query diagram provided by an embodiment of this application. Taking the VPU decoder verification scenario as an example, the target code unit corresponding to the function point to be verified is line 138 of the code in mv_prediction.cpp. First, the coverage records of this code unit are retrieved in the sub-table area (the sub-table dedicated to the code file), and records with a count > 0 are filtered out and the associated identifier (test case ID = 36) is extracted. Then, based on this identifier, the corresponding test case attribute information (the code stream file test25_b8_claire_176x144_Lcommon_src_mv_prediction.json) is matched in the parent table area (the main test case table); finally, this test case is determined as the target test case and output.

[0089] In this embodiment, the association retrieval logic based on the parent and child tables enables precise location of target code units to test cases, improving the accuracy of test case selection. The retrieval process does not require traversing the entire dataset; it quickly extracts association identifiers from the child table and then matches them with the parent table, shortening the test case selection time and improving the overall efficiency of algorithm verification.

[0090] In some embodiments of this application, the method further includes: in response to the addition of a test case to the test case library, running the newly added test case to execute the algorithm model; obtaining the newly added coverage data and updating the coverage database.

[0091] The algorithm model in this application embodiment can be implemented in C / C++, or it can include models implemented in other high-level languages ​​commonly used in the art. In specific embodiments, a C model (such as Cmodel) implemented in C language is used as an example. New test cases refer to the bitstream files or test cases added to the test case library after the coverage database is built. These can be newly compressed bitstreams, new bitstreams generated by open-source tools, etc. When running new test cases, compiler instrumentation configuration and Cmodel compilation adaptation are used to ensure the consistency of coverage data. New coverage data refers to the coverage data generated after running new test cases, including the hit count value of each code unit for that test case; database updates include new records in the parent table and new records in the child table, without modifying existing data.

[0092] For example, taking VPU decoder verification as an example, when a new batch of compressed bitstream files is added to the test case library, verifiers do not need to rerun the original full bitstreams. They only need to run the new bitstream files in the Cmodel code that has been adapted to the compiler instrumentation, and execute the corresponding decoding algorithm. Then, the gcovr tool can be used to collect the JSON format coverage data file after the new bitstream test cases have run. After parsing, the new coverage data is obtained, a unique test case ID is assigned to the new test case, and the attribute information record of the test case is added to the parent table. At the same time, the line code hit count value record corresponding to the test case ID is added to the sub-table of each corresponding code file, completing the incremental update of the coverage database. In this embodiment, lightweight maintenance of the coverage database is achieved, adapting to the actual needs of continuous test case iteration in the project, and reducing the subsequent maintenance cost of the database. It avoids the repeated running of the full test cases, saves server resources and verification time, and improves the overall verification efficiency.

[0093] In some embodiments of this application, the method further includes:

[0094] During the algorithm model verification phase, all test cases are run and a coverage database is established. Before the hardware design verification phase begins, the coverage database is queried based on the target code units corresponding to the functional points to be verified, and test cases covering the target code units are pre-selected. The pre-selected test cases are then provided to the hardware verification environment to obtain hardware verification results.

[0095] The algorithm model verification phase refers to the functional verification phase of the Cmodel code. This phase is a pre-concept phase for hardware design verification. At this stage, the port dump files for the hardware design have not yet been developed, and the Hmodel verification phase has not yet begun. A coverage database is established in this phase, decoupling test case selection from hardware design development. The hardware design verification phase refers to the HLS (High-Level Synthesis) / RTL (Register Transfer Level) hardware logic verification phase. This phase requires port dump files and valid test cases. Pre-selecting test cases before starting this phase allows for advance preparation of test cases, preventing verification stalls due to missing test cases. The hardware verification environment refers to the dedicated environment for HLS / RTL hardware logic verification, including a hardware simulation platform and a test case execution platform. Importing the pre-selected target test cases into this environment and running them yields the execution results of the hardware design for those test cases, i.e., the hardware verification results.

[0096] For example, see Figure 5 It illustrates a flowchart of an algorithm model verification and hardware design verification provided in an embodiment of this application. Figure 5 In this context, decoder represents a decoder, specifically the VPU decoder in this embodiment; prd represents a product requirements document; spec represents a specification document; cmodel represents an algorithm model (a reference model implemented in C language); dump represents a data dump (recording signals / data inside the module or its interface); tb represents a testbench (an environment used to verify hardware designs); hmodel represents a hardware model (a hardware design implemented in RTL or HLS); and port represents a port (the signal interface of a hardware module to the outside world).

[0097] like Figure 5 As shown, between constructing code stream test cases and setting up the test platform tb and verifying the hardware model hmodel, a step has been added to build a coverage database based on gcovr (used to generate and manage coverage reports for C / C++ code) and pre-select test cases, thus refactoring the traditional process.

[0098] During the algorithm model verification stage (i.e.) Figure 5 In the phase following "Constructing test cases for the bitstream" and "Running cmodel to obtain the bitstream dump," all test cases are run and a coverage database is built, without waiting for the port dump file development to complete. During the hardware design verification phase (i.e....) Figure 5Before the "Build TB, Verify HModel" phase begins, the coverage database is queried based on the target code units corresponding to the functionalities to be verified, and test cases covering the target code units are pre-selected. These pre-selected test cases are then directly provided to the hardware verification environment (i.e., Figure 5 In the "Build TB, Verify HModel" section, skip the "Select test cases based on interface dump" dependency step and complete the test case preparation in advance.

[0099] pass Figure 5 As can be seen from the process comparison, the embodiments of this application solve the limitation that test case selection in the traditional process must rely on port dump code, achieve decoupling of test case selection and hardware design and development, and move the test case selection stage forward to before hardware design verification. This solves the problem of the late test case selection stage in the existing solution, significantly shortens the overall verification cycle, and forms a complete test case transfer closed loop of algorithm verification library building and hardware verification pre-selected test cases. This improves the coherence of the overall verification process. Moving the test case selection stage forward to before hardware design verification solves the problem of the late test case selection stage in the existing solution and significantly shortens the overall verification cycle.

[0100] In some embodiments of this application, the method further includes: after obtaining the hardware verification results, analyzing the differences between the hardware verification results and the expected results of the algorithm model; based on the differences, identifying target code units in the algorithm model that have deviations; querying the coverage database for test cases covering the identified target code units, using them as regression verification test cases; and re-providing the regression verification test cases to the hardware verification environment to verify the correctness of the optimized algorithm model.

[0101] The hardware verification result is the execution result after the hardware design runs the target test cases. Taking the Cmodel model as an example, the expected result is the algorithm execution result after the Cmodel code runs the same test cases. The difference between the two refers to inconsistencies in decoding results, data output, and functional execution, indicating a problem in the hardware design or algorithm model. The target code unit with deviation refers to the code unit in the algorithm model that causes the inconsistency between the hardware verification result and the expected algorithm result. It represents the specific location of the problem at the code level, and its granularity is consistent with the granularity of the code units verified in the previous stage (line code level in this embodiment).

[0102] Regression verification test cases refer to all test cases that cover the target code unit with deviations. This includes not only the initial target test cases but also all other test cases that can execute that code unit. These cases are used to verify whether the optimized code logic of the algorithm model can cover all relevant scenarios. Algorithm model optimization refers to modifying and optimizing the code and logic of the identified target code units with deviations to ensure that their algorithm execution results are consistent with the hardware design results. The regression verification test cases are then imported into the hardware verification environment, and after running, it is verified whether the optimized algorithm model can resolve the original differences without generating new problems.

[0103] For example, in the HLS verification of the VPU decoder, after importing the target test cases into the hardware verification environment, it was found that the motion vector prediction results of the hardware decoding were inconsistent with the algorithm prediction results of the Cmodel code. Analyzing the specific manifestations of this difference, the core cause was determined to be the code logic of the algorithm model. Based on the above discrepancies and combined with the logical analysis of the Cmodel code, the code unit causing the deviation in motion vector prediction results was identified as line 138 of `mv_prediction.cpp` and its surrounding related code. This code unit was identified as the target code unit with the deviation. In the coverage database, using line 138 of `mv_prediction.cpp` as the target code unit, all test cases with a count > 0 hitting this code unit were queried. Besides the bitstream with test case ID = 36 from the previous query, five other bitstream test cases that could cover this code unit were retrieved. These six test cases were all identified as regression verification test cases. The logic of line 138 of mv_prediction.cpp was optimized. After modification, the six regression verification test cases were re-imported into the HLS verification environment of the VPU decoder. All test cases were run and hardware verification results were collected to verify whether the execution results of the optimized Cmodel code were consistent with the hardware design.

[0104] This embodiment realizes a complete closed loop of hardware verification issues, code unit location, regression test case selection, and regression verification, which solves problems such as difficulty in locating problems during verification and blind selection of regression verification test cases, improves the efficiency of problem investigation and regression verification, and reduces the verification cost of hardware and algorithm model adaptation.

[0105] In this embodiment, the verification requirements for scenarios where multiple functional points are triggered simultaneously or in a coordinated manner can also be met. Correspondingly, it also includes:

[0106] S501: Receive multiple target code units corresponding to multiple function points to be verified.

[0107] S502. Query the coverage database to find the test cases that cover each target code unit, and obtain the candidate test case set corresponding to each target code unit.

[0108] S503. Process each candidate test case set to obtain a target test case set that meets the coverage requirements of multiple functional points; output the target test case set, which is used to verify the collaborative scenarios of multiple functional points.

[0109] In step S501, the multiple functional points to be verified refer to multiple functional points that need to be verified simultaneously in hardware or algorithm verification, typically functional points with collaborative relationships. Each functional point corresponds to a target code unit, thus forming a combined retrieval requirement for multiple target code units. In step S502, the candidate test case set refers to all test cases that can cover a single target code unit, with each target code unit corresponding to an independent candidate test case set. In step S503, the combined coverage requirement can be flexibly set according to the verification scenario, including but not limited to intersection coverage (simultaneously covering all target code units, i.e., multiple functional points are triggered simultaneously), union coverage (covering at least one target code unit, i.e., any one functional point is triggered), and difference coverage (covering one target code unit but not others). In this embodiment, intersection coverage is used as an example to verify a collaborative scenario where two functional points are triggered simultaneously. The output target test case set contains complete attribute information of the test cases and coverage data for each target code unit, making it easy for verification personnel to understand the coverage of each functional point by each test case.

[0110] For example, in the HLS verification of the VPU decoder, two collaborative functional points need to be verified simultaneously: Functional point 1 is "the current block uses the motion vector (mv) of the left block as the predicted motion vector (pmv), and the left block is valid" (corresponding to line 138 of target code unit A: mv_prediction.cpp), and functional point 2 is "the current block uses the motion vector (mv) of the block above it as the pmv, and the block above it is valid" (corresponding to line 156 of target code unit B: mv_prediction.cpp). The verifier inputs target code units A and B into a general filtering tool and initiates a combined query request. The tool retrieves test cases covering target code units A and B from the coverage database, respectively, and obtains candidate test case set A (containing 6 bitstream test cases) and candidate test case set B (containing 8 bitstream test cases). The tool performs an intersection operation on candidate test case sets A and B, filtering out test cases that simultaneously cover target code units A and B, resulting in three bitstream test cases. These three test cases are then identified as the target test case set that meets the combined coverage requirement. The ID, filename, and hit count values ​​for target code units A and B of the three bitstream test cases obtained from the intersection operation are output uniformly. This target test case set can be directly used for verifying complex scenarios in the VPU decoder where "left block + top block MV work together as PMV".

[0111] In this embodiment, combined queries of multiple functional points to be verified are supported, adapting to the verification needs of complex collaborative scenarios in engineering and overcoming the limitations of single functional point queries. The combined coverage requirements can be flexibly configured, exhibiting strong versatility and applicability to all multi-functional point collaborative verification scenarios, thus improving the efficiency of test case selection in complex scenarios.

[0112] It should be noted that those skilled in the art should understand that the test case determination method described in this application is not limited to algorithm models implemented in C / C++. For other programming languages ​​or development frameworks, as long as their corresponding compilers or toolchains can provide code execution coverage data, those skilled in the art can refer to the technical solution of this application to construct a coverage database and implement query functions in a similar manner. This application is a general coverage-driven test case determination framework, which mainly focuses on implementing a reverse retrieval mechanism for code units and test cases, rather than relying on a specific programming language or tool. Therefore, equivalent substitutions and adaptations of programming languages ​​or coverage tools can be made without departing from the technical concept of this application.

[0113] See Figure 6 This application also provides a test case determination device, comprising:

[0114] The determining unit 601 is used to determine the target code unit to be verified; the target code unit represents the code unit in the algorithm model that corresponds to the function point to be verified.

[0115] The acquisition unit 602 is used to acquire a coverage database, wherein the coverage database represents the degree of coverage of each test case in the test case library to the preset granularity code units in the algorithm model;

[0116] The query unit 603 is used to query the coverage database based on the target code unit to obtain the test cases that cover the target code unit as target test cases.

[0117] In some possible implementations, a database creation unit is also included for creating the coverage database, wherein the database creation unit includes:

[0118] The recording sub-unit is used to run all test cases in the test case library to execute the algorithm model, and automatically insert coverage detection code through compiler options, record the number of times each code unit is executed when each test case runs, and generate a coverage data file;

[0119] The parsing sub-unit is used to parse the coverage data file to obtain the coverage data of each test case for each code unit;

[0120] A sub-unit is constructed to build the coverage database based on the coverage data.

[0121] In some possible implementations, the building subunit is configured as follows:

[0122] Based on the attribute information of each test case in the test case library, a parent table is constructed;

[0123] Based on the coverage data of each code unit for each test case, a sub-table is constructed; the sub-table establishes a mapping relationship with the parent table through association identifiers to represent the correspondence between each test case and the covered code unit;

[0124] The coverage database is constructed based on the parent table and the child table.

[0125] In some possible implementations, the query unit includes:

[0126] The first acquisition sub-unit is used to search the sub-table for a record that covers the target code unit and obtain the association identifier corresponding to the record;

[0127] The first query sub-unit is used to query the parent table for matching test cases based on the association identifier;

[0128] The first determining subunit is used to determine the queried test cases as target test cases covering the target code unit and output them.

[0129] Among the possible implementations are:

[0130] The first execution unit is used to respond to the addition of a test case in the test case library and run the newly added test case to execute the algorithm model;

[0131] The update unit is used to obtain new coverage data and update the coverage database.

[0132] Among the possible implementations are:

[0133] The second execution unit is used to run all test cases and establish the coverage database during the algorithm model verification phase.

[0134] The pre-selection unit is used to query the coverage database based on the target code unit corresponding to the function point to be verified before the hardware design verification phase starts, and pre-select test cases that cover the target code unit.

[0135] The first verification unit is used to provide pre-selected test cases to the hardware verification environment in order to obtain hardware verification results.

[0136] Among the possible implementations are:

[0137] An analysis unit is used to analyze the difference between the hardware verification results and the expected results of the algorithm model after obtaining the hardware verification results;

[0138] An identification unit is used to identify target code units in the algorithm model that have deviations based on the differences.

[0139] The test case query unit is used to query the coverage database for test cases that cover the identified target code unit, as regression verification test cases;

[0140] The second verification unit is used to re-provide the regression verification test cases to the hardware verification environment to verify the correctness of the optimized algorithm model.

[0141] Among the possible implementations are:

[0142] The receiving unit is used to receive multiple target code units corresponding to multiple functional points to be verified.

[0143] The candidate test case set acquisition unit is used to query the coverage database for test cases covering each target code unit to obtain the candidate test case set corresponding to each target code unit.

[0144] The target test case set acquisition unit is used to process each candidate test case set to obtain a target test case set that meets the requirements of multiple function point combination coverage.

[0145] The output unit is used to output the target test case set, wherein the target test case set is used to verify the collaborative scenario of multiple functional points.

[0146] It should be noted that the specific implementation of each unit and subunit in this embodiment can be referred to the corresponding content above, and will not be described in detail here.

[0147] In another embodiment of this application, a readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the test case determination method as described above.

[0148] In another embodiment of this application, an electronic device is also provided for storing a computer program and data generated by the execution of the computer program;

[0149] A processor for executing the computer program to achieve:

[0150] Identify the target code unit to be verified; the target code unit represents the code unit in the algorithm model that corresponds to the functional point to be verified.

[0151] Obtain a coverage database, which represents the degree to which each test case in the test case library covers the preset granularity code units in the algorithm model;

[0152] Based on the target code unit, a query is performed in the coverage database to obtain the test cases that cover the target code unit as the target test cases.

[0153] It should be noted that the specific implementation of the processor in this embodiment can be referred to the corresponding content above, and will not be described in detail here.

[0154] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0155] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0156] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0157] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining test cases, comprising: Identify the target code unit to be verified; The target code unit represents the code unit in the algorithm model that corresponds to the function point to be verified. Obtain a coverage database, which represents the degree to which each test case in the test case library covers the preset granularity code units in the algorithm model; Based on the target code unit, a query is performed in the coverage database to obtain the test cases that cover the target code unit as the target test cases.

2. The method according to claim 1, wherein the process of creating the coverage database includes: Run all test cases in the test case library to execute the algorithm model, and automatically insert coverage detection code through compiler options. Record the number of times each code unit is executed during the runtime of each test case, and generate a coverage data file. Parse the coverage data file to obtain the coverage data of each test case for each code unit; Based on the coverage data, the coverage database is constructed.

3. The method according to claim 2, wherein constructing the coverage database based on the coverage data comprises: Based on the attribute information of each test case in the test case library, a parent table is constructed; Based on the coverage data of each test case for each code unit, construct a sub-table; The sub-table establishes a mapping relationship with the parent table through an association identifier to represent the correspondence between each test case and the covered code unit; The coverage database is constructed based on the parent table and the child table.

4. The method according to claim 3, wherein querying the coverage database based on the target code unit to obtain test cases covering the target code unit as target test cases includes: Search the sub-table for records that cover the target code unit, and obtain the association identifier corresponding to the record; Based on the association identifier, query the parent table for matching test cases; The retrieved test cases are identified as target test cases covering the target code unit and output.

5. The method according to claim 2, further comprising: In response to the addition of a test case to the test case library, the newly added test case is run to execute the algorithm model; Obtain the new coverage data and update the coverage database.

6. The method according to claim 1, further comprising: During the algorithm model verification phase, all test cases are run and the coverage database is established. Before the hardware design verification phase begins, the coverage database is queried based on the target code unit corresponding to the function point to be verified, and test cases covering the target code unit are pre-selected. The pre-selected test cases are provided to the hardware verification environment to obtain hardware verification results.

7. The method according to claim 6, further comprising: After obtaining the hardware verification results, analyze the difference between the hardware verification results and the expected results of the algorithm model; Based on the differences, identify the target code units in the algorithm model that have deviations; Based on the identified target code unit, query the coverage database for test cases that cover the target code unit as regression verification test cases; The regression verification test cases are then re-provided to the hardware verification environment to verify the correctness of the optimized algorithm model.

8. The method according to claim 1, further comprising: Receive multiple target code units corresponding to multiple functional points to be verified; The test cases covering each target code unit are queried in the coverage database to obtain a set of candidate test cases corresponding to each target code unit. Each candidate test case set is processed to obtain a target test case set that meets the coverage requirements of multiple functional points. Output the target test case set, wherein the target test case set is used to verify collaborative scenarios of multiple functional points.

9. A test case determination device, comprising: The determination unit is used to identify the target code unit to be verified. The target code unit represents the code unit in the algorithm model that corresponds to the function point to be verified. The acquisition unit is used to acquire a coverage database, wherein the coverage database represents the degree of coverage of each test case in the test case library to the preset granularity code units in the algorithm model; The query unit is used to query the coverage database based on the target code unit to obtain the test cases that cover the target code unit as target test cases.

10. An electronic device, comprising: A memory for storing computer programs and the data generated by the execution of said computer programs; A processor for executing the computer program to achieve: Identify the target code unit to be verified; the target code unit represents the code unit in the algorithm model that corresponds to the functional point to be verified. Obtain a coverage database, which represents the degree to which each test case in the test case library covers the preset granularity code units in the algorithm model; Based on the target code unit, a query is performed in the coverage database to obtain the test cases that cover the target code unit as the target test cases.

Images