Chip verification method, chip, computing system, and device

Abstract

Embodiments of the present disclosure provide a chip verification method, a chip, a computing system and a device, and relate to the technical field of chips. The chip verification method comprises: obtaining a first domain-specific language text for describing a chip verification plan and a second domain-specific language text for describing a test case corresponding to the chip verification plan; generating a target verification object set corresponding to the chip verification plan and a plurality of test case objects according to the first domain-specific language text and the second domain-specific language text; starting a chip verification process to verify a target chip to be verified according to test execution information in each test case object, and obtaining a case execution result of each test case object; generating verification result information according to the case execution result and the target verification object set; and the verification result information includes a verification state of each function point object and a verification point object. In this way, the efficiency and accuracy of chip verification can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of chip technology, and more specifically, to a chip verification method, a chip, a computing system, and a device. Background Technology

[0002] Chip verification refers to the process of verifying the design logic of a chip during chip design or manufacturing to confirm whether it can achieve the expected functions and performance, thereby improving the correctness and reliability of chip design or manufacturing. In related technologies, chip verification plans are described and maintained using static documents (such as Excel, Word, etc.), while test cases are managed and maintained using scripts (such as Makefile, Tcl scripts, or JSON scripts). This approach to chip verification is inefficient and lacks sufficient statistical accuracy in verification results. Summary of the Invention

[0003] One objective of this disclosure is to provide a new technical solution for chip verification.

[0004] According to a first aspect of the present disclosure, a chip verification method is provided, the method comprising: Obtain a first domain-specific language text for describing the chip verification plan and a second domain-specific language text for describing the test cases corresponding to the chip verification plan; Based on the first domain-specific language text and the second domain-specific language text, a target verification object set and multiple test case objects corresponding to the chip verification plan are generated; wherein, the target verification object set includes multiple function point objects of the chip verification plan, verification point objects corresponding to at least some of the function point objects, and an associated test case set corresponding to the verification point objects, and each test case object includes test execution information and object mapping information representing the verification point object corresponding to the test case object; Based on the test execution information in each of the test case objects, the chip verification process is started to verify the target chip to be verified, and the test case execution results of each of the test case objects are obtained. Based on the execution results of the test cases and the target verification object set, verification result information of the target chip is generated; wherein, the verification result information includes the verification status of each of the function point objects and the verification point objects.

[0005] Optionally, generating a set of target verification objects and multiple test case objects corresponding to the chip verification plan based on the first domain-specific language text and the second domain-specific language text includes: An initial set of verification objects corresponding to the chip verification plan is generated based on the language text of the first domain; wherein, the initial set of verification objects includes multiple function point objects of the chip verification plan and verification point objects corresponding to at least some of the function point objects; Based on the language-specific text of the second domain and the initial set of verification objects, generate multiple test case objects; Based on the object mapping information in the test case object, the test case object is used as the associated test case object of the corresponding verification point object, and an associated test case set of the verification point object is generated based on the associated test case object. The initial set of verification objects generated after generating the set of associated test cases corresponding to the verification point objects is used as the target set of verification objects.

[0006] Optionally, the first domain-specific language text includes function point description text, which includes multiple function point objects, hierarchical attributes of each function point object, and verification point objects corresponding to at least some of the function point objects; the step of generating an initial verification object set corresponding to the chip verification plan based on the first domain-specific language text includes: The first compiler compiles the language text of the first domain to generate multiple function point objects stored in a hierarchical structure. For the lowest-level function point object among multiple function point objects, determine the verification point object corresponding to the lowest-level function point object; The initial set of verification objects is generated based on the multiple function point objects and the verification point objects.

[0007] Optionally, the multiple function point objects in the verification object set include function point objects at least three levels, and the at least three levels of function point objects include: The first-level function point object corresponds to the macro-level classification of the chip verification plan; The second-level function point object corresponds to the sub-function group of the chip verification plan. The second-level function point object is the lower-level node of the first-level function point object. The third-level function point object corresponds to the specific scenario of the chip verification plan. The third-level function point object is a lower-level node of the second-level function point object, and the third-level function point object is the final-level function point object.

[0008] Optionally, the second domain-specific language text includes test case description text, which includes execution information text of the test cases and object mapping text between the test cases and verification point objects; the step of generating multiple test case objects based on the second domain-specific language text and the initial set of verification objects includes: The execution information text is compiled using a second compiler to obtain test execution information; Based on the second compiler, the initial verification object set and the object mapping text are compiled to obtain object mapping information; The test case object is generated based on the test execution information and the object mapping information.

[0009] Optionally, generating the verification result information of the target chip based on the execution result of the use case and the target verification object set includes: For each verification point object in the target verification object set, a first verification state of the verification point object is generated based on the test case execution results of each associated test case object in the associated test case set of the verification point object, and the first verification state is written into the attribute information of the verification point object; Based on the first verification state of each verification point object, determine the second verification state of each function point object; Based on the first verification status and the second verification status, the verification result information of the target chip is generated.

[0010] Optionally, the first verification status includes at least one of the following: uncovered status, not running status, failed status, or passed status; The step of generating a first verification state for the verification point object based on the execution results of each associated test case object in the associated test case set of the verification point object includes: If the set of associated test cases for the verification point object is empty, the first verification state of the verification point object is determined to be an uncovered state; or, If the set of associated test cases for the verification point object is not empty, traverse each associated test case object in the set of associated test cases and determine the first verification state of the verification point object according to the preset rules. The preset rules include: If the execution result of any associated use case object is "not run", then the first verification state of the verification point object is determined to be "not run". If there are no associated test case objects whose execution result is "not run", and the execution result of any associated test case object is "failed", then the first verification state of the verification point object is determined to be a failure state; or, If the execution result of all associated use case objects in the associated use case set is "pass", then the first verification state of the verification point object is determined to be "pass".

[0011] Optionally, the verification result information further includes a coverage report document; the step of generating the verification result information of the target chip based on the first verification status and the second verification status includes: Traverse the set of verification objects to obtain the function point name and the second verification status of each level of function point object, as well as the verification point name and the first verification status of each verification point object; Based on the hierarchical relationship of the set of verification objects, generate hierarchical numbers for each level of function point object and verification point object; A coverage report file is generated based on the level number, the function point name, the second verification status, the verification point name, and the first verification status; wherein, the coverage report file is a table format file, and different color markers are set in the table format file according to different verification statuses.

[0012] According to a second aspect of the present disclosure, a chip is provided, which is a chip that has been successfully verified based on the method described in the first aspect.

[0013] According to a third aspect of the present disclosure, a computing system is provided, the computing system including one or more chips, the chips being chips that have been successfully verified based on the method described in the first aspect.

[0014] According to a fourth aspect of the present disclosure, an electronic device is provided, including a memory and a processor, the memory being configured to store computer instructions, and the processor being configured to invoke the computer instructions from the memory to perform the method described in the first aspect.

[0015] Based on the chip verification method provided in this disclosure, the verification plan and test cases are described using independent domain-specific languages. A set of target verification objects and multiple test case objects are generated according to the domain-specific languages, realizing the mapping and binding of function points, verification points and test cases. During the test execution process, the test status of each function point and verification point can be automatically summarized directly based on the execution results of the test cases, thereby improving the chip verification efficiency and the accuracy of verification result statistics.

[0016] Other features and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0018] Figure 1 This is a schematic diagram of an application scenario that can be used to implement the chip verification method of the embodiments of this disclosure.

[0019] Figure 2 This is a schematic flowchart of a chip verification method provided in an embodiment of this disclosure.

[0020] Figure 3 It is based on Figure 2 The illustrated embodiment provides a flowchart of a method for generating a set of target verification objects and test case objects.

[0021] Figure 4 It is based on Figure 3 The illustrated embodiment is a schematic diagram of a target verification object set and a test case object generated.

[0022] Figure 5 It is based on Figure 2 The illustrated embodiment provides a flowchart of a method for generating verification result information.

[0023] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0024] Various exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0025] The following description of at least one exemplary embodiment is merely illustrative and is not intended to limit this disclosure or its application or use. Techniques, methods, and apparatus known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and apparatus may be considered part of the specification.

[0026] It should be noted that similar labels and letters in the following figures indicate similar items, so once an item is defined in one figure, it need not be discussed further in subsequent figures.

[0027] This disclosure relates to a technical solution for chip verification. Figure 1 This is a schematic diagram of an application scenario that can be used to implement the chip verification method of the embodiments of this disclosure, such as... Figure 1 As shown, this application scenario may include a test platform 100 and an operation terminal 200.

[0028] The test platform 100 can be used for chip verification, such as executing the chip verification method of the embodiments of this disclosure. Exemplarily, the test platform may include a processor, on which the chip verification method is executed. The processor may include at least one of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a General-Purpose Graphics Processing Unit (GPGPU), or a Neural-network Processing Unit (NPU). Exemplarily, the test platform may also include a memory, which may include at least one of Static Random Access Memory (SRAM), Read-Only Memory (ROM), or Erasable Programmable Read-Only Memory (EPROM). The memory can be used to store a computer program that, when executed by the processor, controls the processor to execute the chip verification method of the embodiments of this disclosure.

[0029] The operating terminal 200 and the test platform 100 can establish a connection via an interface device or a communication device. Users can transmit test files or other information to the test platform via the operating terminal, and the processor of the test platform reads the test file and executes the methods of this embodiment. Users can also manage test plans and test cases on the test platform via the operating terminal.

[0030] exist Figure 1 In the application scenario shown, the test platform 100 and the operation terminal 200 are illustrated as different physical devices. It should be noted that the test platform 100 and the operation terminal 200 can also be integrated on the same physical device, and this embodiment does not limit this.

[0031] During chip design or manufacturing, the chip's design logic can be verified using a testing platform to confirm whether it achieves the expected functionality and performance. In related technologies, verification engineers develop and maintain verification plans and test cases during chip verification. Verification plans can be described and maintained using static documents (such as Excel or Word documents), while test cases can be managed and maintained using scripts (such as Makefiles, Tcl scripts, or JSON scripts). For example, after completing the design document, verification engineers extract functional points from a verification perspective, forming a verification plan in Excel or Word format. Then, test cases are developed, and the verification plan document is manually annotated or recorded to indicate which test cases cover a particular functional point. If a feature is covered by multiple test cases, due to the lack of script-level automatic association and aggregation logic, it is impossible to automatically deduce the final state of the feature in real time based on the success or failure status of the test cases; manual association and judgment are usually required. This results in a lack of automated association and status feedback mechanisms between the verification plan and test cases in the chip verification process, leading not only to low verification efficiency but also to insufficient accuracy of manual statistics, which in turn reduces verification accuracy.

[0032] Furthermore, the chip verification solutions in related technologies also have the following problems: chip verification plans are usually maintained in static documents (Excel, Word), while test cases are managed through scripts. The two are scattered in different file systems, making it difficult to establish an effective correlation; verification plan documents lack version traceability, which does not meet engineering management requirements; when a verification point requires coverage by multiple test cases, the test results cannot be automatically aggregated to determine the status of the verification point, and can only rely on manual judgment, which is prone to logical discrepancies; when facing large-scale regression test sets, manually calculating the verification plan coverage is a huge workload and prone to errors, and may result in false pass status of verification points due to overlooking failed tests, causing a large waste of human resources.

[0033] This disclosure provides a novel technical solution for chip verification. By defining a Domain Specific Language (DSL) to describe the verification plan and test cases, a target verification object set is generated, and a mapping information between verification point objects and test case objects is established through an object reference mechanism. Based on the test case execution results and the target verification object set, verification result information of the target chip is automatically generated to solve one or more of the above problems.

[0034] The following combination Figure 1 The illustrated application environment illustrates the specific implementation of the embodiments of this disclosure.

[0035] Figure 2This is a schematic flowchart of a chip verification method provided in an embodiment of this disclosure. The chip verification method can be performed by... Figure 1 The verification platform and / or operating terminal shown are used for execution. For example... Figure 2 As shown, the chip verification method of this embodiment may include the following steps S210 to S240.

[0036] Step S210: Obtain the first domain-specific language text used to describe the chip verification plan and the second domain-specific language text used to describe the test cases corresponding to the chip verification plan.

[0037] The first domain-specific language (Feature DSL) can be used to provide a structured textual description of the chip verification plan. For example, based on the first domain-specific language text, feature points and / or verification points of the verification plan can be defined. For instance, the first domain-specific language text may include feature point description text, which includes multiple feature point objects, hierarchical attributes of each feature point object, and verification point (VPoint) objects corresponding to at least some of the feature point objects.

[0038] The second domain-specific language (Test DSL) can be used to describe the structured textual description of test cases. For example, based on the second domain-specific language text, execution information of test cases and object mapping information between test cases and verification point objects can be defined. For instance, the second domain-specific language text may include test case description text, which may include execution information text and object mapping text between test cases and verification point objects. For example, the execution information text may include at least one of the following: compilation configuration text describing the compilation environment, test parameter text for determining test parameters, and regression grouping text for classifying test cases into specific regression groups.

[0039] The aforementioned first and second domain-specific language texts can be text input by users (e.g., verification engineers), who can input the corresponding text based on the formatting requirements of the domain-specific language. Users can use these first and second domain-specific language texts to describe the verification plan and test cases, respectively. For example, the first domain-specific language text can use hierarchical indentation syntax, representing the parent-child relationship of function points through indentation levels, and distinguishing function point objects and verification point objects through specific keywords (e.g., feature, point, etc.). The second domain-specific language text can use key-value pair syntax, describing compilation configuration, test parameters, regression grouping, and object mapping information through specific fields (e.g., config, args, group, covers, etc.).

[0040] In some examples, domain-specific language text and domain-specific language text can be stored in different file system paths or code repositories. Operating terminals or verification platforms can automatically read these two types of text using pre-configured path information. This allows verification plans and test cases to be maintained by different teams in parallel, without interference, thus improving the collaborative efficiency of large-scale chip verification projects.

[0041] By adopting a two-layer domain-specific language architecture, the description of the verification plan can be decoupled from the execution configuration of the test cases, so that the maintenance of the verification plan and the management of the test cases can be carried out independently, while the relationship between the two can be established at compile time through the object reference mechanism.

[0042] Step S220: Based on the first domain-specific language text and the second domain-specific language text, generate a set of target verification objects and multiple test case objects corresponding to the chip verification plan.

[0043] The target verification object set may include multiple function point objects of the chip verification plan, verification point objects corresponding to at least some of the function point objects, and a set of associated test cases corresponding to the verification point objects.

[0044] In some examples, function point objects can be used to hierarchically categorize chip verification plans. For instance, they can be organized using a tree structure with at least three levels, including first-level function point objects corresponding to macro-functional modules, second-level function point objects corresponding to sub-functional groups, and third-level function point objects (i.e., last-level function point objects) corresponding to specific verification scenarios. The verification status (second verification status) of a function point object is recursively aggregated from the status of its subordinate child nodes, reflecting the overall verification progress from a project management perspective. Verification point objects can be attached to last-level function point objects, providing fine-grained descriptions of specific verification items. Each last-level function point object can correspond to one or more verification point objects. Each verification point object internally maintains a set of associated test cases to store all test case objects mapped to that verification point. The verification status (first verification status) of a verification point object is directly calculated by aggregating the execution results of each test case object in its associated test case set according to preset rules, from a test execution perspective. By designing the above two-level objects of functional points and verification points, the perspectives of verification plan management and test execution are unified. In this way, the hierarchical structure of functional point objects provides a coverage statistics view from macro to micro for project management. The associated test case set and its status aggregation mechanism of verification point objects solve the technical problem that multiple test cases covering the same verification point cannot be automatically aggregated and judged, avoiding the risk of false pass status caused by manual judgment, and significantly improving the accuracy and automation of coverage statistics.

[0045] In some examples, each test case object may include test execution information and object mapping information representing the verification point object corresponding to the test case object. For example, the test execution information may include at least one of the following: compilation configuration information for specifying the compilation environment (e.g., different simulation tool versions or compilation options), test parameter information for determining test parameters, and regression grouping information for classifying test cases into specific regression groups (e.g., smoke test group, full regression group, etc.). Based on this test execution information, flexible test management and execution strategies can be supported.

[0046] In this step, the first domain-specific language text and the second domain-specific language text can be compiled through a compilation mechanism to compile the domain-specific language text into an object structure in memory, and a bidirectional mapping relationship between verification point objects and test case objects can be established based on the object structure. This bidirectional mapping relationship can include the set of associated test cases corresponding to the verification point objects and the object mapping information representing the verification point objects corresponding to the test case objects.

[0047] For example, the verification platform can first compile the language text specific to the first domain, parse out the hierarchical structure of the verification plan, and generate an initial set of verification objects containing multiple function point objects and verification point objects; then, it can compile the language text specific to the second domain, and combine it with the verification point information in the initial set of verification objects to generate multiple test case objects; finally, based on the object mapping information in each test case object, it can associate the test case objects with the corresponding verification point objects to form the final target set of verification objects. Through this method, a clear object-level mapping relationship is established between the verification plan and the test cases, which helps to accurately calculate verification coverage subsequently.

[0048] Step S230: Based on the test execution information in each test case object, start the chip verification process to verify the target chip to be verified, and obtain the test case execution results of each test case object.

[0049] For example, the test execution information may include compilation configuration information, test parameter information, regression grouping, and other information. The verification platform can determine parameters such as simulator type and compilation options based on the compilation configuration information, set the parameter values ​​for simulation runtime based on the test parameter information, and classify test cases into different regression test sets based on the regression grouping information.

[0050] For example, the verification platform can traverse each test case object, sequentially read the test execution information in each test case object, and construct the corresponding simulation runtime environment and simulation runtime commands based on the test execution information, thus initiating the chip verification process to verify the design logic of the target chip. This chip verification process can include a simulation verification process based on the Universal Verification Methodology (UVM). For example, this chip verification process can call simulation tools to perform functional and / or performance verification of the target chip design and generate a verification log file. The verification log file can include test case execution results, which may include states such as not run, failed, or passed. These states can be automatically identified by parsing specific identifiers (such as keywords like PASS and FAIL) in the verification log file.

[0051] After the chip verification process is completed, the verification platform can parse the verification log file generated by the chip verification process, extract the test status information corresponding to each test case object from the verification log file, and update the test case execution result of the corresponding test case object according to the test status information. For example, the test case execution result may include test passed (Pass), test failed (Fail), or not run (NotRun). The test case execution result can be the same as the test status information.

[0052] The above methods can automate the execution of test cases and the collection of results, which helps reduce manual intervention and improve the efficiency of verification execution.

[0053] In some examples, chip verification processes corresponding to multiple test case objects can be started in parallel. By monitoring the running status of each process, the execution results of each test case object can be obtained separately. This can make full use of computing resources, shorten the overall verification time, and further improve chip verification efficiency.

[0054] Step S240: Generate the verification result information of the target chip based on the test case execution result and the target verification object set.

[0055] For example, the verification result information may include the verification status of each functional point object and the verification point object. This verification status can be automatically generated based on the aggregation of test case execution results. For instance, the verification status of each verification point object can be determined based on the test case execution results of each test case object in the associated test case set of each verification point object; then, based on the verification status of each verification point object, the verification status of each level of functional point object is determined by a hierarchical aggregation; and finally, the above verification status information is summarized to generate verification result information characterizing the overall verification coverage of the target chip.

[0056] The chip verification method described in steps S210 to S240 above uses an independent domain-specific language to describe the verification plan and test cases. A set of target verification objects and multiple test case objects are generated based on the domain-specific language, realizing the mapping and binding of function points, verification points and test cases. During the test execution process, the test status of each function point and verification point can be automatically summarized directly based on the execution results of the test cases, thereby improving the chip verification efficiency and the accuracy of verification result statistics.

[0057] Figure 3 It is based on Figure 2 The illustrated embodiment provides a flowchart of a method for generating a set of target verification objects and test case objects. Figure 3 As shown, step S220 may include steps S221 to S224.

[0058] Step S221: Generate an initial set of verification objects corresponding to the chip verification plan based on the language text specific to the first domain.

[0059] The initial verification object set includes multiple function point objects of the chip verification plan, as well as verification point objects corresponding to at least some of the function point objects. It should be noted that this initial verification object set may not include the set of associated test cases for the verification point objects.

[0060] In some examples, this step can perform lexical and / or syntactic analysis on the first domain-specific language text, parse the hierarchical structure of function points described in the text into a tree of function point objects in memory, and automatically create corresponding verification point objects for the function point objects at the lowest level, thereby forming an initial set of verification objects.

[0061] In other examples, the first domain-specific language text may include function point description text, which includes multiple function point objects, hierarchical attributes of each function point object, and verification point objects corresponding to at least some of the function point objects. The method for generating the initial verification object set in this step may include: compiling the first domain-specific language text using a first compiler to generate multiple function point objects stored in a hierarchical structure; determining the verification point object corresponding to the lowest-level function point object among the multiple function point objects; and generating the initial verification object set based on the multiple function point objects and the verification point objects. The aforementioned verification point objects can be explicitly declared directly in the function point description text, or they can be automatically created by the verification platform based on the attributes of the lowest-level function point objects.

[0062] The first compiler can be a compiler used to compile text in a language specific to a first domain. It can parse the text in the language specific to the first domain and compile it to obtain the corresponding initial verification object. The first compiler can use one or more of the compilation techniques such as lexical analysis, syntax analysis, and semantic analysis to parse the text in the language specific to the first domain. The first compiler can identify syntactic elements in the text, such as keywords (e.g., feature, point), hierarchical indentation relationships, attribute fields, etc., and construct an abstract syntax tree according to predefined syntactic rules.

[0063] For example, the first compiler can first perform lexical analysis, decomposing the text into lexical units such as keywords, identifiers, and indentation symbols, and then obtain function point objects and validation point objects based on these lexical units. Next, it performs syntax analysis, constructing a hierarchical relationship tree of function point objects and validation point objects according to the indentation level. Finally, it performs semantic analysis, extracting information such as the name, description, and hierarchical attributes of each function point object, and assigning a unique identifier to each function point object. The function point objects generated by this first compiler can be stored using a tree data structure, where each function point object can contain references to its parent and child nodes for subsequent hierarchical traversal and state aggregation operations.

[0064] Optionally, validation point objects can be defined below last-level function point objects. When parsing first-domain-specific language text, the first compiler can identify specific keywords (such as "point") to distinguish validation point objects from function point objects. For each last-level function point object, the first compiler can extract all validation point objects defined below it and establish a parent-child relationship between the last-level function point object and the validation point object. For example, a validation point object may include attributes such as validation point name, validation point description, and a unique identifier for the validation point. Furthermore, the first compiler can initialize an empty set of associated test cases for each validation point object to store associated test case objects later.

[0065] In some examples, this initial set of verification objects can be stored using an object container (such as a list, dictionary, or other data structure) to facilitate lookup and traversal operations. This initial set of verification objects can simultaneously maintain a hierarchical tree structure of function point objects and a flat list of verification point objects to support access requirements in different scenarios.

[0066] In this way, the hierarchical structure of functional points in the validation plan can be automatically parsed into structured storage objects, which can avoid the hierarchical confusion and omissions that are easy to occur when manually maintaining static documents, and improve the standardization and maintainability of validation plan management.

[0067] In some examples, the multiple function point objects in the above-mentioned set of verification objects include at least three levels of function point objects, which may include the following first-level function point objects to third-level function point objects.

[0068] The first-level function point objects correspond to the macro-level classification of the chip verification plan. For example, the first-level function point objects can correspond to the main functional modules of the chip, such as macro-level classifications like processor core verification, memory controller verification, and bus interface verification.

[0069] The second-level function point object corresponds to the sub-function group of the chip verification plan. The second-level function point object is the lower-level node of the first-level function point object. For example, under the first-level function point object of processor core verification, it can include second-level function point objects such as instruction execution verification, pipeline verification, and cache coherence verification.

[0070] The third-level function point object corresponds to the specific scenario of the chip verification plan. The third-level function point object is a lower-level node of the second-level function point object. For example, under the second-level function point object of instruction execution verification, there may be third-level function point objects such as arithmetic instruction verification, logical instruction verification, and jump instruction verification. Specific verification point objects can be defined below these third-level function point objects. Optionally, this third-level function point object can be a last-level function point object.

[0071] The following is a grammatical example of a first-domain-specific language text (Feature DSL): l1feauture l1_example_001: desc = “l1_example_001 Function Description” l2feature l2_example_001: desc = “l2_example_001 Functionality Description” l3feature l3_example_001: desc = “l3_example_001 Functionality Description” vpoint.add("vp_a: description of test point a") vpoint.add("vp_b: description of test point b") endl3feature endl2feature endl1feature In this way, by employing a functional point object structure with at least three levels, a clear verification plan hierarchy can be constructed. This allows for both a macro-level grasp of verification progress and fine-grained coverage statistics at the specific scenario level. This hierarchical structure can be extended according to the complexity of the actual verification plan, supporting four, five, or even more levels of functional point objects.

[0072] Step S222: Generate multiple test case objects based on the second domain-specific language text and the initial set of verification objects.

[0073] In some examples, this step may perform lexical and / or syntactic analysis on the second domain-specific language text to obtain test case analysis results containing test case execution information and object mapping information. Based on the test case analysis results and the initial verification object set, multiple test case objects are generated. For example, the object mapping information in the test case analysis results can be verified by combining the existing verification point object information in the initial verification object set, ensuring that the verification point objects referenced by the test cases do indeed exist in the initial verification object set, thereby generating verified test case objects.

[0074] In other examples, the second domain-specific language text may include test case description text, which may include test case execution information text and object mapping text between test cases and verification point objects. The method for generating test case objects in this step may include: compiling the execution information text using a second compiler to obtain test execution information; compiling the initial verification object set and object mapping text using a second compiler to obtain object mapping information; and generating test case objects based on the test execution information and object mapping information. The second compiler may be the same as or different from the first compiler.

[0075] For example, the second compiler can identify specific fields (e.g., config, args, group, etc.) in the execution information text and extract the values ​​of these fields as test execution information. This test execution information may include compilation environment configuration information, test parameter information, regression grouping information, etc. The compilation environment configuration information can be used to determine the simulator type (e.g., VCS, Xcelium, etc.), compilation options (e.g., whether to enable code coverage, whether to enable assertion checking, etc.), and simulation parameters (e.g., simulation clock frequency, simulation duration, etc.). The test parameter information can be used to determine the parameter values ​​during test case execution (e.g., packet size, number of transactions, etc.). The regression grouping information can be used to categorize test cases into different regression test sets (e.g., daily regression, weekly regression, pre-release regression, etc.).

[0076] The object mapping text mentioned above may contain a reference identifier to the verification point object (such as a unique identifier or path to the verification point). The second compiler can use this reference identifier to find the corresponding verification point object in the set of verification objects and establish an object reference relationship between the test case object and the verification point object.

[0077] For example, the object mapping text can be in the format of [verification point path 1, verification point path 2, ...], where the verification point path can use a hierarchical path representation (e.g., functional module A. sub-function B. verification point C). The second compiler can parse this path, search along the hierarchical tree of functional point objects, locate the corresponding verification point object, and store a reference to that verification point object in the object mapping information of the test case object.

[0078] By resolving object mappings at compile time, the association between test case objects and verification point objects can be a strongly typed object reference. This approach can detect reference errors (such as referencing a non-existent verification point) at compile time and avoid runtime errors.

[0079] In some examples, a test case object may contain attributes such as the test case name, test execution information, object mapping information, and test case execution results. This test case object can be encapsulated using object-oriented programming techniques such as classes or structs for subsequent access and manipulation.

[0080] The following is a grammatical example of a second domain-specific language text (Test DSL): test test001: dut_config = dutA # Specifies the DUT compilation object corresponding to Test (can include filelist and parameters). run_args.add("+UVM_TESTNAME=uvm_test_A") # Add simulation parameters run_args.add("+customized_plus_args") regression.add("sanity") # Classifies the Test into a specific regression group info.desc = “Test case description” info.vpoint = features.13feature001.vp_a # Object property reference, establishing a mapping from the test case object (Test) to the verification point object (Feature.VPoint). endtest The `dut_config` function links test cases to specific compilation environments, supporting flexible partitioning of different models (such as RTL simulation and gate-level simulation). `regression.add` allows grouping and labeling different test cases `t`, enabling the validation platform to automatically generate regression test groups of different granularities, such as a smoke test (Sanity regression) group or a nightly regression test group, and obtain the corresponding validation plan coverage report. `info.vpoint` can be a command that, in the generated runtime script, registers the test case object of `test001` to the associated test case set of the `vp_a` node (validation point object) of `l3feature001` in the features object tree.

[0081] By employing a second compiler to compile the second domain-specific language text, the dynamic execution configuration of test cases can be transformed into a specific object structure, and the association between the test cases and the verification point objects can be established during compilation. This compilation process works in conjunction with the compilation process of the first compiler to jointly construct a complete verification object model.

[0082] It should be noted that the compiler in this embodiment can also be called a parser. For example, the first compiler can be called the first parser, and the second compiler can be called the second parser. The first compiler can identify nested keywords, function point information, verification point information, etc., and construct nested scopes. The second compiler can identify test case information, such as test block keywords, test-related configuration instructions, cross-DSL reference syntax features.xxx, etc. Among them, test-related configuration instructions can include one or more of the following: device under test configuration (dut config), run parameters (run args), regression test (regression), and info.

[0083] The aforementioned domain-specific language text (LST) and domain-specific language text (LST) can be two sets of languages ​​with different domain concepts. The LST describes the verification plan, i.e., the static structure of what needs to be verified, while the LST describes the test path, i.e., the dynamic execution process of how verification is performed. Because the concepts are different, they are implemented using independent syntax and independent compilers. For example, the compilation target of the first compiler used by the LST can be a Python object tree (one implementation of a target verification object set). This tree stores the function point objects and verification point objects at each level, maintains an associated test case set (e.g., the `cov_tests` list) within each verification point object, and designs a status judgment algorithm (e.g., `get_cov_status`, which can be executed during verification). Based on the verification results of all test case objects in this associated test case set, logical operations are used to determine the first verification status of the verification point object. Furthermore, the Python object tree generated by the LST can be referenced by the LST at compile time. For example, the compilation target of the second compiler used by the second domain-specific language text can be a Python runtime script (containing test execution instructions and back-annotation logic). The compiler can parse the features.xxx in the target verification object set for reference and add the test case object to the associated test case set of the corresponding verification point object. When parsing cross-DSL references, it can rely on the object tree already compiled by the first domain-specific language text to achieve compile-time type checking. In this way, using two independent domain-specific languages ​​can avoid redundancy, reduce complexity, and improve verification efficiency and user experience.

[0084] Step S223: Based on the object mapping information in the test case object, the test case object is used as the associated test case object of the corresponding verification point object, and a set of associated test cases for the verification point object is generated based on the associated test case object.

[0085] For example, all test case objects can be traversed, the verification point mapping information in each test case object can be read, the corresponding verification point object in the initial verification object set can be found, and the test case object can be added to the associated test case set of the verification point object. This associated test case set can be stored using data structures such as lists or sets for subsequent status judgment and / or coverage statistics.

[0086] In this way, a reverse mapping from test case objects to verification point objects is achieved, establishing a two-way mapping relationship between verification point objects and test case objects.

[0087] Step S224: The initial set of verification objects generated after generating the set of associated test cases corresponding to the verification point objects is used as the target set of verification objects.

[0088] In this way, the generated set of target verification objects contains not only the complete hierarchical structure of the verification plan, but also the relationship between each verification point object and the test case object, thus providing a complete data foundation for subsequent verification execution and verification result statistics, which can improve the efficiency of chip verification and the accuracy of verification result statistics.

[0089] Figure 4 It is based on Figure 3 The illustrated embodiment provides a schematic diagram of a target verification object set and test case objects generated. Figure 4 As shown, the target verification object set includes multiple function point objects and multiple verification point objects. The function point objects are divided into three levels. The lower-level nodes of the first-level function point object 11 include the second-level function point objects 21 and 22; the lower-level nodes of the second-level function point object 21 include the third-level function point objects 31 and 32; and the lower-level node of the second-level function point object 22 is the third-level function point object 33. In the third level, verification point objects a and b are attached to the third-level function point object 31, verification point objects c and d are attached to the third-level function point object 32, and verification point object e is attached to the third-level function point object 33. Each verification point object can map to one or more test case objects. Multiple test case objects (the first to fourth test case objects) are mapped to the same or different verification point objects. For example, among multiple test case objects, at least two different verification point objects may be mapped to the same test case object; similarly, among multiple test case objects, two different test case objects may be mapped to the same verification point object. This constitutes a complete combination of target verification objects and test case objects.

[0090] Figure 5 It is based on Figure 2 The illustrated embodiment provides a flowchart of a method for generating verification result information. For example... Figure 5 As shown, step S240 may include steps S241 to S243.

[0091] Step S241: For each verification point object in the target verification object set, generate the first verification state of the verification point object based on the test case execution results of each associated test case object in the associated test case set of the verification point object, and write the first verification state into the attribute information of the verification point object.

[0092] In this step, the execution results of the test cases of the associated test cases can be obtained based on the set of associated test cases of the verification point object. By saving the first verification status in the attribute information of the verification point object, timely feedback and automatic back-annotation of the verification status can be achieved.

[0093] In some examples, the first verification state may include at least one of the following: NotCover, NotRun, Fail, or Pass.

[0094] If the set of associated test cases for a verification point object is empty, the first verification state of the verification point object can be determined to be an uncovered state. This uncovered state indicates that there are currently no test cases associated with the verification point, and it has not yet been covered. New test cases need to be added to verify the verification point.

[0095] If the set of associated test cases for a verification point object is not empty, each associated test case object in the set can be traversed, and the first verification state of the verification point object can be determined according to preset rules. These preset rules may include at least one of the following rules one through three.

[0096] Rule 1: If the execution result of any associated test case object is "not run", then the first verification state of the verification point object is determined to be the "not run" state. This rule adopts the weakest link principle, that is, as long as any test case has not been run, the verification point is considered to be in the "not run" state, with the highest priority.

[0097] Rule 2: If there are no associated test case objects whose execution result is "not run", and the execution result of any associated test case object is "failed", then the first verification state of the verification point object is determined to be "failed". This rule means that assuming all test cases have been run, if even one test case fails, the verification point is considered to be in a "failed" state. Rule 2 has a lower priority than Rule 1 but a higher priority than Rule 3.

[0098] Rule 3: If the execution results of all associated test cases in the associated test case set are all passed, then the first verification state of the verification point object is determined to be the passed state. This rule means that the verification point is considered to be in the passed state only when all test cases pass. Rule 3 has the lowest priority.

[0099] By employing the aforementioned state aggregation rule, the verification states of verification point objects can be automatically aggregated. This rule embodies the design principle of the weakest link principle, where the state of a verification point is determined by its worst-case test case, ensuring the conservatism and reliability of the verification results. Furthermore, this rule can be predefined at compile time and executed automatically at runtime, requiring no manual judgment.

[0100] Step S242: Determine the second verification status of each function point object based on the first verification status of each verification point object.

[0101] In this step, the first verification status of all verification point objects under a certain function point object can be summarized, and the second verification status of the function point object can be determined according to a predetermined rule.

[0102] For example, the second verification state of a function point object can also be aggregated using a similar state aggregation rule to the first verification state of a verification point object. For a last-level function point object, aggregation can be performed based on the first verification state of all its subordinate verification point objects; for non-last-level function point objects, recursive aggregation can be performed based on the second verification state of its child function point objects. This aggregation rule also adopts the weakest link principle, that is, the state of a function point is determined by its worst child node.

[0103] Step S243: Generate verification result information for the target chip based on the first verification state and the second verification state.

[0104] In some examples, the verification object set can be traversed to obtain the function point name and second verification status of each level function point object, as well as the verification point name and first verification status of each verification point object; a level number can be generated for each level function point object and verification point object according to the hierarchical relationship of the verification object set; and a coverage report file can be generated according to the level number, function point name, second verification status, verification point name, and first verification status.

[0105] For example, a depth-first search or breadth-first search algorithm can be used to traverse the hierarchy tree of the verification object set, sequentially visiting each function point object and verification point object to extract their names and verification status information. The generated hierarchy numbers can adopt a multi-level numbering format, such as 1, 1.1, 1.1.1, 1.1.1.1, etc., to reflect the hierarchical and subordinate relationships between function point objects and verification point objects. This hierarchy number can be dynamically generated during the traversal process, or it can be pre-calculated and stored in the object attributes at compile time. For example, first-level function point objects can be numbered 1, 2, 3, etc.; second-level function point objects can be numbered 1.1, 1.2, 2.1, etc.; third-level function point objects can be numbered 1.1.1, 1.1.2, 1.2.1, etc.; and verification point objects can be numbered 1.1.1.1, 1.1.1.2, etc.

[0106] In some examples, the coverage report file described above can be a tabular file, such as an Excel file, a CSV file, or an HTML table file. This tabular file can be configured with different color markers for different verification statuses to visually display the verification progress.

[0107] For example, an uncovered state can be marked in red, an inactive state in yellow, a failed state in orange, and a passed state in green. This color coding can be applied to the background or text color of table cells, allowing validators to quickly identify weaknesses in the validation plan.

[0108] Optionally, the coverage report file may also include statistical information, such as the number of each state, coverage percentage, and a list of uncovered items. This statistical information can be visualized using charts, such as pie charts or bar charts.

[0109] By generating structured coverage report documents, verification personnel can be provided with a clear view of the verification progress, making it easy to quickly locate verification blind spots and failed use cases, and supporting continuous improvement of the verification plan.

[0110] This disclosure also provides a chip, which can be a chip that has been successfully verified based on any of the chip verification methods described in the foregoing embodiments of this disclosure. The chip can be a processor, such as one or more processors including a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a General-Purpose Graphics Processing Unit (GPGPU), or a Neural-network Processing Unit (NPU).

[0111] This disclosure also provides a computing system that may include one or more chips, any of which is a chip that has been successfully verified based on any of the chip verification methods described in the foregoing embodiments of this disclosure. Exemplarily, the chip in the computing system may be a processor, such as one or more of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a General-Purpose Graphics Processing Unit (GPGPU), or a Neural-network Processing Unit (NPU).

[0112] The computing system can be any type of electronic device, such as any type of terminal device, workstation or server, etc., and this embodiment does not limit it.

[0113] This disclosure also provides an electronic device, such as... Figure 6As shown, the electronic device 600 may include a memory 610 and a processor 620. The memory can be used to store computer instructions, and the processor can be used to retrieve computer instructions from the memory to execute all or part of the steps of any of the methods in the foregoing embodiments of this disclosure. The processor may be one or more processors, which may execute instructions individually or jointly. Similarly, the memory may be one or more memories, which may store the aforementioned computer instructions individually or jointly. The processor may be a CPU, or a chip in the form of an Application Specific Integrated Circuit (ASIC), System on Chip (SOC), Field Programmable Gate Array (FPGA), Programmable Logic Array (PLA), etc., which are not limited in this embodiment.

[0114] This disclosure also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements all or part of the steps of any of the methods in the foregoing embodiments of this disclosure. Optionally, the computer-readable storage medium may be a non-transitory storage medium, but is not limited thereto, and may also be a temporary storage medium.

[0115] This disclosure also provides a computer program product that may include a computer program that, when executed by a processor, can implement all or part of the steps of any of the methods in the foregoing embodiments of this disclosure.

[0116] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and apparatuses according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, unit, or part of a circuit. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should be noted that embodiments of the present disclosure may include some or all of the functions marked in the multiple blocks in the drawings, and may also include other functions not shown in the blocks in the drawings. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented in hardware that performs the specified function or action, or in a combination of dedicated hardware and computer instructions. Unless otherwise specified, implementation in hardware, implementation in software, and implementation in a combination of software and hardware may be equivalent.

[0118] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A chip verification method, characterized in that, The method includes: Obtain a first domain-specific language text for describing the chip verification plan and a second domain-specific language text for describing the test cases corresponding to the chip verification plan; Based on the first domain-specific language text and the second domain-specific language text, a target verification object set and multiple test case objects corresponding to the chip verification plan are generated; wherein, the target verification object set includes multiple function point objects of the chip verification plan, verification point objects corresponding to at least some of the function point objects, and an associated test case set corresponding to the verification point objects, and each test case object includes test execution information and object mapping information representing the verification point object corresponding to the test case object; Based on the test execution information in each of the test case objects, the chip verification process is started to verify the target chip to be verified, and the test case execution results of each of the test case objects are obtained. Based on the execution results of the test cases and the target verification object set, verification result information of the target chip is generated; wherein, the verification result information includes the verification status of each of the function point objects and the verification point objects.

2. The method according to claim 1, characterized in that, The step of generating a set of target verification objects and multiple test case objects corresponding to the chip verification plan based on the first domain-specific language text and the second domain-specific language text includes: An initial set of verification objects corresponding to the chip verification plan is generated based on the language text of the first domain; wherein, the initial set of verification objects includes multiple function point objects of the chip verification plan and verification point objects corresponding to at least some of the function point objects; Based on the language-specific text of the second domain and the initial set of verification objects, generate multiple test case objects; Based on the object mapping information in the test case object, the test case object is used as the associated test case object of the corresponding verification point object, and an associated test case set of the verification point object is generated based on the associated test case object. The initial set of verification objects generated after generating the set of associated test cases corresponding to the verification point objects is used as the target set of verification objects.

3. The method according to claim 2, characterized in that, The first domain-specific language text includes function point description text, which includes multiple function point objects, hierarchical attributes of each function point object, and verification point objects corresponding to at least some of the function point objects. The step of generating an initial set of verification objects corresponding to the chip verification plan based on the language-specific text of the first domain includes: The first compiler compiles the language text of the first domain to generate multiple function point objects stored in a hierarchical structure. For the lowest-level function point object among multiple function point objects, determine the verification point object corresponding to the lowest-level function point object; The initial set of verification objects is generated based on the multiple function point objects and the verification point objects.

4. The method according to claim 3, characterized in that, The set of verification objects includes multiple function point objects at least three levels, and the at least three levels of function point objects include: The first-level function point object corresponds to the macro-level classification of the chip verification plan; The second-level function point object corresponds to the sub-function group of the chip verification plan. The second-level function point object is the lower-level node of the first-level function point object. The third-level function point object corresponds to the specific scenario of the chip verification plan. The third-level function point object is a lower-level node of the second-level function point object, and the third-level function point object is the final-level function point object.

5. The method according to claim 2, characterized in that, The second domain-specific language text includes test case description text, which includes test case execution information text and object mapping text between the test case and the verification point object; The step of generating multiple test case objects based on the second domain-specific language text and the initial verification object set includes: The execution information text is compiled using a second compiler to obtain test execution information; Based on the second compiler, the initial verification object set and the object mapping text are compiled to obtain object mapping information; The test case object is generated based on the test execution information and the object mapping information.

6. The method according to any one of claims 1 to 5, characterized in that, The step of generating verification result information for the target chip based on the execution result of the test case and the target verification object set includes: For each verification point object in the target verification object set, a first verification state of the verification point object is generated based on the test case execution results of each associated test case object in the associated test case set of the verification point object, and the first verification state is written into the attribute information of the verification point object; Based on the first verification state of each verification point object, determine the second verification state of each function point object; Based on the first verification status and the second verification status, the verification result information of the target chip is generated.

7. The method according to claim 6, characterized in that, The first verification status includes at least one of the following: uncovered status, not running status, failed status, or passed status; The step of generating a first verification state for the verification point object based on the execution results of each associated test case object in the associated test case set of the verification point object includes: If the set of associated test cases for the verification point object is empty, the first verification state of the verification point object is determined to be an uncovered state; or, If the set of associated test cases for the verification point object is not empty, traverse each associated test case object in the set of associated test cases and determine the first verification state of the verification point object according to the preset rules. The preset rules include: If the execution result of any associated use case object is "not run", then the first verification state of the verification point object is determined to be "not run". If there are no associated test case objects whose execution result is "not run", and the execution result of any associated test case object is "failed", then the first verification state of the verification point object is determined to be a failure state; or, If the execution result of all associated use case objects in the associated use case set is "pass", then the first verification state of the verification point object is determined to be "pass".

8. The method according to claim 6, characterized in that, The verification result information also includes a coverage report document; the step of generating the verification result information of the target chip based on the first verification status and the second verification status includes: Traverse the set of verification objects to obtain the function point name and the second verification status of each level of function point object, as well as the verification point name and the first verification status of each verification point object; Based on the hierarchical relationship of the set of verification objects, generate hierarchical numbers for each level of function point object and verification point object; A coverage report file is generated based on the level number, the function point name, the second verification status, the verification point name, and the first verification status; wherein, the coverage report file is a table format file, and different color markers are set in the table format file according to different verification statuses.

9. A chip, characterized in that, The chip is a chip that has been successfully verified based on the chip verification method according to any one of claims 1 to 8.

10. A computing system, characterized in that, The computing system includes one or more chips, which are chips that have been successfully verified based on the chip verification method according to any one of claims 1 to 8.

11. An electronic device, characterized in that, The method includes a memory and a processor, the memory being used to store computer instructions, and the processor being used to retrieve the computer instructions from the memory to perform the method of any one of claims 1 to 8.

Images