Chip verification data optimization method, chip verification data system and device

Abstract

The embodiment of the application relates to the chip verification technical field, in particular to a kind of optimization method of chip verification data, chip verification data system and equipment, method includes: the command parameter list of target verification team is obtained;Response to the task command input by target verification team, the target test case of target verification team is executed based on command parameter list, chip verification data is obtained, and saved in back-end database;The chip verification data of different verification teams belonging to the same chip verification project is comprehensively summarized in back-end database, and comprehensive summary data is obtained;Task execution sequence is generated based on comprehensive summary data, and task execution sequence is used to indicate the optimal execution order of each test case to shorten total task execution time.The embodiment of the application can realize intelligent scheduling of verification task, shorten total verification period, improve verification efficiency, strengthen the collaborative ability between verification teams, and solve the management problem of verification data in multi-team collaborative verification.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the chip verification technical field, and in particular to a chip verification data optimization method, a chip verification data system and equipment. BACKGROUND

[0002] In modern large-scale integrated circuit design, functional verification is the core link to ensure the correctness of chip design, and its cost and cycle account for the project dominance. With the surge in chip complexity, the verification link generates massive, multi-source data. Currently, the industry generally adopts a decentralized verification data management scheme, that is, each verification team such as unit test (UT), system test (ST), hardware simulation (EMU) and FPGA prototype verification uses independent automation scripts and data storage systems.

[0003] However, this mode has limitations, and its data island problem is prominent. The data of each verification team is scattered and has different standards, making it difficult to correlate information and conduct global analysis. The intelligent analysis capability is severely insufficient, and the verification data of different verification teams cannot be effectively aggregated and utilized, limiting the further improvement of verification efficiency. SUMMARY

[0004] An object of an embodiment of the present application is to provide a chip verification data optimization device and a chip verification data system to alleviate the technical problem that the verification data of different verification teams cannot be effectively utilized, limiting the improvement of verification efficiency in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a chip verification data optimization method, comprising:

[0006] Obtaining a command parameter list of a target verification team, the command parameter list can be configured to integrate configuration parameters of test cases of other verification teams;

[0007] In response to a task command input by the target verification team, executing a target test case of the target verification team based on the command parameter list to obtain chip verification data, and saving the chip verification data in a preset backend database;

[0008] Comprehensively processing chip verification data of different verification teams belonging to the same chip verification project in the backend database to obtain comprehensive induction data, and saving the comprehensive induction data in the backend database;

[0009] Generating a task execution sequence based on the comprehensive induction data, the task execution sequence being used to represent an optimal execution order of each test case to shorten the total task execution time.

[0010] Optionally, the obtaining of the command parameter list of the target verification team comprises:

[0011] obtain a regression list file, the regression list file comprising at least one target test case, the target test case comprising a plurality of parameter tags and attribute values corresponding to the parameter tags, and a file format of the regression list file being a yaml format;

[0012] determine a parent configuration file based on the target test case, the parent configuration file forming an inheritance relationship with the regression list file, and the parent configuration file comprising an extension tag for representing a file name of a base configuration file, the extension tag being a parameter tag with an extension attribute as a tag attribute;

[0013] determine a base configuration file based on the extension tag, the base configuration file forming an inheritance relationship with the parent configuration file;

[0014] obtain attribute values of non-extension tags in the target test case, attribute values of each non-extension tag in the parent configuration file, and attribute values of each non-extension tag in the base configuration file;

[0015] generate a command parameter list based on the attribute values of the non-extension tags in the target test case, the attribute values of each non-extension tag in the parent configuration file, and the attribute values of each non-extension tag in the base configuration file.

[0016] Optionally, the determining the parent configuration file based on the target test case comprises:

[0017] determine a tag attribute of each parameter tag in the target test case in sequence;

[0018] in response to the tag attribute of the parameter tag being an extension attribute, obtain the parent configuration file based on the attribute value of the parameter tag.

[0019] Optionally, the obtaining the parent configuration file based on the attribute value of the parameter tag comprises:

[0020] perform regular matching on the attribute value of the parameter tag to obtain an absolute path of the parent configuration file;

[0021] obtain the parent configuration file based on the absolute path of the parent configuration file.

[0022] Optionally, the generating the command parameter list based on the attribute values of the non-extension tags in the target test case, the attribute values of each non-extension tag in the parent configuration file, and the attribute values of each non-extension tag in the base configuration file comprises:

[0023] determining whether a numerical type of an attribute value of a non-extension label in the target test case or an attribute value of each non-extension label in the parent configuration file or an attribute value of each non-extension label in the base configuration file is a list type;

[0024] in response to the numerical type of the attribute value being the list type, placing the non-extension label and the attribute value in a preset global list in an appending manner;

[0025] in response to the numerical type of the attribute value not being the list type, placing the non-extension label and the attribute value in the preset global list in an overwriting manner;

[0026] recognizing an attribute expression in the global list by using a regular expression, to obtain the non-extension label and the attribute value;

[0027] combining the non-extension label and the attribute value to obtain a command parameter list.

[0028] Optionally, the task execution sequence includes a predicted execution number of each target test case in a task, and the generating the task execution sequence based on the comprehensive induction data includes:

[0029] determining a coverage contribution value of the target test case based on the comprehensive induction data;

[0030] determining a target correlation weight of the target test case based on the comprehensive induction data;

[0031] inputting the coverage contribution value and the target correlation weight into a pre-trained decision model to obtain the predicted execution number of the target test case.

[0032] Optionally, the comprehensive induction data includes a use case execution database table, the use case execution database table is used to record a use case test result corresponding to a function verification feature and a feature name of the function verification feature, the target test case is configured with the feature name, and the determining the target correlation weight of the target test case based on the comprehensive induction data includes:

[0033] searching, based on the feature name, whether there is a use case test result of the target test case by another verification team in the use case execution database table;

[0034] In response to the use case execution database table not having use case test results of the other verification team on the target test case, determining a parent use case set, the parent use case set including a plurality of target test cases of different verification teams sharing a same feature name;

[0035] Optionally, the determining the target correlation weight of the target test case based on the comprehensive induction data comprises:

[0036] In response to the use case execution database table not having use case test results of the other verification team on the target test case, determining a parent use case set, the parent use case set including a plurality of target test cases of different verification teams sharing a same feature name;

[0037] In response to the use case execution database table not having use case test results of the other verification team on the target test case, determining a parent use case set, the parent use case set including a plurality of target test cases of different verification teams sharing a same feature name;

[0038] Optionally, the determining the parent use case set comprises:

[0039] Based on the feature name of the target use case, inversely tracing a parent feature name corresponding to the feature name of the target use case;

[0040] Based on the parent feature name, determining a parent use case set.

[0041] Optionally, the task execution sequence includes an execution order of each target test case in a task, and the generating the task execution sequence based on the comprehensive induction data comprises:

[0042] Based on the comprehensive induction data, obtaining recent execution time and resource consumption data of the target test case;

[0043] Inputting the recent execution time and the resource consumption data into a pre-trained decision model to obtain a predicted execution time of the target test case;

[0044] Based on a shortest processing time first algorithm, processing the predicted execution time of each target test case in a task to obtain a first execution sequence and a first total execution time, the first execution sequence being used to represent a first execution order of each target test case;

[0045] The predicted execution time of each target test case in a task is processed based on a longest processing time first algorithm to obtain a second execution sequence and a second total execution time, the second execution sequence being used to represent a second execution order of each target test case;

[0046] In response to the first total execution time being greater than the second total execution time, the second execution order is determined as the execution order of each target test case in the task execution sequence;

[0047] In response to the first total execution time being less than the second total execution time, the first execution order is determined as the execution order of each target test case in the task execution sequence.

[0048] In a second aspect, an embodiment of the present application provides a computer device, comprising a memory and a processor, the memory being connected to the processor, the processor being used to execute one or more computer programs stored in the memory, and the processor, when executing the one or more computer programs, causes the computer device to implement the chip verification data optimization method as described above.

[0049] In a third aspect, an embodiment of the present application provides a chip verification data system, comprising:

[0050] The computer device as described above;

[0051] A server cluster, which is communicatively connected to the computer device;

[0052] A backend server, which is communicatively connected to the computer device and the server cluster respectively and is configured with a backend database;

[0053] A code server, which is communicatively connected to the computer device;

[0054] An intelligent server, which is communicatively connected to the computer device and is configured with an artificial intelligence module;

[0055] A display client, which is communicatively connected to the backend server.

[0056] The embodiment of the application can achieve the following technical effects: in the chip verification data optimization method provided by the embodiment of the application, the method comprises the following steps: obtaining a command parameter list of a target verification team; in response to a task command input by the target verification team, executing a target test case of the target verification team based on the command parameter list, obtaining chip verification data, and saving the chip verification data in a preset back-end database; comprehensively processing the chip verification data of different verification teams belonging to the same chip verification project in the back-end database to obtain comprehensive induction data, and saving the comprehensive induction data in the back-end database; and generating a task execution sequence based on the comprehensive induction data, wherein the task execution sequence is used to represent an optimal execution order of each test case to shorten the total task execution time.

[0057] The embodiment of the application realizes the unification and sharing of configurations of different verification teams by fusing the command parameter list of the configuration parameters of the test cases of other verification teams, and enhances the collaboration ability of different verification teams. The task execution sequence for shortening the total task execution time is generated based on the comprehensive induction data, the task scheduling is optimized, and the convergence speed of the verification coverage is accelerated. The task waiting and resource idling are effectively reduced, the utilization efficiency of the cluster computing power is improved, the efficiency and chip verification data management problems in the multi-team collaborative verification are effectively solved, and the verification efficiency and resource utilization rate are simultaneously improved. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the application. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without any creative labor.

[0059] Figure 1 The schematic diagram of the architecture of a chip verification data system provided by the embodiment of the application;

[0060] Figure 2 The schematic diagram of the flow of a chip verification data optimization method provided by the embodiment of the application;

[0061] Figure 3 The schematic diagram of a hierarchical configuration mechanism provided by the embodiment of the application;

[0062] Figure 4 The schematic diagram of a use case execution database table provided by the embodiment of the application;

[0063] Figure 5 The schematic diagram of a use case database table provided by the embodiment of the application;

[0064] Figure 6A structural schematic diagram of a double-layer cascaded XGBoost decision model provided by an embodiment of the present application is shown in FIG. 1.

[0065] Figure 7 A structural schematic diagram of a computer device provided by an embodiment of the present application is shown in FIG. 4. DETAILED DESCRIPTION

[0066] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0067] It should be noted that the various features in the embodiments of the present application can be combined with each other without conflict, and all fall within the scope of protection of the present application. In addition, although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the order in the flowchart. Furthermore, the "first", "second", "third" and the like used in the present application do not limit the data and execution order, but only distinguish the same items or similar items with basically the same function and effect.

[0068] Below, please refer to Figure 1 , Figure 1 A structural schematic diagram of a chip verification data system provided by an embodiment of the present application is shown in FIG. 2. The chip verification data system 100 includes a code server 11, a server cluster 12, a backend server 13, a display client 14, an intelligent server 15 and a computer device 16, which are communicatively connected with each other.

[0069] The code server 11 is used to store and manage the related codes of the verification environment, including test cases, verification platforms, design codes, scripts, tool configuration files, etc. When the computer device 16 executes an initialization task command, a verification working environment can be automatically generated, which can be stored on the code server 11 after being further improved by the verification personnel, and the verification working environment can be run on the server cluster 12.

[0070] The server cluster 12 is an LSF (Load Sharing Facility) cluster, which is an LSF distributed cluster. The LSF cluster is used to distribute a large number of test cases (regression tasks) to idle servers in the cluster for execution in parallel, so as to shorten the simulation task which takes weeks to complete to a few hours or a day. The LSF cluster itself does not store the code, but accesses the location specified by the code server 11 to obtain the required files when executing the task.

[0071] The backend server 13 is configured with a preset backend database, which stores the chip verification data generated after the verification task of different verification teams EMU / FPGA / UT / ST / Formal is completed in a structured manner.

[0072] The display client 14 is used to receive and visualize the data sent by the computer device 16. The computer device 16 is configured with a data visualization module. The data visualization module analyzes the chip verification data of different verification teams stored in the backend database, and the results support multi-dimensional visualization, including the number of use cases, the total number of executions, the use case growth trend, the coverage data and its trend, etc. The relevant data is returned by the backend server 13 in the form of HTTP response, and is rendered and displayed by the display client 14, which provides effective support for project management and review.

[0073] The intelligent server 15 is configured with an artificial intelligence module. The artificial intelligence module is used to receive the input question description of the user, and gives reasonable suggestions to the verification personnel based on the historical chip verification data of all modules generated during the project development process, including use cases, test point decomposition, coverage, problem list distribution, etc., to improve work efficiency.

[0074] The computer device 16 is in communication connection with the code server 11, the server cluster 12, the backend server 13, the display client 14 and the intelligent server 15, so as to realize the execution of the chip verification task and the processing and comprehensive analysis of the chip verification data. Specifically, the computer device 16 is configured with a user interface module and a data correlation and enhancement module. The user interface module, as the source of the verification data, is used to realize the following main functions: automatic generation of test case list, configuration mechanism supporting multi-level coverage and inheritance, automatic generation of test environment and execution command interface of test case, etc., and provides a unified user interface for the corresponding functions, encapsulates the task command of different verification teams EMU / FPGA / UT / ST / Formal, facilitates the user to develop the test environment and test cases, executes the regression task, and extracts the chip verification data generated by the regression task at the end of the regression task and writes it into the backend database.

[0075] The data association and enhancement module utilizes historical verification data generated by different verification teams, extracts features from the historical verification data through a script, and trains a double-layer cascaded XGBoost decision model using the extracted features. According to the prediction results of the double-layer cascaded XGBoost decision model, the execution of different team task lists is more efficient, the cluster computing power is saved, and the verification convergence is accelerated. Specifically, the first layer XGBoost model uses coverage contribution value, association weight value and other feature values, and sets the execution frequency of test cases as the training target, so as to train the optimal execution frequency of test cases. The coverage contribution value is automatically extracted by a script, and the association weight value is calculated by associating the execution results of test cases with the same characteristics of other verification teams. The second layer XGBoost model uses test case execution duration, cluster resources, test case type and other feature values as the training target, and according to the predicted execution time of the test case and the predicted optimal execution frequency, the predicted execution time is obtained, and then the best execution order is obtained through a scheduling algorithm, so as to shorten the total execution time TAT, so as to achieve fast convergence of the verification task and improve the verification efficiency.

[0076] Please refer to Figure 2 , Figure 2 is a flowchart of a chip verification data optimization method provided by the embodiment of the present application. The chip verification data optimization method comprises the following steps:

[0077] S21: Obtain the command parameter list of the target verification team. The command parameter list can be configured to fuse the configuration parameters of the test cases of other verification teams.

[0078] In step S21, specifically, the system will first obtain the regression list file, which includes at least one target test case. The system will parse the test point decomposition file through a preset script and generate the regression list file. The test point decomposition file is a file describing the target test case written by the target verification team. The file formats of the test point decomposition files of different target verification teams may not be unified, and the file formats of the test point decomposition files include but are not limited to Excel, CSV, JSON, etc. The system will extract the configuration parameters of each target test case from the test point decomposition file through a preset script, and the configuration parameters include parameter tags and corresponding attribute values. The system will generate a regression list file according to the extracted parameter tags and corresponding attribute values, and the file format of the regression list file is yaml format. The regression list file includes a regression test case list, and the regression test case list includes all target test cases. Each target test case includes a preset parameter tag and a corresponding attribute value.

[0079] It should be noted that the system pre-configures multiple parameter tags and corresponding attribute values for test cases of different verification teams, for example, the parameter tags include team, run_times, uvm, bin, feature_id, and cfg, the attribute value corresponding to the parameter tag team is used to represent the verification team responsible for creating or mainly using the test case, and the attribute value corresponding to the parameter tag run_times is used to represent the execution times of the test case.

[0080] After obtaining the regression list file, the system sequentially determines the tag attribute of each parameter tag in the target test case, the tag attribute includes an extended attribute and a non-extended attribute, if the tag attribute of a parameter tag is an extended attribute, the attribute value corresponding to the parameter tag is subjected to regular matching to obtain the absolute path of the parent configuration file, and then the parent configuration file is obtained according to the absolute path of the parent configuration file.

[0081] It should be noted that the embodiment of the present application designs a hierarchical configuration mechanism, please refer to Figure 3 The hierarchical configuration mechanism includes a test case configuration layer, a private configuration layer, and a public configuration layer, the test case configuration layer includes multiple regression list files xxx.testlist.yaml, each regression list file includes configuration parameters of a test case, the private configuration layer includes multiple parent configuration files xxx_module_cfg.yaml, each parent configuration file includes configuration parameters of a test module, and the public configuration layer includes a basic configuration file base_cfg.yaml, the basic configuration file includes common configuration parameters of the entire project. It can be understood that in the hierarchical configuration mechanism, the parent configuration file and the regression list file form an inheritance relationship, and the basic configuration file and the parent configuration file form an inheritance relationship.

[0082] The files in the hierarchical configuration mechanism are in the yaml format, the yaml format file is the current mainstream configuration format file, and has the advantages of structure and good readability, but the yaml format file itself does not support hierarchical attribute coverage, addition, Cartesian product and other operations. The hierarchical configuration mechanism of the embodiment of the present application uses the extends extension tag and the attribute expression, and customizes the user_constructor function in yaml, realizes the functional extension of the configuration information of the yaml format file, and the extended yaml tag supports the Cartesian product operation of the parent configuration file identification and the attribute expression of the numerical value type.

[0083] After the parent configuration file is identified, the system acquires an extension tag of a file name in the parent configuration file for representing the basic configuration file, the extension tag is a parameter tag with an extension attribute, and it is to be noted that the tag attribute of the extension tag extends is the extension attribute, and the attribute value of the extension tag in the parent configuration file is the file name of the basic configuration file, which is combined with a regular expression for regular matching to obtain an absolute path of the basic configuration file, and the absolute path of the basic configuration file can be used to acquire the basic configuration file.

[0084] After the parent configuration file and the basic configuration file are identified, the system uses a recursive algorithm to cover and append the same parameter tags according to the numerical value types of the attribute values corresponding to the parameter tags. Specifically, the system first acquires the attribute values of non-extension tags in the target test case, the attribute values of the non-extension tags in the parent configuration file, and the attribute values of the non-extension tags in the basic configuration file, the non-extension tags are parameter tags with non-extension attributes, i.e., the parameter tags other than the extends tag.

[0085] The system then determines whether the numerical value types of the attribute values of the non-extension tags in the target test case or the attribute values of the non-extension tags in the parent configuration file or the attribute values of the non-extension tags in the basic configuration file are list types, if the attribute values are list types, the non-extension tags and the attribute values are placed in a preset global list in an appending manner. If the numerical value types of the attribute values are not list types, the non-extension tags and the attribute values are placed in the preset global list in a covering manner.

[0086] The system uses a regular expression to identify the attribute expressions in the global list to obtain the non-extension tags and the attribute values, specifically, the system traverses the global list, uses a regular expression to match and identify attribute expressions with && symbols, combines the attribute values of the same non-extension tags, and obtains all non-extension tags and attribute values. All non-extension tags and attribute values are combined to obtain a command parameter list of all target test cases. The command parameter list is used to instruct the target verification team to execute the target test case.

[0087] S22: In response to a task command input by the target verification team, the target test case of the target verification team is executed based on the command parameter list to obtain chip verification data, and the chip verification data is saved in a preset backend database.

[0088] In step S22, specifically, the embodiment of the present application provides a set of command set ADK (Aiding Development Kit, auxiliary development package) for the verification personnel of different verification teams to use on the Linux Terminal. The command set ADK includes task commands submitted to the server cluster by different verification teams such as UT / ST / EMU / FPGA / FORMAL. Different task commands have different command functions. For example, refer to Table 1.

[0089] Table 1

[0090]

[0091] The target verification team executes the target test case according to the command parameter list and the corresponding task command in the command set. After the target test case of the target verification team is executed, the chip verification data generated is saved in the back-end database in a specified format.

[0092] For example, the back-end database generates a test case execution database table. Please refer to Figure 4 The test case execution database table includes test_id, moudule_id, sim_type, data, updated and other parameter information. Among them, test_id is used to identify the execution number of the test case, moudule_id represents the module number of the test case, sim_type represents the verification team that executes the test case, data represents the execution information of the test case, which is in json format, wherein status represents the execution state of the test case, and feature_id represents the verification feature of the test case; updated represents the latest update time of the chip verification data.

[0093] S23: Comprehensive induction processing of chip verification data of different verification teams belonging to the same chip verification project in the back-end database to obtain comprehensive induction data, and saving the comprehensive induction data in the back-end database.

[0094] In step S23, specifically, the system includes a data correlation and enhancement module. The data correlation and enhancement module can be used to comprehensively analyze the chip verification data generated by different verification teams to improve the collaboration ability of different verification teams and solve the problem of low mutual utilization rate of chip verification data of different verification teams. The system uses the data correlation and enhancement module to comprehensively induce the chip verification data of different verification teams belonging to the same chip verification project, obtains comprehensive induction data, and saves the comprehensive induction data in the back-end database in a specified format.

[0095] For example, the back-end database generates a test case database table. Please refer to Figure 5The use case database table includes test_id, sim_type, data, updated, and the like parameter information. The test_id is used to identify the execution number of the test case, the sim_type is the verification team that executes the test case, the data indicates the inductive analysis information of the test case, and includes the associated feature name feature_id, the execution number exec_num, the coverage contribution cov_weight, the defect weight bug_weight, and the like.

[0096] S24: generating a task execution sequence based on the comprehensive inductive data, the task execution sequence being used to represent the optimal execution order of each test case to shorten the total task execution time.

[0097] In step S24, specifically, the task execution sequence includes the predicted execution number of each target test case in a regression task. The test cases in a regression task will be executed for different execution numbers. Specifically, the command parameter list includes a parameter tag run_times, and the attribute value corresponding to the parameter tag run_times is used to indicate the execution number of the target test case.

[0098] It should be noted that the target test cases of different verification teams have randomness, and the execution number of one target test case is often different. The verification personnel are not clear about the optimal execution number of each target test case. The inventors find that the execution number of the test case is related to the coverage contribution and the probability of discovering defects. In the embodiments of the present application, the coverage contribution is obtained by searching the parameters of the use case execution database table in the back-end database, and the probability value of discovering defects is obtained by searching the execution state of the target test case of different verification teams in the back-end database. Specifically, the system associates the test cases with the same feature name feature_id or the same parent feature name feature_id in the command parameter list. When a test case in one verification team discovers a defect, the associated weight value of the test case including the same feature name feature_id or the parent feature name feature_id in other verification teams is increased. The associated weight value is used to represent the probability of discovering defects by the test case.

[0099] In addition, a regression task includes a large number of different target test cases, and different target test cases have different execution sequences when executed, and different execution sequences produce different total execution times. Generally, the verification personnel are also not clear about the best execution sequence of all target test cases, and in order to obtain the best execution times and the best execution sequence of all target test cases, an embodiment of the present application designs a double-layer cascaded XGBoost decision model, which can predict the best execution times and the best execution sequence of the target test cases in the command parameter list, and shorten the total execution time of the command parameter list in the project.

[0100] Referring to Figure 6 , Figure 6 is an architecture diagram of a double-layer cascaded XGBoost decision model provided by an embodiment of the present application. The double-layer cascaded XGBoost decision model includes two layers of XGBoost models connected in series. The first layer of XGBoost model extracts the coverage contribution features and the associated weight features of the target test cases in different verification teams UT / BT / ST / Formal / EMU / FPGA and other task features by analyzing the comprehensive induction data in the back-end database, and predicts the best execution times of the target test cases based on the coverage contribution features and the associated weight features. In the first layer of XGBoost model, the hyperparameter configuration related to the coverage contribution features is: params_cov = { 'objective':'reg:squarederror', 'learning_rate': 0.1,'max_depth': 6, 'eval_metric': 'rmse'}. The hyperparameter configuration related to the associated weight features is: params_weight = { 'objective':'reg:squarederror', 'learning_rate': 0.3,'max_depth': 3, 'eval_metric': 'rmse'}.

[0101] The second layer XGBoost model obtains the duration of the latest execution of the target test case exec_time, CPU resources and memory resources and the like by analyzing the comprehensive summarized data in the backend database, and trains the second layer XGBoost model parameters in combination with the current resource utilization of the LSF cluster. Then, the duration of the target test case in the best execution number is predicted according to the first layer XGBoost model. The hyperparameter configuration of the second layer XGBoost model is as follows: params = { 'objective':'reg:squarederror', 'learning_rate': 0.1,'max_depth': 8,'subsample': 0.8, 'colsample_bytree': 0.8, 'gamma': 0.1,'reg_alpha': 0.5,'reg_lambda': 1.0, 'n_estimators': 1000, 'early_stopping_rounds': 50, 'eval_metric': 'rmse'}.

[0102] After the system obtains the execution time of each target test case, a first execution order and a first total execution time of all target test cases in a regression task are obtained through an SPT (Shortest Processing Time) algorithm, and a second execution order and a second total execution time of all target test cases in the regression task are obtained through an LPT (Longest Processing Time) algorithm. If the first total execution time is greater than the second total execution time, the second execution order is determined as the execution order of each target test case in the task execution sequence. If the first total execution time is less than the second total execution time, the first execution order is determined as the execution order of each target test case in the task execution sequence.

[0103] The embodiment of the application realizes the unification and sharing of configurations of different verification teams by fusing the command parameter lists of the configuration parameters of the test cases of other verification teams, and enhances the collaboration ability of different teams. The task execution sequence for shortening the total task execution time is generated according to the comprehensive summarized data, the task scheduling is optimized, and the convergence speed of the verification coverage is accelerated. The task waiting and resource idling are effectively reduced, the utilization efficiency of the cluster computing power is improved, the efficiency and chip verification data management problems in the multi-team collaborative verification are effectively solved, and the verification efficiency and resource utilization are simultaneously improved.

[0104] In some embodiments, the command parameter list of the target verification team is acquired, including steps S211 to S215, as shown below:

[0105] S211: A regression list file is acquired, the regression list file including at least one target test case, the target test case including a plurality of parameter tags and attribute values corresponding to the parameter tags, and the file format of the regression list file being a yaml format.

[0106] In step S211, specifically, the system parses the test point decomposition file through a preset script, and generates a regression list file. The test point decomposition file is a file describing the target test case written by the target verification team, and the file formats of the test point decomposition files of different target verification teams may not be unified, including but not limited to Excel, CSV, JSON, etc. The system extracts the configuration parameters of each target test case from the test point decomposition file through a preset script, including parameter tags and corresponding attribute values. The system generates and acquires a regression list file according to the extracted parameter tags and corresponding attribute values, and the file format of the regression list file is a yaml format. The regression list file includes a regression case list, and the regression case list includes all target test cases, each target test case including a preset parameter tag and a corresponding attribute value.

[0107] S212: A parent configuration file is determined based on the target test case, the parent configuration file forming an inheritance relationship with the regression list file, and the parent configuration file including an extension tag for representing a file name of a basic configuration file. The extension tag is a parameter tag with an extension attribute as a tag attribute.

[0108] In step S212, specifically, the extension tag is a parameter tag with an extension attribute as a tag attribute. It should be noted that the tag attribute of the extension tag extends is an extension attribute. After acquiring the regression list file, the system sequentially determines the tag attribute of each parameter tag in the target test case, including an extension attribute and a non-extension attribute. If the tag attribute of the parameter tag is an extension attribute, the attribute value corresponding to the parameter tag is matched by regular expression to obtain the absolute path of the parent configuration file, and then the parent configuration file is acquired according to the absolute path of the parent configuration file.

[0109] S213: A basic configuration file is determined based on the extension tag, the basic configuration file forming an inheritance relationship with the parent configuration file.

[0110] In step S213, after the parent configuration file is identified, the system acquires the extension label of the file name for representing the basic configuration file in the parent configuration file, and the attribute value of the extension label is the file name of the basic configuration file. Regularization matching is performed on the extension label combined with the regular expression to obtain the absolute path of the basic configuration file. Then, the absolute path of the basic configuration file is used to acquire the basic configuration file.

[0111] S214: Acquire the attribute values of the non-extension labels in the target test case, the attribute values of the non-extension labels in the parent configuration file, and the attribute values of the non-extension labels in the basic configuration file.

[0112] In step S214, the system can acquire the attribute values of the non-extension labels in each target test case. Then, the system reversely traces the parent configuration file and the basic configuration file according to the extension label extends, and acquires the attribute values of the non-extension labels in the parent configuration file and the attribute values of the non-extension labels in the basic configuration file.

[0113] S215: Generate a command parameter list based on the attribute values of the non-extension labels in the target test case, the attribute values of the non-extension labels in the parent configuration file, and the attribute values of the non-extension labels in the basic configuration file.

[0114] In step S215, the system further judges whether the numerical type of the attribute values of the non-extension labels in the target test case, the attribute values of the non-extension labels in the parent configuration file, or the attribute values of the non-extension labels in the basic configuration file is a list type.

[0115] If the attribute values are of the list type, the non-extension label and the attribute values are placed in a preset global list in an appending manner. Specifically, the preset global list includes the non-extension labels of each target test case and the corresponding attribute values. If the attribute values of the non-extension labels in the parent configuration file and the non-extension labels in the target test case are of the list type, the attribute values of the non-extension labels in the parent configuration file are appended to the attribute values of the corresponding non-extension labels in the global list through the “&&” symbol to obtain the corresponding attribute expression. The appending manner of the basic configuration file is the same, and thus is not described here.

[0116] If the numerical type of the attribute value is not a list type, the non-extended label and its attribute value are placed in the preset global list in an overriding manner. Specifically, it can be understood that the basic configuration file forms an inheritance relationship with the parent configuration file, and the parent configuration file forms an inheritance relationship with the regression list file, and the regression list file includes the target test case. If the attribute value of the non-extended label in the parent configuration file is not a list type, the attribute value of the non-extended label in the parent configuration file is overwritten. If the attribute value of the non-extended label in the target test case is not a list type, the attribute value of the non-extended label in the target test case is overwritten.

[0117] The system uses regular expressions to identify attribute expressions in the global list to obtain non-extended labels and their attribute values. Specifically, the system traverses the global list and uses regular expressions to match and identify attribute expressions that also contain "&&" symbols. The attribute values of the same non-extended label are combined to obtain all non-extended labels and their attribute values. All non-extended labels and their attribute values are combined to obtain a command parameter list of all target test cases. The command parameter list is used to instruct the target verification team to execute the target test case.

[0118] In some embodiments, the task execution sequence includes the predicted execution times of each target test case in a task, and the task execution sequence is generated based on the comprehensive induction data, including the following steps:

[0119] S251: determining the coverage contribution value of the target test case based on the comprehensive induction data;

[0120] S252: determining the target association weight of the target test case based on the comprehensive induction data;

[0121] S253: inputting the coverage contribution value and the target association weight into a pre-trained decision model to obtain the predicted execution times of the target test case.

[0122] In step S251, the comprehensive induction data includes a use case database table. Specifically, the backend database includes a use case database table, and the use case database table records the coverage contribution value of the target test case. The coverage contribution value is a quantitative indicator that measures the size of the target test case's contribution to the overall test coverage target. The system can directly obtain the coverage contribution value of the target test case from different verification teams.

[0123] In step S252, the comprehensive induction data includes a use case execution database table, and the use case execution database table is used to record use case test results corresponding to the functional verification feature and the feature name of the functional verification feature. The target test case is configured with the feature name, and the use case execution database table can be referred to continuously Figure 4 . The system searches whether there is a use case test result of the target test case of the other verification team in the use case execution database table based on the feature name. If the use case execution database table has the use case test result of the target test case of the other verification team, and the use case test result of the target test case of the other verification team is a failure result FAIL, the associated weight of the target test case of the other verification team is increased, and the target associated weight of the target test case of the target verification team is obtained. The target associated weight is proportional to the execution times of the target test case.

[0124] It should be noted that the associated weight is used to represent the probability of discovering defects of the test case. The probability of discovering defects of the test case can be obtained through the use case database table in the back-end database, and the use case database table can be referred to continuously Figure 5 ,

[0125] The defect weight bug_weight is used to represent the probability of discovering defects of the test case. In step S252, if the use case execution database table has the use case test result of the target test case of the other verification team, and the use case test result of the target test case of the other verification team is a failure result FAIL, the defect weight bug_weight of the target test case of the other verification team is obtained, and the target associated weight of the target test case of the different verification teams is obtained by multiplying the correlation coefficient. The correlation coefficient is a value greater than 1, and the correlation coefficient can be set to 1.5.

[0126] In step S253, specifically, the system inputs the coverage contribution value and the associated weight value of the target test case of the different verification teams into the pre-trained decision model. The pre-trained decision model is a double-layer cascaded XGBoost decision model, and the double-layer cascaded XGBoost decision model includes a first layer XGBoost model. The first layer XGBoost model can predict the best execution times of the target test case, i.e., the predicted execution times, according to the input coverage contribution value and the associated weight value of the target test case of the different verification teams.

[0127] In some embodiments, determining the target associated weight of the target test case based on the comprehensive induction data further includes the following steps:

[0128] S31: In response to the use case execution database table not having the use case test result of the target test case of the other verification team, a parent use case set is determined, and the parent use case set includes a plurality of target test cases of the different verification teams sharing the same feature name.

[0129] S32: In response to the case test result of one target test case in the parent case set being a failure result, increasing the association weight of the target test case corresponding to the failure result to obtain a target association weight of the target test case for the target verification team.

[0130] In step S31, if the case execution database table does not exist the case test result of the target test case for other verification teams, the parent feature name corresponding to the feature name of the target test case is traced back based on the feature name of the target test case. The parent case set is determined based on the parent feature name. The parent case set includes a plurality of target test cases sharing the same feature name by different verification teams. For example, if the feature name of the target test case is FS001.1, the parent feature name is FS001, and the parent case set includes test cases with the feature name prefixed by the parent feature name FS001, such as FS001.1, FS001.2, FS001.3... FS001.n, etc.

[0131] In step S32, if the case test result of one target test case in the parent case set is a failure result, the association weight of the target test case corresponding to the failure result is increased to obtain a target association weight of the target test case for the target verification team.

[0132] In some embodiments, the task execution sequence includes the execution order of each target test case in a task, and the task execution sequence is generated based on the comprehensive induction data, further comprising steps S254 to S259, which are specifically as follows:

[0133] S254: Obtain the latest execution time and resource consumption data of the target test case based on the comprehensive induction data.

[0134] S255: Input the latest execution time and resource consumption data into the pre-trained decision model to obtain the predicted execution time of the target test case.

[0135] S256: Process the predicted execution time of each target test case in a task based on the shortest processing time first algorithm to obtain a first execution sequence and a first total execution time, the first execution sequence being used to represent a first execution order of each target test case.

[0136] S257: Process the predicted execution time of each target test case in a task based on the longest processing time first algorithm to obtain a second execution sequence and a second total execution time, the second execution sequence being used to represent a second execution order of each target test case.

[0137] S258: In response to the first total execution time being greater than the second total execution time, determining the second execution order as the execution order of each target test case in the task execution sequence.

[0138] S259: In response to the first total execution time being less than the second total execution time, determining the first execution order as the execution order of each target test case in the task execution sequence.

[0139] In step S254, the recent execution time is the duration of the target test case in the last execution, exec_time. The resource consumption data includes cpu resource and memory resource. Specifically, the system can obtain the duration of the target test case in the last execution, exec_time, and the cpu resource and memory resource from the backend database.

[0140] In step S255, the double-layer cascaded XGBoost decision model includes a second layer XGBoost model, which obtains the duration of the target test case in the last execution, exec_time, the cpu resource and memory resource, etc. by analyzing the comprehensive induction data in the backend database, and combines the current resource utilization of the LSF cluster, and then obtains the best execution number of the target test case according to the first layer XGBoost model, to predict the duration of the target test case in the best execution number, i.e. the predicted execution time of the target test case.

[0141] In step S256, the task execution sequence includes the execution order of each target test case in a task. Specifically, after obtaining the predicted execution time of each target test case, the system processes the predicted execution time of each target test case in a task by SPT (Shortest Processing Time) algorithm to obtain the first execution sequence of all target test cases in a regression task and the first total execution time, and the first execution sequence is used to represent the first execution order of each target test case.

[0142] In step S257, specifically, the system processes the predicted execution time of each target test case in a task by LPT (Longest Processing Time) algorithm to obtain the second execution sequence and the second total execution time, and the second execution sequence is used to represent the second execution order of each target test case.

[0143] In step S258, if the first total execution time is greater than the second total execution time, the second execution order is determined as the execution order of each target test case in the task execution sequence.

[0144] In step S259, if the first total execution time is less than the second total execution time, it is determined that the first execution order is the execution order of each target test case in the task execution sequence.

[0145] In some embodiments, the system is also configured with a data visualization module for comprehensive and inductive analysis of chip verification data of different verification teams, including quantity statistics and change trend analysis. The data visualization module is implemented using a front-end and back-end separated network architecture. The front-end server is developed based on Vue3 and TypeScript, used to obtain comprehensive and inductive data processed by data correlation and enhancement module analysis from the back-end database, and present the analysis results on the front-end page for reference by verification personnel and project management personnel and adjust strategies. The back-end is developed based on the Flask framework, responsible for extracting chip verification data from the back-end database, comprehensive and intelligent analysis, and writing the analysis results back to the back-end database to support the visual display of the front-end page. In the process of project advancement, managers can intuitively understand the verification progress of each module. If it is found that the module progress is slow, the cause can be found out in time to adjust the strategy or increase manpower to ensure that each module is carried out according to the plan.

[0146] The embodiments of the present application can assist in overall understanding of project progress, identifying potential risks in the verification process, and replacing manual data confirmation at the end of verification, thereby improving project management efficiency through comprehensive analysis and centralized visualization of multi-source data.

[0147] It should be noted that in the above various embodiments, the above steps do not necessarily have a certain sequence, and those skilled in the art can understand from the description of the embodiments of the present application that the above steps can have different execution sequences in different embodiments, that is, they can be executed in parallel, or they can be exchanged and executed, etc.

[0148] Referring to Figure 7 , Figure 7 is a structural schematic diagram of a computer device 16 provided by an embodiment of the present application. The computer device 16 includes one or more processors 71 and a memory 72. The memory 72 is connected to the one or more processors 71, for example, connected to the processor 71 through a bus.

[0149] The processor 71 is configured to support the computer device to perform the corresponding functions in the methods in the above method embodiments. The processor 71 can be a central processing unit (CPU), a network processor (NP), a hardware chip or any combination thereof. The hardware chip can be an application specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0150] The memory 72 is configured to store program codes and the like. The memory 72 can include a volatile memory (VM), such as a random access memory (RAM); the memory can also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory can also include a combination of the above-mentioned memories.

[0151] The memory 72 can be configured to store non-volatile software programs, non-volatile computer-executable programs and modules, such as program instructions / modules corresponding to the chip verification data optimization method in the embodiments of the present application. The processor performs the chip verification data optimization method by running the non-volatile software programs, instructions and modules stored in the memory, i.e. implements the chip verification data optimization method provided in the above method embodiments.

[0152] The memory 72 can include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function.

[0153] The one or more modules are stored in the memory 72 and, when executed by the one or more processors 71, perform the chip verification data optimization method in any of the above method embodiments, for example, perform the method steps described in the above method embodiments.

[0154] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, the computer program comprises program instructions, and the program instructions enable a computer to execute the method according to the foregoing embodiment when the computer executes the program instructions.

[0155] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the program can be stored in a computer readable storage medium. When the program is executed, the program can include the processes of the above-mentioned embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM) and the like.

[0156] The above only describes the preferred embodiments of the present application, and of course cannot limit the scope of the rights of the present application, so the equivalent changes made according to the claims of the present application still fall within the scope of the present application.

Claims

1. A method for optimizing chip verification data, characterized in that, include: Obtain the command parameter list of the target verification team, which integrates the configuration parameters of test cases from other verification teams; In response to the task command input by the target verification team, the target test cases of the target verification team are executed based on the command parameter list to obtain chip verification data, and the chip verification data is stored in a preset backend database. The chip verification data of different verification teams belonging to the same chip verification project are comprehensively summarized in the backend database to obtain comprehensive summary data, which is then stored in the backend database. Based on the comprehensive inductive data, a task execution sequence is generated. This sequence represents the optimal execution order of each test case to shorten the total task execution time. The task execution sequence includes the predicted execution count of each target test case within a task. Generating the task execution sequence based on the comprehensive inductive data includes: Based on the comprehensive inductive data, the coverage contribution value of the target test case is determined; based on the comprehensive inductive data, the target association weight of the target test case is determined, and the association weight is used to characterize the probability of the test case discovering defects; the coverage contribution value and the target association weight are input into a pre-trained decision model to obtain the predicted number of executions of the target test case; The task execution sequence includes the execution order of each target test case in a task. Generating the task execution sequence based on the comprehensive inductive data includes: Based on the comprehensive inductive data, the most recent execution time and resource consumption data of the target test cases are obtained; the most recent execution time and resource consumption data are input into a pre-trained decision model to obtain the predicted execution time of the target test cases; the predicted execution time of each target test case in a task is processed based on the shortest processing time first algorithm to obtain a first execution sequence and a first total execution time, wherein the first execution sequence is used to represent a first execution order of each target test case; the predicted execution time of each target test case in a task is processed based on the longest processing time first algorithm to obtain a second execution sequence and a second total execution time, wherein the second execution sequence is used to represent a second execution order of each target test case; in response to the first total execution time being greater than the second total execution time, the second execution order is determined as the execution order of each target test case in the task execution sequence; in response to the first total execution time being less than the second total execution time, the first execution order is determined as the execution order of each target test case in the task execution sequence.

2. The optimization method according to claim 1, characterized in that, The command parameter list for obtaining the target verification team includes: Obtain a regression list file, which includes at least one target test case. The target test case includes multiple parameter labels and attribute values ​​corresponding to the parameter labels. The regression list file is in YAML format. The parent configuration file is determined based on the target test cases. The parent configuration file and the regression list file form an inheritance relationship. The parent configuration file includes an extended tag that represents the file name of the base configuration file. The extended tag is a parameter tag with the tag attribute being an extended attribute. The basic configuration file is determined based on the extended tags, and the basic configuration file and the parent configuration file form an inheritance relationship. Obtain the attribute values ​​of non-extended tags in the target test case, the attribute values ​​of each non-extended tag in the parent configuration file, and the attribute values ​​of each non-extended tag in the base configuration file; A list of command parameters is generated based on the attribute values ​​of the non-extended tags in the target test case, the attribute values ​​of each non-extended tag in the parent configuration file, and the attribute values ​​of each non-extended tag in the base configuration file.

3. The optimization method according to claim 2, characterized in that, The process of determining the parent configuration file based on the target test case includes: The tag attributes of each parameter tag in the target test case are determined sequentially; In response to the parameter tag's tag attribute being an extended attribute, the parent configuration file is obtained based on the attribute value of the parameter tag.

4. The optimization method according to claim 3, characterized in that, The step of obtaining the parent configuration file based on the attribute value of the parameter tag includes: The attribute values ​​of the parameter tags are matched using regularization to obtain the absolute path of the parent configuration file; The parent configuration file is obtained based on its absolute path.

5. The optimization method according to claim 2, characterized in that, The command parameter list is generated based on the attribute values ​​of the non-extended tags in the target test case, the attribute values ​​of each non-extended tag in the parent configuration file, and the attribute values ​​of each non-extended tag in the base configuration file, including: Determine whether the numeric type of the attribute value of the non-extended tag in the target test case, or the attribute value of each non-extended tag in the parent configuration file, or the attribute value of each non-extended tag in the base configuration file is a list type; In response to the fact that the numeric type of the attribute value is a list type, the non-expanded tag and its attribute value are appended to a preset global list; In response to the fact that the numeric type of the attribute value is not a list type, the non-expanded tag and its attribute value are placed in a preset global list in an overriding manner; The attribute expressions in the global list are identified using regular expressions to obtain the non-extended tags and their attribute values; Combine all the non-extended tags and their attribute values ​​to obtain a list of command parameters.

6. The optimization method according to claim 1, characterized in that, The comprehensive inductive data includes a test case execution database table, which records the test results of test cases corresponding to functional verification features and the feature names of the functional verification features. The target test cases are configured with feature names. Determining the target association weight of the target test cases based on the comprehensive inductive data includes: Based on the feature name, search the test case execution database table to see if there are test results from other verification teams for the target test case; In response to the existence of test results from other verification teams for the target test case in the test case execution database table, and the test results from other verification teams for the target test case are failures, the association weight of the other verification teams for the target test case is increased to obtain the target association weight of the target verification team for the target test case. The target association weight is proportional to the number of times the target test case is executed.

7. The optimization method according to claim 6, characterized in that, The step of determining the target association weight of the target test case based on the comprehensive inductive data further includes: In response to the absence of test results from other verification teams for the target test case in the test case execution database table, a parent test case set is determined, which includes multiple target test cases with the same feature name shared by different verification teams. In response to the existence of a target test case in the parent test case set where the test result is a failure, the association weight of the target test case corresponding to the failure result is increased, thereby obtaining the target association weight of the target verification team for the target test case.

8. The optimization method according to claim 7, characterized in that, The determination of the parent use case set includes: Based on the feature name of the target test case, trace back to the parent feature name corresponding to the feature name of the target test case; The parent use case set is determined based on the parent feature name.

9. A computer device, characterized in that, The device includes a memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, the processor causing the computer device to implement the chip verification data optimization method as described in any one of claims 1-8 when executing the one or more computer programs.

10. A chip verification data system, characterized in that, include: The computer device as described in claim 9; A server cluster is communicatively connected to the computer equipment. The backend server is communicatively connected to both the computer device and the server cluster, and is configured with a backend database. A code server, which is communicatively connected to the computer device; The intelligent server, which is communicatively connected to the computer device, is equipped with an artificial intelligence module; The client displays the communication connection with the backend server.

Images