Verification strategy determination method, device, electronic device and storage medium

Abstract

The present application provides a method, device, electronic device and storage medium for determining a verification strategy, wherein the method includes: obtaining a candidate verification strategy, wherein the candidate verification strategy includes each feature to be verified of the simulated code to be verified; running the simulated code to be verified based on the candidate verification strategy, verifying each feature to be verified in the candidate verification strategy, and generating a verification coverage report; reverse-marking the candidate verification strategy according to the verification coverage report to obtain the verified coverage rate of each feature to be verified; generating a target verification strategy based on the verified coverage rate of each feature to be verified. In the present application, the verification coverage report is reverse-marked to the candidate verification strategy, so that the verification result is presented more intuitively, effectively realizing the optimization and adjustment of the candidate verification strategy, improving the adjustment efficiency of the candidate verification strategy, and optimizing the quality of the target verification strategy.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, electronic device, and storage medium for determining a verification strategy. Background Art

[0002] With the development of chip technology, the design of chip bus is becoming more and more complex. In implementation, the design of chip bus plays an important role in the realization of chip functions. Therefore, the determination of verification strategy has a certain degree of influence on the realization of chip functions.

[0003] In the related art, the determination cycle of chip verification strategy is long and the efficiency is low. Therefore, how to improve the efficiency of bus verification is a problem that needs to be solved at present. Summary of the Invention

[0004] The purpose of this application is to solve one of the technical problems in the above technology at least to a certain extent.

[0005] The first aspect of the present application provides a method for determining a verification strategy, including: obtaining a candidate verification strategy, wherein the candidate verification strategy includes each feature to be verified in a simulated code to be verified; running the simulated code to be verified based on the candidate verification strategy, verifying each feature to be verified in the candidate verification strategy, and generating a verification coverage report; reverse-marking the candidate verification strategy according to the verification coverage report to obtain the verified coverage rate of each feature to be verified; and generating a target verification strategy based on the verified coverage rate of each feature to be verified.

[0006] The first aspect of the present application provides a method for determining a verification strategy, which also has the following technical features, including:

[0007] According to an embodiment of the present application, the candidate verification strategy is reverse-marked according to the verification coverage report to obtain the verified coverage rate of each of the characteristics to be verified, including: obtaining the first identification information of the verified performance and the corresponding verified performance under each of the characteristics to be verified from the verification coverage report; obtaining the second identification information of all the performance to be verified under each of the characteristics to be verified from the candidate verification strategy; matching the first identification information with the second identification information, and reverse-marking the verified performance to each of the characteristics to be verified; obtaining the unverified performance under each of the characteristics to be verified, and generating the verified coverage rate of each of the characteristics to be verified based on the verified performance and the unverified performance.

[0008] According to an embodiment of the present application, the target verification strategy is generated based on the verified coverage of each characteristic to be verified, including: determining whether to adjust the candidate verification strategy based on the verified coverage of each characteristic to be verified; in response to determining that no adjustment is required, determining the candidate verification strategy as the target verification strategy; in response to determining that adjustment is required, adjusting the candidate verification strategy according to the unverified performance of each characteristic to be verified, and when the performance to be verified included in each characteristic to be verified in the adjusted candidate verification strategy is verified, the target verification strategy is generated.

[0009] According to an embodiment of the present application, the determination of whether to adjust the candidate verification strategy based on the verified coverage of each of the characteristics to be verified includes: in response to the fact that the performance to be verified under each of the characteristics to be verified is verified, determining that the candidate verification strategy does not need to be adjusted; in response to the fact that there is any performance to be verified under each of the characteristics to be verified that has not been verified, determining that the candidate verification strategy needs to be adjusted.

[0010] According to an embodiment of the present application, obtaining the candidate verification strategy includes: constructing a test case and obtaining each of the characteristics to be verified of the simulated code to be verified under the environment of the test case; and generating the candidate verification strategy based on each of the characteristics to be verified.

[0011] According to an embodiment of the present application, the simulated code to be verified is run based on the candidate verification strategy, each of the characteristics to be verified in the candidate verification strategy is verified, and a verification coverage report is generated, including: based on each characteristic to be verified in the candidate verification strategy, the simulated code to be verified is run, and the performance under each characteristic to be verified is verified according to the running result to obtain the verified performance under each characteristic to be verified; based on the verified performance under each characteristic to be verified and the verification result of the verified performance, the verification coverage report of the candidate verification strategy is generated.

[0012] The second aspect of the present application provides a device for determining a verification strategy, including: an acquisition module for acquiring a candidate verification strategy, wherein the candidate verification strategy includes each feature to be verified in the simulated code to be verified; a verification module for running the simulated code to be verified based on the candidate verification strategy, verifying each feature to be verified in the candidate verification strategy, and generating a verification coverage report; a marking module for reverse marking the candidate verification strategy according to the verification coverage report to obtain the verified coverage rate of each feature to be verified; and a generation module for generating a target verification strategy based on the verified coverage rate of each feature to be verified.

[0013] The second aspect of the present application provides a device for determining a verification strategy, which further has the following technical features, including:

[0014] According to one embodiment of the present application, the marking module is also used to: obtain the first identification information of the verified performance and the corresponding verified performance under each of the characteristics to be verified from the verification coverage report; obtain the second identification information of all the performance to be verified under each of the characteristics to be verified from the candidate verification strategy; match the first identification information with the second identification information, and reversely mark the verified performance to each of the characteristics to be verified; obtain the unverified performance under each of the characteristics to be verified, and generate the verified coverage rate of each of the characteristics to be verified based on the verified performance and the unverified performance.

[0015] According to one embodiment of the present application, the generation module is further used to: determine whether to adjust the candidate verification strategy based on the verified coverage of each characteristic to be verified; in response to determining that no adjustment is required, determine the candidate verification strategy as the target verification strategy; in response to determining that adjustment is required, adjust the candidate verification strategy according to the unverified performance of each characteristic to be verified, and when the performance to be verified included in each characteristic to be verified in the adjusted candidate verification strategy is verified, generate the target verification strategy.

[0016] According to one embodiment of the present application, the generation module is further used to: in response to the fact that the performance to be verified under each of the characteristics to be verified is verified, determine that the candidate verification strategy does not need to be adjusted; in response to the fact that the performance to be verified under each of the characteristics to be verified is not verified, determine that the candidate verification strategy needs to be adjusted.

[0017] According to an embodiment of the present application, the acquisition module is further used to: construct a test case and obtain each of the characteristics to be verified of the simulated code to be verified under the environment of the test case; and generate the candidate verification strategy based on each of the characteristics to be verified.

[0018] According to one embodiment of the present application, the verification module is further used to: run the simulation code to be verified based on each of the characteristics to be verified in the candidate verification strategy, and verify the performance under each of the characteristics to be verified based on the running results to obtain the verified performance under each of the characteristics to be verified; generate the verification coverage report of the candidate verification strategy based on the verified performance under each of the characteristics to be verified and the verification results of the verified performance.

[0019] An embodiment of the third aspect of the present application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for determining the verification strategy provided in the first aspect of the present application.

[0020] The fourth aspect of the present application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method for determining the verification strategy provided in the first aspect of the present application.

[0021] The fifth aspect of the present application provides a computer program product. When the instruction processor in the computer program product is executed, the method for determining the verification strategy provided in the first aspect of the present application is executed.

[0022] The method and device for determining the verification strategy provided in the present application run a simulation of the code to be verified based on the candidate verification strategy to obtain a verification coverage report of the candidate verification strategy. The verification coverage report is reverse-marked into the candidate verification strategy, and then the verified coverage corresponding to each feature to be verified in the candidate verification strategy is obtained, and the candidate verification strategy is adjusted according to the verified coverage until the set standard is met, thereby generating a target verification strategy. In the present application, by reverse-marking the verification coverage report into the candidate verification strategy, the verification results of the candidate verification strategy can be presented more intuitively, thereby more effectively realizing the optimization and adjustment of the candidate verification strategy, effectively improving the adjustment efficiency of the candidate verification strategy, and optimizing the quality of the target verification strategy.

[0023] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0025] Figure 1 A flowchart of a method for determining a verification strategy according to an embodiment of the present application is shown;

[0026] Figure 2 A flowchart of a method for determining a verification strategy according to another embodiment of the present application is shown;

[0027] Figure 3 A flowchart of a method for determining a verification strategy according to another embodiment of the present application is shown;

[0028] Figure 4A flowchart of a method for determining a verification strategy according to another embodiment of the present application is shown;

[0029] Figure 5 A flowchart of a method for determining a verification strategy according to another embodiment of the present application is shown;

[0030] Figure 6 This is a schematic diagram of the structure of a device for determining a verification strategy according to an embodiment of the present application;

[0031] Figure 7 This is a schematic structural diagram of a device for determining a verification strategy according to another embodiment of the present application;

[0032] Figure 8 It is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0034] The following describes the verification strategy determination method, device, electronic device, and storage medium according to embodiments of the present application with reference to the accompanying drawings.

[0035] Figure 1 This is a flow chart of a method for determining a verification strategy according to an embodiment of the present application. Figure 1 As shown, the method includes:

[0036] S101 , obtaining a candidate verification strategy, wherein the candidate verification strategy includes simulating each feature to be verified of the code to be verified.

[0037] During implementation, in order to realize the performance verification of the chip in a more systematic and complete manner, before entering the verification phase, it is necessary to formulate corresponding verification strategies for the chip's characteristics to be verified.

[0038] To ensure a more systematic and complete verification of the chip's performance under verification, a corresponding verification environment, such as the Universal Verification Methodology (UVM) verification environment, can be established for the chip. Based on the established verification environment and the features that the chip can achieve under that environment, a verification strategy for the chip can be developed.

[0039] In the embodiment of the present application, various parameters of the chip to be verified can be expressed by code, and the code segment can be determined as the simulated code to be verified corresponding to the chip to be verified.

[0040] In the implementation, the simulated code to be verified has multiple characteristics. In order to fully verify the simulated code to be verified and thus ensure the verification effect of the chip characteristics, corresponding verification strategies can be formulated based on the multiple characteristics to be verified.

[0041] Furthermore, the verification strategy includes each feature to be verified, and each feature to be verified has several features to be verified that simulate the code to be verified. This can be understood as the full range of functions that can be realized in the scenario corresponding to each feature to be verified.

[0042] Optionally, the characteristics to be verified that simulate the code to be verified can be extracted from a design document containing all source data of the chip and the verification environment.

[0043] Because a chip has multiple features to be verified, and different features are implemented by different modules, the features to be verified can be extracted based on the different modules on the chip. This means selecting the corresponding description of the features to be verified from the design document based on the chip features that can be implemented by a module on the chip that currently needs to be verified.

[0044] Furthermore, in order to achieve a better chip verification effect, the established verification strategy may be optimized and adjusted. Therefore, a candidate verification strategy may be established based on the characteristics of the simulated code to be verified.

[0045] The description of the characteristics to be verified extracted from the design documents may include all of the following information:

[0046] It can realize the partial simulation code of the feature to be verified;

[0047] This part simulates the states that can be achieved by the code to be verified, where the states can include the operation logic, working mode, state changes, etc. of the functional module;

[0048] This part simulates the current configuration of the code to be verified, as well as the corresponding configurations in different states;

[0049] This part simulates the input rules and output rules of the code to be verified;

[0050] This section simulates the impact of the output of the code to be verified after running on the status of the current simulated code to be verified.

[0051] Furthermore, relevant statements of the characteristics to be verified can be stored in relevant locations of the design document. Therefore, information extraction is required from different levels of the design document, such as extracting from the entire design document, or extracting information from each chapter in the design document, or extracting information from each section in each chapter, or extracting information from each paragraph in each section, or extracting information from each sentence in each paragraph, and so on.

[0052] During implementation, for features to be verified in special scenarios, there is a possibility that effective information extraction cannot be achieved from the design document. Therefore, it is possible to separate the content in the design document and perform separate information editing based on the features to be verified in the special scenario.

[0053] It should be noted that in design documents, there are two types of relationships between the features to be verified that simulate the code to be verified: orthogonal and non-orthogonal. Within the same candidate verification strategy, only features to be verified with orthogonal relationships exist. An orthogonal relationship can be understood as features to be verified that have an associated relationship, while a non-orthogonal relationship can be understood as features to be verified that do not have an associated relationship.

[0054] Furthermore, based on the effective extraction of the description information of the characteristics to be verified, corresponding candidate verification strategies are generated.

[0055] S102 , running a simulation of the code to be verified based on the candidate verification strategy, verifying each feature to be verified in the candidate verification strategy, and generating a verification coverage report.

[0056] In order to adjust and optimize the candidate verification strategy, the code to be verified may be simulated and run based on the candidate verification strategy to verify each feature to be verified in the candidate verification strategy.

[0057] Run the code based on the input and working mode corresponding to each feature to be verified, and verify each feature to be verified based on the running results of the simulated code to be verified.

[0058] Optionally, there may be multiple functions that need to be verified under each feature to be verified. The simulation code to be verified may run the code separately according to different functions that need to be verified and generate corresponding outputs.

[0059] Furthermore, after completing the verification of all the features to be verified, a verification coverage report of the candidate verification strategy is generated based on the running results of the simulated code to be verified under each feature to be verified.

[0060] Optionally, the verification coverage report may be stored based on a set format, such as a table, which may include relevant parameters of each feature to be verified, verification results of each feature to be verified, and matching relationships between parameters of each feature to be verified and verification results.

[0061] S103 , reverse-marking the candidate verification strategies according to the verification coverage report to obtain the verified coverage of each feature to be verified.

[0062] The verification coverage report includes the verification results of each feature to be verified that has been verified. Therefore, it is possible that when a feature to be verified has not been verified, its related information will not be presented in the verification coverage report.

[0063] In an embodiment of the present application, in order to adjust the candidate verification strategy based on the verification coverage report, the candidate verification strategy can be reverse-marked. It can be understood that the content in the obtained verification coverage report is reverse-marked into the candidate verification strategy, and then the unverified features to be verified, insufficiently verified features to be verified, etc. are determined from the marked candidate verification strategy.

[0064] Insufficient verification can be understood as only partially verifying the various functions that can be implemented under the feature to be verified, or failing to verify some of the verified functions under the feature to be verified.

[0065] Furthermore, after the verification coverage report is reversely marked to the candidate verification strategy, based on the marked verification results under each feature to be verified in the candidate verification strategy, the verified coverage of each feature to be verified in the candidate verification strategy can be generated.

[0066] For example, if all achievable functions under a feature to be verified are verified, the verification coverage of the feature to be verified is 100%.

[0067] For example, there are 80 achievable functions under a feature to be verified. After reverse marking the verification coverage report, it can be seen that there are 60 verified functions under the feature to be verified, so the verified coverage rate of the feature to be verified is 75%.

[0068] For another example, if a feature to be verified is not verified in the current round of simulated code to be verified, and the reverse marking result is blank, the verified coverage of the feature to be verified is 0%.

[0069] It should be noted that verified coverage can include code coverage and functional coverage.

[0070] The code coverage can be obtained based on the executed and unexecuted code in the simulated code corresponding to the feature to be verified. The function coverage can be obtained based on the verified and unverified functions of the feature to be verified.

[0071] In the implementation, among the chip's to-be-verified performance, there are performances that require complete and effective verification coverage of all their functions. Therefore, in the to-be-verified performance of the simulated to-be-verified code, the functional coverage rate can be used to determine whether the complete and effective verification coverage of the to-be-verified performance has been completed.

[0072] For example, set a feature to be verified as a configuration that simulates the code to be verified.

[0073] To ensure proper operation of the verified chip, all chip configurations must be verified. Therefore, each configuration of the simulated code to be verified can be verified, and the verification completion can be determined by the verified coverage of the corresponding feature to be verified.

[0074] For another example, a feature to be verified is set to simulate the state of the code to be verified.

[0075] To ensure that the verified chip can operate normally in different states, it is necessary to verify all states of the chip. Therefore, it is possible to verify all states of the simulated code to be verified, and determine whether the current verification is complete by comparing the verified coverage of the corresponding feature to be verified.

[0076] For another example, a feature to be verified is set as a simulation port that simulates the code to be verified.

[0077] To ensure that the verified chip's ports can transmit data normally, the chip's analog ports need to be functionally verified. Therefore, all functions of the analog ports in the simulated code to be verified can be verified, and the verification is complete based on the verified coverage of the verified features.

[0078] S104: Generate a target verification strategy based on the verified coverage of each feature to be verified.

[0079] In the embodiment of the present application, the availability of the current candidate verification strategy can be judged based on the verified coverage of each feature to be verified.

[0080] Optionally, when the verified coverage corresponding to each feature to be verified in the candidate verification strategy reaches the set standard, it can be determined that in the actual chip verification process, complete and effective verification of the chip's function to be verified can be achieved based on the current candidate verification strategy.

[0081] Optionally, when the verified coverage corresponding to some items of the to-be-verified features in the candidate verification strategy does not meet the set standard, the current candidate verification strategy needs to be optimized and adjusted until the set standard is met.

[0082] Furthermore, the candidate verification strategy whose verified coverage of each feature to be verified reaches the set standard is determined as the target verification strategy.

[0083] The method for formulating a verification strategy provided in the present application runs a simulation of the code to be verified based on the candidate verification strategy to obtain a verification coverage report of the candidate verification strategy. The verification coverage report is reverse-marked into the candidate verification strategy, and then the verified coverage corresponding to each feature to be verified in the candidate verification strategy is obtained, and the candidate verification strategy is adjusted according to the verified coverage until the set standard is met, thereby generating a target verification strategy. In the present application, by reverse-marking the verification coverage report into the candidate verification strategy, the verification results of the candidate verification strategy can be presented more intuitively, and then the optimization adjustment of the candidate verification strategy can be more effectively realized, which effectively improves the adjustment efficiency of the candidate verification strategy and optimizes the quality of the target verification strategy.

[0084] In the above embodiment, the verification coverage report is reversely marked to the candidate verification strategy, which can be combined with Figure 2 Further understanding, Figure 2 This is a flow chart of a method for formulating a verification strategy according to another embodiment of the present application, such as Figure 2 As shown, the method includes:

[0085] S201: Obtain, from the verification coverage report, first identification information of the verified performance and the corresponding verified performance for each feature to be verified.

[0086] In the embodiment of the present application, each characteristic to be verified includes several performances to be verified, wherein each performance to be verified has set identification information.

[0087] The identification information corresponding to the verified performance under each feature to be verified can be determined as the first identification information. The first identification information can be used to accurately search and mark the verified performance.

[0088] Therefore, the verification coverage report can be accurately reverse-labeled into the candidate verification strategy through the first identification information.

[0089] Among them, the first identification information is unique.

[0090] Optionally, the first identification information may be stored in the attribute list of each verified characteristic. Therefore, the first identification information of each verified performance may be obtained by reading the attribute list of each verified performance.

[0091] S202: Obtain second identification information of all to-be-verified capabilities under each to-be-verified characteristic from the candidate verification strategy.

[0092] In the embodiment of the present application, each performance to be verified under each characteristic to be verified in the candidate verification strategy has set identification information, and the identification information can be determined as the second identification information of each characteristic to be verified.

[0093] The second identification information is unique.

[0094] Therefore, accurate search for each to-be-verified performance under each to-be-verified characteristic in the candidate verification strategy can be achieved through the second identification information.

[0095] Optionally, the second identification information can be stored in the attribute list of each performance to be verified under each characteristic to be verified. Therefore, by reading the attribute information list of each performance to be verified under each characteristic to be verified, the second identification information of each characteristic to be verified can be obtained.

[0096] It should be noted that the first identification information and the second identification information are based on the same format setting and can be matched.

[0097] S203: Match the first identification information with the second identification information, and reverse-mark the verified performance under each characteristic to be verified.

[0098] In an embodiment of the present application, matching can be performed based on the format, parameters, content, etc. in the first identification information and the second identification information, and based on the matching results, the association relationship between the verified performance corresponding to the first identification information and the to-be-verified performance corresponding to the second identification information can be determined.

[0099] For example, assuming that the first identification information A and the second identification information B match, the verified performance a corresponding to the first identification information A and the to-be-verified performance b corresponding to the second identification information B are in a corresponding relationship, and the verification result X of the verified performance a is the verification result corresponding to the to-be-verified performance b.

[0100] It can be understood that the verified performance a is the same as the to-be-verified performance b. Therefore, when performing reverse marking, the verification result X of the verified performance a in the verification coverage report needs to be marked under the to-be-verified performance b in the candidate strategy.

[0101] Furthermore, all first identification information and all second identification information are matched one by one, and based on the matching results, the verification results of each verified performance in the verification coverage report are marked under the corresponding performance to be verified in the candidate strategy.

[0102] S204: Obtain unverified performance under each feature to be verified, and generate a verified coverage rate for each feature to be verified based on the verified performance and the unverified performance.

[0103] Based on the marking of the verification result of each to-be-verified performance under each to-be-verified characteristic, it can be determined that there is no verified performance in the candidate verification strategy.

[0104] Optionally, the verification result may be marked at a set position under each performance item to be verified, and the position without a verification result mark may be filled with blank data.

[0105] Therefore, after completing the marking of all verification results in the verification coverage report, if the location of the verification result under a certain performance to be verified is blank data, it can be determined that the performance to be verified corresponding to the location of the blank data has not been verified.

[0106] Furthermore, the unverified performance of each feature to be verified in the candidate verification strategy can be obtained, and based on the verified performance and unverified performance of each feature to be verified, the verified coverage of each feature to be verified is generated.

[0107] For example, assuming that under the feature to be verified P, the number of verified properties is 80 and the number of unverified properties is 20, then the verified coverage rate corresponding to the feature to be verified P is 80%.

[0108] For another example, if the number of verified properties is 0 and the number of unverified properties is 50 under the property Q to be verified, the verified coverage rate corresponding to the property Q to be verified is 0%.

[0109] The verification strategy determination method provided in this application reverse-tags the verification coverage report to the performance to be verified under each feature to be verified in the candidate verification strategy, thereby generating the verified coverage rate of each feature to be verified. By reverse-tagging, the verified coverage rate of each feature to be verified in the candidate verification strategy is accurately obtained, thereby effectively adjusting and optimizing the candidate verification strategy and improving the quality of the target verification strategy.

[0110] Furthermore, regarding the generation of target verification strategy, it can be combined with Figure 3 , Figure 3 This is a flow chart of a method for formulating a verification strategy according to another embodiment of the present application, such as Figure 3 As shown, the method includes:

[0111] S301 : Determine whether to adjust the candidate verification strategy based on the verified coverage of each feature to be verified.

[0112] In the embodiment of the present application, the verification status of each feature to be verified can be determined based on the verified coverage of each feature to be verified.

[0113] In order to fully and comprehensively verify the functions of the chip, it is necessary to verify each performance to be verified under each feature to be verified.

[0114] Optionally, in response to the fact that the performance to be verified under each characteristic to be verified is verified, it is determined that the candidate verification strategy does not need to be adjusted.

[0115] If the verified coverage of each feature to be verified is 100%, it can be determined that, in the current candidate verification strategy, each performance to be verified under each feature to be verified has been verified.

[0116] It can be understood that based on the current candidate verification strategy, during the chip verification process, the characteristics of the chip can be completely and comprehensively verified. Therefore, there is no need to adjust the current candidate verification strategy.

[0117] In response to the fact that the performance to be verified under each characteristic to be verified is not verified, it is determined that the candidate verification strategy needs to be adjusted.

[0118] If the verified coverage report of the feature to be verified is not 100%, it can be known that there are unverified features to be verified in the feature to be verified corresponding to the verification result.

[0119] It can be understood that based on the current candidate verification strategy, some characteristics cannot be verified during the chip verification process, and therefore, the current candidate verification strategy needs to be adjusted.

[0120] S302 : In response to determining that no adjustment is required, determining the candidate verification strategy as the target verification strategy.

[0121] In the embodiment of the present application, when it is determined that the candidate verification strategy does not need to be adjusted, it can be understood that the current candidate verification strategy can achieve comprehensive verification of the chip function.

[0122] Therefore, the current candidate verification strategy can be determined as the target verification strategy.

[0123] It should be noted that the candidate verification strategy can be set based on some functional modules, so the final verification strategy can be composed of at least one target verification strategy.

[0124] S303 , in response to determining that adjustment is required, adjusting the candidate verification strategy according to the unverified performance of each feature to be verified, and generating a target verification strategy when all the unverified performance included in each feature to be verified in the adjusted candidate verification strategy is verified.

[0125] In the embodiment of the present application, the current candidate verification strategy can be adjusted according to the unverified performance of each feature to be verified.

[0126] Optionally, for features to be verified with a verified coverage rate less than 100%, effective information can be re-extracted from the design document, and by enriching the description information of the extracted features to be verified, the current candidate verification strategy can be optimized and adjusted.

[0127] Furthermore, verification is continued for the adjusted candidate verification strategy.

[0128] Run simulations of the code to be verified based on the adjusted candidate verification strategy and obtain the verified coverage of each feature in the adjusted candidate verification strategy. Continue to determine the verified coverage of each feature to be verified in the adjusted candidate verification strategy until the verified coverage of each feature to be verified in the adjusted candidate verification strategy reaches 100%, then stop adjusting.

[0129] Furthermore, based on the candidate verification strategies with 100% verified coverage of each feature to be verified, a target verification strategy is generated.

[0130] The method for determining the verification strategy provided by the present application determines whether the candidate verification strategy needs to be adjusted based on the verified coverage. For the candidate verification strategy that does not need to be adjusted, it is determined as the target verification strategy output. For the candidate verification strategy that needs to be adjusted, the candidate verification strategy is adjusted based on the unverified performance under each feature to be verified, and the adjusted candidate verification strategy is verified until the verified coverage of each feature to be verified in the adjusted candidate verification strategy is 100%, then the adjustment is stopped, and the target verification strategy is generated based on the candidate verification strategy with a verified coverage of 100% for each feature to be verified. Through the verified coverage, the verification results of each performance to be verified under each feature to be verified are accurately obtained, which effectively improves the adjustment efficiency of the candidate verification strategy and optimizes the quality of the target verification strategy.

[0131] In the above embodiment, regarding the generation of the verification coverage report, it can be combined with Figure 4 Further understanding, Figure 4 This is a flow chart of a method for determining a verification strategy according to another embodiment of the present application. Figure 4 As shown, the method includes:

[0132] S401 , based on each feature to be verified in the candidate verification strategy, running the simulation code to be verified, and verifying the performance of each feature to be verified according to the running result, so as to obtain the verified performance of each feature to be verified.

[0133] In the embodiment of the present application, the code to be verified can be simulated based on the operating environment and requirements required for each performance to be verified under each characteristic to be verified, and the corresponding output can be generated based on the input corresponding to each performance to be verified.

[0134] Here, it can be determined based on the output result whether the performance to be verified currently being verified is successfully verified.

[0135] Optionally, a verification result standard for each performance to be verified under each feature to be verified may be set. When the output result of simulating the running of the code to be verified meets the set standard, it can be determined that the performance to be verified currently being verified is successfully verified.

[0136] Furthermore, based on each performance to be verified under each feature to be verified, the simulated code to be verified is run, and the output result of each time and the performance to be verified corresponding to each output result are recorded. When the current verification is judged to be successful based on the output result, the successfully verified performance to be verified is marked as the verified performance under the feature to which it belongs.

[0137] S402 : Generate a verification coverage report for the candidate verification strategy based on the verified performance of each feature to be verified and the verification results of the verified performance.

[0138] In the embodiment of the present application, the verification results of the verified performance under each feature to be verified can be obtained based on the verification coverage report.

[0139] Optionally, a verification coverage report, such as a table, may be generated based on a set format that defines each feature to be verified, the verified performance under each feature to be verified, related attribute parameters, and the filling location of the verification results.

[0140] After obtaining each feature to be verified, the verified performance under each feature to be verified, related attribute parameters and verification results, the blank table can be filled in based on the limited positions to generate a verification coverage report of the candidate verification strategy in a set format.

[0141] It should be noted that in the method provided in this application, the verification coverage report is reverse-marked into the candidate verification strategy. Therefore, in order to improve the efficiency of generating the verification coverage report and save relevant resources, during the process of filling in the blank report, only relevant information on the performance of generating the verification results can be filled in.

[0142] The verification strategy determination method provided in this application generates a verification coverage report for candidate verification strategies based on the verified performance of each feature to be verified. This provides a data basis for subsequent reverse marking of verification results, effectively improving the efficiency of adjusting candidate verification strategies.

[0143] In the above embodiment, the acquisition of candidate verification strategies can be combined with Figure 5 Further understanding, Figure 5 This is a flow chart of a method for determining a verification strategy according to another embodiment of the present application. Figure 5 As shown, the method includes:

[0144] S501, constructing a test case, and obtaining each feature to be verified by simulating the code to be verified in the environment of the test case.

[0145] Optionally, the verification process of simulating the code to be verified can be started by constructing a test case. It can be understood that the test case provides a verification environment for simulating the feature verification of the code to be verified.

[0146] In the environments provided by different test cases, simulating the code to be verified can achieve different features. Therefore, for a certain test case constructed, it is necessary to determine from the design document each feature that can be achieved by simulating the code to be verified in this environment based on the verification environment it provides.

[0147] Furthermore, based on the set extraction rules of the features to be verified, the features to be verified in the current test case environment are determined from all the features.

[0148] S502: Generate a candidate verification strategy based on each feature to be verified.

[0149] In an embodiment of the present application, each feature to be verified has related settings, such as the implementation fragment of the simulated code to be verified corresponding to each feature to be verified, the working mode of the simulated code to be verified corresponding to each feature to be verified, the status and configuration corresponding to each feature to be verified, etc.

[0150] Furthermore, relevant parameters corresponding to each feature to be verified are obtained, and a corresponding relationship between the feature to be verified and the relevant parameters is constructed, thereby generating a candidate verification strategy for each feature to be verified of the current simulated code to be verified.

[0151] The verification strategy determination method provided in this application uses test cases to determine each feature of the simulated code to be verified, thereby determining the corresponding candidate verification strategy. This provides a pre-planning for the verification of the simulated code to be verified, effectively improving the verification efficiency of the simulated code to be verified.

[0152] Corresponding to the verification strategy determination method provided in the above-mentioned embodiments, an embodiment of the present application also provides a verification strategy determination device. Since the verification strategy determination device provided in the embodiment of the present application corresponds to the verification strategy determination method provided in the above-mentioned embodiments, the implementation method of the above-mentioned verification strategy determination method is also applicable to the verification strategy determination device provided in the embodiment of the present application, and will not be described in detail in the following embodiments.

[0153] Figure 6 This is a structural diagram of a device for determining a verification strategy according to an embodiment of the present application. Figure 6 As shown, the verification strategy determination device 600 includes an acquisition module 61, a verification module 62, a marking module 63, and a generation module 64, wherein:

[0154] An acquisition module 61 is configured to acquire a candidate verification strategy, wherein the candidate verification strategy includes simulating each feature to be verified of the code to be verified;

[0155] A verification module 62 is configured to run a simulation of the code to be verified based on the candidate verification strategy, verify each feature to be verified in the candidate verification strategy, and generate a verification coverage report;

[0156] a marking module 63 for reversely marking the candidate verification strategies according to the coverage report to obtain the verified coverage of each feature to be verified;

[0157] The generating module 64 is configured to generate a target verification strategy based on the verified coverage of each feature to be verified.

[0158] Figure 7 This is a structural diagram of a device for determining a verification strategy according to another embodiment of the present application. Figure 7 As shown, the verification strategy determination device 700 includes an acquisition module 71, a verification module 72, a marking module 73, and a generation module 74, wherein:

[0159] It should be noted that the acquisition module 61 , the verification module 62 , the marking module 63 , and the generation module 64 have the same structure and function as the acquisition module 71 , the verification module 72 , the marking module 73 , and the generation module 74 .

[0160] In an embodiment of the present application, the marking module 73 is also used to: obtain the first identification information of the verified performance and the corresponding verified performance under each feature to be verified from the verification coverage report; obtain the second identification information of all the performance to be verified under each feature to be verified from the candidate verification strategy; match the first identification information with the second identification information, and reversely mark the verified performance under each feature to be verified; obtain the unverified performance under each feature to be verified, and generate the verified coverage rate of each feature to be verified based on the verified performance and the unverified performance.

[0161] In an embodiment of the present application, the generation module 74 is also used to: determine whether to adjust the candidate verification strategy based on the verified coverage of each feature to be verified; in response to determining that no adjustment is required, determine the candidate verification strategy as the target verification strategy; in response to determining that adjustment is required, adjust the candidate verification strategy according to the unverified performance of each feature to be verified, and when the performance to be verified included in each feature to be verified in the adjusted candidate verification strategy is verified, generate the target verification strategy.

[0162] In an embodiment of the present application, the generation module 74 is further used to: in response to the fact that the performance to be verified under each characteristic to be verified is verified, determine that the candidate verification strategy does not need to be adjusted; in response to the fact that the performance to be verified under each characteristic to be verified is not verified, determine that the candidate verification strategy needs to be adjusted.

[0163] In the embodiment of the present application, the acquisition module 71 is further used to: construct a test case and obtain each feature to be verified that simulates the code to be verified in the test case environment; and generate a candidate verification strategy based on each feature to be verified.

[0164] In an embodiment of the present application, the verification module 72 is also used to: run a simulation of the code to be verified based on each feature to be verified in the candidate verification strategy, and verify the performance under each feature to be verified according to the running results to obtain the verified performance under each feature to be verified; generate a verification coverage report for the candidate verification strategy based on the verified performance under each feature to be verified and the verification results of the verified performance.

[0165] The verification strategy formulation device provided in the present application runs a simulation of the code to be verified based on the candidate verification strategy to obtain the verification coverage report of the candidate verification strategy. The verification coverage report is reverse-marked into the candidate verification strategy, and then the verified coverage corresponding to each feature to be verified in the candidate verification strategy is obtained, and the candidate verification strategy is adjusted according to the verified coverage until the set standard is met, and then the target verification strategy is generated. In the present application, by reverse-marking the verification coverage report into the candidate verification strategy, the verification result of the candidate verification strategy can be presented more intuitively, and then the optimization adjustment of the candidate verification strategy can be realized more effectively, which effectively improves the adjustment efficiency of the candidate verification strategy and optimizes the quality of the target verification strategy.

[0166] To achieve the above embodiments, the present application also provides an electronic device, a computer-readable storage medium, and a computer program product.

[0167] Figure 8 This is a block diagram of an electronic device according to an embodiment of the present application. Figure 8 The electronic device shown can be used to perform Figures 1 to 5 A method for determining a verification strategy of an embodiment.

[0168] In order to implement the above embodiment, the present application also provides a non-transitory computer-readable storage medium storing computer instructions, which is used to enable the computer to execute Figures 1 to 5 A method for determining a verification strategy of an embodiment.

[0169] In order to implement the above embodiment, the present application also provides a computer program product, when the instruction processor in the computer program product executes Figures 1 to 5 A method for determining a verification strategy of an embodiment.

[0170] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0171] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0172] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0173] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0174] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0175] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0176] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0177] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A method for determining a verification strategy, characterized in that: include: Obtaining a candidate verification strategy, wherein the candidate verification strategy includes simulating each feature to be verified of the code to be verified; Running the simulated code to be verified based on the candidate verification strategy, verifying each of the features to be verified in the candidate verification strategy, and generating a verification coverage report; Reverse-marking the candidate verification strategy according to the verification coverage report to obtain the verified coverage of each feature to be verified; generating a target verification strategy based on the verified coverage of each feature to be verified; The reverse marking of the candidate verification strategy according to the verification coverage report to obtain the verified coverage of each feature to be verified includes: Obtaining, from the verification coverage report, first identification information of a verified performance under each of the characteristics to be verified and the corresponding verified performance, wherein the first identification information is used to enable searching and marking of the verified performance; Obtaining, from the candidate verification strategy, second identification information of all to-be-verified performances under each to-be-verified characteristic; Matching the first identification information with the second identification information, and reversely marking the verified performance under each of the characteristics to be verified; The unverified performance under each of the characteristics to be verified is obtained, and the verified coverage of each of the characteristics to be verified is generated based on the verified performance and the unverified performance.

2. The method according to claim 1, characterized in that Generating a target verification strategy based on the verified coverage of each of the features to be verified includes: Determining whether to adjust the candidate verification strategy based on the verified coverage of each of the characteristics to be verified; In response to determining that no adjustment is required, determining the candidate verification strategy as the target verification strategy; In response to determining that adjustment is required, the candidate verification strategy is adjusted according to the unverified performance of each characteristic to be verified, and when the performance to be verified included in each characteristic to be verified in the adjusted candidate verification strategy is verified, the target verification strategy is generated.

3. The method according to claim 2, characterized in that The determining whether to adjust the candidate verification strategy based on the verified coverage of each of the to-be-verified features includes: In response to the to-be-verified performance under each of the to-be-verified characteristics being verified, determining that the candidate verification strategy does not need to be adjusted; In response to the fact that the performance to be verified under each of the characteristics to be verified is not verified, it is determined that the candidate verification strategy needs to be adjusted.

4. The method according to claim 3, characterized in that The obtaining of the candidate verification strategy includes: Constructing a test case, and obtaining each of the to-be-verified characteristics of the simulated to-be-verified code under the environment of the test case; Based on each of the characteristics to be verified, the candidate verification strategy is generated.

5. The method according to claim 1, wherein The step of running the simulated code to be verified based on the candidate verification strategy, verifying each feature to be verified in the candidate verification strategy, and generating a verification coverage report includes: Based on each of the characteristics to be verified in the candidate verification strategy, running the simulation code to be verified, and verifying the performance under each of the characteristics to be verified according to the running results, so as to obtain verified performance under each of the characteristics to be verified; The verification coverage report of the candidate verification strategy is generated based on the verified performance under each of the characteristics to be verified and the verification results of the verified performance.

6. A device for determining a verification strategy, characterized in that: include: An acquisition module, configured to acquire a candidate verification strategy, wherein the candidate verification strategy includes simulating each feature to be verified of the code to be verified; a verification module, configured to run the simulated code to be verified based on the candidate verification strategy, verify each of the characteristics to be verified in the candidate verification strategy, and generate a verification coverage report; a marking module, configured to reversely mark the candidate verification strategy according to the coverage report to obtain a verified coverage rate of each feature to be verified; A generating module, configured to generate a target verification strategy based on the verified coverage of each feature to be verified; The marking module is further used for: Obtaining, from the verification coverage report, first identification information of a verified performance under each of the characteristics to be verified and the corresponding verified performance, wherein the first identification information is used to enable searching and marking of the verified performance; Obtaining, from the candidate verification strategy, second identification information of all to-be-verified performances under each to-be-verified characteristic; Matching the first identification information with the second identification information, and reversely marking the verified performance under each of the characteristics to be verified; The unverified performance under each of the characteristics to be verified is obtained, and the verified coverage of each of the characteristics to be verified is generated based on the verified performance and the unverified performance.

7. The device according to claim 6, characterized in that The generating module is further configured to: Determining whether to adjust the candidate verification strategy based on the verified coverage of each of the characteristics to be verified; In response to determining that no adjustment is required, determining the candidate verification strategy as the target verification strategy; In response to determining that adjustment is required, the candidate verification strategy is adjusted according to the unverified performance of each characteristic to be verified, and when the performance to be verified included in each characteristic to be verified in the adjusted candidate verification strategy is verified, the target verification strategy is generated.

8. The device according to claim 7, characterized in that The generating module is further configured to: In response to the to-be-verified performance under each of the to-be-verified characteristics being verified, determining that the candidate verification strategy does not need to be adjusted; In response to the fact that the performance to be verified under each of the characteristics to be verified is not verified, it is determined that the candidate verification strategy needs to be adjusted.

9. The device according to claim 8, characterized in that The acquisition module is further used to: Constructing a test case, and obtaining each of the to-be-verified characteristics of the simulated to-be-verified code under the environment of the test case; Based on each of the characteristics to be verified, the candidate verification strategy is generated.

10. The device according to claim 6, characterized in that The verification module is further configured to: Based on each of the characteristics to be verified in the candidate verification strategy, running the simulation code to be verified, and verifying the performance under each of the characteristics to be verified according to the running results, so as to obtain verified performance under each of the characteristics to be verified; The verification coverage report of the candidate verification strategy is generated based on the verified performance under each of the characteristics to be verified and the verification results of the verified performance.

11. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

Images