Verification methods and equipment

Abstract

The present disclosure provides a verification method and device, relating to the field of semiconductor technology, the method comprising: obtaining a first target command sequence formed by a test case in the process of executing a test on a memory chip, the first target command sequence including a plurality of target commands arranged in sequence; obtaining a first target command sequence formed by a test case in the process of executing a test on a memory chip, the first target command sequence including a plurality of target commands arranged in sequence; and determining a coverage parameter of the test case based on an occurrence parameter. Compared to manually analyzing a test case to determine its quality, the solution of determining the quality of a test case through occurrence parameters in the embodiment of the present disclosure has lower labor costs and is more objective, will not cause errors in the analysis results due to human problems, and has higher accuracy.

Description

Technical Field

[0001] The present disclosure relates to, but is not limited to, a verification method and device. Background Art

[0002] Memory chips are semiconductor devices used to store data. To ensure proper performance, they can be tested using test cases. Only those that pass these tests can be put into operation. Test cases contain various commands that, when executed, perform corresponding operations on the memory chip. Therefore, whether the results match the expected results can be used to determine whether the memory chip is functioning properly.

[0003] In existing technologies, chip verification engineers can analyze test cases to determine whether they are complete. If not, they need to adjust the test cases to improve them. Testing memory chips with these complete test cases can detect as many defects as possible.

[0004] However, this approach results in high costs and cannot accurately represent the quality of test cases. Summary of the Invention

[0005] The embodiments of the present disclosure provide a verification method and device to reduce costs, and represent the coverage of test cases through coverage parameters to accurately indicate the quality of the test cases.

[0006] In a first aspect, an embodiment of the present disclosure provides a verification method, the method comprising:

[0007] Acquire a first target command sequence generated by a test case during a process of executing a test on a memory chip, wherein the first target command sequence includes a plurality of target commands arranged in sequence;

[0008] determining occurrence parameters of a preset number of first reference command sequences in the first target command sequence, wherein the first reference command sequence includes a plurality of reference commands arranged in sequence;

[0009] Determine the coverage parameter of the test case according to the occurrence parameter.

[0010] In some embodiments, determining occurrence parameters of a preset number of first reference command sequences in the first target command sequence includes:

[0011] Acquire a command mapping table, wherein the command mapping table includes: a mapping relationship between a command name of the reference command and a command name of the target command, and a mapping relationship between command parameters of the reference command and command parameters of the target command;

[0012] Determine occurrence parameters of a preset number of first reference command sequences in the first target command sequence according to the command mapping table.

[0013] In some embodiments, the occurrence parameter includes an occurrence quantity and / or an occurrence number, and determining, according to the command mapping table, the occurrence parameters of a preset number of first reference command sequences in the first target command sequence includes:

[0014] Acquire a second reference command sequence, where the second reference command sequence is obtained by mapping the first reference command sequence in advance through the command mapping table;

[0015] determining the number of different second reference command sequences included in the first target command sequence as the occurrence number;

[0016] And / or, determining the number of occurrences of each second reference command sequence in the first target command sequence.

[0017] In some implementations, determining the coverage parameter of the test case according to the occurrence parameter includes:

[0018] Determining a first proportion of the number of occurrences in the preset number;

[0019] A coverage parameter of the test case is determined according to the first proportion and / or the number of occurrences, where the coverage parameter is positively correlated with the first proportion and the coverage parameter is positively correlated with the number of occurrences.

[0020] In some implementations, the process of generating the first reference command sequence includes:

[0021] Acquire all reference commands that comply with the command rule, each of the reference commands including a command name of the reference command and all command parameters supported by the reference command;

[0022] The same reference commands or different reference commands are combined to obtain multiple first reference command sequences, wherein the same reference commands include at least two reference commands with the same command name and the same command parameters, and the different reference commands include at least two reference commands with different command names and / or different command parameters.

[0023] In some implementations, after combining the same reference command or different reference commands to obtain multiple first reference command sequences, the method further includes:

[0024] The command sequences that do not meet the preset sequence rule are deleted from the first reference command sequence.

[0025] In some embodiments, the method further comprises:

[0026] The coverage parameters of the test case and / or the corresponding occurrence parameters are written into the coverage report of the test case.

[0027] In some embodiments, after writing the coverage parameter of the test case and / or the corresponding occurrence parameter into the coverage report of the test case, the method further includes:

[0028] Processing each of the first reference command sequences to obtain at least one third reference command sequence, wherein the processing includes: removing commands and / or removing command parameters;

[0029] Counting the number of occurrences of the first reference command sequence including the third reference command sequence to obtain the number of occurrences of the third reference command sequence;

[0030] The third reference command sequence and the corresponding number of occurrences are written into the coverage report of the test case.

[0031] In some embodiments, obtaining a first target command sequence generated by a test case during execution of a test on a memory chip includes:

[0032] Performing a simulation test on the model of the memory chip using the test case to obtain a command tracking file corresponding to the test case; the command tracking file includes at least one target command used in the process of executing the simulation test and the corresponding execution time;

[0033] The at least one target command is sorted according to the execution time to obtain the first target command sequence.

[0034] In a second aspect, an embodiment of the present disclosure provides a verification device, the device comprising:

[0035] A first command sequence acquisition module is configured to acquire a first target command sequence generated when a test case is executed on a memory chip for testing, wherein the first target command sequence includes a plurality of target commands arranged in sequence;

[0036] an occurrence determination module, configured to determine occurrence parameters of a preset number of first reference command sequences in the first target command sequence, wherein the first reference command sequence includes a plurality of reference commands arranged in sequence;

[0037] The coverage determination module is used to determine the coverage parameters of the test case according to the occurrence parameters.

[0038] In some embodiments, the occurrence determination module is further configured to:

[0039] Acquire a command mapping table, wherein the command mapping table includes: a mapping relationship between a command name of the reference command and a command name of the target command, and a mapping relationship between command parameters of the reference command and command parameters of the target command;

[0040] Determine occurrence parameters of a preset number of first reference command sequences in the first target command sequence according to the command mapping table.

[0041] In some embodiments, the occurrence parameter includes the number of occurrences and / or the number of occurrences, and the occurrence determination module is further configured to:

[0042] Acquire a second reference command sequence, where the second reference command sequence is obtained by mapping the first reference command sequence in advance through the command mapping table;

[0043] determining the number of different second reference command sequences included in the first target command sequence as the occurrence number;

[0044] And / or, determining the number of occurrences of each second reference command sequence in the first target command sequence.

[0045] In some embodiments, the coverage determination module is further configured to:

[0046] Determining a first proportion of the number of occurrences in the preset number;

[0047] A coverage parameter of the test case is determined according to the first proportion and / or the number of occurrences, where the coverage parameter is positively correlated with the first proportion and the coverage parameter is positively correlated with the number of occurrences.

[0048] In some implementations, the process of generating the first reference command sequence includes:

[0049] A reference command acquisition module, configured to acquire all reference commands that comply with command rules, each of the reference commands including a command name of the reference command and all command parameters supported by the reference command;

[0050] A command combination module is used to combine the same reference commands or different reference commands to obtain multiple first reference command sequences, wherein the same reference commands include at least two reference commands with the same command name and the same command parameters, and the different reference commands include at least two reference commands with different command names and / or different command parameters.

[0051] In some embodiments, the above apparatus further comprises:

[0052] The command sequence deletion module is configured to delete command sequences that do not meet preset sequence rules from a plurality of first reference command sequences after combining the same reference commands or different reference commands.

[0053] In some embodiments, the device further comprises:

[0054] The first report writing module is used to write the coverage parameters of the test case and / or the corresponding occurrence parameters into the coverage report of the test case.

[0055] In some embodiments, the device further comprises:

[0056] a command sequence processing module, configured to, after writing the coverage parameters and / or corresponding occurrence parameters of the test cases into the coverage report of the test cases, process each of the first reference command sequences to obtain at least one third reference command sequence, wherein the processing includes: removing commands and / or removing command parameters;

[0057] an occurrence count statistics module, configured to count the occurrence counts of the first reference command sequence including the third reference command sequence to obtain the occurrence count of the third reference command sequence;

[0058] The second report writing module is used to write the third reference command sequence and the corresponding number of occurrences into the coverage report of the test case.

[0059] In some implementations, the first command sequence acquisition module is further configured to:

[0060] Performing a simulation test on the model of the memory chip using the test case to obtain a command tracking file corresponding to the test case; the command tracking file includes at least one target command used in the process of executing the simulation test and the corresponding execution time;

[0061] The at least one target command is sorted according to the execution time to obtain the first target command sequence.

[0062] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: at least one processor and a memory;

[0063] The memory stores computer-executable instructions;

[0064] The at least one processor executes the computer-executable instructions stored in the memory, so that the electronic device implements the method according to the first aspect.

[0065] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, which are used to implement the method described in the first aspect when the computer-executable instructions are executed by an electronic device.

[0066] In a fifth aspect, an embodiment of the present disclosure further provides a computer program for implementing the method of the first aspect.

[0067] The present disclosure provides a verification method and device, including: obtaining a first target command sequence formed by a test case during the test of a memory chip, wherein the first target command sequence includes a plurality of target commands arranged in sequence; obtaining a first target command sequence formed by a test case during the test of a memory chip, wherein the first target command sequence includes a plurality of target commands arranged in sequence; and determining a coverage parameter of the test case based on an occurrence parameter. Compared to manually analyzing test cases to determine their quality, the present disclosure embodiment uses the occurrence parameter to determine the quality of test cases, which has lower labor costs and is more objective. It will not cause errors in the analysis results due to human errors and has higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0069] Figure 1 This is a schematic diagram of a command sequence provided by an embodiment of the present disclosure;

[0070] Figure 2 is a flowchart of the steps of a verification method provided by an embodiment of the present disclosure;

[0071] Figure 3 This is a detailed verification process diagram provided by an embodiment of the present disclosure;

[0072] Figure 4 This is a structural block diagram of a verification device provided by an embodiment of the present disclosure;

[0073] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present disclosure.

[0074] The above drawings illustrate specific embodiments of the present disclosure, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the present disclosure in any way, but rather to illustrate the concepts of the present disclosure to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0075] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0076] The embodiment of the present disclosure is used to analyze a test case (Test Case), determine the quality of the test case in a quantitative manner, and realize the quality verification of the test case. In order to solve the problem of high cost and poor accuracy of manual analysis of test cases, the embodiment of the present disclosure can obtain the command sequence corresponding to the test case, called the first target command sequence, and then determine the coverage of the test case according to the occurrence of the preset first reference command sequence in the first target command sequence to describe the quality of the test case. It can be understood that when the first reference command sequence appears in the first target command sequence, it means that the test case covers more test scenarios. At this time, the higher the coverage of the test case, the better the test case. Conversely, when the first reference command sequence appears in the first target command sequence, it means that the test case covers fewer test scenarios. At this time, the lower the coverage of the test case, the worse the test case. Compared with manual analysis of test cases to determine their quality, the solution of determining the quality of test cases by the occurrence parameters of the embodiment of the present disclosure has lower labor costs and is more objective. It will not cause analysis results errors due to human problems and has higher accuracy.

[0077] It can be seen that the embodiment of the present disclosure involves multiple command sequences, including the aforementioned first target command sequence and the first reference command sequence. The command sequence includes multiple commands executed sequentially, Figure 1 This is a schematic diagram of a command sequence provided by an embodiment of the present disclosure. Figure 1 As shown, the command sequence includes 5 commands CMD1, CMD2, CMD3, CMD4 and CMD5. When executed, according to Figure 1 The commands CMD1, CMD2, CMD3, CMD4 and CMD5 are executed in the order shown.

[0078] It should be noted that the order of commands in a command sequence cannot be swapped. After swapping the order of commands, a completely different command sequence will be obtained. For example, Figure 1In the command sequence CMD1->CMD2->CMD3->CMD4->CMD5, swapping the order of commands CMD1 and CMD2 results in another command sequence: CMD2->CMD1->CMD3->CMD4->CMD5. It can be understood that the first target command sequence is the command sequence consisting of the commands to be executed by the test case during the test execution, and the first reference command sequence is the set of all command sequences supported by the memory.

[0079] Figure 2 This is a flowchart of a verification method provided by an embodiment of the present disclosure. Figure 2 As shown, the above verification method includes S201 to S203.

[0080] S201: Acquire a first target command sequence generated when a test case is executed to test a memory chip, where the first target command sequence includes a plurality of target commands arranged in sequence.

[0081] The first target command sequence includes all target commands to be executed by the test case during the test execution process, and these target commands are arranged in sequence in the first target command sequence.

[0082] To accurately and quickly obtain the first target command sequence, the first target command sequence can be extracted through simulation testing. Specifically, first, a simulation test is performed on a memory chip model using a test case to obtain a command tracking file corresponding to the test case; the command tracking file includes at least one target command used during the simulation test and the corresponding execution time; then, the at least one target command is sorted according to the execution time to obtain the first target command sequence.

[0083] The model may be a Golden Model, the simulation may be a Verilog simulation, and the simulation may be performed on an EDA (Electronic Design Automation) tool. An EDA tool is a software tool used for functions such as IC (Integrated Circuit) design, routing, verification, and simulation. By using the EDA tool to perform Verilog simulation of the Golden Model of the memory chip using a test case, a command trace file may be output, from which the first target command sequence may be extracted.

[0084] The command tracking file includes, but is not limited to, the command name of the target command, at least one command parameter of the target command, the start execution time of the target command, and the end execution time of the target command. In the disclosed embodiment, the start execution time can be used as the execution time for sorting the target commands. The sorting here is generally in ascending order of time, with target commands that were executed earlier being sorted after target commands that were executed later.

[0085] The target command may be any command supported by the memory, and the command name may represent the function of the command. For example, the command name ACT is used for row activation, the command name WR is used for writing data, the command name RD is used for reading data, the command name REFab is used for data refresh, and the command name PREab is used for precharging. The command names in the embodiments of the present disclosure may be all command names supported by the memory, and are not limited to ACT, WR, REFab, RD, and PREab in the above examples.

[0086] The command parameters may include, but are not limited to: a bank group (BG) identifier, a bank identifier, a row identifier, a column identifier, a sort parameter, a burst length (BL), and a mask indication parameter (DM).

[0087] It should be understood that a memory chip may generally include multiple memory groups, each memory group may include multiple banks, each bank may include multiple rows, each row may include multiple columns of memory cells, and each memory cell may correspond to a logical address.

[0088] Therefore, the BG identifier in the command parameter may be a storage group address, which is used to indicate the storage group targeted by the command.

[0089] The bank identifier in the command parameter may be a bank address, which is used to indicate the bank targeted by the command.

[0090] The row identifier in the command parameter may be a row address, which is used to indicate the row targeted by the command.

[0091] The column identifier in the command parameter may be a column address, which is used to indicate the column targeted by the command.

[0092] The burst length in the command parameter is used to indicate the length of data to be read or written at one time by the command. For example, the burst length can be 8 bits.

[0093] The sort parameter in the command parameter is used to indicate the sorting rule of the data.

[0094] The mask indication parameter in the command parameter is used to indicate whether to perform mask processing on the data.

[0095] Based on the above description, the commands in the embodiments of the present disclosure are uniformly represented in the following manner: Command Name_BG Identifier_Bank Identifier_Row Identifier_Column Identifier_Burst Length_Sorting Parameter_Mask Indicator Parameter. Of course, some commands do not require all of the above parameters, but only require some of them. For example, the refresh command does not require the above BG Identifier, Row Identifier, Column Identifier, Burst Length, Sorting Parameter, and Mask Indicator Parameter, but only requires the Bank Identifier.

[0096] S202: Determine occurrence parameters of a preset number of first reference command sequences in a first target command sequence, where the first reference command sequence includes a plurality of reference commands arranged in sequence.

[0097] The first reference command sequence is a sequence of all commands supported by a preset memory. In some embodiments, the first reference command sequence can be generated by the following process: first, all reference commands that conform to the command rule are obtained, each reference command including a command name of the reference command and all command parameters supported by the reference command; then, the same reference commands or different reference commands are combined to obtain multiple first reference command sequences, where the same reference commands include at least two reference commands with the same command name and the same command parameters, and the different reference commands include at least two reference commands with different command names and / or different command parameters.

[0098] For example, the two reference commands RD_BGA_BKA_CA_BC8_Odr0 and WR_BGA_BKA_CA_BC8_Ord0 can be combined to obtain the first reference command sequence: RD_BGA_BKA_CA_BC8_Odr0->WR_BGA_BKA_CA_BC8_Ord0. RD and WR are the command names of the two reference commands, used for reading and writing data, respectively. BGA is the command parameter of the reference command: the BG identifier, indicating that the reference command operates on the BG with the BG identifier BGA. BKA is the command parameter of the reference command: the bank identifier, indicating that the reference command operates on the bank with the bank identifier BKA. CA (columnA) is the command parameter of the reference command: the column identifier, indicating that the reference command operates on column A. BC8 (burstchop 8, burst length 8 bits) is the command parameter of the reference command: the burst length, indicating that the burst length of the reference command parameter is 8 bits. Ord0 (a pre-set sorting rule number) is a command parameter of the reference command: a sorting parameter, used to indicate that the data sorting rule of the reference command is Ord0.

[0099] The reference commands are all commands that can be supported by the memory, and one or more identical or different reference commands among all the reference commands are combined to obtain a first reference command sequence.

[0100] The disclosed embodiment can combine all reference commands in any manner to obtain a large number of first reference command sequences, thereby avoiding missing first reference command sequences that can be supported by the memory, thereby improving the coverage and parameter accuracy of test cases.

[0101] In some embodiments, reference commands may be combined in pairs to obtain a first reference command sequence comprising two reference commands. This ensures that the first reference command sequence is as short as possible while covering the most combination scenarios, thereby better verifying the quality of the test case.

[0102] In some embodiments, after obtaining the first reference command sequence, command sequences that do not satisfy the preset sequence rules can also be deleted from the first reference command sequence. Among them, the command sequence that does not satisfy the preset sequence rules can be understood as a command sequence that will not appear in the test case, that is, there will be no command sequence that does not satisfy the preset sequence rules in the first target command sequence. Therefore, such command sequences that do not satisfy the preset sequence rules in the first reference command sequence can be deleted to reduce the number of first reference command sequences, thereby reducing the time required for the first reference command sequence to match the first target command sequence, which can improve the matching efficiency and thereby improve the efficiency of determining the coverage parameters of the test case.

[0103] Specifically, a command sequence list that does not meet the preset sequence rule can be set to match the first reference command sequence obtained by combining the reference commands with each command sequence in the command sequence list, and delete the first reference command sequence that successfully matches.

[0104] Of course, a sequence rule may also be set to match the first reference command sequence with the sequence rule, and delete the first reference command sequence that is successfully matched.

[0105] After a preset number of first reference command sequences are generated, occurrence parameters of the first reference command sequences in the first target command sequence of the test case may be determined.

[0106] Among them, the occurrence parameter is used to represent the occurrence of the first reference command sequence in the first target command sequence from at least one dimension, including at least one of the following: the number of occurrences of the first reference command sequence in the first target command sequence, and the number of occurrences of each first reference command sequence in the first target command sequence.

[0107] In actual applications, commands with the same function may have different names in the first reference command sequence and the command tracking file. For example, the command names RD and RDS8 in the command tracking file are both used to read data, but the read command in the first reference command sequence has the command name RD. Similarly, the same command parameter may be represented differently in the first reference command sequence and the command tracking file. For example, the BG identifier is represented as 00 in the command tracking file, but as BGA in the first reference command sequence.

[0108] Due to the differences in the representation of command names and command parameters, the first reference command sequence and the first target command sequence for the same function are represented differently. Consequently, if the first reference command sequence and the first target command sequence are matched to determine the occurrence parameters, the match will fail, resulting in a lower accuracy of the occurrence parameters.

[0109] To address the aforementioned issues, the disclosed embodiments utilize a pre-established command mapping table to achieve mapping between the two for matching. Specifically, a command mapping table is first acquired, which includes mappings between the command names of reference commands and target commands, and between the command parameters of the reference commands and the command parameters of the target commands. The command mapping table is then used to determine the occurrence parameters of a preset number of first reference command sequences in the first target command sequence.

[0110] Specifically, when determining the occurrence parameters of a preset number of first reference command sequences in the first target command sequence according to the command mapping table, there are two mapping strategies.

[0111] In the first mapping strategy, each first reference command sequence can be mapped to a second reference command sequence according to the command mapping table. That is, the command name and command parameters of the reference command in the first reference command sequence are replaced with the command name and command parameters in the command mapping table to obtain the second reference command sequence corresponding to each first reference command sequence. Therefore, each second reference command sequence can be matched with the first target command sequence to obtain the parameters of the occurrence of each second reference command sequence in the first target command sequence, which are used as the parameters of the occurrence of the first reference command sequence in the first target command sequence.

[0112] In the first mapping strategy described above, since the reference commands supported by the memory vary little, the first reference command sequence also varies little. Therefore, the first reference command sequence can be pre-mapped to a second reference command sequence according to a command mapping table and stored for reuse. In this way, each time the coverage parameters of a test case need to be determined, the second reference command sequence can be directly matched with the first target command sequence, which can save the time required for the mapping process and effectively improve the efficiency of determining the coverage parameters of the test case.

[0113] The occurrence condition parameter includes the number of occurrences and / or the number of occurrences. When determining the occurrence condition parameter based on the second reference command sequence and the first target command sequence, the number of different second reference command sequences included in the first target command sequence may be determined as the number of occurrences; and / or the number of occurrences of each second reference command sequence in the first target command sequence may be determined.

[0114] It is understood that when the first target command sequence includes a second reference command sequence, it means that the second reference command sequence is a subsequence of the first target command sequence. In other words, all commands in the second reference command sequence are identical to multiple commands that appear consecutively in the first target command sequence.

[0115] For example, if the multiple second reference command sequences include: CMD1->CMD2, CMD2->CMD1, CMD2->CMD2, CMD1->CMD1, and the first target command sequence is CMD1->CMD2->CMD2->CMD2->CMD1->CMD2, then the second reference command sequence CMD1->CMD2 appears 2 times in the first target command sequence, the second reference command sequence CMD2->CMD1 appears 1 times in the first target command sequence, the second reference command sequence CMD2->CMD2 appears 2 times in the first target command sequence, and the second reference command sequence CMD1->CMD1 appears 0 times in the first target command sequence. Therefore, the number of different second reference command sequences included in the first target command sequence, that is, the number of occurrences of the second reference command sequences, is 3.

[0116] It should be noted that since commands include not only command names but also command parameters, two commands are considered identical only if both command names and command parameters are identical. For example, if the third reference command CMD1 in the second reference command sequence CMD1->CMD2 is identical to CMD1 in the first target command sequence, this means that both commands have the same command name and command parameters. If the third reference command CMD1 and the target command CMD2 are different, this means that their command names and / or command parameters are different.

[0117] In the second mapping strategy, the first target command sequence can be mapped to the second target command sequence according to the command mapping table. That is, the command name and command parameters of the target command in the first target command sequence are replaced with the command name and command parameters in the command mapping table to obtain the second target command sequence. Therefore, each first reference command sequence can be matched with the second target command sequence to obtain the parameters of the occurrence of each first reference command sequence in the second target command sequence.

[0118] When determining the occurrence parameter based on the second target command sequence and the first reference command sequence, the number of different first reference command sequences included in the second target command sequence can be determined as the occurrence number; and / or the number of occurrences of each first reference command sequence in the second target command sequence can be determined. The process for determining the occurrence number and the number of occurrences is the same as that in the first mapping strategy described above and is not further described here.

[0119] S203: Determine coverage parameters of the test case according to the occurrence parameters.

[0120] The coverage parameter is used to indicate the coverage of the test case. A larger coverage parameter indicates a wider coverage of the test case and a better test case. A smaller coverage parameter indicates a smaller coverage of the test case and a worse test case.

[0121] As can be seen from the foregoing description, the occurrence parameter includes the number of occurrences and / or the number of occurrences. The coverage parameter can be calculated from the number of occurrences and / or the number of occurrences. The coverage parameter is positively correlated with both the number of occurrences and the number of occurrences. That is, when the number of occurrences increases, the coverage parameter increases; when the number of occurrences increases, the coverage parameter increases. And vice versa. Therefore, any function that satisfies this positive correlation can be constructed. The occurrence parameter corresponding to a test case can be input into this function to obtain the coverage parameter for that test case. For example, this function can be addition, multiplication, etc.

[0122] In one embodiment, the process of determining the coverage parameter may include: first, determining the number of occurrences of each first reference command sequence in the first target command sequence corresponding to the test case, and a first proportion in a preset number; then, determining the coverage parameter of the test case based on the first proportion and / or the number of occurrences, wherein the coverage parameter is positively correlated with the first proportion, and the coverage parameter is positively correlated with the number of occurrences. In other words, when the first proportion increases, the coverage parameter increases, which means that the coverage of the test case is wider and the test case is better; when the number of occurrences increases, the coverage parameter increases, which means that the coverage of the test case is wider and the test case is better. And vice versa.

[0123] Based on the above positive correlation, a positive correlation function between the coverage parameter of the test case and the first proportion and the number of occurrences may also be pre-established, for example, by addition or multiplication.

[0124] As can be seen from the preceding description, the preset number is the total number of first reference command sequences. Therefore, the first percentage represents the percentage of all first reference command sequences that appear in the first target command sequence. This eliminates the effect of the total number of first reference command sequences on the coverage parameter, thereby improving the stability of the coverage parameter's value range when the total number of first reference command sequences varies, and making the coverage parameters corresponding to different numbers of first reference command sequences comparable.

[0125] After obtaining the coverage parameters and / or corresponding occurrence parameters of the test case through the above S201 to S203, the coverage parameters and / or corresponding occurrence parameters of the test case can be written into the coverage report of the test case. The coverage report can be a file in any form, for example, a text file and a table file. The coverage report can include the coverage parameters and / or corresponding occurrence parameters of multiple test cases, and the multiple test cases can be all the test cases corresponding to the memory, or they can be test cases with smaller coverage parameters. The test cases in the coverage report can be arranged in order of coverage parameters, including ascending or descending order, which can facilitate the auxiliary chip verification engineer to quickly select one or more test cases with the smallest coverage parameters to improve the test cases and improve the quality of the test cases.

[0126] Furthermore, the test case occurrence parameters in the disclosed embodiments can assist chip verification engineers in refining test cases. Based on the occurrence parameters, chip verification engineers can determine a first reference command sequence that does not appear in the test case, thereby improving test case refinement more quickly and further increasing the efficiency of chip verification engineers in refining test cases.

[0127] In some embodiments, each first reference command sequence can also be processed to obtain at least one third reference command sequence, where the processing includes: removing commands and / or removing command parameters; then, counting the number of occurrences of the first reference command sequence including the third reference command sequence to obtain the number of occurrences of the third reference command sequence; finally, writing the third reference command sequence and the corresponding number of occurrences into the coverage report of the test case.

[0128] In the disclosed embodiment, each reference command in the first reference command sequence includes all supported parameters for the reference command. These parameters must be provided when executing the command to ensure successful execution. By removing commands and / or command parameters from these first reference command sequences, a third reference command sequence can be obtained. For example, for the first reference command sequence comprising two reference commands: WR_BGA_BKA_CA_BC8_ORD0_DM->RD_BGC_BKA_CA_BC6_ORD0_DM, the last four parameters can be removed to obtain the third reference command sequence: WR_BGA_BKA->RD_BGC_BKA.

[0129] After the above processing, one or more first reference command sequences may correspond to the same third reference command sequence. This third reference command sequence can be understood as the reference command sequence above these first reference command sequences. Therefore, the number of occurrences of this third reference command sequence can be calculated based on the number of occurrences of the corresponding one or more first reference command sequences, where the calculation is summed.

[0130] For example, the first reference command sequences WR_BGA_BKA_CA_BC8_ORD0_DM->RD_BGC_BKA_CA_BC6_ORD0_DM and WR_BGA_BKA_CB_BC8_ORD3_DM->RD_BGC_BKA_CA_BC16_ORD2_DM both correspond to the third reference command sequence WR_BGA_BKA->RD_BGC_BKA. Of course, in addition to these two, there are many other first reference command sequences that correspond to this third reference command sequence. The sum of the occurrence counts of all first reference command sequences corresponding to a third reference command sequence is the occurrence count of the third reference command sequence.

[0131] It should be noted that the aforementioned first reference command sequences WR_BGA_BKA_CA_BC8_ORD0_DM->RD_BGC_BKA_CA_BC6_ORD0_DM and WR_BGA_BKA_CB_BC8_ORD3_DM->RD_BGC_BKA_CA_BC16_ORD2_DM also correspond to the third reference command sequence WR->RD. In this case, if no other first reference command sequence corresponds to the third reference command sequence WR->RD, the number of occurrences of the third reference command sequence WR->RD is the sum of the number of occurrences of the two first reference command sequences.

[0132] Through the above-mentioned processing and statistics, the embodiment of the present disclosure can provide multiple levels of statistical results of the number of occurrences, which can assist chip verification engineers in better improving test cases. For example, the number of occurrences based on storage groups, the number of occurrences based on banks, the number of occurrences based on burst lengths, the number of occurrences based on rows, the number of occurrences based on columns, and other levels can be obtained. In this way, chip verification engineers can better improve test cases based on these multiple levels of occurrences. These multiple levels of occurrences can form a tree structure, and users can analyze the occurrences layer by layer starting from the root node.

[0133] Figure 3 This is a detailed verification process diagram provided by the embodiment of the present disclosure. Figure 3 As shown, the detailed verification process may include the following steps:

[0134] S301: Acquire all reference commands that comply with command rules, where each reference command includes a command name of the reference command and all command parameters that the reference command can support.

[0135] S302: Combine identical reference commands or different reference commands to obtain a first reference command sequence, where identical reference commands include at least two reference commands with identical command names and identical command parameters, and different reference commands include at least two reference commands with different command names and / or different command parameters; the first reference command sequence includes multiple reference commands arranged in sequence.

[0136] S303: Deleting command sequences that do not meet preset sequence rules from the first reference command sequence.

[0137] S304: Execute simulation test on the model of the memory chip through the test case to obtain a command tracking file corresponding to the test case; the command tracking file includes at least one target command used in the process of executing the simulation test and the corresponding execution time.

[0138] S305: Sort at least one target command according to execution time to obtain a first target command sequence, where the first target command sequence includes a plurality of target commands arranged in sequence.

[0139] S306: Acquire a command mapping table, where the command mapping table includes: a mapping relationship between a command name of a reference command and a command name of a target command, and a mapping relationship between command parameters of the reference command and command parameters of the target command.

[0140] S307: Map the first reference command sequence to a second reference command sequence according to the command mapping table.

[0141] S308: Determine the number of different second reference command sequences included in the first target command sequence as the number of occurrences; and / or determine the number of occurrences of each second reference command sequence in the first target command sequence.

[0142] S309: Determine a first proportion of the number of occurrences in a preset number, where the preset number is the total number of the first reference command sequence.

[0143] S310: Determine a coverage parameter of the test case according to the first proportion and / or the number of occurrences, where the coverage parameter is positively correlated with the first proportion and the coverage parameter is positively correlated with the number of occurrences.

[0144] S311: Write the coverage parameters of the test case and / or the corresponding occurrence parameters into the coverage report of the test case.

[0145] S312: Process each first reference command sequence to obtain at least one third reference command sequence, where the processing includes: removing commands and / or removing command parameters.

[0146] S313: Count the number of occurrences of the first reference command sequence including the third reference command sequence to obtain the number of occurrences of the third reference command sequence.

[0147] S314: Write the third reference command sequence and the corresponding number of occurrences into the coverage report of the test case.

[0148] It is understandable that the order of the above steps S310 to S314 can be flexibly adjusted on the basis of not being dependent on each other, and the present disclosure does not limit the order thereof. S310 to S314 can refer to the description of the corresponding steps in S201 to S203, and will not be repeated here.

[0149] Corresponding to the verification method of the above embodiment, Figure 4 This is a structural block diagram of a verification device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 4 , the verification device 400 includes:

[0150] The first command sequence acquisition module 401 is configured to acquire a first target command sequence generated when a test case executes a test on a memory chip, wherein the first target command sequence includes a plurality of target commands arranged in sequence.

[0151] The occurrence determination module 402 is configured to determine occurrence parameters of a preset number of first reference command sequences in the first target command sequence, where the first reference command sequence includes a plurality of reference commands arranged in sequence.

[0152] The coverage determination module 403 is configured to determine the coverage parameters of the test case according to the occurrence parameters.

[0153] In some implementations, the occurrence determination module 402 is further configured to:

[0154] A command mapping table is obtained, wherein the command mapping table includes: a mapping relationship between a command name of the reference command and a command name of the target command, and a mapping relationship between command parameters of the reference command and command parameters of the target command.

[0155] Determine occurrence parameters of a preset number of first reference command sequences in the first target command sequence according to the command mapping table.

[0156] In some embodiments, the occurrence parameter includes the number of occurrences and / or the number of occurrences, and the occurrence determination module 402 is further configured to:

[0157] Acquire a second reference command sequence, where the second reference command sequence is obtained by mapping the first reference command sequence in advance through the command mapping table;

[0158] determining the number of different second reference command sequences included in the first target command sequence as the occurrence number;

[0159] And / or, determining the number of occurrences of each second reference command sequence in the first target command sequence.

[0160] In some implementations, the coverage determination module 403 is further configured to:

[0161] A first proportion of the occurrence quantity in the preset quantity is determined.

[0162] A coverage parameter of the test case is determined according to the first proportion and / or the number of occurrences, where the coverage parameter is positively correlated with the first proportion and the coverage parameter is positively correlated with the number of occurrences.

[0163] In some implementations, the process of generating the first reference command sequence includes:

[0164] The reference command acquisition module is used to acquire all reference commands that comply with the command rules, each of the reference commands including the command name of the reference command and all command parameters that the reference command can support.

[0165] A command combination module is used to combine the same reference commands or different reference commands to obtain multiple first reference command sequences, wherein the same reference commands include at least two reference commands with the same command name and the same command parameters, and the different reference commands include at least two reference commands with different command names and / or different command parameters.

[0166] In some embodiments, the above apparatus further comprises:

[0167] The command sequence deletion module is configured to delete command sequences that do not meet preset sequence rules from a plurality of first reference command sequences after combining the same reference commands or different reference commands.

[0168] In some embodiments, the device further comprises:

[0169] The first report writing module is used to write the coverage parameters of the test case and / or the corresponding occurrence parameters into the coverage report of the test case.

[0170] In some embodiments, the device further comprises:

[0171] A command sequence processing module is used to process each of the first reference command sequences to obtain at least one third reference command sequence after writing the coverage parameters and / or corresponding occurrence parameters of the test case into the coverage report of the test case, wherein the processing includes: removing commands and / or removing command parameters.

[0172] The occurrence count module is configured to count the occurrence counts of the first reference command sequence including the third reference command sequence to obtain the occurrence count of the third reference command sequence.

[0173] The second report writing module is used to write the third reference command sequence and the corresponding number of occurrences into the coverage report of the test case.

[0174] In some implementations, the first command sequence acquisition module is further configured to:

[0175] A simulation test is performed on the model of the storage chip through the test case to obtain a command tracking file corresponding to the test case; the command tracking file includes at least one target command used in the process of executing the simulation test and the corresponding execution time.

[0176] The at least one target command is sorted according to the execution time to obtain the first target command sequence.

[0177] The above-mentioned verification device is a device embodiment corresponding to the above-mentioned method embodiment, and has the same implementation principle as the method embodiment. Detailed description can refer to the above-mentioned method embodiment, which will not be repeated here.

[0178] Figure 5 This is a structural block diagram of an electronic device provided by an embodiment of the present disclosure. Figure 5 As shown, the electronic device 600 includes a memory 602 and at least one processor 601 .

[0179] The memory 602 stores computer-executable instructions.

[0180] At least one processor 601 executes the computer-executable instructions stored in the memory 602 , so that the electronic device 600 implements the aforementioned verification method.

[0181] In addition, the electronic device 600 may further include a receiver 603 and a transmitter 604. The receiver 603 is configured to receive information from other devices or equipment and forward it to the processor 601. The transmitter 604 is configured to send the information to the other devices or equipment.

[0182] The electronic device 600 is an embodiment of a device corresponding to the verification method. For details, please refer to the detailed description of the embodiment of the verification method, which will not be repeated here.

[0183] The embodiment of the present disclosure further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by the electronic device 600, they are used to implement the aforementioned verification method.

[0184] The computer-readable storage medium here is an embodiment of the device corresponding to the verification method. For details, please refer to the detailed description of the aforementioned verification method embodiment, which will not be repeated here.

[0185] The embodiment of the present disclosure also provides a computer program, which is used to implement the aforementioned verification method.

[0186] The computer program here is an embodiment of the device corresponding to the verification method. For details, please refer to the detailed description of the embodiment of the verification method, which will not be repeated here.

[0187] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0188] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A verification method, characterized in that: The method comprises: Acquire a first target command sequence generated by a test case during a process of executing a test on a memory chip, wherein the first target command sequence includes a plurality of target commands arranged in sequence; determining occurrence parameters of a preset number of first reference command sequences in the first target command sequence, wherein the first reference command sequence includes a plurality of reference commands arranged in sequence; Determining coverage parameters of the test case according to the occurrence parameters; The determining of occurrence parameters of a preset number of first reference command sequences in the first target command sequence includes: Acquire a command mapping table, wherein the command mapping table includes: a mapping relationship between a command name of the reference command and a command name of the target command, and a mapping relationship between command parameters of the reference command and command parameters of the target command; determining, according to the command mapping table, occurrence parameters of a preset number of first reference command sequences in the first target command sequence; The occurrence parameter includes a number of occurrences, and determining the occurrence parameters of a preset number of first reference command sequences in the first target command sequence according to the command mapping table includes: Acquire a second reference command sequence, where the second reference command sequence is obtained by mapping the first reference command sequence in advance through the command mapping table; The number of occurrences of each second reference command sequence in the first target command sequence is determined.

2. The method according to claim 1, characterized in that The occurrence condition parameter includes an occurrence quantity, and determining the occurrence condition parameters of a preset number of first reference command sequences in the first target command sequence according to the command mapping table includes: Acquire a second reference command sequence, where the second reference command sequence is obtained by mapping the first reference command sequence in advance through the command mapping table; The number of different second reference command sequences included in the first target command sequence is determined as the occurrence number.

3. The method according to claim 2, characterized in that The determining of the coverage parameter of the test case according to the occurrence parameter includes: Determining a first proportion of the number of occurrences in the preset number; A coverage parameter of the test case is determined according to the first proportion and / or the number of occurrences, where the coverage parameter is positively correlated with the first proportion and the coverage parameter is positively correlated with the number of occurrences.

4. The method according to claim 3, characterized in that The process of generating the first reference command sequence includes: Acquire all reference commands that comply with the command rule, each of the reference commands including a command name of the reference command and all command parameters supported by the reference command; The same reference commands or different reference commands are combined to obtain multiple first reference command sequences, wherein the same reference commands include at least two reference commands with the same command name and the same command parameters, and the different reference commands include at least two reference commands with different command names and / or different command parameters.

5. The method according to claim 4, characterized in that After combining the same reference commands or different reference commands to obtain a plurality of first reference command sequences, the method further includes: The command sequences that do not meet the preset sequence rule are deleted from the first reference command sequence.

6. The method according to claim 5, characterized in that The method further comprises: The coverage parameters of the test case and / or the corresponding occurrence parameters are written into the coverage report of the test case.

7. The method according to claim 6, characterized in that After writing the coverage parameter of the test case and / or the corresponding occurrence parameter into the coverage report of the test case, the method further includes: Processing each of the first reference command sequences to obtain at least one third reference command sequence, wherein the processing includes: removing commands and / or removing command parameters; Counting the number of occurrences of the first reference command sequence including the third reference command sequence to obtain the number of occurrences of the third reference command sequence; The third reference command sequence and the corresponding number of occurrences are written into the coverage report of the test case.

8. The method according to any one of claims 1 to 7, characterized in that The obtaining of a first target command sequence formed by the test case during the process of executing the test on the memory chip includes: Performing a simulation test on the model of the memory chip using the test case to obtain a command tracking file corresponding to the test case; the command tracking file includes at least one target command used in the process of executing the simulation test and the corresponding execution time; The at least one target command is sorted according to the execution time to obtain the first target command sequence.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed by an electronic device.

Images