Barrel scheme for functional verification

Abstract

The invention relates to a barreled scheme for functional verification. An example is a non-transitory computer-readable storage medium that includes stored instructions. The instructions, when executed by the one or more processors, cause the one or more processors to: obtain an assignment of an assigned bucket; generating an incentive value aiming at one or more uncovered facets of the assigned buckets; simulating operation of the circuit design using the excitation value; determining whether the assigned bucket has been covered by the simulated operation of the circuit design; and enlarging the size of the assigned bucket after determining that the assigned bucket has been covered. The assigned buckets when assigned have a subset of facets of functional coverage space.

Description

Technical Field

[0001] The present disclosure relates to functional verification of circuit designs, and more particularly, to bucketizing bins of a functional coverage space for simulation in functional verification. Background Art

[0002] Designing a circuit can be a daunting process, especially for today's complex system-on-chip (SoC) circuits. Designs are typically thoroughly tested to ensure functionality, specifications, and reliability. Conducting these tests before tape-out and fabrication is very important because of the high cost and complexity of tape-out and fabrication. Fabricating a circuit that does not meet the necessary specifications, does not operate as expected, and / or is unreliable can result in a significant waste of costs. Brief Description of the Drawings

[0003] The present disclosure will be more fully understood from the detailed description given below and from the accompanying drawings of the embodiments of the present disclosure. These drawings are provided to provide knowledge and understanding of the embodiments of the present disclosure and do not limit the scope of the present disclosure to these particular embodiments. In addition, these drawings are not necessarily drawn to scale.

[0004] Figure 1A is computer code of an example test bench.

[0005] Figure 1B is computer code of an example test bench according to some examples.

[0006] Figure 2 depicts a functional verification environment according to some examples.

[0007] Figure 3 is a flowchart of a method for functional verification according to some examples.

[0008] Figure 4 depicts a flowchart of various processes used during the design and manufacture of an integrated circuit according to some examples.

[0009] Figure 5 depicts a diagram of an example computer system in which examples may operate. Detailed Description

[0010] Aspects of the present disclosure relate to a binning scheme for functional verification. Functional verification of a circuit design (which may also be referred to as a design under test (DUT)) may be performed during the design process of a circuit or system to test whether the DUT operates as expected. In this functional verification, during simulation of the operation of the DUT, stimuli may be used as inputs to the DUT such that a user may observe the functionality of the DUT generated by the stimuli. Generally, comprehensive functional verification may include simulating the operation of the DUT using a large set of stimuli, where the set of stimuli is diverse to cover many possible operation scenarios. Functional verification may use a constrained random stimulus generator to generate diverse stimuli to efficiently execute a verification plan and achieve a functional coverage goal.

[0011] In functional verification, such as in regression testing, multiple concurrent simulations may be run. For example, hundreds or even thousands of concurrent simulations may be run. Each simulation may not know the state of other concurrent simulations. Thus, each concurrent simulation targeting the same uncovered functional coverage space results in an overlap of the targeted holes. A coverage hole may be or include a set of user-defined or inferred observation points or values in a design or test bench that are not reached during simulation. A hole means that the corresponding condition is not generated and not observed during simulation. A targeted hole may be executed to verify whether the simulation has generated the condition indicated by the hole. In the case of considering a full regression, having multiple concurrent simulations target the same hole (e.g., overlap of targeted holes) may reduce the rate of coverage convergence.

[0012] Figure 1A is the computer code of an example test bench. The test bench includes a definition of a class for a functional coverage space. The class of type coverage is defined starting from line 101 to line 116. Random variables r1 and r2 are declared in lines 102 and 103. In line 105, the random variable r1 is connected to the coverage point CP1, and in line 108, the random variable r2 is connected to the coverage point CP2. As shown in line 106, the coverage point CP1 has bins with corresponding values in the range from 0 to 2047, and as shown in line 109, the coverage point CP2 has bins with corresponding values in the range from 0 to 1023. In line 111, the cross coverage point CR1 is defined as the cross product of the coverage points CP1 and CP2. In line 118, an object cov of type coverage is instantiated. In line 120, the class object cov is randomized, and in line 121, the coverage group is sampled after randomization.

[0013] In the case where multiple concurrent simulations target the same coverage space, a particular bin (such as a particular bin for covering any one of the coverage points CP1, CP2) can be hit in more than one simulation. If the unique bin is hit during a concurrent simulation run, the coverage throughput of the regression run may increase, which can be possible if each simulation run targets a different functional coverage space.

[0014] According to the examples described herein, a binning scheme is implemented, by which bins are separated from the bins of the functional coverage space. Each simulation of one or more concurrent simulations is assigned a corresponding bin. The corresponding simulation targets the uncovered space (such as holes or uncovered bins) of the assigned bin until the simulation completely covers the assigned bin. The assigned bin can be expanded (such as increasing the size) to include additional bins. Then, the simulation can target the uncovered space of the assigned bin. The simulation and expansion can be iteratively performed until the assigned bin is the functional coverage space.

[0015] The technical advantages of the present disclosure include, but are not limited to, reducing the overlap of the functional coverage space targeted across concurrent simulations. Binning enables each simulation to initially target different, separate portions of the functional coverage space. This can reduce the overlap of concurrent simulations targeting the same uncovered space. The reduction of overlap may result in faster coverage closure in the regression environment and increase the throughput of coverage closure in the regression. Therefore, the computing resources used by the simulation can be saved and used more efficiently. Other benefits and advantages can be achieved.

[0016] Figure 1B Is the computer code of an example test bench according to some examples. According to some examples, the test bench includes a definition of a class of the functional coverage space. This test bench is referred to in the context of the following various examples. The class of type coverage_reg is defined starting from line 151 to line 164. The random variable addr1 is declared on line 152. On line 154, the random variable addr1 is connected to the coverage point CP1_Low_Reg, and on line 157, the random variable addr1 is connected to the coverage point CP2_High_Reg. As shown on line 155, the coverage point CP1_Low_Reg has bins with corresponding values in the range from 0 to 2047, and as shown on line 158, the coverage point CP2_High_Reg has bins with corresponding values in the range from 2048 to 4095. On line 166, an object cov of the class of type coverage_reg is instantiated. On line 168, the class object cov is randomized, and on line 169, the coverage group is sampled after the randomization.

[0017] Figure 2FIG. illustrates a functional verification environment 200 according to some examples. The functional verification environment 200 includes a first computer system 202 on which a testbench engine 204 executes, and N second computer systems 212-1, 212-2 to 212-N (collectively or individually referred to as the "(multiple) second computer systems 212") on which corresponding simulation engines 214-1, 214-2 to 214-N (collectively or individually referred to as the "(multiple) simulation engines 214") execute. The testbench engine 204 may be or include a process executing on a processor of the first computer system 202, and the simulation engines 214 may be or include corresponding processes executing on corresponding processors of the second computer systems 212, each of which processors may be different from the processor of the first computer system 202 on which the testbench engine 204 executes. Each simulation engine 214 includes a random stimulus generator 222, a coverage biaser 224, a simulator 226, and a simulation monitor 228, although the simulation engine 214-1 is only illustrated in Figure 2 as including these components. The functional verification environment 200 also includes a database 232.

[0018] The testbench engine 204 is embodied as instructions stored on a non-transitory computer-readable medium that are executable by one or more processors of the first computer system 202. Similarly, the simulation engines 214 (and their components) are embodied as instructions stored on a non-transitory computer-readable medium that are executable by one or more processors of the corresponding second computer systems 212. Although one simulation engine 214 is shown on a corresponding second computer system 212, any of the second computer systems 212 may have one or more simulation engines 214 executing or executable thereon.

[0019] Each of the random stimulus generator 222, the coverage biaser 224, the simulator 226, and the simulation monitor 228 may be embodied as instructions stored on a non-transitory computer-readable medium that are executable by one or more processors of the corresponding second computer systems 212. Each of the random stimulus generator 222, the coverage biaser 224, the simulator 226, and the simulation monitor 228 may be a subset of the instructions (e.g., instruction modules) of the corresponding simulation engine 214.

[0020] The instructions embodying the testbench engine 204 may perform various functionalities described herein when executed by one or more processors. The instructions embodying the corresponding simulation engines 214 (including the random stimulus generator 222, the coverage biaser 224, the simulator 226, and the simulation monitor 228) may perform various functionalities described herein when executed by one or more processors.

[0021] The test bench engine 204 receives a design under test (DUT) file 242, which includes an electronic (e.g., digital) representation of the DUT. The DUT is a circuit design that is being designed and tested with the aim of fabricating a circuit represented by the circuit design. The DUT file 242 may include a hardware description language (HDL) description (such as Verilog, etc.), a register transfer level (RTL) description, a gate level description, a layout level description, etc. The test bench engine 204 also obtains (e.g., as a file or as user input) a definition of a functional coverage space, such as that shown as an example in Figure 1B The test bench engine 204 may store the DUT file 242 and the definition of the functional coverage space in the database 232.

[0022] The test bench engine 204 distributes the bins of the functional coverage space into buckets, where each bucket includes a plurality of bins. Each bin is a range of values of a corresponding observed variable. For example, the observed variable may completely cover a range from 0 to 127. Code in the test bench engine 204 (which may be provided by the user) may distribute the range into bins, e.g., such that bin 1 = [0 - 31], bin 2 = [32 - 63], bin 3 = [64 - 95], bin 4 = [96 - 127]. The range may be specified in any way to meet the target specifications, such as by the user distributing the range or the test bench engine 204 automatically distributing the range into a default number of bins. Each bucket has a subset (e.g., a proper subset) of the bins of the functional coverage space. Additionally, the bins of each bucket are different or distinct from the bins of each other bucket. No bucket includes a bin that is also included in another bucket, and no bin is included in more than one bucket.

[0023] Each bucket may be assigned a unique bucket identification (ID) number. The functional coverage space may be distributed into M buckets, where each bucket is assigned a corresponding unique ID number from 1 to M. The test bench engine 204 may distribute the functional coverage space into buckets based on any distribution scheme. In some examples, the functional coverage space is generally evenly distributed into buckets. In such examples, the buckets may have the same S bins of the functional coverage space. A given bucket with ID number X may span from bin (S*(X - 1)) to bin (S*X - 1). Consecutive buckets with ID numbers X and (X + 1) may span from bin (S*(X - 1)) to bin (S*X - 1) and from bin (S*X) to bin (S*(X + 1) - 1). For example, when the number of bins S is 10, the bucket with ID number 1 includes bins 0 to 9; the bucket with ID number 2 includes bins 10 to 19; etc. The bins of different coverage points may be digitally concatenated such that the bins may be distributed into consecutive buckets. For example, referring to Figure 1B, CP1_Low_Reg and CP2_High_Reg are digitally coupled, where CP1_Low_Reg has the lower half range (0 - 2047) of the value of the random variable addr1, while CP2_High_Reg has the upper half range (2048 - 4095) of the value.

[0024] In some examples, the testbench engine 204 may allow a user to input the desired number of bins within each bucket, and then the testbench engine 204 may determine the number of buckets based on the input number of bins within each bucket. In some examples, the testbench engine 204 may allow a user to input the desired number of buckets, and then the testbench engine 204 may determine the number of bins within the buckets based on the input number of buckets. Other distribution schemes may be implemented, such as grouping bins of coverage points based on value distribution, sampling similar expressions, or cross-sharing the same coverage points.

[0025] The testbench engine 204 may create an object that is stored in the database 232 (which may be a simple file) or other memory, and the object indicates which bins are included in the corresponding bucket. For example, the object may be indexed by the unique ID number of the bucket to indicate the lower and upper limits of the range of bins included in the corresponding bucket with that unique ID number.

[0026] The testbench engine 204 also starts the simulation engine 214 to concurrently perform a simulation of the operations on the DUT. The testbench engine 204 assigns a unique ID number to the simulation engine 214 as part of the startup. The testbench engine 204 transmits the unique ID number of the simulation to the corresponding simulation engine 214 at startup. The testbench engine 204 may transfer the DUT file 242 to the simulation engine 214 and / or may provide access to the location in the database 232 where the DUT file 242 is stored by the simulation engine 214. After the simulation, the testbench engine 204 may access the database 232 to view the results of the simulation.

[0027] The simulator 226 simulates the operations of the DUT. The simulator 226 receives the DUT file 242 (e.g., from the testbench engine 204 or the database 232). The simulator 226 uses the excitation values of the signals output from the random stimulus generator 222 to simulate the operations of the DUT of the DUT file 242.

[0028] Generally, the random stimulus generator 222 generates excitation values as signal input values, and the excitation values are used in the simulation of the operations on the DUT performed by the simulator 226. In some examples, the random stimulus generator 222 includes a problem generator, a random number generator, and a constraint solver.

[0029] A problem generator can generate a constrained problem to be solved. The problem generator can receive a constraint file that can include declarative statements of constraints for various signals for which stimulus values are to be generated for simulation. The problem generator can convert the constraints into a form readable by a constraint solver and can dynamically expand the constraints (if any such constraints are included in the constraint file) to generate a constrained problem. The problem generator can output the constrained problem to the constraint solver.

[0030] A random number generator can produce random values that can be used as seed values by a constraint solver. Random values can be output from the random number generator to a constraint solver. A constraint solver can query the random number generator to generate and output random values.

[0031] The constraint solver can receive a constraint problem and a random value. Based on the received constraint problem and random value, the constraint solver can generate a stimulus value for the signal within the constraints of the constraint file. The constraint solver can solve the constraint problem based on commands from the test bench engine 204. The constraint solver can be or include a constraint satisfiability solver. The generated stimulus value can be applied to the corresponding driver to simulate the operation of the DUT by the simulator 226.

[0032] The coverage biaser 224 accesses a database 232 of buckets assigned to the corresponding simulation engine 214. Using the unique ID number of the corresponding simulation engine 214, the coverage biaser 224 can implement a heuristic method to determine the unique ID number of the bucket assigned to the corresponding simulation engine 214. For example, the test bench engine 204 can transmit the unique ID number of the simulation engine 214 and the bucket size (e.g., the number of evenly distributed bins in each bucket) or the number of buckets to the simulation engine 214 after initializing the corresponding simulation engine 214. The coverage biaser 224 can plug the ID number of the simulation engine 214 and the bucket size or the number of buckets into a formula to determine the unique ID number of the bucket assigned to the simulation engine 214. An example formula can be assigned_ID=(sim_ID< <n)%totalBuckets,其中assigned_ID是被指派的桶的唯一ID,sim_ID是模拟引擎214的唯一ID号,totalBuckets是总的桶的数目,并且n是被指派的桶之间的间距常数。为了清楚起见,在该示例中,“<<”指示sim_ID向左移n位,并且“%”指示模数或余数运算。可以实现其他公式和启发式方法。利用被指派的桶的唯一ID号,覆盖偏置器224可以访问存储在数据库232中的对象以获取被指派的桶的面元。在一些示例中,覆盖偏置器224可以访问存储在数据库232中的完整功能覆盖空间。

[0033] The coverage biaser 224 operates with a random stimulus generator 222 (such as a constraint solver) to bias a process implemented by the random stimulus generator 222 (such as a constraint satisfaction solver process) to force one or more values to be obtained. The coverage biaser 224 may limit the random stimulus generator 222 to generate stimulus values within the bins assigned to the corresponding simulation engine 214. Additionally, the coverage biaser 224 may bias the random stimulus generator 222 to generate stimulus values in the uncovered bins of the bins assigned to the corresponding simulation engine 214 (such as previously un-simulated stimulus values). A bin is an uncovered bin if it has any value within the range of values of the bin that has not been observed (such as not been covered) in the simulation. The coverage biaser 224 may implement a machine learning process to identify coverage within the mathematical space available for stimulus values and bias the process implemented by the random stimulus generator 222 to force values to diversify the generated stimulus values. In some examples, the coverage biaser 224 implementing the machine learning process may be trained based on multiple simulations (such as different simulations) such that: when the coverage biaser 224 determines that, for example, the random stimulus generator 222 (such as a constraint solver) may converge to a set of possible repeating stimulus values, the coverage biaser 224 may bias the process implemented by the random stimulus generator 222 (such as a constraint solver).

[0034] The simulation monitor 228 observes the generated stimulus values output by the random stimulus generator 222 (such as an input to the simulator 226) and the state of the DUT during simulation by the simulator 226. The simulation monitor 228 is configured to observe the state of the DUT during simulation. The simulation monitor 228 creates a record of the results of the operation of the simulation engine 214 and writes it to the database 232. The record may include the trace of the simulation, including the stimulus values. Thus, the simulation monitor 228 may write to the database 232 which bins within the functional coverage space have been simulated and covered.

[0035] For a given simulation engine 214, once the simulation engine 214 achieves full coverage of the bins within the bin assigned to the simulation engine 214 (e.g., every value of the bins within the bin has been simulated by the simulator 226), the coverage biaser 224 can expand or increase the size of the assigned bin to include additional bins. For example, using the unique ID number of the assigned bin, the coverage biaser 224 can access the object stored in the database 232 and overwrite which bins are included in the assigned bin (e.g., such as the upper and / or lower limits of the range of the bins). Expanding or increasing the size of the bin can allow the bin to include the bins of the (multiple) adjacent bins in the functional coverage space. The coverage biaser 224 then biases the random stimulus generator 222 to target the uncovered bins within the assigned bin. When full coverage of the bins within the assigned bin is achieved, the coverage biaser 224 can expand or increase the size of the assigned bin again. The simulation engine 214 can iteratively continue to achieve full coverage of the bins within the assigned bin and expand or increase the size of the assigned bin until the size of the assigned bin is the full functional coverage space. In some examples, each time the size of the bin is increased, the size of the assigned bin is doubled. Other examples can increase the bin by other amounts.

[0036] In some examples, the size of the assigned bin can be increased by changing the upper limit of the range of the assigned bin to accommodate the increase. The upper limit can be iteratively increased to accommodate the increase in size until the upper limit reaches the upper limit of the full functional coverage space, at which point the lower limit of the range of the assigned bin can be changed to accommodate the increase. The lower limit can be iteratively decreased to accommodate the increase in size until the lower limit reaches the lower limit of the full functional coverage space. Similarly, in some examples, the lower limit can be iteratively decreased to accommodate the increase in size until the lower limit reaches the lower limit of the full functional coverage space, at which point the upper limit of the range of the assigned bin can be changed to accommodate the increase. Other examples can change both the upper and lower limits of the range of the assigned bin simultaneously. Any method for increasing the size of the assigned bin can be implemented. Once full coverage of the bin with the range of the full functional coverage space is achieved, the simulation can end.

[0037] Implementing a binning scheme can reduce the simulation engines 214 that simultaneously target the same uncovered space within the functional coverage space. When using bins, initially, each simulation engine 214 targets a portion of the functional coverage space (e.g., the bins of the corresponding assigned bin), which is different from the corresponding portions (e.g., the bins of the corresponding bin) targeted by other simulation engines 214. Thus, at least initially, there is no overlap in targeting the same uncovered space by different simulation engines 214. When the bin size is expanded to include the same uncovered space, some overlap in targeting the same uncovered space by different simulation engines 214 may subsequently occur.

[0038] In addition, in some examples, assigning buckets to the simulation engines 214 when a relatively large number of unassigned buckets are continuously located between adjacent assigned buckets can further reduce the probability that the same uncovered space within the functional coverage space is targeted by different simulation engines 214. When a relatively large number of unassigned buckets are located between adjacent assigned buckets, a given bucket may have to be expanded multiple times before it overlaps with another assigned bucket, which can provide sufficient opportunity for the simulation engine 214 to which that other bucket is assigned to fully cover the bins of that bucket. Thus, in some cases, expanding a bucket to overlap with another assigned bucket may not overlap a large amount of uncovered space within the overlapping buckets, which reduces the probability that the same uncovered space is targeted by different simulation engines 214. Thus, in some examples, the number M of buckets is greater than the number N of simulation engines 214 (e.g., concurrent simulations). Additionally, a greater ratio of the number M of buckets to the number N of simulation engines 214 can result in a more efficient simulation process.

[0039] Figure 3 is a flowchart of a method 300 for functional verification according to some examples. At 302, a DUT is obtained. The DUT can be obtained as a DUT file 242. At 304, a definition of the functional coverage space is obtained. The definition of the functional coverage space can be obtained as a file or as user input. At 306, the bins of the functional coverage space are distributed into buckets with unique ID numbers. In some examples, distributing the bins can include obtaining the bucket size of the buckets. As described above, the bins can be distributed substantially evenly among the buckets, or can be distributed by any other scheme. At 308, unique ID numbers are assigned to the concurrent simulations of the DUT to be launched. At 310, the concurrent simulations of the DUT are launched. Launching the concurrent simulations can include transmitting the unique ID number of the corresponding concurrent simulation to the corresponding concurrent simulation. The operations of 302 - 310 can be performed by the testbench engine 204. The testbench engine 204 can cause the functional coverage space and the buckets (with ID numbers) to be stored in the database 232. Additionally, the testbench engine 204 can cause the DUT (e.g., DUT file 242) to be stored in the database 232. The testbench engine 204 can launch the concurrent simulations of the DUT on the simulation engines 214.

[0040] N concurrent simulations 314-1, 314-2 to 314-N (collectively or individually referred to as "(a plurality of) simulations 314") are started, for example, at corresponding simulation engines 214. Each concurrent simulation 314 is used to simulate the operation of a DUT. Each simulation 314 includes: obtaining, at 316, the ID number of the bucket assigned to the simulation 314 of the DUT. Obtaining the ID number of the bucket may include: determining the ID number of the bucket based on the assigned ID number of the simulation 314 of the DUT. Determining the assigned ID number of the bucket may include using a heuristic method. The ID number of the simulation 314 may be used in the heuristic method to determine the assigned ID number of the bucket. Additionally, in some examples, the bucket size may be used in the heuristic method to determine the assigned ID number of the bucket.

[0041] At 318, an excitation value for simulating the operation of the DUT is generated by targeting one or more uncovered bins of the assigned bucket. For example, the coverage biaser 224 may access a functional coverage space from the database 232 and determine the uncovered bins within the assigned bucket in the functional coverage space. The coverage biaser 224 biases the random stimulus generator 222 to generate an excitation value that covers the uncovered bins within the assigned bucket.

[0042] At 320, the generated excitation value is used to simulate the operation of the DUT. For example, the simulator 226 uses the generated excitation value as an input signal to the DUT to simulate the operation of the DUT. At 322, the results from simulating the operation of the DUT are recorded. The results may include a trace that includes the excitation value. Each simulation engine 214 that executes the corresponding simulation 314 may record the corresponding results in the database 232; that is, the concurrent simulations 314 may record the corresponding results in the same database 232. Recording the results indicates which bins have been simulated and covered.

[0043] At 324, it is determined whether the assigned bucket has been fully covered. To fully cover the bucket, the operation of the DUT has been simulated using the excitation values corresponding to each bin within the bucket. If at 324 it is determined that the bucket has not been fully covered, then simulation 314 iterates to 318 to generate excitation values by targeting one or more uncovered bins of the assigned bucket, and these excitation values are used to simulate the operation of the DUT at 320. This loop continues until at 324 it is determined that the assigned bucket has been fully covered, at which point at 326 it is determined whether the assigned bucket is a fully functionally covered space. If at 326 it is determined that the assigned bucket is a fully functionally covered space, then simulation 314 terminates at 328. Although the simulation of the functional coverage space can terminate at 328, other simulations or operations for functional verification can continue. If at 326 it is determined that the assigned bucket is not a fully functionally covered space, then at 330 the size of the assigned bucket is expanded, and simulation 314 iterates to 318 to generate excitation values by targeting one or more uncovered bins of the assigned bucket, and these excitation values are used to simulate the operation of the DUT at 320. At 330 the size of the assigned bucket is expanded to include additional bins that may be located in one or more other buckets. The size of the bucket can be expanded by any amount. In some examples, the size is doubled. In some cases, the amount by which the assigned bucket can be expanded can be limited to the upper and / or lower limits of the functional coverage space.

[0044] As Figure 3A simple example of the operation of simulation 314 is assumed to start five simulations 314 (e.g., on the respective simulation engines 214 with ID numbers 1, 2, 3, 4, and 5); the functional coverage space has 200 bins (e.g., from bin 0 to bin 199); the bucket size is 10; after complete coverage of the assigned bucket, the bucket size is doubled. After initialization, assume that simulation 314-1 (with ID number 1) is assigned the bucket with ID number 1 (with a range from bin 0 to bin 9); simulation 314-2 (with ID number 2) is assigned the bucket with ID number 5 (with a range from bin 40 to bin 49); simulation 314-3 (with ID number 3) is assigned the bucket with ID number 9 (with a range from bin 80 to bin 89); simulation 314-4 (with ID number 4) is assigned the bucket with ID number 13 (with a range from bin 120 to bin 129); and simulation 314-5 (ID number 5) is assigned the bucket with ID number 17 (with a range from bin 160 to bin 169). The buckets with ID numbers 2-4, 6-8, 10-12, 14-16, 18-20 are not assigned to simulation 314. Bins 0 to 49 are continuously located between the assigned bucket with ID number 1, the unassigned buckets with ID numbers 2-4, and the assigned bucket with ID number 5; bins 40 to 89 are continuously located between the assigned bucket with ID number 5, the unassigned buckets with ID numbers 6-8, and the assigned bucket with ID number 9; and so on.

[0045] Referring to simulation 314-1, an excitation value is generated at 318, the operation of the DUT is simulated at 320, and the results are recorded at 322 until bins 0 to 9 of the bucket with ID number 1 are completely covered. Then, the size of the bucket with ID number 1 is enlarged or increased at 330 to have a range from bin 0 to bin 19. Then, an excitation value is generated at 318, the operation of the DUT is simulated at 320, and the results are recorded at 322 until bins 0 to 19 of the bucket with ID number 1 are completely covered. Then, the size of the bucket with ID number 1 is enlarged or increased at 330 to have a range from bin 0 to bin 39. Then, an excitation value is generated at 318, the operation of the DUT is simulated at 320, and the results are recorded at 322 until bins 0 to 39 of the bucket with ID number 1 are completely covered.

[0046] Then, at 330, the bin with ID number 1 is expanded or increased to have a range from bin 0 to bin 79. At this time, the bin is expanded or increased to overlap with the bin with ID number 5 assigned to simulation 314-2. Depending on the progress of simulation 314-2, the overlap of the bins with corresponding ID numbers 1 and 5 can be fully covered by simulation 314-2. If the overlap of the bins with corresponding ID numbers 1 and 5 includes uncovered bins, the uncovered bins can be simultaneously targeted at 318 by different simulations 314-1, 314-2. By initially having a sufficient number of unassigned bins between the assigned bins with corresponding ID numbers 1 and 5, simulation 314-2 can be allowed to proceed such that when the assigned bins with corresponding ID numbers 1 and 5 overlap, the overlap is fully covered, which can increase the efficiency of simulation 314. The bins assigned to simulation 314 can be similarly expanded or increased, and simulation 314 can proceed until the bins reach full functional coverage of the space.

[0047] An example of the test has two coverage points and a cross-coverage point. The two coverage points are respectively connected to random variables with 2048 bins and 1024 bins. The cross-coverage point has 2 million bins. When using the binning scheme to target the functional coverage space, each simulation performs 32 concurrent simulations and 1024 randomizations and samplings, and it takes about 7000 simulations to reach the 100% coverage target. Compared with the simulation without implementing the binning scheme, the binning scheme provides a 125% gain.

[0048] Figure 4 A set of example processes 400 used during the design, verification, and fabrication of an article such as an integrated circuit for transforming and verifying design data and instructions representing the integrated circuit is illustrated. Each of these processes can be constructed and enabled as multiple modules or operations. The term "EDA" represents the term "electronic design automation". These processes start with creating a product idea 410 using information supplied by the designer, which is transformed to create an article using a set of EDA processes 412. When the design is complete, the design is taped out 434, at which time the artwork (e.g., geometric patterns) of the integrated circuit is sent to a fabrication plant to manufacture a mask set, which is then used to manufacture the integrated circuit. After tape-out, the semiconductor die is fabricated 436, and packaging and assembly processes 438 are performed to produce 440 the finished integrated circuit.

[0049] The specification of a circuit or electronic structure can range from low-level transistor material layouts to high-level description languages. High-level representations can be used to design circuits and systems, using hardware description languages (HDLs) such as VHDL, Verilog, SystemVerilog, SystemC, MyHDL, or OpenVera. The HDL description can be transformed into a logic-level register transfer level (RTL) description, a gate-level description, a layout-level description, or a mask-level description. Each lower representation level, which is a more detailed description, adds more useful details to the design description. For example, it includes more details of the modules containing the description. The lower-level representation, which is a more detailed description, can be computer-generated, exported from a design library, or created by another design automation process. An example of a specification language at the lower-level representation language for specifying a more detailed description is SPICE, which is used to detail circuits with many analog components. The descriptions at each representation level can be used by the corresponding systems (such as formal verification systems) at that layer. The design process can use Figure 4 the sequence depicted in

[0050] During system design 414, the functionality of the integrated circuit to be fabricated is specified. The design can be optimized for desired characteristics such as power consumption, performance, area (physical and / or lines of code), and reduction in cost. At this stage, the design can be partitioned into different types of modules or components.

[0051] During logic design and functional verification 416, the modules or components in the circuit are specified using one or more description languages, and the functional accuracy of the specifications is checked. For example, the components of the circuit can be verified to generate outputs that match the requirements of the specifications of the circuit or system being designed. Functional verification can use simulators and other programs such as test bench generators, static HDL checkers, and formal verifiers. In some embodiments, a special system of components called an "emulator" or "prototyping system" is used to accelerate functional verification.

[0052] During synthesis and design for testing 418, the HDL code is transformed into a netlist. In some embodiments, the netlist can be a graphical structure where the edges of the graphical structure represent the components of the circuit and where the nodes of the graphical structure represent how the components are interconnected. Both the HDL code and the netlist are hierarchical artifacts that EDA products can use to verify whether the integrated circuit will perform according to the specified design when fabricated. The netlist can be optimized for the target semiconductor manufacturing technology. Additionally, the fabricated integrated circuit can be tested to verify that the integrated circuit meets the requirements of the specifications.

[0053] During netlist verification 420, the netlist is checked to comply with timing constraints and correspond to the HDL code. During design planning 422, an overall floorplan for the integrated circuit is constructed and analyzed for timing and top-level routing.

[0054] During placement or physical implementation 424, physical placement (the positioning of circuit components such as transistors or capacitors) and routing (connecting circuit components via multiple conductors) occur, and cells can be selected from a library to enable specific logic functions. As used herein, the term "cell" can specify a collection of transistors, other components, and interconnects that provide a Boolean logic function (e.g., AND, OR, NOT, XOR) or a storage function (such as a flip-flop or latch). As used herein, a circuit "block" can refer to two or more cells. Both cells and circuit blocks can be referred to as modules or components and can be enabled as physical structures and in simulations. Parameters (such as size) are specified for selected cells (based on "standard cells") and made accessible in a database for use by EDA products.

[0055] During analysis and extraction 426, the circuit functionality is verified at the layout level, which allows for refinement of the layout design. During physical verification 428, the layout design is checked to ensure that manufacturing constraints (such as DRC constraints, electrical constraints, lithography constraints) are correct and that the circuit device functionality matches the HDL design specifications. During resolution enhancement 430, the geometry of the layout is transformed to improve the way the circuit design is manufactured.

[0056] During tapeout, data is created for the production of lithography masks (after applying lithography enhancements where appropriate). During mask data preparation 432, the "tapeout" data is used to produce lithography masks, which are used for the finished integrated circuit.

[0057] The storage subsystem of a computer system (such as Figure 5 the computer system 500 therein) can be used to store some or all of the EDA products described herein, as well as the programs and data structures used by products that develop libraries and use the physical and logical designs of the cells in the library.

[0058] Figure 5Illustrated is an example machine of computer system 500, within which a set of instructions can be executed to cause the machine to perform any one or more of the methods discussed herein. In alternative implementations, the machine can be connected (e.g., networked) to other machines in a LAN, an intranet, an extranet, and / or the Internet. The machine can operate as a server or a client machine in a client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client machine in a cloud computing infrastructure or environment.

[0059] The machine can be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular telephone, a network device, a server, a network router, a switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, although a single machine is shown, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methods discussed herein.

[0060] Example computer system 500 includes a processing device 502, a main memory 504 (e.g., read only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM)), a static memory 506 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage device 518, which communicate with each other via a bus 530.

[0061] Processing device 502 represents one or more processors, such as a microprocessor, a central processing unit, etc. More particularly, the processing device can be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets, or a processor implementing a combination of instruction sets. Processing device 502 can also be one or more special-purpose processing devices, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. Processing device 502 can be configured to execute instructions 526 to perform the operations and steps described herein.

[0062] The computer system 500 may also include a network interface device 508 to communicate over a network 520. The computer system 500 may also include a video display unit 510 (such as a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 512 (such as a keyboard), a cursor control device 514 (such as a mouse), a graphics processing unit 522, a signal generation device 516 (such as a speaker), a graphics processing unit 522, a video processing unit 528, and an audio processing unit 532.

[0063] The data storage device 518 may include a machine-readable storage medium 524 (also referred to as a non-transitory computer-readable storage medium) having stored thereon a set or sets of instructions 526 or software embodying any one or more of the methods or functions described herein. The instructions 526 may also reside, completely or at least partially, within the main memory 504 and / or within the processing device 502 during execution thereof by the computer system 500, the main memory 504 and the processing device 502 also constituting machine-readable storage media.

[0064] In some implementations, the instructions 526 include instructions for implementing the functionality corresponding to the present disclosure. Although the machine-readable storage medium 524 is shown as a single medium in the exemplary implementation, the term "machine-readable storage medium" should be understood to include a single medium or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) storing one or more sets of instructions. The term "machine-readable storage medium" should also be understood to include any medium that is capable of storing or encoding a set of instructions for execution by a machine and that causes the machine and the processing device 502 to perform any one or more of the methods of the present disclosure. Thus, the term "machine-readable storage medium" should be understood to include, but not be limited to, solid-state memories, optical media, and magnetic media.

[0065] Some of the foregoing detailed descriptions have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm may be a series of operations that result in a desired outcome. These operations are those requiring physical manipulation of physical quantities. Such quantities may take the form of electrical or magnetic signals capable of being stored, combined, compared, and otherwise manipulated. Such signals may be referred to as bits, values, elements, symbols, characters, items, numbers, etc.

[0066] It should be remembered, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless expressly stated otherwise, as will be apparent from this disclosure, it should be recognized that throughout the description certain terms refer to the actions and processes of a computer system or similar electronic computing device that manipulates data represented as physical (electronic) quantities within the computer system's registers and memories and converts it into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage devices.

[0067] The present disclosure also relates to a device for performing the operations described herein. The device may be specially constructed for the intended purpose, or it may include a computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a non-transitory computer-readable storage medium, such as, but not limited to, any type of disk (including floppy disks, optical disks, CD-ROMs, and magneto-optical disks), read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical cards, or any type of medium suitable for storing electronic instructions, each of which is coupled to a computer system bus.

[0068] The algorithms and displays presented herein are not inherently related to any particular computer or other device. Various other systems may be used in conjunction with programs according to the teachings herein, or it may prove convenient to construct more specialized devices to perform the method. Furthermore, the present disclosure is not described with reference to any particular programming language. It will be appreciated that various programming languages may be used to implement the teachings of the present disclosure as described herein.

[0069] The present disclosure may be provided as a computer program product or software, which may include a machine-readable storage medium having instructions stored thereon that may be used to program a computer system (or other electronic device) to perform a process according to the present disclosure. A machine-readable storage medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). For example, a machine-readable (e.g., computer-readable) storage medium includes a machine-readable (e.g., computer-readable) storage medium (such as a read-only memory (ROM), a random access memory (RAM)), a magnetic disk storage medium, an optical storage medium, a flash memory device, etc.

[0070] One example is a non-transitory computer-readable storage medium that includes stored instructions. The instructions, when executed by one or more processors, cause the one or more processors to: obtain an assignment of an assigned bucket; generate incentive values for one or more uncovered bins that target the assigned bucket; use the incentive values to simulate the operation of a circuit design; determine whether the assigned bucket has been covered by the simulated operation of the circuit design; and, after determining that the assigned bucket has been covered, expand the size of the assigned bucket. The assigned bucket, when assigned, has a subset of the bins of the functional coverage space.

[0071] Another example is a system that includes a memory and a processor. The memory stores instructions. The processor is coupled to the memory and is to execute the instructions. The instructions, when executed, cause the processor to: obtain a definition of a functional coverage space for simulation of a circuit design; distribute the bins of the functional coverage space into corresponding buckets; assign an identification number to a concurrent simulation to be launched; and launch the concurrent simulation, including sending the identification number to the corresponding concurrent simulation. Each bucket among the buckets has a proper subset of the bins of the functional coverage space that is different from each other bucket among the buckets.

[0072] Yet another example is a method. The operation of a circuit design is simulated using concurrent processes on respective first processors. Each concurrent process among the concurrent processes includes: obtaining an assignment of an assigned bucket and iterating until the assigned bucket is the functional coverage space: (i) iterating until the assigned bucket has been fully covered: (a) generating incentive values for simulating the operation of the circuit design by targeting one or more uncovered bins of the assigned bucket; (b) using the incentive values to simulate the operation of the circuit design; and (c) recording the results of simulating the operation of the circuit design; and (ii) expanding the size of the assigned bucket to include additional bins of the functional coverage space. The assigned bucket of each concurrent process has a subset of the bins of the functional coverage space that is different from the assigned bucket of each other concurrent process.

[0073] In the foregoing disclosure, implementations of the present disclosure have been described with reference to their specific example implementations. Obviously, various modifications can be made thereto without departing from the broader spirit and scope of the implementations of the present disclosure as set forth in the appended claims. When the present disclosure refers to some elements in the singular tense, more than one element may be depicted in the figures, and like elements are labeled with like numbers. Accordingly, the present disclosure and the figures should be regarded in an illustrative rather than a restrictive sense.

Claims

1. A non-transitory computer-readable storage medium, including stored instructions that, when executed by one or more processors, cause the one or more processors to: Obtain an assignment of an assigned bucket, where the assigned bucket has a subset of the bins of the functional coverage space; Generate incentive values for one or more uncovered bins targeting the assigned bucket; Use the incentive values to simulate the operation of a circuit design; Determine whether the assigned bucket has been covered by the simulated operation of the circuit design; And After determining that the assigned bucket has been covered, expand the size of the assigned bucket.

2. The non-transitory computer-readable storage medium according to claim 1, where when executed by the one or more processors, the instructions cause the one or more processors to: generate the incentive values and iteratively simulate the operation of the circuit design until the assigned bucket has been covered.

3. The non-transitory computer-readable storage medium according to claim 2, where when executed by the one or more processors, the instructions cause the one or more processors to: iteratively expand the size of the assigned bucket after correspondingly determining that the assigned bucket has been covered until the assigned bucket is the functional coverage space.

4. The non-transitory computer-readable storage medium according to claim 1, where the bins of the functional coverage space are distributed into different buckets, the assigned bucket is a bucket among the different buckets, and adjacent buckets adjacent to the assigned bucket among the different buckets are not assigned.

5. The non-transitory computer-readable storage medium according to claim 1, wherein the instructions that, when executed by the one or more processors, cause the one or more processors to obtain the assignment of the assigned bucket comprise: Instructions that, when executed by the one or more processors, cause the one or more processors to use a heuristic method to determine the assigned bucket.

6. The non-transitory computer-readable storage medium according to claim 1, where when executed by the one or more processors, the instructions cause the one or more processors to: record the results of the simulation of the operation of the circuit design.

7. A system, including: A memory storing instructions; And A processor coupled to the memory and to execute the instructions, the instructions when executed causing the processor to: Obtain a definition of a functional coverage space for simulation of a circuit design; Distribute the bins of the functional coverage space into corresponding buckets, each bucket among the buckets having a proper subset of the bins of the functional coverage space different from each other bucket among the buckets; Assign an identification number to a concurrent simulation to be launched; And Launch the concurrent simulation, including sending the identification number to the corresponding concurrent simulation.

8. The system according to claim 7, where the bins are evenly distributed in the buckets.

9. The system according to claim 7, where the number of buckets is greater than the number of concurrent simulations.

10. A method, including: Using concurrent processes on respective first processors to simulate the operation of a circuit design, where each of the concurrent processes includes: Obtain the assignment of the assigned buckets, where the assigned buckets of the corresponding concurrent processes have a subset of the bins with a different functional coverage space than each of the assigned buckets of the other concurrent processes; and Iterate until the assigned buckets are the functional coverage space: Iterate until the assigned buckets have been fully covered: Generate stimulus values for simulating the operation of the circuit design by targeting one or more uncovered bins of the assigned buckets; Use the stimulus values to simulate the operation of the circuit design; and Record the results of simulating the operation of the circuit design; and Enlarge the size of the assigned buckets to include additional bins of the functional coverage space.

11. The method according to claim 10, further comprising: Obtain the definition of the functional coverage space; And Distribute the bins of the functional coverage space into buckets, each bucket of the buckets having a subset of the bins of the functional coverage space different from each other bucket in the buckets, wherein the assigned buckets of each of the concurrent processes in the concurrent processes are the corresponding buckets of the buckets.

12. The method according to claim 11, wherein the bins are evenly distributed into the buckets.

13. The method according to claim 11, wherein the number of buckets is greater than the number of concurrent processes.

14. The method according to claim 11, wherein some of the buckets are not assigned to any of the concurrent processes in the concurrent processes.

15. The method according to claim 11, wherein for each pair of adjacent buckets assigned to a corresponding concurrent process in the concurrent processes, a plurality of unassigned buckets are located between the corresponding pair of adjacent buckets, and the bins of the functional coverage space are continuous between the corresponding pair of adjacent buckets and the plurality of unassigned buckets.

16. The method according to claim 11, wherein distributing the surface elements into the bins further comprises: Assign a unique bucket identification ID number to the buckets.

17. The method according to claim 16, wherein, For each of the concurrent processes, obtaining the assigned buckets includes: determining the corresponding unique bucket ID number of the assigned buckets using a heuristic method based on the unique bucket ID number.

18. The method according to claim 17 further comprises: Assign a unique simulation ID number to the concurrent processes, wherein for each of the concurrent processes, the heuristic method uses the corresponding unique simulation ID number of the corresponding concurrent process.

19. The method according to claim 11, wherein obtaining the definition of the functional coverage space and distributing the bins of the functional coverage space into the buckets are performed by a process on a second processor different from each of the processors in the first processor.

20. The method according to claim 10, wherein recording the results of simulating the operation of the circuit design includes writing the results to a database, and the concurrent processes write the results to the same database.

Images