Method, electronic device, and medium for batch running target test cases

Abstract

The present application relates to the technical field of chip, especially to a method for batch running target test cases, electronic equipment and medium, the method comprises the following steps: step S1, obtaining user A n Corresponding parallelism B n And B n Is divided into first parallelism B1 n And second parallelism B2;Step S2, obtaining user A n Corresponding target test case list L n L n ={C1 n ,C2 n ,...,C i n ,...,C f(n) n},C i n =(C1 in ,C2 in ,C3 in );Step S3, sequentially executing each C i n Judgment: whether the current available first parallelism number corresponding to A n Is greater than or equal to C3 in , if greater, then C i n Assign C3 in Computing resource units, based on C1 in And C2 in Execute C i n , until all C i n The present application reduces the jamming of computing resources in the process of batch running target test cases, and improves the chip verification efficiency.

Description

Technical Field

[0001] This invention relates to the field of chip technology, and in particular to a method, electronic device, and medium for batch running target test cases. Background Technology

[0002] In chip verification, there are various scenarios where users need to run target test cases in batches. Target test cases can specifically include test cases that failed in regression testing, test cases requiring targeted coverage improvements, and so on. In existing technologies, each user is allocated a fixed amount of parallelism, and multiple users share a certain amount of computing resources. When multiple users run target test cases in batches, it is impossible to flexibly specify test case seeds, and when multiple users submit batch runs of target test cases, it can easily cause congestion of computing resources, reducing chip verification efficiency. Therefore, how to reduce the congestion of computing resources during the batch execution of target test cases and improve chip verification efficiency has become an urgent technical problem to be solved. Summary of the Invention

[0003] The purpose of this invention is to provide a method, electronic device, and medium for batch running target test cases, which reduces the congestion of computing resources during the batch running of target test cases and improves chip verification efficiency.

[0004] According to a first aspect of the present invention, a method for batch running target test cases is provided, comprising:

[0005] Step S1: Obtain user A n The corresponding parallelism B n And B n Divided into first parallelism B1 n Second parallelism B2 n B n For user A n The corresponding total number of computing resource units, with a first parallelism of B1. n For user A n The corresponding number of computational resource units used to execute the target test cases, the second degree of parallelism B2 n For user A n The corresponding number of computing resource units used to execute non-target test cases, where n ranges from 1 to N, and N is the total number of users, B n =B1 n +B2 n ;

[0006] Step S2: Obtain user A n The corresponding target test case list L n L n ={C1 n C2 n,...,C i n ,...,C f(n) n}, where C i n For L n The i-th target test case, where i ranges from 1 to f(n), and f(n) is A. n The corresponding total number of target test cases, C i n =(C1 in C2 in C3 in C1 in C i n The corresponding test case name is C2. in C i n Corresponding test case seed information, C3 in C i n The corresponding number of required computing resource units;

[0007] Step S3: Sequentially process each C i n Execution: Determine A n Is the current available first parallelism number greater than or equal to C3? in If it is greater than, then it is C. i n Assign C3 in Each computing resource unit, based on C1 in and C2 in Execute C i n Until all C processes are executed. i n .

[0008] According to a second aspect of the present invention, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform the method described in the first aspect of the present invention.

[0009] According to a third aspect of the present invention, a computer-readable storage medium is provided, storing computer-executable instructions for performing the method described in the first aspect of the present invention.

[0010] Compared with existing technologies, this invention has significant advantages and beneficial effects. Through the above technical solution, the method, electronic device, and medium for batch running target test cases provided by this invention achieve considerable technological advancement and practicality, and have broad industrial application value. It has at least the following beneficial effects:

[0011] This invention divides the parallelism of each user into a first parallelism for executing target test cases and a second parallelism for executing non-target test cases. During the batch execution of target test cases, computing resources are allocated only to target test cases based on the first parallelism, while the first parallelism is reserved to execute other non-target test cases. This reduces the congestion of computing resources during the batch execution of target test cases and improves chip verification efficiency. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A flowchart for batch running target test cases provided in an embodiment of the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] This invention provides a method for batch running target test cases, such as... Figure 1 As shown, it includes:

[0016] Step S1: Obtain user A n The corresponding parallelism B n And B n Divided into first parallelism B1 n Second parallelism B2 n B n For user A n The corresponding total number of computing resource units, with a first parallelism of B1. n For user A n The corresponding number of computational resource units used to execute the target test cases, the second degree of parallelism B2 n For user An The corresponding number of computing resource units used to execute non-target test cases, where n ranges from 1 to N, and N is the total number of users, B n =B1 n +B2 n .

[0017] In a scheduling system, a slot typically refers to an allocable unit of computing resources on a node (such as a CPU core, GPU slot, or container instance). Each slot can independently run a task or process. The number of slots directly determines the system's parallel processing capability, i.e., its degree of parallelism. It's important to note that the scheduling system assigns a corresponding degree of parallelism to each user, and all users share all the computing resource units within the scheduling system.

[0018] Step S2: Obtain user A n The corresponding target test case list L n L n ={C1 n C2 n ,...,C i n ,...,C f(n) n}, where C i n For L n The i-th target test case, where i ranges from 1 to f(n), and f(n) is A. n The corresponding total number of target test cases, C i n =(C1 in C2 in C3 in C1 in C i n The corresponding test case name is C2. in C i n The corresponding test case seed information, C3 in C i n The corresponding number of required computing resource units.

[0019] It should be noted that the target test case list L n The target test case in the test is user A. n The target test cases that need to be executed in batches.

[0020] Step S3: Sequentially process each C i n Execution: Determine A nIs the current available first parallelism number greater than or equal to C3? in If it is greater than, then it is C. i n Assign C3 in Each computing resource unit, based on C1 in and C2 in Execute C i n Until all C processes are executed. i n .

[0021] It should be noted that the above settings allow each user to be assigned an independent degree of parallelism for running target test cases in batches, reducing resource congestion.

[0022] As one embodiment, step S1 can directly specify each B n Set a fixed first parallelism B1 n Second parallelism B2 n It can also be customized for each B based on different time periods. n Dynamically set the first parallelism B1 n Second parallelism B2 n As one embodiment, step S1 includes:

[0023] Step S11: Obtain user A n The corresponding parallelism B n .

[0024] It should be noted that the scheduling system assigns a task to each user A. n The corresponding parallelism B was set. n When n takes different values, B n They can be equal or unequal.

[0025] Step S12, B n Divide the first time period into B1 n and the preset first time period B2 n And B1 corresponding to the preset second time period. n B2 corresponding to the preset second time period n The first time period is assumed to handle more non-target test cases than the second time period is assumed to handle more non-target test cases. The B1 value corresponding to the first time period is assumed to be... n Less than the B1 corresponding to the preset second time period n .

[0026] It should be noted that step S12 is only one example, and more B1 corresponding to different time periods can be set according to application requirements. n and B2 nIn step S12, a 24-hour day can be divided into busy and off-peak periods. In the example above, the first time period is processed as the busy period, and the second time period is the off-peak period. This is achieved by dynamically setting B1. n and B2 n This allows for more efficient use of computing resources while avoiding impact on non-target test cases, thus improving the efficiency of batch processing the target test case list.

[0027] As one embodiment, step S2 includes:

[0028] Step S21: Obtain candidate test cases. Candidate test cases include the corresponding test case name, the original seed information of the test case, and the number of computing resource units required.

[0029] Specifically, candidate test cases can be test cases that fail to run in regression testing, test cases that need targeted improvement in coverage, and so on.

[0030] Step S22: Determine if the test case name corresponding to each candidate test case exists in the preset test case definition file. If it does not exist, delete the candidate test case and generate the target test case list L. n .

[0031] It should be noted that step S22 can filter out invalid candidate test cases, ensuring that each candidate test case is a valid test case.

[0032] As one embodiment, step S2 includes

[0033] Step C21: Obtain candidate test cases. Candidate test cases include the corresponding test case name, the original seed information of the test case, and the number of computing resource units required.

[0034] Specifically, candidate test cases can be test cases that fail to run in regression testing, test cases that need targeted improvement in coverage, and so on.

[0035] Step C22: Determine if the test case name corresponding to each candidate test case exists in the preset test case definition file. If it does not exist, delete the candidate test case and generate a basic test case list {D1}. n D2 n ,...,D j n ,..,D g(n) n}, D j n Let g(n) be the j-th basic test case, where j ranges from 1 to g(n), and g(n) is the total number of basic test cases. <f(n),D jn = (D1 jn , D2 jn , D3 jn ), D1 jn is the name of the corresponding test case, D2 j n is the seed information of the corresponding basic test case, D3 jn is the number of required computing resource units corresponding to D j n jn j n [j+e×g(n)]n It should be noted that by step S22, invalid candidate test cases can be filtered out to ensure that each candidate test case is a valid test case.

[0036]

[0037] Step C23: Obtain the preset random number of times E, E > 1, f(n) = E × g(n).

[0038] Step C24: Set the current random serial number e = 0, set C1 [j+e×g(n)]n jn = D1 [j+e×g(n)]n , C2 jn = D2 [j , C3 +e×g(n)]n jn = D3 [j+e×g(n)] , and generate C n jn .

[0039] Step C25: If e < E, then set e = e + 1 and execute step C26; otherwise, execute step C27.

[0040] Step C26: Randomly generate the test case seed information H2 for D1 jn , C1 [j+e×g(n)]n = D1 jn , C2 [j+e×g(n)]n = H2 jn , C3 [j+e×g(n)]n = D3 jn , and generate C [j+e×g(n)] n , and then return to execute step C25.

[0041]

[0042] Step C27: Based on all the currently generated C [j+e×g(n)] n , generate the target test case list L in the generation order of C [j+e×g(n)] n . n

[0042] ​It should be noted that through the above settings, the randomness of the target test cases can be increased, which is convenient for improving the coverage rate. And the target test case list L is generated in the order of C [j+e×g(n)] n , making the distribution of the target test cases more balanced. n

[0043] As an embodiment, step S3 includes:

[0044] Step S31: Set i = 1.

[0045] Step S32: Judge whether the current corresponding available first parallelism degree quantity of A n is greater than or equal to C3 in . If so, execute step S33; otherwise, wait until the current corresponding available first parallelism degree quantity of A in is greater than or equal to C3, and then execute step S33.

[0046] It should be noted that the target test cases need to run based on the allocated computing resource units, and when the execution is completed, the corresponding computing resource units will be released. Therefore, the current corresponding available first parallelism degree quantity of A n is dynamically changing.

[0047] Step S33: Allocate C3 i n computing resource units for C in , obtain the corresponding simulation configuration parameters based on C1 in , and execute C i n based on the corresponding C2 i , computing resource units and simulation configuration parameters of C i n . If C i n abnormally terminates, add C i n to the re-run list, and the re-run list is initially empty.

[0048] Step S34: If i < f(n), set i = i + 1, and return to step S32; if i = f(n), execute step S35.

[0049] Step S35: If the re-run list is empty, end the process; otherwise, re-allocate the computing resource units corresponding to the first parallelism degree for each target test case in the re-run list and re-run them one by one.

[0050] It should be noted that rerunning the target test cases in the rerun list reduces simulation termination due to machine malfunctions. It should also be noted that if rerunning a target test case in the rerun list still fails, an error should be reported based on that target test case.

[0051] As one embodiment, step S32 includes:

[0052] Step S321: Obtain B1 corresponding to the current time period. n And A n The number of computing resource units X currently being used by the target test cases. n .

[0053] Step S322, if B1 n >X n And the total remaining computing resource units of the chip are greater than or equal to B1. n -X n Then B1 n -X n Determined as A n The number of available first parallelism degrees.

[0054] It should be noted that step S322 can balance the computing resources of users and the scheduling system.

[0055] As one embodiment, step S33 further includes:

[0056] Step S331: If A is preset n The corresponding waveform identifier will be used when executing C. i n During the process, C is generated i n The corresponding waveform data.

[0057] Step S332, if A is preset n The corresponding print level is then executed in C. i n During the process, according to C i n The corresponding print level is C. i The corresponding log information.

[0058] The print level is directly proportional to the amount of corresponding log information.

[0059] Step S333, place C i n The corresponding waveform data and / or C i The corresponding log information is stored in a preset file path.

[0060] It should be noted that waveform markings and print levels can be flexibly set according to specific application requirements. All C... i n The corresponding waveform data and / or C i The corresponding log information is stored in a preset file path for easy subsequent analysis.

[0061] As one embodiment, step S33 further includes:

[0062] Step S334: Trigger the statistical function in the preset statistical interface.

[0063] Step S335: Display the currently executed C in real time on the preset statistics interface. i n Waveform data and / or log information.

[0064] It should be noted that the preset statistics interface is different from the currently running C. i n The interfaces are different. Steps S334 and S335 allow for real-time acquisition of the execution status of batch-processed target test cases.

[0065] It should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. A process can be terminated when its operation is complete, but it may also have additional steps not included in the figures. A process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0066] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform the method described in this invention.

[0067] This invention also provides a computer-readable storage medium storing computer-executable instructions for performing the methods described in this invention.

[0068] This invention divides the parallelism of each user into a first parallelism for executing target test cases and a second parallelism for executing non-target test cases. During batch execution of target test cases, computing resources are allocated only to the target test cases based on the first parallelism, reserving the first parallelism for executing other non-target test cases. This reduces the congestion of computing resources during batch execution of target test cases and improves chip verification efficiency. This invention can automatically complete the simulation of planned test cases under dynamic monitoring and counting of resources, making full use of resources and reducing manual workload. For individual users, it can batch filter out failed test cases and reproduce specific failed test cases, and can also selectively supplement coverage.

[0069] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method of running target test cases in batches, characterized by, Comprising: Step S1, obtaining a user A n Corresponding parallelism B n And B n Is divided into a first parallelism B1 n And a second parallelism B2 n , B n Corresponding to the total number of computing resource units for user A n The first parallelism is B1 n Corresponding to the number of computing resource units for executing target test cases for user A n The second parallelism B2 n Corresponding to the number of computing resource units for executing non-target test cases for user A n N is the total number of users, B n =B1 n +B2 n ; Step S2: Obtain user A n The corresponding target test case list L n L n ={C1 n C2 n ,...,C i n ,...,C f(n) n }, where C i n For L n The i-th target test case, where i ranges from 1 to f(n), and f(n) is A. n The corresponding total number of target test cases, C i n =(C1 in C2 in C3 in C1 in C i n The corresponding test case name is C2. in C i n Corresponding test case seed information, C3 in C i n The corresponding number of required computing resource units; Step S3, sequentially for each C i n Perform: judge A n Is the current corresponding available first parallelism number greater than or equal to C3 in , if greater, C i n Assign C3 in Computing resource units, based on C1 in And C2 in Execute C i n , until all C i n ; The step S2 comprises: Step C21, obtaining a candidate test case, the candidate test case comprising a corresponding test case name, test case original seed information and a required number of computing resource units; Step C22, judging whether the test case name corresponding to each candidate test case exists in the preset test case definition file, if not, deleting the candidate test case, generating a basic test case list list{D1 n ,D2 n ,...,D j n ,..,D g(n) n},D j n is the jth basic test case, the value range of j is 1 to g(n), g(n) is the total number of basic test cases, g(n)<f(n), D j n =(D1 jn ,D2 jn ,D3 jn ), D1 jn is the test case name corresponding to D j n , D2 jn is the basic test case seed information corresponding to D j n , D3 jn is the required number of computing resource units corresponding to D j n ; Step C23, obtaining a preset random number of times E, E>1, f(n)=E×g(n); Step C24, set current random sequence number e = 0, set C1 [j+e×g(n)]n = D1 jn , C2 [j+e×g(n)]n = D2 jn , C3 [j+e×g(n)]n = D3 jn , generate C [j+e×g(n)] n ; Step C25, if e<E, setting e=e+1, executing step C26, otherwise, executing step C27; Step C26, for D1 jn Randomly generate test case seed information H2 jn , C1 [j+e×g(n)]n = D1 jn , C2 [j+e×g(n)]n = H2 jn , C3 [j+e×g(n)]n = D3 jn , generate C [j+e×g(n)] n Return to execute Step C25; Step C27, based on all the generated C [j+e×g(n)] n , according to C [j+e×g(n)] n the target test case list L n is generated in the generation order.

2. The method of claim 1, wherein, The step S1 comprises: Step S11, acquiring user A n Corresponding parallelism B n ; Step S12, B n corresponding to the preset first time period B1 n and the preset first time period B2 n corresponding to the preset second time period B1 n corresponding to the preset second time period B2 n , the number of non-target test cases processed in the preset first time period is more than the number of non-target test cases processed in the preset second time period, the B1 n corresponding to the preset first time period is less than the B1 n corresponding to the preset second time period.

3. The method of claim 1, wherein, The step S2 comprises: Step S21, obtaining a candidate test case, the candidate test case comprising a corresponding test case name, test case original seed information and a required number of computing resource units; Step S22, judging whether the test case name corresponding to each candidate test case exists in the preset test case definition file, if not, deleting the candidate test case, and generating the target test case list L n .

4. The method of claim 1, wherein, The step S3 comprises: Step S31, setting i=1; Step S32, judging A n whether the current corresponding available first parallelism quantity is greater than or equal to C3 in If yes, step S33 is executed, otherwise, when the current corresponding available first parallelism quantity is greater than or equal to C3 in , step S33 is executed; Step S33, C i n Assign C3 in a computing resource unit, based on C1 in Obtain the corresponding simulation configuration parameter, based on C i n The corresponding C2 i , the computing resource unit and the simulation configuration parameter execute C i n , if C i n Abnormal termination, C i n is added to the re-run list, and the re-run list is initially empty; Step S34, if i<f(n), setting i=i+1, returning to step S32, if i=f(n), executing step S35; Step S35, if the re-run list is empty, ending the flow, otherwise, re-running each target test case in the re-run list by re-allocating the computing resource units corresponding to the first parallel degree.

5. The method of claim 4, wherein, The step S32 comprises: Step S321, acquire B1 corresponding to the current time period n and A n The number of computing resource units occupied by the target test case currently running X n ; Step S322, if B1 n X n and the total remaining amount of the current computing resource units of the chip is greater than or equal to B1 n X n B1 n X n is determined as A n the current corresponding available first parallelism quantity.

6. The method of claim 4, wherein, The step S33 further comprises: Step S331: If A is preset n The corresponding waveform identifier will be used when executing C. i n During the process, C is generated i n The corresponding waveform data; Step S332, if A n corresponding print level, then in performing C i n corresponding print level, then in performing C i n corresponding print level, then in performing C i corresponding log information; Step S333, C i n Corresponding waveform data and / or C i The corresponding log information is stored under a preset file path.

7. The method of claim 4, wherein, The step S33 further comprises: Step S334, triggering a statistical function in a preset statistical interface; Step S335, presenting the current executed C i n waveform data and / or log information in the preset statistical interface in real time.

8. An electronic device, comprising: Comprising: At least one processor; And a memory connected in communication with the at least one processor; Wherein, the memory stores instructions executed by the at least one processor, the instructions are set to execute the method of any one of the preceding claims 1-7.

9. A computer-readable storage medium, characterized in that, Computer executable instructions are stored, the computer executable instructions are used to execute the method of any one of the preceding claims 1-7.

Images