Regression verification method and device

Abstract

The invention discloses a regression verification method and device, which can be applied to the technical field of chip verification. The regression verification method comprises the steps that the running state of each test case is monitored in real time, and when running of the test cases is finished, running state information of the test cases is stored in a first file; the first file is accessed in real time, the running state information of one test case in the first file is taken out every time, the running state information of the test case is verified, and if verification is passed, the running state information of the test case is stored in a second file; the second file is accessed in real time, the running state information of one test case in the second file is taken out each time, and the simulation waveform of the test case is generated based on the running state information of the test case. According to the regression verification method and device provided by the invention, the real-time performance and efficiency of regression verification can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the chip verification technical field, and in particular to a regression verification method and device. BACKGROUND

[0002] In the integrated circuit technical field, before a chip is manufactured, a series of verifications are usually performed to ensure that the manufactured chip can meet the design requirements. With the increase of the design scale of integrated circuit systems, chip verification becomes more and more important to ensure the quality of the chip. Therefore, not only a large number of test cases are used to perform more sufficient verification on the design under test, but also repeated regression testing is performed on the failed test cases to find more design defects, or the latest design code modification is confirmed not to affect the original functions of the design through repeated regression testing.

[0003] In order to improve the simulation speed of regression testing in verification and save hard disk resources, the regression testing performed in the project development process generally does not generate simulation waveforms. However, due to defects of the design under test and the environment, especially in the early stage of the project, a large number of test cases fail to pass the verification, which means that the verification personnel need to spend a lot of time and effort on the analysis and debugging of the test cases that fail to pass the regression testing. However, the analysis and debugging of the test cases that fail to pass the regression testing are mainly performed after the regression testing, and often need to be recompiled to reproduce the problem, and then the debugging work can be started, which occupies a large amount of time of the verification personnel and greatly reduces the work efficiency. SUMMARY

[0004] To solve at least part of the above technical problems, the embodiments of the present application provide a regression verification method and device.

[0005] In one aspect, the embodiments of the present application provide a regression verification method, comprising: monitoring a running state of each test case in real time, and saving the running state information of the test case to a first file when the test case ends running; accessing the first file in real time, taking out the running state information of a test case in the first file each time, checking the running state information of the test case, and if the checking passes, saving the running state information of the test case to a second file; accessing the second file in real time, taking out the running state information of a test case in the second file each time, and generating a simulation waveform of the test case based on the running state information of the test case.

[0006] In some embodiments, at the end of the test case running, saving the running status information of the test case to the first file includes: at the end of the test case running, if the test case fails to run, saving the running status information of the test case to the first file, wherein the running status information of the failed test case includes identity information, running error information and identification information of the failed test case.

[0007] In some embodiments, the running status information of the test case is verified, and if the verification passes, the running status information of the test case is saved to a second file, including: checking the running error information of the failed test case for consistency with the running error information of the failed test case already in the second file, and if they are inconsistent, saving the retrieved running status information of the test case to the second file.

[0008] In some embodiments, after checking the consistency between the running error information of the failed test case and the running error information of the failed test case already in the second file, the method further includes: if they are consistent, the running status information of the retrieved test case is not saved.

[0009] In some embodiments, at the end of the test case running, the running status information of the test case is saved to the first file, including: at the end of the test case running, if the test case runs successfully, the running status information of the test case is saved to the first file, wherein the running status information of the test case that passes the run includes identity information and identification information of the run passed.

[0010] In some embodiments, the running status information of the test case is verified, and if the verification passes, the running status information of the test case is saved to a second file, including: verifying the running result of the test case based on the running status information of the test case; if it is determined that the running result of the test case is passed, the running status information of the test case is saved to a second file.

[0011] In some embodiments, generating a simulation waveform of the test case based on the running status information of the test case includes: determining the running result of the test case based on the running status information of the test case; if the running result of the test case is a running failure, generating a simulation waveform of the test case.

[0012] In some embodiments, if the running result of the test case is a running failure, the method further comprises: if the running result of the test case is a running failure, generating a running command of the test case based on the running state information of the test case; generating an execution file of the test case according to the running command of the test case; and calling the execution file to execute the test case to obtain the simulation waveform of the test case.

[0013] In some embodiments, after determining the running result of the test case based on the running state information of the test case, the method further comprises: if the running result of the test case is a running pass, deleting the simulation log of the test case in the simulation log file.

[0014] In some embodiments, the running error information comprises error step information and error cause information.

[0015] In some embodiments, the second file only allows one of the following operations to be executed at the same time: a write operation and a read operation.

[0016] In some embodiments, the first thread performs the following operations:

[0017] The first file is accessed in real time, and each time the running state information of a test case in the first file is read out, the running state information of the test case is verified, and if the verification is passed, the running state information of the test case is saved to a second file.

[0018] In some embodiments, the second thread performs the following operations:

[0019] The second file is accessed in real time, and each time the running state information of a test case in the second file is read out, and based on the running state information of the test case, a simulation waveform of the test case is generated.

[0020] In some embodiments, the method further comprises: when the first thread execution occurs unexpectedly, the first thread sets the value of a shared variable to a failure indication value and ends the thread itself; and when the second thread execution occurs unexpectedly, the second thread sets the value of a shared variable to a failure indication value and ends the thread itself.

[0021] In some embodiments, the method further comprises: when the first thread monitors that the value of a shared variable is a failure indication value, the first thread ends the thread itself; and when the second thread monitors that the value of a shared variable is a failure indication value, the second thread ends the thread itself.

[0022] On the other hand, an embodiment of the present application provides a regression verification device, including: a monitoring module, used to monitor the running status of each test case in real time, and when the test case ends, save the running status information of the test case to a first file; a first processing module, used to access the first file in real time, take out the running status information of a test case in the first file each time, verify the running status information of the test case, and if the verification passes, save the running status information of the test case to a second file; a second processing module, used to access the second file in real time, take out the running status information of a test case in the second file each time, and generate a simulation waveform of the test case based on the running status information of the test case.

[0023] An embodiment of the present application further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any of the above embodiments when executing the computer program.

[0024] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any of the above embodiments is implemented.

[0025] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method described in any of the above embodiments is implemented.

[0026] The regression verification method and device provided by the embodiments of the present application, during the regression testing process, as long as a test case is completed, the test case is processed accordingly based on the running status information of the test case in real time or quasi-real time, such as reproducing the test case and downloading the simulation waveform, without having to wait until the simulation or the entire test set is completed before searching in the simulation log file, which greatly advances the analysis and processing of the test case, especially the analysis and processing of the failed test case, and improves the real-time and efficiency of the regression verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0028] Figure 1 This is a flowchart of a regression verification method provided in one embodiment of the present application.

[0029] Figure 2 is a part flow diagram of a regression verification method provided by an embodiment of the present application.

[0030] Figure 3 is a part flow diagram of a regression verification method provided by an embodiment of the present application.

[0031] Figure 4 is a part flow diagram of a regression verification method provided by an embodiment of the present application.

[0032] Figure 5 is a part flow diagram of a regression verification method provided by an embodiment of the present application.

[0033] Figure 6 is a flow diagram of a regression verification method provided by an embodiment of the present application.

[0034] Figure 7 is a structure diagram of a regression verification device provided by an embodiment of the present application.

[0035] Figure 8 is a physical structure diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0036] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application are further described in detail below with reference to the drawings. Herein, the illustrative embodiments of the present application and the descriptions thereof are used to explain the present application, but are not used as limitations to the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be arbitrarily sequenced without conflicts.

[0037] As to the "first", "second", and the like used herein, they are not specially intended to mean the order or sequence, nor are used to limit the present application. They are only used to distinguish the elements or operations described by the same technical terms.

[0038] As to the "include", "comprise", "have", "contain", and the like used herein, they are all open terms, that is, they mean including but not limited to.

[0039] As to the "and / or" used herein, it includes any or all of the orders of the described things.

[0040] The execution subject of the regression verification method provided by the embodiments of the present application includes but is not limited to a computer.

[0041] During the regression test execution, each test case follows a uniform flow, which includes at least the stages of compilation, emulation, simulation, and checker, and the simulation log records the execution of the entire flow of all test cases in the regression test. Therefore, the file content is usually large, and the search efficiency is low.

[0042] To solve the above technical problems, Figure 1 is a flowchart of a regression verification method provided by an embodiment of the present application, as shown in the figure, the regression verification method provided by the embodiment of the present application comprises the following steps. Figure 1

[0043] S1, real-time monitoring of the running state of each test case, and saving the running state information of the test case to a first file when the test case ends running;

[0044] In step S1, some events can be registered in the flow in advance, such as a test case start running event and a test case end running event, which are used to monitor and extract the key information of the test case running process. When the test case ends running, the event is triggered regardless of whether it passes or fails, and the running state information of the test case is saved to the first file.

[0045] S2, real-time access to the first file, taking out the running state information of a test case in the first file each time, verifying the running state information of the test case, and saving the running state information of the test case to a second file if the verification passes;

[0046] In step S2, the first file is accessed in real time during the regression test, for example, it is accessed once every 5 seconds. When the first file has the running state information of a test case, the running state information of a test case is taken out from the first file. Here, to avoid repeatedly saving the running state information of the same test case to the second file, a temporary list can be used to store the identity information (such as the name of the case) of the test case that has been saved in the second file. When the running state information of the test case is read from the first file subsequently, it is verified first whether the identity information of the test case exists in the temporary list. If yes, the running state information of the test case is not saved in the second file. Then, the running state information of the test case is verified, which can include test case result verification and / or test case error type verification, etc. If the verification passes, the running state information of the test case is saved to the second file.

[0047] ​S3, accessing the second file in real time, taking out running state information of a test case in the second file each time, and generating a simulation waveform of the test case based on the running state information of the test case.

[0048] In step S3, in the regression test process, the second file is accessed in real time, for example, the second file is accessed every 10 seconds, when there is running state information of a test case in the second file, the running state information of a test case is taken out from the second file, and here, to avoid reading duplicate data, the data in the second file can be removed at the same time as the data is taken out; then, a simulation waveform of the test case is generated based on the running state information of the test case.

[0049] The regression verification method provided in the embodiments of the present application can, in the regression test process, perform corresponding processing on a test case based on running state information of the test case in real time or quasi-real time as soon as the test case ends running, for example, reproduce the test case and download a simulation waveform, without searching in a simulation log file after the simulation ends or the entire test set ends, which greatly advances the analysis and processing of the test case, especially the analysis and processing of a failed case, and improves the real-time performance and efficiency of regression verification.

[0050] In some embodiments, in step S1, the saving of the running state information of the test case to the first file when the test case ends running can include: when the test case ends running, if the test case fails to run, saving the running state information of the test case to the first file, wherein the running state information of the failed test case includes identity information, running error information and identification information of running failure.

[0051] Specifically, if the test case fails to run, the state of the failed test case is recorded in the first file, for example, running error information and identification information of running failure, wherein the running error information can include error step information (an action causing failure) and error reason information (a reason causing error, also referred to as error reporting information), and the identification information of running failure is also referred to as state information of a failed case; in addition, the identity information of the test case also needs to be recorded, which can specifically include a name of the test case, and can also include a seed number and / or a version number of the test case.

[0052] In some embodiments, when the running state information of the test case that fails to run is included in the first file, in the step S2, the verification of the running state information of the test case, if the verification passes, the running state information of the test case is saved to the second file, which can include: the running error information of the test case that fails to run is verified for consistency with the running error information of the test case that fails to run already existing in the second file, if consistent, the extracted running state information of the test case is not saved, if inconsistent, the extracted running state information of the test case is saved to the second file.

[0053] Specifically, the running error information of the test case that fails to run is verified for consistency with the running error information of the test case that fails to run already existing in the second file, if consistent, it can be considered as the same type of error, and does not need to be added to the second file, if inconsistent, it is added to the second file, in this way, it can be ensured that the test cases that fail to run added to the second file are test cases of different error types.

[0054] In some embodiments, in the step S1, the running state information of the test case is saved to the first file when the test case runs, which can include: when the test case runs, if the test case runs successfully, the running state information of the test case is saved to the first file, wherein the running state information of the test case that runs successfully includes identity information and identification information of running successfully.

[0055] Specifically, if the test case running result is successful, the identity information of the test case and the identification information of running successfully can be recorded in the first file, wherein the identity information of the test case can include the name of the test case, and can also include the seed number and / or version number of the test case, etc., and the identification information of the test case that runs successfully can also be called pass state information.

[0056] In some embodiments, when the running state information of the test case that fails to run is included in the first file, in the step S2, the verification of the running state information of the test case, if the verification passes, the running state information of the test case is saved to the second file, which can include: the running error information of the test case that fails to run is verified for consistency with the running error information of the test case that fails to run already existing in the second file, if consistent, the extracted running state information of the test case is not saved, if inconsistent, the extracted running state information of the test case is saved to the second file.

[0057] Specifically, in the step S2, after taking out the running state information of a test case each time, the running result of the test case is checked based on the running state information of the test case. For example, when the running state information of the test case contains the identification information of running through, it is determined that the running result of the test case is running through; when the running state information of the test case contains the identification information of running failure, it is determined that the running result of the test case is running failure. When it is determined that the running result of the test case is running through, the running state information of the test case is saved to the second file.

[0058] As shown in FIG. 1, in some embodiments, when the second file only contains the running state information of the test case of running failure, in the step S3, the simulation waveform of the test case is generated based on the running state information of the test case, including: Figure 2

[0059] S31a, the running command of the test case is generated based on the running state information of the test case; and the execution file of the test case is generated according to the running command of the test case.

[0060] S32a, the test case is executed by calling the execution file to obtain the simulation waveform of the test case.

[0061] Specifically, the running state information of the test case can be organized into the running command of the test case and written into an execution file run.csh, and then run.csh is called to execute the test case and download the simulation waveform. For example, the execution file run.csh can be a general execution file of all test cases, and only the name of the test case in the execution file run.csh needs to be changed to generate the execution file of the corresponding test case. At this time, in the step S3, only the name of the test case in the running state information of the test case needs to be extracted and written into the execution file run.csh to obtain the execution file of the test case.

[0062] As shown in FIG. 1, in some embodiments, when the second file contains the running state information of the test case of running failure and the test case of running success, in the step S3, the simulation waveform of the test case is generated based on the running state information of the test case, including: Figure 3

[0063] S31b, the running result of the test case is determined based on the running state information of the test case.

[0064] S32b, if the running result of the test case is running failure, the simulation waveform of the test case is generated. ​​

[0065] Specifically, when the second file includes the running status information of test cases that failed to run and test cases that successfully ran, in the above step S3, after each time the running status information of a test case in the second file is retrieved, it is necessary to first verify the running result of the test case based on the running status information of the test case. If the running result of the test case is a running failure, the simulation waveform of the test case is generated. In step S32b, the specific method of generating the simulation waveform of the test case can be referred to the description of the above steps S31a to S32a, and will not be repeated here.

[0066] In some embodiments, after the above step S31b, after determining the running result of the test case based on the running status information of the test case, the method further includes: if the running result of the test case is passed, deleting the simulation log of the test case in the simulation log file.

[0067] Specifically, simulation logs of test cases that have passed and that you do not want to keep can be deleted in real time, which can reduce the peak space usage of the disk and save disk space.

[0068] In some embodiments, the second file only allows one of the following operations to be performed at a time: a write operation or a retrieve operation. Specifically, the second file can be a locked shared list shared_list, which ensures that only write operations or retrieve operations can be performed on the shared list at a time, thereby better maintaining the shared list.

[0069] In some embodiments, the above step S2 is performed by a first thread, and the above step S3 is performed by a second thread.

[0070] like Figure 4 As shown, in some embodiments, when the above step S2 is executed by the first thread and the above step S3 is executed by the second thread, the method further includes:

[0071] S4. When an unexpected event occurs during the execution of the first thread, the first thread sets the value of the shared variable to a fault indication value and terminates its own thread;

[0072] S5. When an unexpected event occurs during the execution of the second thread, the second thread sets the value of the shared variable to a fault indication value and terminates its own thread.

[0073] Specifically, the shared variable can be a shared Boolean variable with a lock, which can control both threads to end when any of the first thread and the second thread executes unexpectedly, avoiding the spread of unexpected situations or causing dead threads. Specifically, when the first thread executes unexpectedly, such as a memory shortage, network instability, etc., the first thread sets the value of the shared variable to a failure indication value, and ends its own thread; the second thread is the same, and will not be repeated here.

[0074] As shown in Figure 5 In some embodiments, when the above step S2 is performed by the first thread and the above step S3 is performed by the second thread, the method further comprises:

[0075] S6, when the first thread monitors that the value of the shared variable is the failure indication value, the first thread ends its own thread;

[0076] S7, when the second thread monitors that the value of the shared variable is the failure indication value, the second thread ends its own thread.

[0077] Specifically, when the first thread monitors that the value of the shared variable is set to the failure indication value, it indicates that the execution of the other thread is unexpected, and then the first thread also ends its own thread; the second thread is the same, and will not be repeated here.

[0078] To better understand the present application, the regression verification method provided by the present application is described in detail below through a specific embodiment.

[0079] During the regression test execution process, each test case (case) will follow a set of unified processes (flow), which at least includes compilation, emulation on the model, simulation on the actual design, and result comparison (checker) these stages, and the simulation log is to record the execution of all case in the regression test entire flow process, so the file content is usually large, and the search efficiency is low. Therefore, some events (event) can be registered in the flow in advance, such as the event of starting running the case, the event of ending running the case, and the event of completing the flow, which are used to monitor and extract the key information of the case running process and the flow running process.

[0080] Figure 6 is a flow diagram of the regression verification method provided by an embodiment of the present application, as Figure 6As shown, in the regression test process, as soon as a failed case occurs, it can be directly processed in real time, and the case can be classified and the simulation waveform can be selectively downloaded, without the need to search in the simulation log file after writing the simulation log or after the end of the entire test set, so that the analysis and processing of the case are greatly advanced, especially for the failed case, and the verification efficiency is improved. The detailed process is as follows:

[0081] The flow registers the event of starting the case, and when a case starts running, the event is triggered, and then the information of the case is printed in the simulation log, including the name, version number, seed number and configuration information of the case; the flow also registers the event of ending the case, and when the case ends running, the event is triggered regardless of pass or fail, if the case running result is pass, the name of the case and the pass state information are recorded in the passed case shared list (a first file), if the case fails, the failed state of the case is recorded in the failed case shared list (a first file), including: the name of the case, the seed number, the version number, the step (action) causing the failure, the error information and the state information of the case failure; in addition, the flow completion event is registered, and when a round of regression test ends and exits, the event is triggered, and a flow end identifier is generated, for example, the identifier is "flow done", and the identifier is recorded in the passed case shared list and the failed case shared list.

[0082] Before each regression test starts, the passed case shared list and the failed case shared list are emptied to avoid conflicts with the contents of the last round. A script is prepared, taking the script run.csh as an example, which only needs to fill in a case command to be called to automatically run the case to generate results.

[0083] During the regression test process, two lightweight daemon threads are started and a shared list (a second file) with a lock and a shared variable with a lock are prepared. For example, they are named insert_case_process and pop_case_process, and shared_list, shared_error_flag.

[0084] The insert_case_process accesses the failed case shared list every 5 seconds, and when there is a case in the failed case shared list, the name, seed number, error step (action), and error reason of the case are extracted to form an array, which is added as an element to the head of the shared list shared_list. When the insert_case_process accesses the "flow done" output, it indicates that all the failed cases have been processed, and the string "flow done" is added as an element to the head of the shared list shared_list, and the thread ends. Specifically, after the insert_case_process extracts the name, seed number, error step (action), and error reason of the case to form an array, the array is first saved as an element in a local temporary array. When the next case is extracted, the error step (action) and error reason of the case are compared with the content recorded in the local temporary array to determine whether there is a repetition. If the error step (action) and error reason are exactly the same as those of the previously recorded case, it is considered to be the same type of error, and does not need to be added to the shared list shared_list. In this way, it is ensured that the cases added to the shared list shared_list are cases of different error types. When an unexpected event occurs in the execution of the insert_case_process, such as insufficient memory, unstable network, etc., the insert_case_process sets the shared_error_flag to true, and ends the thread itself. Or when the insert_case_process detects that the shared_error_flag is set to true, indicating that an unexpected event occurs in the execution of other threads, the insert_case_process also ends the thread itself.

[0085] pop_case_process accesses shared_list every 10 seconds. If there is an element, it takes it from the tail and organizes it into a case run command written into run.csh. Then, run.csh is called to execute the case and download the simulation waveform. After completion, the above process is repeated, and shared_list is accessed again to try to read the next element. When pop_case_process takes the "flow done" element from the tail, it indicates that all failed cases have been processed, and pop_case_process ends its own thread. If an unexpected problem occurs during the execution of pop_case_process, such as insufficient memory or unstable network, pop_case_process sets shared_error_flag to true and ends its own thread. Alternatively, if pop_case_process detects that shared_error_flag is set to true, indicating that an unexpected situation has occurred in the execution of other threads, pop_case_process will also end its own thread.

[0086] shared_list is a locked shared list that ensures only one thread can operate on it at a time, effectively maintaining the shared list. shared_error_flag can be a locked shared Boolean variable that can terminate both threads if an unexpected error occurs in either thread, preventing the unexpected situation from spreading or causing thread deadlocks.

[0087] like Figure 6 As shown, a method similar to the failed case processing method can be used to export passed cases, extract passed case information, and then delete the passed case log in the simulation log file to save disk space. In some embodiments, the passed case shared list and the failed case shared list can be two lists, storing the passed case information and the failed case information respectively; in other embodiments, the passed case shared list and the failed case shared list can be the same list, that is, the passed case information and the failed case information can be stored in the same list. When the cases in the list are subsequently processed, they can be classified according to the case operation results.

[0088] The regression verification method provided in the embodiment guarantees that once a case fails, information can be exported, the case can be reproduced and the waveform can be downloaded during the whole regression test process, without waiting for the information of the failed case to be output to the simulation log in a lagging manner, and more importantly, without searching for the information of the failed case in a large amount of simulation logs, which has very good real-time performance and efficiency; the shared list with lock and the shared variable with lock are used to control the execution and interaction of the two threads of insert_case_process and pop_case_process, and the whole process is efficient and controllable. Meanwhile, the same method can be used to delete the logs that are not intended to be retained, so as to reduce the peak space occupation of the disk and save the disk space.

[0089] Figure 7 is a structural schematic diagram of a regression verification device provided in an embodiment of the application, as shown in the figure, the regression verification device provided in the embodiment of the application comprises: Figure 7

[0090] The monitoring module 201 is configured to monitor the running state of each test case in real time, and save the running state information of the test case to a first file when the test case ends running;

[0091] The first processing module 202 is configured to access the first file in real time, take out the running state information of a test case in the first file each time, check the running state information of the test case, and save the running state information of the test case to a second file if the check passes;

[0092] The second processing module 203 is configured to access the second file in real time, take out the running state information of a test case in the second file each time, and generate a simulation waveform of the test case based on the running state information of the test case.

[0093] The regression verification device provided in the embodiment of the application performs corresponding processing on a test case in real time or quasi-real time based on the running state information of the test case as long as the test case ends running during the regression test process, for example, reproducing the test case and downloading a simulation waveform, without searching in a simulation log file after the simulation ends or the whole test set ends, which greatly advances the analysis and processing of the test case, especially the analysis and processing of a failed test case, and improves the real-time performance and efficiency of regression verification.

[0094] The device provided in the embodiment of the application can be specifically used to perform the processing flow of each regression verification method embodiment described above, and the functions thereof will not be described herein again, and the detailed description can be referred to the description of the method embodiments.

[0095] ​Figure 8 An entity structure diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 3. As shown in FIG. 3, the electronic device can include a processor 301, a communications interface 302, a memory 303, and a communications bus 304, wherein the processor 301, the communications interface 302, and the memory 303 can complete communications with each other through the communications bus 304. The processor 301 can invoke a logical instruction in the memory 303 to execute the method described in any of the above embodiments. Figure 8

[0096] In addition, the logical instruction in the memory 303 described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially, or the part that contributes to the prior art, or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disk, and various media that can store program codes.

[0097] The embodiment of the present application discloses a computer program product, which comprises a computer program stored on a non-transitory computer-readable storage medium, and the computer program comprises program instructions. When the program instructions are executed by a computer, the computer can execute the method provided by the above-mentioned method embodiments.

[0098] The embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program causes the computer to execute the method provided by the above-mentioned method embodiments.

[0099] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0100] ​The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps in the flowchart block or blocks.

[0101] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps in the flowchart block or blocks.

[0102] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps in the flowchart block or blocks.

[0103] In the description of the specification, the description of the terms "one embodiment", "one specific embodiment", "some embodiments", "for example", "exemplary", "specific exemplary", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are contained in at least one embodiment or example of the application. In the specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0104] The above-described specific embodiments further illustrate the purposes, technical solutions and beneficial effects of the application. It should be understood that the above-described specific embodiments are only for the specific embodiments of the application and are not used to limit the protection scope of the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application should be included in the protection scope of the application.

Claims

1. A regression verification method, characterized in that: include: Monitor the running status of each test case in real time, and save the running status information of the test case to a first file when the test case finishes running; accessing the first file in real time, extracting the running status information of a test case in the first file each time, verifying the running status information of the test case, and saving the running status information of the test case to a second file if the verification passes; The second file is accessed in real time, the running status information of a test case in the second file is retrieved each time, and a simulation waveform of the test case is generated based on the running status information of the test case.

2. The method according to claim 1, characterized in that When the test case is finished running, saving the running status information of the test case to the first file includes: At the end of the test case execution, if the test case execution fails, the execution status information of the test case is saved to a first file, wherein the execution status information of the failed test case includes identity information, execution error information and execution failure identification information.

3. The method according to claim 2, characterized in that The verifying the running status information of the test case, and if the verification passes, saving the running status information of the test case to a second file, includes: The running error information of the failed test case is checked for consistency with the running error information of the failed test case already in the second file. If they are inconsistent, the running status information of the taken test case is saved to the second file.

4. The method according to claim 3, characterized in that After performing consistency check between the execution error information of the test case that failed to run and the execution error information of the test case that failed to run in the second file, the method further includes: If they are consistent, the running status information of the retrieved test case will not be saved.

5. The method according to any one of claims 1 to 4, characterized in that When the test case is finished running, saving the running status information of the test case to the first file includes: At the end of the test case execution, if the test case is successfully executed, the execution status information of the test case is saved to a first file, wherein the execution status information of the successfully executed test case includes identity information and identification information of successfully executed test case.

6. The method according to claim 5, characterized in that The verifying the running status information of the test case, and if the verification passes, saving the running status information of the test case to a second file, includes: Verifying the running result of the test case based on the running status information of the test case; If it is determined that the running result of the test case is passed, the running status information of the test case is saved in a second file.

7. The method according to claim 1, characterized in that The step of generating a simulation waveform of the test case based on the running status information of the test case includes: Determining the running result of the test case based on the running status information of the test case; If the running result of the test case is a running failure, a simulation waveform of the test case is generated.

8. The method according to claim 7, characterized in that If the running result of the test case is a running failure, generating a simulation waveform of the test case includes: If the running result of the test case is a running failure, generating a running command for the test case based on the running status information of the test case; Generate an execution file for the test case according to the running command of the test case; The execution file is called to execute the test case to obtain a simulation waveform of the test case.

9. The method according to claim 7, characterized in that After determining the running result of the test case based on the running status information of the test case, the method further includes: If the running result of the test case is passed, the simulation log of the test case in the simulation log file is deleted.

10. The method according to claim 2, characterized in that The operation error information includes error step information and error cause information.

11. The method according to claim 1, wherein The second file is only allowed to perform one of the following operations at the same time: a write operation and a retrieve operation.

12. The method according to claim 1 or 11, characterized in that The following operations are performed by the first thread: accessing the first file in real time, extracting the running status information of a test case in the first file each time, verifying the running status information of the test case, and saving the running status information of the test case to a second file if the verification passes; The following operations are performed by the second thread: The second file is accessed in real time, the running status information of a test case in the second file is retrieved each time, and a simulation waveform of the test case is generated based on the running status information of the test case.

13. The method according to claim 12, characterized in that The method further comprises: When an unexpected event occurs during the execution of the first thread, the first thread sets the value of the shared variable to a fault indication value and terminates its own thread; When an unexpected event occurs during the execution of the second thread, the second thread sets the value of the shared variable to a fault indication value and terminates its own thread.

14. The method according to claim 13, wherein: The method further comprises: When the first thread detects that the value of the shared variable is a fault indication value, the first thread terminates its own thread; When the second thread detects that the value of the shared variable is a fault indication value, the second thread ends its own thread.

15. A regression verification device, characterized in that: include: A monitoring module is used to monitor the running status of each test case in real time, and save the running status information of the test case to a first file when the test case is finished running; a first processing module, configured to access the first file in real time, retrieve the running status information of a test case from the first file each time, verify the running status information of the test case, and save the running status information of the test case to a second file if the verification passes; The second processing module is used to access the second file in real time, extract the running status information of a test case in the second file each time, and generate a simulation waveform of the test case based on the running status information of the test case.

16. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 14 is implemented.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

18. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

Images