Verification method and device for whole-process test, storage medium and computer device

Abstract

The present application relates to the technical field of chip design, and particularly relates to a verification method and device of full-process testing, a storage medium and computer equipment. The verification method is applied to a layout and wiring tool, and comprises the following steps: obtaining a testable version of full-process testing in a test directory at a fixed time, selecting a to-be-tested version according to the testable version, and obtaining a to-be-tested file corresponding to the to-be-tested version; writing the to-be-tested file into an initial script to obtain a full-process script; running the full-process script, and obtaining process content through an initialization method after the running is completed; reading data text in the process content, extracting result content from the data text; selecting at least a previous version of the to-be-tested version to obtain previous content of the previous version, and comparing the result content with the previous content. The present application solves the technical problem that the existing single-point testing cannot verify full-process testing as a whole, so that comprehensive testing results are obtained.

Description

[Technical Field]

[0001] This invention relates to the field of chip design technology, and in particular to a verification method, apparatus, storage medium, and computer equipment for end-to-end testing. [Background Technology]

[0002] In the development of digital backend tools, in order to ensure the integrity and functionality of the tools, full-process testing is required in each daily development version.

[0003] The entire digital backend testing process includes initialization, placement and routing, circuit delay information extraction, static timing analysis, and layout physical verification. It also requires understanding the output of each stage, obtaining parameters such as runtime and version number. By using this data, daily version quality can be quickly analyzed. Throughout the development cycle, this daily data stored in the database helps determine version quality changes, providing engineers with data-driven references and support.

[0004] Existing EDA tools or other Linux desktop software testing processes can only verify placement and routing tools through single-point testing, and cannot verify them through comprehensive testing. In response, this invention takes a holistic approach, including testing, data collection, parsing, storage, and comparison, to verify the full-process testing results of placement and routing tools. [Summary of the Invention]

[0005] To address the technical problem that existing single-point testing cannot verify the entire process testing as a whole and thus obtain comprehensive test results, this invention provides a verification method, apparatus, storage medium, and computer equipment for the entire process testing.

[0006] The solution to the technical problem of this invention is to provide a verification method for full-process testing, applied to placement and routing tools, for overall verification of the entire process testing of EDA tools or other Linux desktop software, including the following steps:

[0007] A Jenkins tool is provided that can build triggers on a schedule. The triggers periodically retrieve testable versions of the full-process tests in the test directory, select a version to be tested based on the testable versions, and obtain the test file corresponding to the version to be tested.

[0008] Write the file to be tested into the initial script to obtain the full process script;

[0009] Run the full-process script, and after completion, obtain the process content through the initialization method of the Python language;

[0010] The data text in the process content is read using the Python language, and the result content is extracted from the data text using regular expressions.

[0011] Select at least one previous version of the version to be tested, obtain the previous content of the previous version, and then compare the result content with the previous content.

[0012] Preferably, the full-process script is run, and its steps include layout planning, clock routing, and general signal routing.

[0013] Preferably, the results include the performance metrics of the version under test, which include at least the chip utilization, minimum performance, and performance range.

[0014] Preferably, the regular expression is selected from the regular expression standard library; the result content and the previous content are stored in the database, the previous content is stored in the database earlier than the result content, and the version corresponding to the previous content is any one or more versions prior to the version to be tested.

[0015] Preferably, after extracting the result content from the data text, the method further includes the following steps:

[0016] The results will be output as a current parameter report.

[0017] Preferably, after performing the verification method, the method further includes the following steps:

[0018] Output the current test status and send the current test status via message push interface. The current test status includes at least one of the following: test results, test report, and runtime.

[0019] Preferably, when the full-process script executes at least one step of layout planning, clock routing, and general signal routing in sequence, it exports a corresponding log file, which is used to view system problems during the execution of the full-process script.

[0020] The present invention also provides a verification apparatus for full-process testing, used to perform the steps of the above verification method, including:

[0021] The version acquisition module is used to periodically acquire the testable versions of the full-process test in the test directory, select the version to be tested based on the testable versions, and obtain the test file corresponding to the version to be tested.

[0022] The script update module is used to write the file to be tested into the initial script to obtain the full-process script.

[0023] The script execution module is used to run the entire process script and obtain the process content through the initialization method after the execution is completed.

[0024] The content reading module is used to read the data text in the process content and extract the result content from the data text.

[0025] The data comparison module is used to select at least one previous version of the version under test, obtain the previous content of the previous version, and then compare the result content with the previous content.

[0026] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the processor performs the steps of the above-described verification method.

[0027] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores program instructions;

[0028] When the program instructions are executed by the processor, the processor performs the steps of the verification method described above.

[0029] Compared with existing technologies, the verification method, apparatus, storage medium, and computer equipment for end-to-end testing provided by this invention have the following advantages:

[0030] 1. This invention provides a verification method for full-process testing, applied to a placement and routing tool, comprising the following steps: periodically obtaining testable versions of the full-process test in the test directory; selecting a version to be tested based on the testable versions to obtain a test file corresponding to the version to be tested; writing the test file to be tested into an initial script to obtain a full-process script; running the full-process script, and obtaining the process content through an initialization method after the script has run; reading the data text in the process content and extracting the result content from the data text; selecting at least one previous version of the version to be tested, obtaining the previous content of the previous version, and then comparing the result content with the previous content.

[0031] Understandably, the key point of this invention lies in verifying the entire process testing of EDA tools or other Linux desktop software as a whole. Through this design, it is possible to quickly analyze whether the version quality is up to standard. Throughout the development cycle, changes in version quality can be judged by data comparison to provide data reference and support. This solves the technical problem that existing single-point testing cannot verify the entire process testing as a whole and thus obtain comprehensive test results.

[0032] 2. The verification method for full-process testing provided in this embodiment of the invention runs a full-process script, and its steps include layout planning, clock routing, and ordinary signal routing.

[0033] It should be noted that the process of running the full-process script involves at least floorplanning, clock routing, and general signal routing. Floorplanning is the process of placing the control unit module of the chip and determining the overall placement of various functional circuits. Floorplanning can affect the final area of ​​the chip. Clock routing, also known as clock tree synthesis (CTS), is the separate routing of the clock signal. General signal routing includes the routing between various standard cells (i.e., basic logic gates).

[0034] 3. The verification method for full-process testing provided in this embodiment of the invention includes performance index values ​​of the version under test. These performance index values ​​include at least chip utilization, minimum performance, and performance range. It should be noted that the performance index values ​​include, but are not limited to, chip utilization, minimum performance, and performance range. Chip utilization is presented using a graphical interface to analyze whether the layout and routing area is sufficient. The chip's minimum performance (Worst Negative Slack, WNS) and performance range (Total Negative Slack, TNS) are timing analysis metrics primarily used to measure chip performance.

[0035] 4. The verification method for full-process testing provided in this embodiment of the invention uses regular expressions to extract the result content from the data text; the result content and the previous content are stored in the database, and the previous content was stored in the database earlier than the result content.

[0036] Understandably, by defining a model to complete the creation of the database, and then by establishing a connection between the database and the result content, the result content extracted by the regular expression is stored in the database. The database can store both previous content and result content at the same time. After multiple storage processes, they are combined to form a large data table, thereby obtaining the improvement process of key data in the iteration process of the version under test, and providing a visual chart structure for reference.

[0037] 5. The verification method for the full-process testing provided in this embodiment of the invention, after extracting the result content from the data text, further includes the following step: outputting the result content as a current parameter report. It should be noted that the current parameter report contains at least the data of the result content, but in order to compare the version under test with the previous version and quickly analyze the quality effect of the version under test, relevant data of the previous content can also be included. Then, the current parameter report is rendered on the page using a dedicated page display template.

[0038] 6. The verification method for full-process testing provided in this embodiment of the invention further includes the following steps after executing the verification method: outputting the current test status and sending the current test status through a message push interface. The current test status includes at least one of the following: test result, test report, and running time.

[0039] Understandably, in the process of executing the verification method, in addition to providing a detailed report on the current parameters, the current test status is also provided. The current test status focuses on the final result, and the test results of each stage in the process can be shared with employees in different departments according to the different divisions of labor in the enterprise, so as to provide data support for the next verification.

[0040] 7. The verification method for full-process testing provided in this embodiment of the invention exports log files when the full-process script executes at least one step of layout planning, clock routing, and general signal routing. These log files are used to view system problems encountered during the execution of the full-process script. It should be noted that corresponding log files can be exported at each stage, and these log files can be used for tracing and comparing problems encountered during the execution of the full-process script.

[0041] 8. This embodiment of the invention also provides a verification device for full-process testing, used to execute the steps of the above verification method, including: a version acquisition module, used to periodically acquire testable versions of full-process testing in the test directory, select a version to be tested according to the testable versions, and obtain a test file corresponding to the version to be tested; a script update module, used to write the test file to be tested into an initial script to obtain a full-process script; a script execution module, used to run the full-process script, and obtain the process content through an initialization method after execution; a content reading module, used to read the data text in the process content, and extract the result content from the data text; and a data comparison module, used to select at least one previous version of the version to be tested, obtain the previous content of the previous version, and then compare the result content with the previous content. This verification device for full-process testing has the same beneficial effects as the above verification method, and will not be described in detail here.

[0042] 9. This embodiment of the invention also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program causes the processor to perform the steps of the verification method described above. This computer-readable storage medium has the same beneficial effects as the verification method described above, and will not be repeated here.

[0043] 10. This embodiment of the invention also provides a computer device, including a memory and a processor, wherein the memory stores program instructions; when the program instructions are executed by the processor, the processor performs the steps of the verification method described above. This computer device has the same beneficial effects as the verification method described above, and will not be repeated here. [Attached Image Description]

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating a verification method for full-process testing provided in the first embodiment of the present invention.

[0046] Figure 2 This is a flowchart illustrating the execution of step S43 after step S42 in a verification method for full-process testing provided in the first embodiment of the present invention.

[0047] Figure 3 This is a flowchart illustrating step S6 in a full-process testing verification method provided in the first embodiment of the present invention.

[0048] Figure 4 This is a schematic diagram of the framework of a verification device for full-process testing provided in the second embodiment of the present invention.

[0049] Figure 5 This is a schematic diagram of a computer-readable storage medium provided in the third embodiment of the present invention.

[0050] Figure 6 This is a schematic diagram of the framework of a computer device provided in the fourth embodiment of the present invention.

[0051] Explanation of reference numerals in the attached diagram:

[0052] 1. Verification device; 11. Script acquisition module; 12. Script update module; 13.

[0053] 14. Script execution module; 15. Content reading module; 16. Data comparison module;

[0054] 2. Computer-readable storage medium; 21. Computer program;

[0055] 3. Computer equipment; 31. Memory; 311. Program instructions; 32. Processor.

Detailed Implementation Methods

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0057] In the embodiments provided by this invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.

[0058] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to the invention.

[0059] In various embodiments of the present invention, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0060] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It is particularly important to note that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0061] With the massive development of integrated circuit scale, today's chip design has reached a trillion-gate level of integration, which is extremely difficult to complete by hand. As a result, placement and routing tools have emerged.

[0062] EDA tools, specifically placement and routing tools, are crucial for chip design and play a vital role in the chip design process. They are primarily used to map logic circuits onto the actual hardware of the chip, including the allocation and placement of internal chip resources, signal line routing, and timing optimization.

[0063] In the past, the entire testing process relied heavily on a large amount of manpower to manually run scripts on the server and quantitatively search for key log and report information, which was time-consuming, labor-intensive, and involved a lot of repetitive work.

[0064] Please see Figure 1 The first embodiment of the present invention provides a verification method for full-process testing, applied to a placement and routing tool, including the following steps:

[0065] S1: Periodically retrieve the testable versions of the full-process test in the test directory, select the version to be tested based on the testable versions, and obtain the test file corresponding to the version to be tested;

[0066] S2: Write the file to be tested into the initial script to obtain the full process script;

[0067] S3: Run the entire process script and retrieve the process content through the initialization method after the script is completed;

[0068] S4: Read the data text in the process content and extract the result content from the data text;

[0069] S5: Select at least one previous version of the version to be tested, obtain the previous content of the previous version, and then compare the result content with the previous content.

[0070] Understandably, the key point of this invention lies in verifying the entire process testing of EDA tools or other Linux desktop software as a whole. Through this design, it is possible to quickly analyze whether the version quality is up to standard. Throughout the development cycle, changes in version quality can be judged by data comparison to provide data reference and support. This solves the technical problem that existing single-point testing cannot verify the entire process testing as a whole and thus obtain comprehensive test results.

[0071] It should be noted that in step S1 of this embodiment, the periodic acquisition of the testable version is achieved through the Jenkins tool. Jenkins, as a commonly used open-source continuous integration (CI) tool, can be used for continuous and automated build tasks, including but not limited to compilation and packaging, and automated testing.

[0072] Specifically, obtaining the testable version in the test directory is done through Jenkins' scheduled build trigger, which automatically retrieves the testable version file in a fixed path. Jenkins supports setting a time period to retrieve the testable version and run the full-process script. The path of the testable version file remains unchanged. Since Jenkins compiles executable files, by configuring the path of the testable version file as a Linux path variable, the file can be run directly. The trigger needs to determine at least whether the version file for the current day has been generated.

[0073] In step S2 of this embodiment, the initial script is a script that does not store any data. The data in the file to be tested is written into the initial script using TCL (Tool Command Language). The initial data can be written using this scripting language. The initial data includes, but is not limited to, the relevant environmental parameters of the test environment and the data source path corresponding to the initial data.

[0074] In step S3 of this embodiment, after the entire process script has been run, the process content can be obtained through the initialization method. The initialization method, i.e., init(), is a special method in the Python language. It can be used to obtain relevant data in the entire process script, including parameters such as task name, task information, and output results.

[0075] It should be noted that, as another implementation of the first embodiment of the present invention, the relevant data in the full-process script can also be obtained through other languages, such as Java, C#, etc. The task name, task information, output results and other parameters can be obtained by simply using the Jenkins API package of these languages.

[0076] Comparatively, Python is more concise in terms of code, and it has good and extensive support for commonly used tools that run full-process scripts, such as CI tools like Jenkins, databases like MySQL, and regular expressions. It is highly compatible, simple, universal, and suitable for subsequent maintenance and modification.

[0077] Optionally, the data text in the process content can be read using the Python language, and then the result content can be extracted from the data text using regular expressions. Before extraction, the result content is in a relatively complex and messy state. Regular expressions can be used to extract and integrate the data required by each stage of the script execution. It should be noted that the regular expressions are selected from the regular expression standard library.

[0078] In step S5 of this embodiment, the version corresponding to the previous content can be any one or more versions prior to the version to be tested. In other words, the previous content can be related data of each version.

[0079] Furthermore, the results are compared with previous results to reveal the improvement process of key data in the test version during the iteration process, and a visual chart structure is provided for users to refer to.

[0080] As an optional implementation, a full-process script is run, the steps of which include layout planning, clock routing, and general signal routing.

[0081] It should be noted that the process of running the full-process script includes at least the following steps in sequence: layout planning, clock routing, and general signal routing.

[0082] The full-process script first performs floorplanning, which is the process of placing the chip's control unit module and determining the overall placement of various functional circuits. During the floorplanning stage, the placement of power lines, memory, and I / O devices is completed sequentially, and this stage can affect the final area of ​​the chip.

[0083] It should be noted that routing is carried out after the layout planning is completed. During routing, clock routing is carried out first, and then ordinary signal routing is carried out. This is determined by the special nature of the technology in digital back-end.

[0084] Clock routing, also known as clock tree synthesis (CTS), is the separate routing of clock signals; while ordinary signal routing includes the routing between various standard units (i.e., basic logic gates).

[0085] Understandably, layout planning, clock routing, and general signal routing will directly affect the size of the chip area. Therefore, it is necessary to understand the output of each step in the entire process script execution, and finally analyze whether the quality of the version under test is up to standard.

[0086] Furthermore, the results include performance metrics for the tested version, which at least include chip utilization, minimum performance, and performance range.

[0087] It should be noted that performance metrics include, but are not limited to, chip utilization, minimum performance, and performance range.

[0088] Chip utilization is the proportion of the total chip size occupied by components such as memory and storage units. Utilization is presented through a graphical interface and used to analyze whether the area of ​​the layout and routing is sufficient. The chip's Worst Negative Slack (WNS) and Total Negative Slack (TNS) are timing analysis metrics primarily used to measure chip performance.

[0089] Specifically, regular expressions are used to extract the result content from the data text; the result content is stored in the database along with the previous content, with the previous content being stored in the database earlier than the result content.

[0090] Understandably, by defining a model to complete the creation of the database, and then by establishing a connection between the database and the result content, the result content extracted by the regular expression is stored in the database. The database can store both previous content and result content at the same time. After multiple storage processes, they are combined to form a large data table, thereby obtaining the improvement process of key data in the iteration process of the version under test, and providing a visual chart structure for reference.

[0091] Optionally, the technical solution provided in the first embodiment of the present invention uses a MySQL database to store the result content.

[0092] Understandably, a database is a way to "persistently" store the above results. Persistence is a mechanism for converting program data between persistent and transient states. In layman's terms, it is to persist transient data into persistent data. Transient data can be data in memory, which cannot be permanently saved under normal circumstances. Persistent data is the transient data stored in the database to achieve long-term preservation.

[0093] The results persisted in the database can be queried. Furthermore, by combining the data with existing open-source BI (Business Intelligence) tools, the data can be transformed into visual information to help companies make decisions. Mainstream BI tools include Metabase, Superset, and Redash, which can be used to present data on web dashboards.

[0094] Specifically, in the first embodiment of the present invention, the official Docker image of the Metabase tool is selected. Through one-click configuration, login and connection to the MySQL database, various types of data visualization charts and reports, such as bar charts, line charts, pie charts, maps and so on, can be created using the persistent results in the database.

[0095] For example, this invention analyzes the number of buffer units and inverter units in the clock routing process using a visual chart (line graph). The horizontal axis of the line graph represents the time the result content is stored in the database, and the vertical axis represents the number of buffer units and inverter units. Buffers and inverters are circuits that amplify or attenuate the input signal and invert the logic state of the input signal, respectively. As chip development progresses, reducing the number of these two types of circuit units can save chip layout resources while achieving the same effect. This means that the clock routing algorithm requires fewer buffer units or inverter units, and over time, the chip yield and quality can be effectively improved.

[0096] Please see Figure 2 In the method provided in the first embodiment above, step S4 includes S41 and S42:

[0097] S41: Read the data text from the process content;

[0098] S42: Extract the result content from the data text.

[0099] As another optional implementation, after extracting the result content from the data text (i.e., after step S42), the following steps are also included:

[0100] S43: Output the result as the current parameter report.

[0101] It should be noted that the current parameter report contains at least the data of the results, but in order to compare the version under test with the previous version and quickly analyze the quality effect of the version under test, relevant data of the previous content can also be included. Then, the current parameter report is rendered on the page using a dedicated page display template.

[0102] Optionally, by outputting the current parameter report, the running results and previous content to be compared can be presented together on the view page, and these parameters can be distinguished according to their importance to facilitate comparison by users; specifically, the parameters can be distinguished by controlling the background color, bold text, and underline display to achieve special marking, thereby making it easier for users to view the current parameter report.

[0103] The current parameter report includes, but is not limited to, parameters such as runtime, leakage power, total number of cells and utilization rate in the layout stage, total wire length of the wiring stage, number of vias, and number of design rule violations.

[0104] Please see Figure 3 After performing the above verification method, the following steps are also included:

[0105] S6: Output the current test status and send the current test status via the message push interface.

[0106] It should be noted that the current test status includes at least one of the following: test results, test report, and runtime.

[0107] Understandably, in the process of executing the verification method, in addition to providing a detailed report on the current parameters, the current test status is also provided. The current test status focuses on the final result, and the test results of each stage in the process can be shared with employees in different departments according to the different divisions of labor in the enterprise, so as to provide data support for the next verification.

[0108] In a preferred embodiment of the first embodiment of the present invention, the message push interface is Lark Robot. After the steps of the above method are completed, the final result is obtained and notified to the relevant responsible personnel within the enterprise as the current test status.

[0109] Furthermore, the full-process script exports a log file when performing at least one of the steps of layout planning, clock routing, and general signal routing.

[0110] Understandably, log files are used to view system problems during the execution of the entire script process. Corresponding log files can be exported at each stage, and these log files can be used to trace and compare problems encountered during the execution of the entire script process.

[0111] It should be noted that the log file includes detailed checks at each stage, specifically system information checks, CPU checks, memory checks, hard disk checks, and network checks.

[0112] Please see Figure 4 The second embodiment of the present invention provides a verification device 1 for full-process testing, which is used to perform the steps of the above verification method, including a version acquisition module 11, a script update module 12, a script execution module 13, a content reading module 14, and a data comparison module 15.

[0113] In the second embodiment of the present invention, firstly, the version acquisition module 11 periodically acquires the testable versions of the full-process test in the test directory, selects the version to be tested according to the testable versions, and obtains the test file corresponding to the version to be tested;

[0114] In the second embodiment of the present invention, the script is written in TCL language. TCL (Tool Command Language) is a scripting language. The script writing process will not be described in detail here. The focus of the present invention is on the updating and running of the script. The script update module 12 writes the file to be tested obtained by the version acquisition module 11 into the initial script, thereby obtaining the full-process script.

[0115] It should be noted that the version update module 12 can write the initial data in the file to be tested into the initial script that has not stored data. The initial data includes, but is not limited to, the relevant environment parameters of the test environment and the data source path corresponding to the initial data.

[0116] After obtaining the full-process script, the script execution module 13 runs the full-process script. After the execution is completed, the process content is obtained through the initialization method. In this process, the relevant data, including task name, task information, output results, etc., are obtained through the initialization method of Python language.

[0117] Optionally, the content reading module 14 reads the data text in the process content using Python, and then uses regular expressions to extract the result content from the data text. The regular expressions are used in the regular expression standard library. The extracted result content is stored in the database. By establishing a database connection, the data extracted by the regular expressions can be stored in the database. After being stored multiple times, this data forms a large data table.

[0118] The data comparison module 15 and the result content stored in the database in the content reading module 14 work together. First, at least one previous version of the version to be tested is selected to obtain the previous content of the previous version. Then, the result content is compared with the previous content to obtain the improvement process of key data of the version to be tested during the iteration process, and a visual chart structure is provided for reference.

[0119] Please see Figure 5 The third embodiment of the present invention provides a computer-readable storage medium 2 on which a computer program 21 is stored. When the computer program 21 is executed by a processor, the processor performs the steps of the above-described verification method.

[0120] Understandably, the computer-readable storage medium 2 in the third embodiment of the present invention stores a computer program 21, which can be called by a processor to execute a full-process testing verification method described in the first embodiment above.

[0121] It should be noted that the computer-readable storage medium 2 of the third embodiment of the present invention has the same beneficial effects as the verification method of the full-process test described above, and will not be repeated here.

[0122] Specifically, the computer-readable storage medium 2 of the third embodiment of the present invention may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, disk, optical disk, etc.

[0123] Optionally, the computer-readable storage medium 2 of the third embodiment of the present invention includes a non-volatile computer-readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs, and modules.

[0124] Specifically, the computer-readable storage medium 2 has storage space for computer programs 21 that perform any of the method steps described above, which can be read from or written to one or more computer program products.

[0125] Optionally, computer program 21 can compress the data in an appropriate manner.

[0126] Please see Figure 4 and Figure 6 The fourth embodiment of the present invention provides a computer device 3, including a memory 31 and a processor 32, wherein the memory 31 stores program instructions 311;

[0127] When program instruction 311 is executed by processor 32, processor 32 performs the steps of the above verification method.

[0128] Understandably, when the computer device 3 in the fourth embodiment of the present invention is running, the processor 32 can implement the full-process testing verification method described in the first embodiment when running program instructions 311.

[0129] Specifically, the computer device 3 in the fourth embodiment of the present invention can have data processing capabilities, and the memory 31 of the computer device 3 can store various program modules that constitute the verification device 1 in the third embodiment, for example, Figure 4 The module shown includes version acquisition module 11, script update module 12, script execution module 13, content reading module 14, and data comparison module 15. The program instructions 311, comprised of these modules, cause the processor 32 to execute the steps of the full-process testing verification method described in the first embodiment of this application.

[0130] It should be noted that the computer device 3 of the fourth embodiment of the present invention has the same beneficial effects as the verification method of the above-described full-process test, and will not be described in detail here.

[0131] Optionally, the processor 32 in the fourth embodiment of the present invention can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, which can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0132] Preferably, the method steps disclosed in the embodiments of this application can be implemented by a hardware processor, or by a combination of hardware and software modules in the processor.

[0133] Compared with existing technologies, the verification method, apparatus, storage medium, and computer equipment for end-to-end testing provided by this invention have the following advantages:

[0134] 1. This invention provides a verification method for full-process testing, applied to a placement and routing tool, comprising the following steps: periodically obtaining testable versions of the full-process test in the test directory; selecting a version to be tested based on the testable versions to obtain a test file corresponding to the version to be tested; writing the test file to be tested into an initial script to obtain a full-process script; running the full-process script, and obtaining the process content through an initialization method after the script has run; reading the data text in the process content and extracting the result content from the data text; selecting at least one previous version of the version to be tested, obtaining the previous content of the previous version, and then comparing the result content with the previous content.

[0135] Understandably, the key point of this invention lies in verifying the entire process testing of EDA tools or other Linux desktop software as a whole. Through this design, it is possible to quickly analyze whether the version quality is up to standard. Throughout the development cycle, changes in version quality can be judged by data comparison to provide data reference and support. This solves the technical problem that existing single-point testing cannot verify the entire process testing as a whole and thus obtain comprehensive test results.

[0136] 2. The verification method for full-process testing provided in this embodiment of the invention runs a full-process script, and its steps include layout planning, clock routing, and ordinary signal routing.

[0137] It should be noted that the process of running the full-process script involves at least floorplanning, clock routing, and general signal routing. Floorplanning is the process of placing the control unit module of the chip and determining the overall placement of various functional circuits. Floorplanning can affect the final area of ​​the chip. Clock routing, also known as clock tree synthesis (CTS), is the separate routing of the clock signal. General signal routing includes the routing between various standard cells (i.e., basic logic gates).

[0138] 3. The verification method for full-process testing provided in this embodiment of the invention includes performance index values ​​of the version under test. These performance index values ​​include at least chip utilization, minimum performance, and performance range. It should be noted that the performance index values ​​include, but are not limited to, chip utilization, minimum performance, and performance range. Chip utilization is presented using a graphical interface to analyze whether the layout and routing area is sufficient. The chip's minimum performance (Worst Negative Slack, WNS) and performance range (Total Negative Slack, TNS) are timing analysis metrics primarily used to measure chip performance.

[0139] 4. The verification method for full-process testing provided in this embodiment of the invention uses regular expressions to extract the result content from the data text; the result content and the previous content are stored in the database, and the previous content was stored in the database earlier than the result content.

[0140] Understandably, by defining a model to complete the creation of the database, and then by establishing a connection between the database and the result content, the result content extracted by the regular expression is stored in the database. The database can store both previous content and result content at the same time. After multiple storage processes, they are combined to form a large data table, thereby obtaining the improvement process of key data in the iteration process of the version under test, and providing a visual chart structure for reference.

[0141] 5. The verification method for the full-process testing provided in this embodiment of the invention, after extracting the result content from the data text, further includes the following step: outputting the result content as a current parameter report. It should be noted that the current parameter report contains at least the data of the result content, but in order to compare the version under test with the previous version and quickly analyze the quality effect of the version under test, relevant data of the previous content can also be included. Then, the current parameter report is rendered on the page using a dedicated page display template.

[0142] 6. The verification method for full-process testing provided in this embodiment of the invention further includes the following steps after executing the verification method: outputting the current test status and sending the current test status through a message push interface. The current test status includes at least one of the following: test result, test report, and running time.

[0143] Understandably, in the process of executing the verification method, in addition to providing a detailed report on the current parameters, the current test status is also provided. The current test status focuses on the final result, and the test results of each stage in the process can be shared with employees in different departments according to the different divisions of labor in the enterprise, so as to provide data support for the next verification.

[0144] 7. The verification method for full-process testing provided in this embodiment of the invention exports log files when the full-process script executes at least one step of layout planning, clock routing, and general signal routing. These log files are used to view system problems encountered during the execution of the full-process script. It should be noted that corresponding log files can be exported at each stage, and these log files can be used for tracing and comparing problems encountered during the execution of the full-process script.

[0145] 8. This embodiment of the invention also provides a verification device for full-process testing, used to execute the steps of the above verification method, including: a version acquisition module, used to periodically acquire testable versions of full-process testing in the test directory, select a version to be tested according to the testable versions, and obtain a test file corresponding to the version to be tested; a script update module, used to write the test file to be tested into an initial script to obtain a full-process script; a script execution module, used to run the full-process script, and obtain the process content through an initialization method after execution; a content reading module, used to read the data text in the process content, and extract the result content from the data text; and a data comparison module, used to select at least one previous version of the version to be tested, obtain the previous content of the previous version, and then compare the result content with the previous content. This verification device for full-process testing has the same beneficial effects as the above verification method, and will not be described in detail here.

[0146] 9. This embodiment of the invention also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program causes the processor to perform the steps of the verification method described above. This computer-readable storage medium has the same beneficial effects as the verification method described above, and will not be repeated here.

[0147] 10. This embodiment of the invention also provides a computer device, including a memory and a processor, wherein the memory stores program instructions; when the program instructions are executed by the processor, the processor performs the steps of the verification method described above. This computer device has the same beneficial effects as the verification method described above, and will not be repeated here.

[0148] The foregoing has provided a detailed description of a full-process testing verification method, apparatus, storage medium, and computer equipment disclosed in the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention. Any modifications, equivalent substitutions, and improvements made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A verification method for full-process testing, applied to placement and routing tools, used to verify the entire process testing of EDA tools or other Linux desktop software as a whole, characterized in that, Includes the following steps: A Jenkins tool is provided that can build triggers on a schedule. The triggers periodically retrieve testable versions of the full-process tests in the test directory, select a version to be tested based on the testable versions, and obtain the test file corresponding to the version to be tested. Write the file to be tested into the initial script to obtain the full process script; Run the full-process script, and after completion, obtain the process content through the initialization method of the Python language; The data text in the process content is read using the Python language, and the result content is extracted from the data text using regular expressions. Select at least one previous version of the version to be tested, obtain the previous content of the previous version, and then compare the result content with the previous content.

2. The verification method as described in claim 1, characterized in that: The full-process script is run, and its steps include layout planning, clock routing, and general signal routing.

3. The verification method as described in claim 2, characterized in that: The results include the performance metrics of the version under test, which include at least chip utilization, minimum performance, and performance range.

4. The verification method as described in claim 3, characterized in that: The regular expressions are selected from the standard regular expression library; The result content and the previous content are stored in the database. The previous content was stored in the database earlier than the result content. The version corresponding to the previous content is any one or more versions prior to the version to be tested.

5. The verification method as described in claim 1, characterized in that, After extracting the result content from the data text, the following steps are also included: The results will be output as a current parameter report.

6. The verification method as described in claim 5, characterized in that, After executing the verification method, the following steps are also included: Output the current test status and send the current test status via message push interface. The current test status includes at least one of the following: test results, test report, and runtime.

7. The verification method as described in claim 2, characterized in that: When the full-process script executes at least one of the steps of layout planning, clock routing, and general signal routing in sequence, it exports a corresponding log file. The log file is used to view system problems during the execution of the full-process script.

8. A verification apparatus for full-process testing, used to perform the steps of the verification method as described in any one of claims 1-7, characterized in that, include: The version acquisition module is used to provide a Jenkins tool that can build triggers on a schedule. The triggers periodically acquire testable versions of the full-process test in the test directory, select the version to be tested based on the testable version, and obtain the test file corresponding to the version to be tested. The script update module is used to write the file to be tested into the initial script to obtain the full-process script. The script execution module is used to run the entire process script and obtain the process content through the initialization method of the Python language after the execution is completed. The content reading module is used to read the data text in the process content using the Python language, and to extract the result content from the data text using regular expressions. The data comparison module is used to select at least one previous version of the version under test, obtain the previous content of the previous version, and then compare the result content with the previous content.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it causes the processor to perform the steps of the verification method as described in any one of claims 1-7.

10. A computer device, characterized in that: It includes a memory and a processor, wherein the memory stores program instructions; When the program instructions are executed by the processor, the processor performs the steps of the verification method as described in any one of claims 1-7.

Images