A chip verification method, computer program product, device, and medium

Abstract

The present application discloses a chip verification method, a computer program product, a device and a medium, which relate to the field of computer technology and are applied to a chip verification device. The device includes a partitioning module, a management module, a communication module and a testing module. The method includes: partitioning test cases into different test categories through the partitioning module and sending them to the management module; determining target test cases through the management module; sending the target test cases to the subsystem test unit in the testing module through the communication module; different subsystem test units have different subsystem characteristics, and the subsystem characteristics include the business relevance between the subsystem test unit and the core business of the chip and the software-hardware coupling degree of the subsystem test unit; executing the target test cases through the subsystem test unit to verify the internal logic of the software-hardware interaction in the subsystem test unit. The present application can discover design problems at the bottom layer of hardware interaction, eliminate potential hazards before tape-out, and significantly reduce the risk of tape-out failure.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to a chip verification method, a computer program product, a device, and a medium. Background Art

[0002] In recent years, application scenarios of high-performance and high-reliability chips such as artificial intelligence and data centers have emerged continuously. This has made the design of chip hardware and firmware increasingly complex, and the interaction between chip hardware and firmware has also become more complex. For chip design companies, how to verify such complex chips is extremely challenging.

[0003] Generally, chip hardware can use mature Electronic Design Automation (EDA) tools for hardware function verification, and chip firmware can conduct pure software function verification through mature tools such as Unit Testing (UT) before tape-out. However, how to fully discover hardware design problems before tape-out and effectively verify the functions of the software-hardware interaction part has always been a difficult problem in the industry. Currently, System Testing (ST) is generally used in the industry to verify this part of the function, but its essence belongs to black-box testing. Black-box testing is a testing method that evaluates whether the software function meets the requirements by input and output without understanding the internal structure and characteristics of the program. Although black-box testing can cover functional requirements, it cannot detect internal logic errors in the code or defects hidden deep in the program, and it is easy to miss some hardware design problems, bringing risks to chip tape-out. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a chip verification method, a computer program product, a device, and a medium, which can discover design problems at the bottom layer of hardware interaction, eliminate potential hazards before tape-out, and significantly reduce the risk of tape-out failure. The specific solutions are as follows:

[0005] In a first aspect, the present application discloses a chip verification method, including:

[0006] By dividing modules, multiple test cases are divided into different test categories, and the test cases carrying the test categories are sent to the management module;

[0007] Through the management module, target test cases are determined from the test cases based on a selection instruction;

[0008] Through the communication module, the target test cases are sent to the corresponding subsystem test units in the test module according to the test categories; wherein, the test module contains multiple subsystem test units with different subsystem characteristics, and the subsystem characteristics include the business relevance of the subsystem test unit to the chip core business and the software-hardware coupling degree of the subsystem test unit.

[0009] Execute the target test case through the corresponding subsystem test unit to verify the internal logic of the software and hardware interaction in the subsystem test unit and obtain the test case execution result.

[0010] Optionally, the test category corresponds to the subsystem test unit or the simulation platform.

[0011] Optionally, divide multiple test cases into different test categories, including:

[0012] Divide multiple test cases into different subsystem test units according to the degree of association between each test case and the business functions of each subsystem test unit and the degree of dependence of each test case on the software and hardware of each subsystem test unit.

[0013] Optionally, divide multiple test cases into different test categories, including:

[0014] Determine the simulation environment required for each test case, and divide each test case into different simulation platforms according to the simulation environment required for each test case.

[0015] Optionally, determine the target test case from the test cases based on the selection instruction, including:

[0016] Obtain the operation information configured by the user through the operation options provided by the visual configuration interface in the management module, and determine the selection instruction according to the operation information;

[0017] Determine the target test case from the test cases according to the selection instruction.

[0018] Optionally, the selection instruction represents selecting to execute the test cases corresponding to at least one subsystem test unit or selecting to execute the test cases corresponding to at least one simulation platform.

[0019] Optionally, the subsystem test unit includes a first subsystem test unit with a business relevance score lower than the first threshold and a software and hardware coupling degree score higher than the second threshold, a second subsystem test unit with a business relevance score higher than the third threshold and a software and hardware coupling degree score lower than the fourth threshold, and a third subsystem test unit with a business relevance score higher than the fifth threshold and a software and hardware coupling degree score higher than the sixth threshold;

[0020] Among them, the first threshold is less than the third threshold, the first threshold is less than the fifth threshold, the second threshold is greater than the fourth threshold, the fourth threshold is less than the sixth threshold, and the second threshold is less than the sixth threshold.

[0021] Optionally, the first subsystem test unit includes a subsystem determined based on the central processing unit and the operating system.

[0022] Optionally, execute the target test case through the corresponding subsystem test unit to verify the internal logic of the software and hardware interaction in the subsystem test unit, including:

[0023] Execute the target test case through the first subsystem test unit based on the simulation platform corresponding to the target test case, and insert print statements at the internal interface and target state of the first subsystem test unit;

[0024] Output the processing flow information of the internal interface and target state through the print statements;

[0025] Compare the processing flow information with the corresponding expected values to verify the internal logic of the software and hardware interaction of the first subsystem test unit.

[0026] Optionally, the second subsystem test unit is a subsystem determined based on a pure software test framework, and the second subsystem test unit simulates the hardware modules relied on by the second subsystem test unit through stub modules.

[0027] Optionally, execute the target test case through the corresponding subsystem test unit to verify the internal logic of the software and hardware interaction in the subsystem test unit, including:

[0028] Determine the business internal process and the interface to be tested of the second subsystem test unit;

[0029] Perform stubbing on the hardware modules relied on by the second subsystem test unit based on the business internal process and the interface to be tested to obtain stub modules;

[0030] Modify the compilation framework and tool parameters of the test tool through the stub modules;

[0031] Execute the target test case through the corresponding second subsystem test unit and the modified test tool to verify the internal logic of the software and hardware interaction in the second subsystem test unit.

[0032] Optionally, the third subsystem test unit includes a disk dropping subsystem.

[0033] Optionally, execute the target test case through the corresponding subsystem test unit to verify the internal logic of the software and hardware interaction in the subsystem test unit, including:

[0034] Execute the target test case through the third subsystem test unit based on the simulation platform corresponding to the target test case, and monitor the communication parameters between the third subsystem test unit and the solid state drive to verify the internal logic of the software and hardware interaction in the third subsystem test unit.

[0035] Optionally, the chip verification method further includes:

[0036] Through the communication module, the use case execution result is fed back to the use case execution report module, and the use case execution report module generates a corresponding execution report according to the use case execution result; wherein, the use case execution result includes use case coverage and use case execution success rate.

[0037] Optionally, generating a corresponding execution report by the use case execution report module according to the use case execution result includes:

[0038] Through the use case execution report module, taking the simulation platform as the first dimension, a first execution report is generated according to the use case execution result under each simulation platform.

[0039] Optionally, generating a corresponding execution report by the use case execution report module according to the use case execution result includes:

[0040] Through the use case execution report module, taking the subsystem test unit as the second dimension, a second execution report is generated according to the use case execution result under each subsystem test unit.

[0041] Optionally, generating a corresponding execution report by the use case execution report module according to the use case execution result includes:

[0042] Through the use case execution report module, taking the subsystem test unit and the simulation platform as the third dimension, a third execution report is generated according to the use case execution result of each subsystem test unit under the preset simulation platform.

[0043] In a second aspect, the present application discloses a computer program product, including computer programs / instructions, and when the computer programs / instructions are executed by a processor, the steps of the foregoing chip verification method are implemented.

[0044] In a third aspect, the present application discloses an electronic device, including:

[0045] A memory for storing computer programs;

[0046] A processor for executing computer programs to implement the foregoing disclosed chip verification method.

[0047] In a fourth aspect, the present application discloses a computer-readable storage medium for storing computer programs; wherein, when the computer programs are executed by a processor, the foregoing disclosed chip verification method is implemented.

[0048] It can be seen that the present application proposes a chip verification method, which is applied to a chip verification device. The device includes a partitioning module, a management module, a communication module, and a test module. The method includes: through the partitioning module, dividing multiple test cases into different test categories and sending the test cases carrying the test categories to the management module; through the management module, determining target test cases from the test cases based on a selection instruction; through the communication module, sending the target test cases to the corresponding subsystem test units in the test module according to the test categories; where the test module contains multiple subsystem test units with different subsystem characteristics, and the subsystem characteristics include the business relevance between the subsystem test unit and the core business of the chip and the software-hardware coupling degree of the subsystem test unit; through the corresponding subsystem test unit, executing the target test case to verify the internal logic of the software-hardware interaction in the subsystem test unit and obtaining the use case execution result.

[0049] Beneficial effects: The partitioning module in the present application classifies multiple test cases and sends them to the management module. The management module determines the target test cases based on the selection instruction. The communication module sends the target test cases to the corresponding subsystem test units in the test module according to the test categories. These subsystem test units have different business relevance and software-hardware coupling degrees. Finally, the subsystem test units execute the target test cases to verify the internal logic of the software-hardware interaction and obtain the use case execution result. That is to say, the present application accurately matches and verifies the test cases based on the business relevance and software-hardware coupling degree by setting multiple subsystem test units with different characteristics. Compared with the traditional method that only relies on system testing (black box testing), the present application can cover various situations of the chip software-hardware interaction more comprehensively and deeply test details such as data processing logic and signal transmission timing when interacting with the hardware. This white box testing method of the present application can discover potential design problems from the code level and the bottom layer of the hardware interaction, so as to eliminate corresponding hidden dangers before tape-out, thereby significantly reducing the risk of tape-out failure and greatly saving R & D costs and R & D time. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0051] Figure 1 It is a flowchart of a chip verification method disclosed in the present application;

[0052] Figure 2 It is a schematic diagram of the relationship between a subsystem test unit and a simulation platform disclosed in the present application;

[0053] Figure 3 Flow chart of the first specific chip verification method disclosed in this application;

[0054] Figure 4 Schematic diagram of a CPU and OS integrated test framework disclosed in this application;

[0055] Figure 5 Flow chart of the second specific chip verification method disclosed in this application;

[0056] Figure 6 Schematic diagram of a UT Plus integrated test framework disclosed in this application;

[0057] Figure 7 Flow chart of the third specific chip verification method disclosed in this application;

[0058] Figure 8 Schematic diagram of a disk dropping subsystem integrated test framework disclosed in this application;

[0059] Figure 9 Schematic diagram of an integrated test framework for software and hardware interaction based on a chip simulation platform disclosed in this application;

[0060] Figure 10 Schematic diagram of a communication interface disclosed in this application;

[0061] Figure 11 Schematic diagram of a structure of an electronic device disclosed in this application. Detailed implementation manners

[0062] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0063] How to fully discover hardware design problems before tape-out and effectively verify the functions of the software and hardware interaction part has always been a difficult problem in the industry. Currently, the industry generally uses system testing to verify this part of the functions. However, it belongs to black-box testing. Although it can cover functional requirements, it cannot discover internal code logic errors or defects hidden deep in the program, and it is easy to miss some hardware design problems, bringing risks to chip tape-out.

[0064] Therefore, the embodiments of this application propose a chip verification solution, which can discover design problems at the bottom layer of hardware interaction and eliminate potential hazards before tape-out, significantly reducing the risk of tape-out failure.

[0065] An embodiment of the present application discloses a chip verification method, which is applied to a chip verification device. The device includes a partitioning module, a management module, a communication module, and a testing module. See Figure 1 as shown, the method includes:

[0066] Step S11: Through the partitioning module, divide multiple test cases into different test categories, and send the test cases carrying the test categories to the management module.

[0067] In this embodiment, after obtaining the test cases sent by the host, through the partitioning module, divide multiple test cases into different test categories, and send the test cases carrying the test categories to the management module.

[0068] In one implementation, according to the degree of association between each test case and the business functions of each subsystem test unit, and the degree of dependence of the test case on the software and hardware of each subsystem test unit, divide the test cases into different subsystem test units. Since different test cases have differences in functions and software and hardware requirements, dividing according to the above principles can make the testing work more targeted and efficient.

[0069] In another embodiment, the simulation environment required for each test case is determined, and each test case is divided into different simulation platforms according to the simulation environment required for each test case. During the verification process of RAID (Redundant Arrays of Independent Disks chip) chips, simulation platforms include VP (Versal Premium, a chip prototype method that uses software to simulate hardware), ZEBU (ZeBu Emulation Platform, which belongs to a real hardware simulation system), and HAPS (Hardware-Accelerated Processing System), etc. Since the support capabilities of different simulation platforms are different, even within the same subsystem test unit, different test cases may rely on different simulation platforms for testing. Taking the host interface subsystem test unit as an example, for the AEM (Acceleration Engine Manager) module, which is used to receive the IO (Input / Output) issued by the host and perform data calculation and transfer acceleration, its test cases need to be tested on the real hardware simulation platform ZEBU, and the VP pure software simulation platform cannot provide support. This is because the testing of the AEM module involves the actual operation and performance verification of hardware devices, and the VP platform cannot simulate a real hardware environment. And for the NTS (NVMe Target Service) module, which also belongs to the host interface subsystem test unit, since it does not rely on a real hardware simulation platform, it can be tested not only on the ZEBU platform but also on the VP platform.

[0070] If a certain test case can only be executed on a specific platform, then this test case belongs to a single-platform use case. If a certain test case can be executed on all simulation platforms, then this test case belongs to a general-platform use case. According to the above rules, testers can accurately match test cases and simulation platforms. See Figure 2 , a subsystem test unit can correspond to multiple simulation platforms, including general platforms and non-general platforms.

[0071] Step S12: Through the management module, determine the target test case from the test cases based on the selection instruction.

[0072] In this embodiment, the management module determines a target test case from test cases based on a selection instruction. Specifically, operation options provided by a visual configuration interface in the management module are used, and operation information configured by the user is obtained. Further, a selection instruction is determined according to the operation information, and a target test case is determined from test cases according to the selection instruction. The selection instruction represents selecting and executing test cases corresponding to at least one subsystem test unit, or selecting and executing test cases corresponding to at least one simulation platform.

[0073] That is to say, in this embodiment, it is possible to select and execute test cases corresponding to one subsystem test unit, or select and execute test cases corresponding to all subsystem test units. It is possible to select test cases corresponding to one simulation platform, or select test cases corresponding to all simulation platforms. It can be seen that by setting a visual configuration interface in the management module, in this embodiment, there is no need to screen one by one among a large number of test cases, which greatly saves time, meets diverse test scenarios, and enhances the flexibility of testing.

[0074] Step S13: Through the communication module, the target test case is sent to the corresponding subsystem test unit in the test module according to the test category; wherein, the test module includes multiple subsystem test units with different subsystem characteristics, and the subsystem characteristics include the business relevance between the subsystem test unit and the core business of the chip and the software-hardware coupling degree of the subsystem test unit.

[0075] In this embodiment, through the communication module, the target test cases are sent to the corresponding subsystem test units in the test module according to the test categories. The test module includes multiple subsystem test units with different subsystem characteristics. The subsystem characteristics include the business relevance between the subsystem test unit and the core business of the chip and the software-hardware coupling degree of the subsystem test unit. Specifically, optionally, the subsystem test units include a first subsystem test unit with a business relevance score lower than a first threshold and a software-hardware coupling degree score higher than a second threshold, a second subsystem test unit with a business relevance score higher than a third threshold and a software-hardware coupling degree score lower than a fourth threshold, and a third subsystem test unit with a business relevance score higher than a fifth threshold and a software-hardware coupling degree score higher than a sixth threshold. Among them, the first threshold is less than the third threshold, the first threshold is less than the fifth threshold, the second threshold is greater than the fourth threshold, the fourth threshold is less than the sixth threshold, and the second threshold is less than the sixth threshold. Exemplarily, the first subsystem test unit includes a subsystem determined based on a central processing unit (CPU) and an operating system (OS). This subsystem has weak business relevance but high software-hardware coupling. The second subsystem test unit can be a subsystem determined based on a pure software test framework, such as a RAID AC (RAID Acceleration Cluster) subsystem. This subsystem has strong business relevance and weak software-hardware coupling. The third subsystem test unit includes a disk drop subsystem. This subsystem has strong business relevance and strong software-hardware coupling.

[0076] To accurately set the above thresholds, this embodiment needs to count the number of times the subsystem is called by the service within a unit time as the quantitative basis for the service participation frequency. At the same time, count the frequency of calling hardware interface functions in the subsystem code, the complexity of data interaction, etc., to obtain the software-hardware coupling degree score. For example, in a chip development project, the first subsystem test unit is responsible for basic instruction processing and resource scheduling. Although it does not directly participate in specific business logic operations, it provides basic support for business operations. After testing, its business relevance score is stable at 20-25, lower than the first threshold, which can be 30; the software-hardware coupling degree score is between 55-60, higher than the second threshold, which can be 50. The second subsystem test unit can be a subsystem determined based on a pure software test framework. This subsystem realizes data storage acceleration through algorithm optimization and is closely related to the business data storage requirements. The business relevance score is between 65-70, higher than the third threshold, which can be 60. At the same time, its operation mainly depends on software algorithms and has less direct interaction with hardware. The software-hardware coupling degree score is, for example, between 30-32, lower than the fourth threshold, which can be 35. The third subsystem test unit includes a disk dropping subsystem. The disk dropping subsystem is responsible for persistently storing business data to a physical disk. It not only deeply participates in the business data storage process, and the business relevance score is between 80-85, higher than the fifth threshold, which can be 75, but also closely interacts with the disk hardware. The software-hardware coupling degree score is between 70-75, higher than the sixth threshold, which can be 65.

[0077] With the dynamic change of business requirements, it is necessary to periodically re-evaluate the thresholds and the classification of subsystems. For example, when introducing new hardware technologies, it may cause changes in the software-hardware coupling degree of some subsystems, or when the business expands into new fields, it may change the relevance between the subsystem and the business. Therefore, this embodiment can establish a set of automated data collection and analysis mechanisms to continuously monitor the business participation and software-hardware interaction of subsystems during actual operation, so as to adjust the thresholds and subsystem classification in a timely manner.

[0078] By setting multiple subsystem test units with different characteristics, this application can more comprehensively cover various situations of chip software-hardware interaction. By testing subsystem test units with different characteristics, potential design problems can be discovered from the code level and the hardware interaction bottom layer, so as to eliminate hidden dangers before tape-out, thus significantly reducing the risk of tape-out failure and greatly saving R & D costs and R & D time.

[0079] Step S14: Execute the target test case through the corresponding subsystem test unit to verify the internal logic of the software-hardware interaction in the subsystem test unit and obtain the use case execution result.

[0080] In this embodiment, through the corresponding subsystem test unit, the target test case is executed to verify the internal logic of the software and hardware interaction in the subsystem test unit, and the test case execution result is obtained. Further, through the communication module, the test case execution result is fed back to the test case execution report module, and different-dimensional execution reports are generated by the test case execution report module according to the test case execution result; wherein, the test case execution result includes the test case coverage rate and the test case execution success rate. The test reports in different dimensions are described below:

[0081] First, through the test case execution report module, taking the simulation platform as the first dimension, a first execution report is generated according to the test case execution results under each simulation platform. The format of this report is shown in Table 1:

[0082] Table 1 First Execution Report Table

[0083]

[0084] Second, through the test case execution report module, taking the subsystem test unit as the second dimension, a second execution report is generated according to the test case execution results under each subsystem test unit. The format of this report is shown in Table 2:

[0085] Table 2 Second Execution Report Table

[0086]

[0087] Third, through the test case execution report module, taking the subsystem test unit and the simulation platform as the third dimension, a third execution report is generated according to the test case execution results of each subsystem test unit under the preset simulation platform. The format of this report is shown in Table 3:

[0088] Table 3 Third Execution Report Table

[0089]

[0090] The above mechanism for generating test reports from three dimensions plays a crucial role in the chip testing process and can produce significant effects in multiple aspects. Specifically, based on the first execution report, R & D personnel can quickly understand the coverage of test cases by each simulation platform and the execution success rate. For example, if it is found that the case coverage rate of a certain simulation platform is significantly lower than that of other platforms, it is possible to check whether the settings of this platform are reasonable or supplement test cases under specific platforms to ensure that the chip can be fully verified in different simulation environments and improve the comprehensiveness of testing. Based on the second execution report, this embodiment can clearly show the test results of each subsystem through indicators such as the total number of cases, case coverage rate, and execution success rate. Suppose the execution success rate of the cases of a certain subsystem is low, then this subsystem can be focused on to deeply investigate software-hardware interaction problems and optimize the subsystem design. Based on the third execution report, this embodiment can help R & D personnel understand the specific performance of different subystems under each simulation platform. For example, if a certain subsystem has a high execution success rate on the VP simulation platform but a low one on the ZEBU simulation platform, this provides clues for further analyzing the compatibility between platform characteristics and subsystems, helps solve the problem of unstable operation of subsystems under specific platforms, and comprehensively ensures the correctness of software-hardware interaction logic and the stability of system operation of the chip in complex and diverse test scenarios.

[0091] It can be seen that this application proposes a method for verifying the software-hardware interaction function of a chip based on a simulation platform, realizing white-box testing of some functions of the software-hardware interaction part. This method first divides the business domain according to business relevance and software-hardware coupling degree, and then carries out use case design and development work. According to the characteristics of different subsystems, a test framework is built. Finally, by executing test cases with one key and automatically summarizing test results, the quality maintenance of some functions of the software-hardware interaction part is realized. Compared with the traditional black-box testing method, the software-hardware interaction verification method based on the simulation platform provided by this application can significantly improve the quality of the software-hardware interaction part of the chip and reduce the risk of tape-out in advance. In addition, by means of the two indicators of case coverage rate and case execution success rate, the goal of improving and monitoring the firmware quality can also be achieved.

[0092] It can be seen that the present application proposes a chip verification method, which is applied to a chip verification device. The device includes a partitioning module, a management module, a communication module, and a testing module. The method includes: through the partitioning module, dividing multiple test cases into different test categories, and sending the test cases carrying the test categories to the management module; through the management module, determining a target test case from the test cases based on a selection instruction; through the communication module, sending the target test case to the corresponding subsystem test unit in the testing module according to the test category; wherein, the testing module includes multiple subsystem test units with different subsystem characteristics, and the subsystem characteristics include the business relevance between the subsystem test unit and the core business of the chip and the software-hardware coupling degree of the subsystem test unit; through the corresponding subsystem test unit, executing the target test case to verify the internal logic of the software-hardware interaction in the subsystem test unit, and obtaining the use case execution result.

[0093] Beneficial effects: The partitioning module in the present application classifies multiple test cases and sends them to the management module. The management module determines the target test case based on the selection instruction. The communication module sends the target test case to the corresponding subsystem test unit in the testing module according to the test category. These subsystem test units have different business relevance and software-hardware coupling degrees. Finally, the subsystem test unit executes the target test case to verify the internal logic of the software-hardware interaction and obtain the use case execution result. That is to say, the present application accurately matches and verifies test cases based on business relevance and software-hardware coupling degree by setting multiple subsystem test units with different characteristics. Compared with the traditional method that only relies on system testing (black box testing), the present application can cover various situations of the chip software-hardware interaction more comprehensively, and deeply test details such as data processing logic and signal transmission timing when interacting with the hardware. This white box testing method of the present application can discover potential design problems from the code level and the bottom layer of the hardware interaction, so as to eliminate corresponding hidden dangers before tape-out, thereby significantly reducing the risk of tape-out failure and greatly saving R & D costs and R & D time.

[0094] The test frameworks and test processes of subsystem test units with different characteristics are described below:

[0095] In the first case, the first subsystem test unit includes a subsystem determined based on a central processing unit and an operating system. Then, referring to Figure 3 and Figure 4 shown, the specific refinement process of step S14 specifically includes:

[0096] Step S14A1: Through the first subsystem test unit, execute the target test case based on the simulation platform corresponding to the target test case, and insert print statements at the internal interface and target state of the first subsystem test unit.

[0097] Step S14A2: Output the processing flow information of the internal interface and the target state through a print statement.

[0098] Step S14A3: Compare the processing flow information with the corresponding expected value to verify the internal logic of the software-hardware interaction of the first subsystem test unit.

[0099] In this embodiment, through the first subsystem test unit, the use case is executed based on the simulation platform corresponding to the target test case. At the same time, print statements are inserted at the internal interface and the target state of the first subsystem test unit to output the processing flow information of the internal interface (such as CPU register access interface, OS kernel scheduling interface, etc.) and the target state (such as CPU load state, memory state, etc.). Then, these processing flow information are compared with the corresponding expected values to verify the internal logic of the software-hardware interaction of the first subsystem test unit. That is to say, the CPU and OS integrated test framework built in this embodiment, in addition to testing the operating system, the adaptation layer, and the upper-layer interface as a whole, also judges whether the processing flow meets the expectations by printing some internal interface and status information of the system, so as to achieve the purpose of white-box testing of the CPU and OS subsystems. In a redundant array of independent disks (RAID) controller chip, the CPU is mainly responsible for program instruction execution and data processing, while the OS is responsible for tasks such as resource management, task scheduling, memory allocation, and interrupt scheduling. This part of the content has a weak correlation with the business but is closely related to the hardware. The testing of the CPU and OS is not only for themselves, because the CPU usually has a mature architecture, the operating system is mostly open-source code, and the stability and quality of both have passed professional certification. However, the CPU and OS in the chip need to be adapted to different platforms, which requires adaptation layer coding. During this process, there will be interactions with hardware such as registers, timers, MMUs (Memory Management Units), and caches, and these interactions will affect the switching and scheduling of thread contexts. Therefore, for the testing of the CPU and OS subsystems in a system-on-chip, it is necessary to comprehensively test and cover such software-hardware interaction operations. Based on the above process, the accurate verification of the internal logic of the software-hardware interaction of the first subsystem test unit is realized, ensuring that its functions are accurate and error-free. In addition, through the comprehensive testing and coverage of the software-hardware interaction operations of the CPU and OS subsystems with the hardware, the stability and reliability of the chip when adapting to different platforms are ensured, and the risks caused by software-hardware interaction problems are avoided in advance, providing guarantee for the normal operation of the chip.

[0100] Exemplarily, the target test case aims to test the performance of the CPU and OS subsystems in complex scenarios. Then, with the help of the first subsystem test unit, this case is executed based on the simulation platform, and print statements are inserted at the internal interface and the target state to output the processing flow information. Then, the instruction execution order of the CPU under high load and the resource allocation strategy of the OS for memory stress situations are recorded, and this information is compared with the expected values. In this way, it is ensured that the CPU and OS subsystems can operate stably when the chip is adapted to different platforms.

[0101] In the second case, the second subsystem test unit is a subsystem determined based on a pure software test framework, and the second subsystem test unit simulates the hardware modules it depends on through stub modules. Then, referring to Figure 5 and Figure 6 as shown, the specific refinement process of step S14 specifically includes:

[0102] Step S14B1: Determine the business internal process and the interface to be tested of the second subsystem test unit.

[0103] Step S14B2: Based on the business internal process and the interface to be tested, perform stubbing on the hardware modules that the second subsystem test unit depends on to obtain stub modules;

[0104] Step S14B3: Modify the compilation framework and tool parameters of the test tool through the stub modules;

[0105] Step S14B4: Execute the target test case through the corresponding second subsystem test unit and the modified test tool to verify the internal logic of the software-hardware interaction in the second subsystem test unit.

[0106] In this embodiment, the business internal process and the interface to be tested of the second subsystem test unit are determined. Based on the business internal process and the interface to be tested, stubbing is performed on the hardware modules that the second subsystem test unit depends on to obtain stub modules. The compilation framework and tool parameters of the test tool are modified with the help of the stub modules, and the target test case is executed through the corresponding second subsystem test unit and the modified test tool to verify the internal logic of the software-hardware interaction in the second subsystem test unit.

[0107] Figure 6It is an integration test framework for this type of subsystem. For the UT Plus (Unit Test Plus) integration test framework, by stubbing the external dependent hardware modules, the internal logic of the system can be tested without relying on a simulation platform. The test framework for this type of subsystem can be built on the basis of a mature unit test tool (such as the Cunit unit test tool) in combination with the business characteristics. Taking the RAID AC (RAID Acceleration Cluster) subsystem in the RAID (Redundant Array of Independent Disks) chip as an example, this subsystem is mainly responsible for the service configuration of the control plane and the IO (Input / Output) processing of the data plane in various scenarios, and is a subsystem with strong business relevance and low hardware coupling. Stub the hardware modules such as the driver layer STUB (Driver Layer Stub), OSAL layer STUB (Operating System Abstraction Layer Stub), and platform basic dependencies STUB (Platform Basic Dependencies Stub), and on the basis of the original CUnit unit test framework, implement full compilation of business code and extended parameters, so as to realize the test of the RAID AC subsystem.

[0108] Exemplarily, the target test case is to verify the performance of the RAID AC subsystem in the scenario of high-concurrency data writing. In this scenario, a large number of users upload data to the storage system at the same time, and the RAID AC subsystem is responsible for the service configuration of the control plane, such as planning the data storage location, and the IO processing of the data plane. By stubbing the hardware modules such as the driver layer STUB, OSAL layer STUB, and platform basic dependencies STUB, the hardware environment is simulated. On the original CUnit unit test framework, the business code is fully compiled, and the parameters are extended to simulate different data volumes and concurrency numbers. Observe whether the subsystem can accurately process a large number of concurrent write requests, whether there is data loss or write errors, so as to verify the internal logic and performance of the RAID AC subsystem.

[0109] In the third case, if the third subsystem test unit includes the disk landing subsystem, then refer to Figure 7 and Figure 8 As shown, the specific refinement process of step S14 specifically includes:

[0110] Step S14C1: Through the third subsystem test unit, execute the target test case based on the simulation platform corresponding to the target test case, and monitor the communication parameters between the third subsystem test unit and the solid-state drive to verify the internal logic of the software and hardware interaction in the third subsystem test unit.

[0111] In the RAID (Redundant Array of Independent Disks) chip, the disk dropping subsystem plays an important role. This subsystem interacts with the SSD (Solid State Drive) disk, including IO commands in the data plane, admin commands in the management plane, and some information query commands such as disk status. The disk dropping subsystem is closely related to the business and has a very high coupling degree with the hardware, and there are frequent interactions with the hardware during operation. Given these characteristics, the testing of this type of subsystem requires the help of a simulation platform. To achieve effective verification of this type of subsystem, a software and hardware interaction integration test framework based on the simulation platform can be built. For the schematic diagram of the test framework, see Figure 8 as shown.

[0112] Exemplarily, the target test case is to simulate massive data concurrent writing to the solid-state drive. When executing this case, use the third subsystem test unit to start the operation based on the corresponding simulation platform. At this time, focus on monitoring the following communication parameters: (1) Data transfer rate, observe how much data can be transferred per second. If the rate is stable at a high level, such as reaching 500 MB / s, it indicates that the data writing is smooth; (2) Transmission delay, record the time from when the subsystem issues a write command to when the solid-state drive completes the write. If the delay is within 10 milliseconds, it indicates timely response; (3) Pay attention to the error rate. If the error rate is less than one in ten thousand in one million write operations, the internal logic of the software and hardware interaction can be initially verified to be normal, otherwise, the problem needs to be investigated.

[0113] See Figure 9 as shown, Figure 9Disclosed is an integrated test framework for software and hardware interaction based on a chip simulation platform, including: 1. Division module (also known as use case management component). All test cases are managed through the use case management component, and the test cases are divided according to different dimensions. For example, when divided by subsystems, different subsystem use cases can be classified, or when divided by the simulation platform, general platform use cases or use cases that can only be executed on a fixed simulation platform can be classified. 2. Management module (also known as the integrated test unified test framework). The integrated test unified test framework provides a unified execution entry for different subsystems. The framework supports one-key execution of all subsystem test cases, and also supports individual execution of all use cases of a certain subsystem or all use cases under a certain simulation platform, and outputs relevant test results. 3. Communication module (also known as the host interface common component). The host interface common component is used to implement communication between the chip and the host. An adaptation layer is added to the chip firmware, which is responsible for receiving, parsing, forwarding and replying after completion of host commands. For example, in a RAID chip, an NVMe shell (Non-Volatile Memory express shell, non-volatile memory host controller interface specification command line tool) module can be added as the adaptation layer to complete the communication between the host and the chip through the processing of NVMe shell commands. Taking the RAID chip as an example, the specific interaction flowchart is as Figure 10As shown in the figure: The NTS module receives the NVMe command sent by the host and then transparently transmits the command to the NVMe Shell public component. After the NVMe Shell parses and encapsulates the command, it distributes the command to each module. After each module completes the execution, it returns the result to the NVMe shell, and then transmits the message to the NTS. The NTS sends a CQE (Completion Queue Entry) to notify the host to obtain the command execution result. 4. The integration test component of the software and hardware interaction subsystem. It mainly conducts test verification on three typical software and hardware interaction subsystems, including: a) Subsystems that are not related to the business but involve software and hardware interaction. Such subsystems are not related to the business but need to rely on hardware to complete the verification of their own functions. For example, the CPU and OS subsystems mainly verify the operating system. b) Subsystems that are strongly related to the business but have less software and hardware coupling. Such subsystems can complete the verification of their own business logic without relying on hardware through pure software testing of a single subsystem. For example, the RAID AC subsystem in the RAID chip. c) Subsystems that are strongly related to the business and have more software and hardware coupling. Such subsystems are strongly related to both the business and hardware and need to rely on hardware to complete the verification of their own logic. For example, the disk subsystem and UAA subsystem in the RAID chip. For the above three typical software and hardware interaction subsystems, different test frameworks are built respectively, and subsystem testing is carried out based on the simulation platform or the extended UT test framework, so as to achieve the white-box testing of the chip's software and hardware interaction function. 5) The use case execution report module (also known as the use case execution report component) is mainly used to present the results of use case execution. It summarizes the test results of the chip's software and hardware interaction part and generates result reports in different dimensions. For example, viewing the use case execution status of each subsystem in terms of subsystems, or viewing the use case execution status under different simulation platforms in terms of the simulation platform.

[0114] It can be seen that the present application aims to provide a chip hardware and software interaction verification method based on a simulation platform. According to the business coupling degree and the hardware and software interaction coupling degree, three typical hardware and software interaction subsystems are divided, and three different test frameworks are built. From use case design, use case execution, unified test entry, and the construction of the internal test framework of the code to the final execution report statistics, white-box testing of the chip hardware and software interaction functions is realized. Compared with the traditional black-box testing method, the testing method provided by the present application has the following advantages for chip design: 1) Before tape-out, through white-box testing of the hardware and software interaction part, the internal logic of the business is tested from the code level, and the hardware design problems of the hardware and software interaction part of the chip are discovered in advance, thereby reducing tape-out risks. 2) Through in-depth testing of the internal logic of the business, problems with the chip's hardware-software interaction interface can be discovered in advance, such as inconsistent understanding between hardware and software personnel, problems with the hardware-software interaction process, etc., to reduce the risk of tape-out; 3) During the test process, use case coverage, use case execution success rate and other means to ensure that all use cases can be correctly executed and tested, thereby protecting the chip firmware quality and effectively improving the firmware quality; 4) This method has a high degree of reuse and can be used for software-hardware interaction verification based on the simulation platform before tape-out, and can also be used for verification based on EVB (Evaluation Board) after the chip is returned, and can also be used for software-hardware interaction verification of other related chips. In addition, this application proposes secondary development of mature unit testing tools to achieve a method for completing subsystem external interface and business process testing in the unit testing framework.

[0115] Further, the present embodiment can also introduce a dynamic priority determination mechanism in the management module. When facing many test cases, it is not only necessary to select instructions to determine the target test cases, but also possible to use the real-time monitoring system to collect the current operating status data of the chip in an all-round manner, such as the CPU load rate, memory usage rate, etc. At the same time, the resource occupancy is recorded in detail, including the busyness of hardware resources such as GPU and hard disk I / O, and the usage of software resources such as thread pool. The historical execution results of the test cases are deeply analyzed. If a test case fails frequently in multiple tests in the past, its priority will be increased accordingly. In addition, combined with the impact assessment on the core business of the chip, for example, the priority of test cases involving key data processing and core algorithm calculations will also be increased. Through a set of complex algorithm models, the priority score of each test case is calculated in real time. For test cases with high priority, the system automatically puts them into the priority execution queue, and within the limited test time, the key functional modules and potential high-risk areas of the chip are fully verified. In addition, as the test progress advances, the resource release is monitored in real time. Once new resources are available, the test case priority is immediately re-evaluated and the execution order is dynamically adjusted, so that the overall efficiency and pertinence of the test are greatly improved.

[0116] Further, an embodiment of the present application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the foregoing chip verification method.

[0117] Further, an embodiment of the present application also provides an electronic device. Figure 11 FIG. 20 is a structural diagram of an electronic device 20 shown according to an exemplary embodiment. The content in the figure should not be construed as any limitation on the scope of use of the present application.

[0118] Figure 11 FIG. 20 is a schematic structural diagram of an electronic device 20 provided by an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the chip verification method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0119] In this embodiment, the power supply 26 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and no specific limitation is imposed on it here; the input / output interface 24 is used to obtain external input data or output data to the outside, and the specific interface type can be selected according to specific application needs, and no specific limitation is imposed here.

[0120] In addition, as a carrier for resource storage, the memory 22 may be a read-only memory, a random access memory, a magnetic disk, or an optical disc, etc. The resources stored thereon may include a computer program 221, and the storage method may be temporary storage or permanent storage. Among them, in addition to the computer program that can be used to complete the chip verification method executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 221 may further include a computer program that can be used to complete other specific tasks.

[0121] Further, an embodiment of the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the foregoing disclosed chip verification method.

[0122] For the specific steps of this method, reference may be made to the corresponding content disclosed in the foregoing embodiments, and details are not described herein again.

[0123] In the present application, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method section.

[0124] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0125] The steps of the methods or algorithms described in combination with the embodiments disclosed herein can be directly implemented by hardware, software modules executed by a processor, or a combination of the two. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0126] Finally, it should be noted that in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0127] The above has introduced in detail a chip verification method, computer program product, device, and storage medium provided by the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A chip verification method, characterized in that, Applied to a chip verification device, the device includes a partitioning module, a management module, a communication module, and a testing module. The method includes: Through the partitioning module, partition multiple test cases into different test categories, and send the test cases carrying the test categories to the management module; Through the management module, determine target test cases from the test cases based on a selection instruction; Through the communication module, send the target test cases to the corresponding subsystem test units in the testing module according to the test categories; wherein, the testing module contains multiple subsystem test units with different subsystem characteristics, and the subsystem characteristics include the business relevance between the subsystem test unit and the core business of the chip and the software-hardware coupling degree of the subsystem test unit; Through the corresponding subsystem test unit, execute the target test case to verify the internal logic of the software-hardware interaction in the subsystem test unit, and obtain the use case execution result; Among them, the subsystem test unit includes a first subsystem test unit with a business relevance score lower than a first threshold and a software-hardware coupling degree score higher than a second threshold, a second subsystem test unit with a business relevance score higher than a third threshold and a software-hardware coupling degree score lower than a fourth threshold, and a third subsystem test unit with a business relevance score higher than a fifth threshold and a software-hardware coupling degree score higher than a sixth threshold; Among them, the first threshold is less than the third threshold, the first threshold is less than the fifth threshold, the second threshold is greater than the fourth threshold, the fourth threshold is less than the sixth threshold, and the second threshold is less than the sixth threshold; Among them, the partitioning of multiple test cases into different test categories includes: According to the degree of association between each test case and the business functions of each subsystem test unit, and the degree of dependence between each test case and the software and hardware of each subsystem test unit, partition multiple test cases into different subsystem test units.

2. The chip verification method according to claim 1, characterized in that, The test category corresponds to the subsystem test unit or to the simulation platform.

3. The chip verification method according to claim 2, wherein The partitioning of multiple test cases into different test categories includes: Determine the simulation environment required for each test case, and partition each test case into different simulation platforms according to the simulation environment required for each test case.

4. The chip verification method according to claim 2, wherein The determining of target test cases from the test cases based on the selection instruction includes: Obtain the operation information configured by the user through the operation options provided by the visual configuration interface in the management module, and determine the selection instruction according to the operation information; Determine the target test cases from the test cases according to the selection instruction.

5. The chip verification method according to claim 4, wherein, The selection instruction represents selecting to execute the test cases corresponding to at least one of the subsystem test units, or selecting to execute the test cases corresponding to at least one of the simulation platforms.

6. The chip verification method according to claim 1, wherein The first subsystem test unit includes a subsystem determined based on a central processing unit and an operating system.

7. The chip verification method according to claim 6, wherein The executing of the target test case through the corresponding subsystem test unit to verify the internal logic of the software-hardware interaction in the subsystem test unit includes: Execute the target test case based on the simulation platform corresponding to the target test case through the first subsystem test unit, and insert print statements at the internal interface and target state of the first subsystem test unit; Output the processing flow information of the internal interface and the target state through the print statement; Compare the processing flow information with the corresponding expected value to verify the internal logic of the software and hardware interaction of the first subsystem test unit.

8. The chip verification method according to claim 1, wherein The second subsystem test unit is a subsystem determined based on a pure software test framework, and the second subsystem test unit simulates the hardware modules relied on by the second subsystem test unit through stub modules.

9. The chip verification method according to claim 8, wherein Execute the target test case through the corresponding subsystem test unit to verify the internal logic of the software and hardware interaction in the subsystem test unit, including: Determine the business internal process and the interface to be tested of the second subsystem test unit; Based on the business internal process and the interface to be tested, perform stubbing on the hardware modules relied on by the second subsystem test unit to obtain stub modules; Modify the compilation framework and tool parameters of the test tool through the stub module; Execute the target test case through the corresponding second subsystem test unit and the modified test tool to verify the internal logic of the software and hardware interaction in the second subsystem test unit.

10. The chip verification method according to claim 1, wherein The third subsystem test unit includes a disk dropping subsystem.

11. The chip verification method according to claim 10, wherein Execute the target test case through the corresponding subsystem test unit to verify the internal logic of the software and hardware interaction in the subsystem test unit, including: Execute the target test case based on the simulation platform corresponding to the target test case through the third subsystem test unit, and monitor the communication parameters between the third subsystem test unit and the solid-state drive to verify the internal logic of the software and hardware interaction in the third subsystem test unit.

12. The chip verification method according to any one of claims 1 to 11, characterized in that, Further includes: Feedback the test case execution result to the test case execution report module through the communication module, and generate a corresponding execution report through the test case execution report module according to the test case execution result; wherein, the test case execution result includes test case coverage and test case execution success rate.

13. The chip verification method according to claim 12, wherein Generating a corresponding execution report by the test case execution report module according to the test case execution result includes: Generate a first execution report by the test case execution report module with the simulation platform as the first dimension according to the test case execution result under each simulation platform.

14. The chip verification method according to claim 12, wherein Generating a corresponding execution report by the test case execution report module according to the test case execution result includes: Generate a second execution report by the test case execution report module with the subsystem test unit as the second dimension according to the test case execution result under each subsystem test unit.

15. The chip verification method according to claim 12, wherein Generating a corresponding execution report by the test case execution report module according to the test case execution result includes: Through the use case execution report module, taking the subsystem test unit and the simulation platform as the third dimension, a third execution report is generated according to the use case execution results of each subsystem test unit under the preset simulation platform.

16. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by a processor, the steps of the chip verification method according to any one of claims 1 to 15 are implemented.

17. An electronic device, characterized in that, Including: A memory for storing a computer program; A processor for executing the computer program to implement the chip verification method according to any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that, For storing a computer program; wherein, when the computer program is executed by a processor, the chip verification method according to any one of claims 1 to 15 is implemented.

Images