Test method and related device

Abstract

According to the test method provided by the embodiment of the invention, when a DUT newly adds functions due to iteration or optimization and the like, a test case can be newly added for an original test excitation set through a target interface, the original test excitation set does not need to be repeatedly developed, the workload required for generating the test excitation set is greatly reduced, and the test efficiency is improved. And the test efficiency is improved. Moreover, the test excitation set is generated based on the target instruction, so that the test excitation set can perform hierarchical testing on the target instruction, and the test requirements of each stage in the DUT design process can be met; besides, even if the cross effect exists among the multiple functions, the cross effect among the functions can be tested and verified through the test case generated for the target instruction, and the development difficulty of the test excitation set can be reduced.

Description

Technical Field

[0001] The present specification relates to the field of computer application technology, specifically, to chip testing technology under the field of computer application technology, and more specifically, to a testing method and related devices. Background Art

[0002] In the design verification process of chips (Integrated Circuit, IC), etc., a test case can be a set of input sequences or events that can be used to drive the chip to test the correctness of its functions and performance characteristics. Testing the device under test (DUT) such as chips through test cases can detect potential design defects or errors in a timely manner, so that designers can correct the defects or errors and ensure the normal operation of the DUT under various operating conditions. However, at present, there is a problem of low efficiency when testing DUT. Summary of the invention

[0003] The embodiments of this specification provide a testing method and related devices to achieve the purpose of improving the efficiency of testing a DUT.

[0004] To achieve the above technical objectives, the embodiments of this specification provide the following technical solutions:

[0005] In a first aspect, an embodiment of the present specification provides a testing method, comprising:

[0006] In response to a test operation on the device under test (DUT), a test process is executed;

[0007] The testing process includes:

[0008] Based on a function to be tested of the DUT, determining a target instruction corresponding to the function to be tested, wherein the function to be tested includes an architectural function or a micro-architectural function of the DUT;

[0009] Based on the target instruction, a test stimulus set is generated; the test stimulus set includes a target interface, and the target interface is used to add a test case to the test stimulus set when being called.

[0010] Optionally, there are multiple test stimulus sets, and different test stimulus sets have different degrees of relevance to the function to be tested corresponding to the target instruction.

[0011] Optionally, generating a test stimulus set based on the target instruction includes:

[0012] Based on the target instruction, generating a first set, wherein the first set is used to verify the encoding and basic functions of the target instruction;

[0013] Based on the target instruction, generating a second set, wherein the second set is used to verify the calculation and processing process of the target instruction on the data;

[0014] Based on the target instruction, generating a third set, the third set is used to verify the calculation and processing process of the target instruction in a target scenario, the target scenario including a data bypass scenario;

[0015] The first set, the second set and the third set have increasing relevance to the functions to be tested corresponding to the target instructions, and the first set, the second set and the third set respectively correspond to different target interfaces.

[0016] Optionally, generating the first set based on the target instruction includes:

[0017] Determining an operation code and an operand of the target instruction;

[0018] The first set is generated based on an opcode and operands of the target instruction.

[0019] Optionally, generating the second set based on the target instruction includes:

[0020] Determining data to be tested and test conditions, wherein the data to be tested includes different data types corresponding to the target instruction, and the test conditions include overflow conditions;

[0021] The second set is generated based on the data to be tested and the test condition.

[0022] Optionally, generating a third set based on the target instruction includes:

[0023] In the target scenario, a third set corresponding to the target instruction is generated, and in the test cases in the third set, the operands of the target instruction are derived from the processing results of other instructions.

[0024] Optionally, the process of determining the function to be tested of the DUT includes:

[0025] Determining, according to the value of the target system register, the degree to which the DUT supports the function represented by the target system register;

[0026] When the DUT supports the function represented by the target system register, the function represented by the target register is determined as the function to be tested, the target instruction corresponding to the function to be tested is in a target test state, and the target instruction representation in the target test state has been defined and can be executed by the DUT instruction.

[0027] Optionally, when the target instruction includes a memory access instruction, the test stimulus set includes a fourth set, and the test process further includes:

[0028] Determine the target memory address and the type of memory data to be accessed;

[0029] Based on the target memory address and the memory access data type, the fourth set corresponding to the memory access instruction is generated, and the fourth set is used to verify whether the memory access instruction can correctly access the target memory address and / or whether it can correctly operate data of the memory access data type.

[0030] Optionally, if there is a historical test stimulus set, the test process further includes:

[0031] The target interface included in the historical test stimulus set is called to add the test cases in the currently generated test stimulus set to the historical test stimulus set.

[0032] In a second aspect, an embodiment of the present specification provides a testing device, the testing device comprising:

[0033] A test module, configured to execute a test process in response to a test operation on a device under test (DUT);

[0034] The testing process includes:

[0035] Based on a function to be tested of the DUT, determining a target instruction corresponding to the function to be tested, wherein the function to be tested includes an architectural function or a micro-architectural function of the DUT;

[0036] Based on the target instruction, a test stimulus set is generated; the test stimulus set includes a target interface, and the target interface is used to add a test case to the test stimulus set when being called.

[0037] In a third aspect, an embodiment of the present specification further provides a computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the testing method as described above when executing the computer program.

[0038] In a fourth aspect, an embodiment of the present specification further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the test method described above is implemented.

[0039] In a fifth aspect, the embodiments of this specification provide a computer program product or a computer program, wherein the computer program product includes a computer program, wherein the computer program is stored in a computer-readable storage medium; the processor of the computer device reads the computer program from the computer-readable storage medium, and the processor implements the steps of the above-mentioned test method when executing the computer program. Optionally, the computer program can be stored in a computer-readable storage medium or in the cloud; the processor of the computer device reads the computer program from the computer-readable storage medium or in the cloud.

[0040] It can be seen from the above technical scheme that the test method provided in the embodiment of this specification determines the target instruction corresponding to the function to be tested based on the function to be tested of the DUT during the test process of the DUT, and generates a test stimulus set based on the target instruction, and the test stimulus set includes a target interface, and the target interface is used to add test cases to the test stimulus set when it is called. In this way, the test method achieves the purpose of expanding the function to be tested related to the target instruction around the target instruction, and generating a test stimulus set based on the target instruction, and reserves the target interface in the test stimulus set to ensure the scalability of the test stimulus set. When the DUT adds a new function due to iteration or optimization, the target interface can be used to add a test case to the original test stimulus set without repeatedly developing the original test stimulus set, which greatly reduces the workload required to generate the test stimulus set and improves the test efficiency. Moreover, since the test stimulus set is generated based on the target instructions, the test stimulus set can perform hierarchical testing on the target instructions, which can meet the testing requirements of each stage in the DUT design process; in addition, even when there are cross-interactions between multiple functions, the cross-interactions between functions can be tested and verified through test cases generated for the target instructions, which is conducive to reducing the difficulty of developing the test stimulus set. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0042] Figure 1 A schematic diagram of a test method provided for one embodiment of this specification;

[0043] Figure 2 A schematic diagram of the structure of a computing device provided for one embodiment of the present specification. DETAILED DESCRIPTION

[0044] Unless otherwise defined, the technical terms or scientific terms used in the embodiments of this specification shall have the common meanings understood by persons with ordinary skills in the field to which this specification belongs. The words "first", "second" and similar words used in the embodiments of this specification do not indicate any order, quantity or importance, but are only used to avoid confusion of constituent elements.

[0045] Unless the context requires otherwise, throughout the specification, "plurality" means "at least two", and "including" is interpreted as an open, inclusive meaning, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "examples", "specific examples" or "some examples" are intended to indicate that a particular feature, structure, material or characteristic associated with the embodiment or example is included in at least one embodiment or example of the specification. The schematic representation of the above terms does not necessarily refer to the same embodiment or example.

[0046] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0047] Overview

[0048] At present, in the testing process of DUTs such as processors, system chips and embedded systems, the development of test stimuli is to write test stimulus programs according to functional categories, and each functional category has its corresponding test stimulus set. This means that when the DUT is updated or optimized, the verification team needs to develop new test stimulus programs to cover the new or optimized functions. In addition, when it is necessary to consider the cross-effects between different functions, the test cases need to be redesigned and developed, which may involve the integration of multiple independent test scripts or programs, resulting in high overall maintenance costs and complexity, requiring the verification team to do repetitive work, resulting in low testing efficiency.

[0049] In order to solve this problem, the inventors discovered through research that, in the test process for the DUT, based on the function to be tested of the DUT, the target instruction corresponding to the function to be tested is determined, and based on the target instruction, a test stimulus set is generated, and the test stimulus set includes a target interface, and the target interface is used to add test cases to the test stimulus set when it is called. In this way, the testing method achieves the purpose of expanding the function to be tested related to the target instruction around the target instruction, and generating a test stimulus set based on the target instruction, and reserves the target interface in the test stimulus set, thereby ensuring the scalability of the test stimulus set. When the DUT adds a new function due to iteration or optimization, new test cases can be added to the original test stimulus set through the target interface, without the need to repeatedly develop the original test stimulus set, which greatly reduces the workload required to generate the test stimulus set and improves the testing efficiency. Moreover, since the test stimulus set is generated based on the target instructions, the test stimulus set can perform hierarchical testing on the target instructions, which can meet the testing requirements of each stage in the DUT design process; in addition, even when there are cross-interactions between multiple functions, the cross-interactions between functions can be tested and verified through test cases generated for the target instructions, which is conducive to reducing the difficulty of developing the test stimulus set.

[0050] Based on the above concept, the embodiment of this specification provides a testing method, and the testing method provided in the embodiment of this specification will be exemplarily described below in conjunction with the accompanying drawings.

[0051] Exemplary Methods

[0052] To be applied to Figure 1 Taking the DUT in as an example, some embodiments of this specification exemplarily illustrate the test method, and the test method includes:

[0053] S101: In response to a test operation on a device under test (DUT), a test process is executed;

[0054] The testing process includes:

[0055] S1011: Based on a function to be tested of the DUT, determining a target instruction corresponding to the function to be tested, wherein the function to be tested includes an architecture function or a micro-architecture function of the DUT;

[0056] S1012: Generate a test stimulus set based on the target instruction; the test stimulus set includes a target interface, and the target interface is used to add a test case to the test stimulus set when called.

[0057] DUT includes but is not limited to processors, system chips and embedded systems, etc. Architectural Features may refer to functions defined at the instruction set architecture (ISA) level, which can determine how the program interacts with the hardware. The instruction set architecture is the hardware interface seen by the compiler, which defines the format, addressing mode, register set and memory model of the instruction. Microarchitectural Features may refer to functions defined at the hardware implementation level, which can determine how the instruction is executed in the hardware. The microarchitectural function is a specific implementation of the ISA, which focuses on how to efficiently execute instructions and improve performance and effectiveness. In some embodiments, the microarchitectural function may include data bypassing. The test operation for the DUT may refer to an operation that triggers the DUT to be tested, such as a click operation on the test start button provided for the human-computer interaction interface, or an input operation of entering a specific instruction or code, which is not limited in this specification.

[0058] The function to be tested may refer to a function that needs to be tested in the DUT. In one embodiment, if a test stimulus set has not been generated for the DUT (i.e., there is no historical test stimulus set), the function to be tested may include at least one architecture function or micro-architecture function possessed by the DUT. If a test stimulus set has been generated for the DUT, the function to be tested may include a newly added or modified architecture function or micro-architecture function in the DUT. In this way, when a test stimulus set is generated for the newly added or modified architecture function or micro-architecture function, the historical test stimulus set can be reused to avoid repeated development of a test stimulus set for a certain function, which is conducive to improving test efficiency.

[0059] During the test process, firstly, for the function to be tested of the DUT, the target instruction corresponding to the function to be tested is determined. The target instruction corresponding to the function to be tested may refer to the instruction related to the function to be tested. For example, assuming that the DUT can implement the FEAT_Pauth (Pointer Authentication) feature, the target instruction may include the PACDA (Pointer Authentication with Cryptographic Derivation and Application) instruction related to the feature, etc. Among them, the FEAT_Pauth feature is a feature used to enhance security in the DUT. It authenticates the pointer by introducing special instructions to prevent control flow attacks. The PACDA instruction can be one of a series of instructions that can be used to implement pointer authentication. While performing data operations, it adds an authentication code to ensure the security of the data.

[0060] After determining the target instruction, a test stimulus set can be generated based on the target instruction. Since the test stimulus set is generated based on the target instruction, the test stimulus set can verify the encoding and basic functions of the instruction itself, and can also verify the calculation and processing of data in combination with data verification instructions. It can also verify the calculation and processing of instructions in complex scenarios, so as to achieve all-round verification of instructions and functions through test case sets at different levels. Even when there is cross-interaction between multiple functions, the cross-interaction between functions can be tested and verified through test cases generated for the target instruction, which helps to reduce the difficulty of developing the test stimulus set.

[0061] In general, in the test process for DUT, based on the function to be tested of DUT, the target instruction corresponding to the function to be tested is determined, and based on the target instruction, a test stimulus set is generated, and the test stimulus set includes a target interface, and the target interface is used to add test cases to the test stimulus set when called. In this way, the test method realizes the purpose of expanding the function to be tested related to the target instruction around the target instruction, and generating the test stimulus set based on the target instruction, and the target interface is reserved in the test stimulus set to ensure the scalability of the test stimulus set. When the DUT adds a new function due to iteration or optimization, a new test case can be added to the original test stimulus set through the target interface, without the need to repeatedly develop the original test stimulus set, which greatly reduces the workload required to generate the test stimulus set and improves the test efficiency. In addition, since the test stimulus set is generated based on the target instruction, the test stimulus set can be tested hierarchically for the target instruction, which can meet the test requirements of each stage in the DUT design process.

[0062] In an optional implementation, in order to test the DUT at different stages in the hierarchical development process, there are multiple test stimulus sets, and different test stimulus sets have different degrees of relevance to the function to be tested corresponding to the target instruction.

[0063] Thus, in this embodiment, through the test stimulus sets with different relevance to the function to be tested, the test requirements of the DUT at different stages of the hierarchical development process can be met. For example, in the initial stage of DUT development, the test stimulus set with the lowest relevance to the function to be tested can be used to encode the target instruction itself and test the basic functions. As the development progresses, the test stimulus sets with gradually increasing relevance to the function to be tested can be used for testing to meet the functional verification requirements of the development results at different stages. Optionally, in one embodiment, each of the test stimulus sets includes a corresponding target interface. Thus, after the DUT is updated or optimized, the target interfaces of different test stimulus sets can be used to enrich and supplement test cases with different relevance to meet the test requirements of new functions.

[0064] In an optional implementation, a feasible generation method of multiple test stimulus sets is proposed. Specifically, generating the test stimulus set based on the target instruction includes:

[0065] Based on the target instruction, generating a first set, wherein the first set is used to verify the encoding and basic functions of the target instruction;

[0066] Based on the target instruction, generating a second set, wherein the second set is used to verify the calculation and processing process of the target instruction on the data;

[0067] Based on the target instruction, generating a third set, the third set is used to verify the calculation and processing process of the target instruction in a target scenario, the target scenario including a data bypass scenario;

[0068] The first set, the second set and the third set have increasing relevance to the functions to be tested corresponding to the target instructions, and the first set, the second set and the third set respectively correspond to different target interfaces.

[0069] In this embodiment, through the hierarchical design of the first set, the second set and the third set, the verification and testing requirements for the encoding and basic functions of the target instructions, the calculation and processing process of the target instructions on data, and the calculation and processing process of the target instructions in the target scenario are met, thereby achieving the purpose of hierarchical verification.

[0070] In some implementations, a feasible generation method of the first set, the second set, and the third set is provided. Specifically, in one implementation, generating the first set based on the target instruction includes:

[0071] Determining an operation code and an operand of the target instruction;

[0072] The first set is generated based on an opcode and operands of the target instruction.

[0073] Assuming that the target instruction is an addition instruction (ADD instruction), the addition instruction can be used to add the values ​​of two registers and store the result in another register.

[0074] As for the operation code (Opcode), in a feasible implementation, the operation code of the addition instruction may be "0b000000". The operation code is a part of the instruction, which tells the processor what operation should be performed.

[0075] For operands, the operands can contain the memory location of the data to be operated on in a register or directly the data itself. Operands can be immediate values ​​(data given directly in the instruction), register operands (data stored in the CPU's register), or memory operands (data stored in memory, the address of the memory unit is given in the instruction). For the addition instruction, the operands can be three registers, such as "R1", "R2" and "R3".

[0076] The first set generated based on the opcode and operands may include the following test cases:

[0077] - Test case 1: `R1=5`, `R2=10`, expect `R3=15`;

[0078] - Test case 2: `R1=-3`, `R2=2`, expect `R3=-1`;

[0079] - Test case 3: `R1=0`, `R2=0`, expect `R3=0`;

[0080] After the first set is generated, the first level of verification and testing can be performed based on the first set. Specifically, instructions can be executed and the results can be verified: that is, each test case is converted into machine code, and the addition instruction is executed through a simulator or hardware environment, the value of the R3 register is read, and compared with the expected value in the test case.

[0081] Then confirm the instruction recognition and execution status: observe the status of instruction execution and ensure that no exceptions occur. Ensure that the instruction encoding is correctly parsed and there are no errors in the execution path.

[0082] Finally, record and analyze the results: collect the results of each test case, record the success and failure. If the test fails, you can analyze the possible reasons (such as coding errors, data dependencies, etc.), and make corresponding repairs and retest.

[0083] Through this process, developers can verify the encoding and basic functions of the target instruction (such as the addition instruction) to ensure that it can be executed correctly in the actual environment. Such a first set is not only applicable to a single target instruction, but can also be extended to more complex instructions and functional modules.

[0084] In one embodiment, generating the second set based on the target instruction includes:

[0085] Determining data to be tested and test conditions, wherein the data to be tested includes different data types corresponding to the target instruction, and the test conditions include overflow conditions;

[0086] The second set is generated based on the data to be tested and the test condition.

[0087] Based on the first set, the second set can introduce more test scenarios, such as involving the calculation and processing of data by the target instruction. For example, still taking the addition instruction as an example, its response to different data ranges, overflow conditions, etc. can be verified.

[0088] For example: the test scenarios for the addition instruction can be enriched: Based on the first set, we will introduce more complex test scenarios for the addition instruction to verify its response under different data ranges and overflow conditions.

[0089] 1) Determine the test scenario:

[0090] Determine the data to be tested and introduce different data types (for addition instructions, different data types may include positive numbers, negative numbers, zero, and limit values, etc.);

[0091] Design test conditions, such as designing overflow condition tests to verify the behavior of addition instructions in overflow and non-overflow situations.

[0092] 2) Generate test cases:

[0093] Generate each test case in the second set based on the data to be tested and the test condition:

[0094] -Scene 1: Normal addition;

[0095] - Test case 1: `R1=10`, `R2=20`, expected `R3=30`;

[0096] -Scene 2: Addition of negative numbers;

[0097] - Test case 2: `R1=-10`, `R2=5`, expect `R3=-5`;

[0098] - Scenario 3: Zero-involvement addition;

[0099] - Test case 3: `R1=0`, `R2=100`, expected `R3=100`;

[0100] -Scenario 4: Overflow situation;

[0101] - Test case 4: `R1=2147483647`, `R2=1` (assuming 32-bit integers), expecting overflow flag to be checked;

[0102] - Test case 5: `R1=-2147483648`, `R2=-1` (assuming 32-bit integers), expecting overflow flag to be checked.

[0103] After the second set is generated, the second level of verification and testing can be performed based on the second set. Specifically, the instructions can be executed and the results can be verified: that is, each test case is converted into machine code, and the addition instruction is executed through a simulator or hardware environment, and the test result is compared with the expected value in the test case. For overflow, check whether the overflow flag is set correctly.

[0104] Then analyze the results: Collect and analyze the results to ensure that the addition instructions behave as expected for different data ranges and overflow conditions.

[0105] Confirm instruction recognition and execution status: Observe the status of instruction execution to ensure that no exceptions occur. Ensure that the instruction encoding is correctly parsed and there are no errors in the execution path.

[0106] Finally, record and analyze the results: collect the results of each test case, record the success and failure. If the test fails, you can analyze the possible reasons, and make corresponding repairs and retest.

[0107] The main differences between the test cases in the second set and the first set are:

[0108] 1. Scenario complexity: The test cases in the second set cover more types of data (such as negative numbers, zero, and extreme values), while the test cases in the first set may only involve simple addition of positive numbers.

[0109] 2. Overflow conditions: The second set pays special attention to overflow conditions, which not only verifies whether the results are correct, but also checks whether the DUT can correctly identify the overflow state.

[0110] 3. Data range: In the second set, the test cases include boundary conditions of the data (such as maximum and minimum integers) to ensure the stability and reliability of the instructions under extreme conditions.

[0111] With these extensions, testing will be more comprehensive, helping to uncover potential edge cases and shortcomings in DUT design.

[0112] In one embodiment, generating the third set based on the target instruction includes:

[0113] In the target scenario, a third set corresponding to the target instruction is generated, and in the test cases in the third set, the operands of the target instruction are derived from the processing results of other instructions.

[0114] Based on the second set, the third set introduces more complex target scenarios, which can include data bypass. This means that in the test cases of the third set, the operands of the target instruction may come from the results of other instructions executed previously, rather than just reading from registers or memory.

[0115] For addition instructions, a data bypass scenario can be introduced for the addition instruction to test the situation when the operand of the target instruction is derived from the result of the previous instruction execution.

[0116] 1) Determine the target scenario: Design a data bypass scenario in which the operands of the addition instruction in the test case come from the previous calculation results instead of being read directly from registers or memory.

[0117] 2) Test cases in the third set generated:

[0118] -Scenario 1: Simple data bypass:

[0119] -Test Case 1:

[0120] - Instruction 1: `ADD R1, R2, R3` (assuming `R2=5`, `R3=10`, then `R1=15`)

[0121] - Instruction 2: `ADD R4, R1, R5` (assuming `R5 = 20`, expect `R4 = 35`, this

[0122] When `R4` depends on the result of `R1`); ADD means addition;

[0123] -Scenario 2: Chained data bypass:

[0124] - Test case 2:

[0125] - Instruction 1: `ADD R1, R2, R3` (`R2 = 5`, `R3 = 10`, `R1 = 15`);

[0126] - Instruction 2: `SUB R6,R1,R5` (assuming `R5=5`, then `R6=10`);

[0127] - Instruction 3: `ADD R4, R6, R1` (in this case `R4 = 25`, depending on the results of `R6` and `R1`); where SUB means subtraction;

[0128] -Scene 3: Check bypass effect:

[0129] - Test case 3:

[0130] - Instruction 1: `ADD R1, R2, R3` (`R2 = 3`, `R3 = 4`, `R1 = 7`);

[0131] - Instruction 2: `MUL R5, R1, R1` (multiplication, expecting `R5 = 49`, checking whether data bypass works correctly). MUL means multiplication.

[0132] After the third set is generated, the third level of verification and testing can be performed based on the third set. Specifically, the instructions can be executed and the results can be verified: that is, each test case is converted into machine code, and the addition instruction is executed through a simulator or hardware environment, and the test result is compared with the expected value in the test case to verify whether the instruction result meets the expectation. Record whether the data bypass is used correctly during execution, and confirm whether the result reflects the dependent instructions.

[0133] Then analyze the results: Collect and analyze the results to ensure that the addition instruction can be executed correctly in the data bypass scenario.

[0134] Confirm instruction recognition and execution status: Observe the status of instruction execution to ensure that no exceptions occur. Ensure that the instruction encoding is correctly parsed and there are no errors in the execution path.

[0135] Finally, record and analyze the results: collect the results of each test case to ensure that the target instructions can be executed correctly in the data bypass scenario.

[0136] In some embodiments, when generating the third set, some executable conditions may also be considered, such as the executable conditions of encryption control (for example, the EnDA control bit 27 in the system control register SCTLR_ELx will affect the execution behavior of the PACDA instruction, indicating whether the general registers will be encrypted). When generating test cases, the execution condition can be achieved through other instructions before the target instruction is executed.

[0137] The main differences between the test cases in the third set and the second set are:

[0138] 1. Data source: The test cases in the third set emphasize that the operands come from the results of previous instructions, rather than being directly obtained from registers or memory. This data dependency increases the complexity of the test and the simulation of real scenarios.

[0139] 2. Instruction chaining: This step introduces interdependencies between multiple instructions (e.g., chained data bypass). The test not only focuses on the functionality of a single instruction, but also verifies the correctness of the instruction sequence.

[0140] 3. Bypass effect: The test cases pay special attention to the bypass effect to ensure that the result of the previous instruction can be used by the subsequent instructions in a timely manner, verifying the efficiency and accuracy of the microarchitecture in data processing.

[0141] By introducing data bypass testing, the actual application scenarios can be better simulated and the comprehensiveness and accuracy of instruction function verification can be improved.

[0142] In one implementation of the present specification, a process for determining a function to be tested is provided. Specifically, the process may include:

[0143] Determining, according to the value of the target system register, the degree to which the DUT supports the function represented by the target system register;

[0144] When the DUT supports the function represented by the target system register, the function represented by the target register is determined as the function to be tested, the target instruction corresponding to the function to be tested is in a target test state, and the target instruction representation in the target test state has been defined and can be executed by the DUT instruction.

[0145] For example, assuming that the DUT can implement the FEAT_PAuth feature, the value in the system register ID_AA64ISAR1_EL1.{GPI, GPA, API, APA} (target system register) will indicate the degree of support for the feature, and the instructions PACDA and others related to the feature will be changed from the original undef test state to the define space test state. In the system register ID_AA64ISAR1_EL1, different values ​​of GPI (Generic Pointer Authentication Implementation) indicate whether the DUT implements the generic authentication using the IMPLEMENTATION DEFINED algorithm; different values ​​of GPA (QARMA5Generic Pointer Authentication Algorithm) indicate whether the DUT supports the implementation of the generic authentication using the QARMA5 algorithm; different values ​​of API (Pointer Authentication Code for Instruction Address) indicate whether the DUT supports the implementation of pointer authentication for instruction addresses; different values ​​of APA (Pointer Authentication Code for Data Address) indicate whether the DUT supports the implementation of pointer authentication for data addresses (including instructions such as PACDA).

[0146] By measuring the values ​​of the target system registers, the degree to which the DUT supports each function can be accurately determined, and by recording changes in the values ​​of the target system registers, changes in the functions supported by the DUT can be quickly determined.

[0147] In one embodiment, when the target instruction includes a memory access instruction, the test stimulus set includes a fourth set, and the test process further includes:

[0148] Determine the target memory address and the type of memory data to be accessed;

[0149] Based on the target memory address and the memory access data type, the fourth set corresponding to the memory access instruction is generated, and the fourth set is used to verify whether the memory access instruction can correctly access the target memory address and / or whether it can correctly operate data of the memory access data type.

[0150] In this embodiment, the expandable test scenarios include: memory address access testing, data integrity and consistency of memory access instructions, memory access conflicts, memory access delays and sequences, testing of special memory areas, and exception handling of memory access instructions.

[0151] Taking the memory address access test as an example, the memory address access test is designed to verify how the memory access instruction interacts with the memory correctly. The following is a feasible implementation process for generating the fourth set:

[0152] 1) Determine the test objectives: Verify that memory access instructions (such as `LDR` (Load Register, load memory contents into registers) and `STR` (Store Register, store register contents into memory)) can correctly access the specified memory address; check reading and writing under different data types (such as bytes, half words, words) and alignment requirements.

[0153] 2) Design test cases in the fourth set:

[0154] -Test case 1: Basic memory access

[0155] - `STR R1,[R2]`: Write the value of `R1` to address `R2`.

[0156] - `LDR R3,[R2]`: reads data from address `R2` to `R3`.

[0157] -Verify that the value of `R3` is equal to `R1`.

[0158] -Test case 2: Access of different data types

[0159] - `STRB R1,[R2]`: Write the lowest byte of `R1` to address `R2`.

[0160] - `LDRB R3,[R2]`: Read a byte from address `R2` to `R3`.

[0161] - Verify that `R3` is equal to the lowest byte of `R1`.

[0162] -Test case 3: memory alignment test

[0163] - `STR R1,[R2]` (assuming `R2` is misaligned and points to an odd address).

[0164] - Check if the system throws an alignment exception (if the system requires alignment).

[0165] After the fourth set is generated, the DUT can be verified and tested for memory access based on the fourth set. Specifically, instructions can be executed and the results can be verified: that is, each test case is converted into machine code, and memory access instructions are executed through a simulator or hardware environment, and memory status is checked using a memory monitoring tool (such as a debugger) to confirm whether data is read and written correctly.

[0166] Handle special cases during testing: Memory boundary conditions: Test the boundary values ​​of memory access (such as minimum and maximum addresses) to confirm that the DUT behaves as expected. Illegal memory access: Attempt to access unallocated or protected memory areas to verify that the DUT can handle these exceptions correctly.

[0167] Finally, record and analyze the results: collect the results of each test case to ensure that all memory accesses are as expected. If the test fails, analyze the cause of the error, fix the code and retest.

[0168] The main considerations in the generation process of the fourth set may include: Address range: Ensure that the test cases cover valid and invalid memory addresses. Data consistency: Verify the consistency and integrity of data in multiple operations to ensure that there are no data errors caused by multiple writes. Exception handling: Ensure that the system can correctly handle alignment errors and illegal accesses and provide appropriate exception responses.

[0169] Through such a testing process, the memory interaction capability of memory access instructions can be effectively verified, and the stability and reliability of the system can be ensured.

[0170] In an optional implementation, if there is a historical test stimulus set, the test process further includes:

[0171] The target interface included in the historical test stimulus set is called to add the test cases in the currently generated test stimulus set to the historical test stimulus set.

[0172] In this embodiment, a feasible method is provided for adding the currently generated test case to the historical test stimulus set when there is a historical test stimulus set. In this way, the historical test stimulus set can be utilized, duplication of development work can be avoided, and testing efficiency can be improved.

[0173] In general, by using the test method provided in the embodiments of this specification, the complexity and richness of the test are gradually increased through a hierarchical development method, so that the correctness and performance of the DUT design can be more comprehensively verified. This method helps to capture and resolve potential design defects and ensure the quality and reliability of the final product.

[0174] Exemplary Devices

[0175] In an exemplary embodiment of the present specification, a testing device is also provided, the testing device comprising:

[0176] A test module, configured to execute a test process in response to a test operation on a device under test (DUT);

[0177] The testing process includes:

[0178] Based on a function to be tested of the DUT, determining a target instruction corresponding to the function to be tested, wherein the function to be tested includes an architectural function or a micro-architectural function of the DUT;

[0179] Based on the target instruction, a test stimulus set is generated; the test stimulus set includes a target interface, and the target interface is used to add a test case to the test stimulus set when being called.

[0180] For the specific definition of the test device, please refer to the definition of the test method above, which will not be repeated here. Each module in the above-mentioned test device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0181] Exemplary Computing Devices

[0182] Another embodiment of the present application further provides a computing device, see Figure 2 As shown, an exemplary embodiment of the present specification also provides a computing device, including: a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the testing method according to various embodiments of the present specification described in the above embodiments of the present specification are executed.

[0183] The internal structure of the computing device can be as follows Figure 2 As shown, the computing device includes a processor, a memory, a network interface and an input device connected through a system bus. Among them, the processor of the computing device is used to provide computing and control capabilities. The memory of the computing device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computing device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps of the test method according to various embodiments of the present specification described in the above embodiments of the present specification are performed.

[0184] The processor may include a main processor and may also include a baseband chip, a modem, etc.

[0185] The memory stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes computer operation instructions. More specifically, the memory may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0186] The processor may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0187] The input device may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0188] Output devices may include means that allow information to be output to a user, such as display screens, printers, speakers, etc.

[0189] The communication interface may include using any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0190] The processor executes the program stored in the memory and calls other devices, which can be used to implement each step of any test method provided in the above embodiments of the present application.

[0191] The computing device may also include a display component and a voice component. The display component may be a liquid crystal display or an electronic ink display. The input device of the computing device may be a touch layer covered on the display component, or a button, trackball or touchpad provided on the housing of the computing device, or an external keyboard, touchpad or mouse.

[0192] Those skilled in the art will understand that Figure 2The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of this specification, and does not constitute a limitation on the computing device to which the scheme of this specification is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0193] Exemplary computer program products and storage media

[0194] In addition to the above-mentioned methods and devices, the testing method provided in the embodiments of this specification may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the testing method according to various embodiments of this specification described in the above "Exemplary Method" section of this specification.

[0195] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK).

[0196] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the embodiments of the present specification, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0197] In addition, an embodiment of the present specification also provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to execute the steps of the testing method according to various embodiments of the present specification described in the above "Exemplary Method" section of the present specification.

[0198] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this specification can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0199] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0200] The above-mentioned embodiments only express several implementation methods of this specification, and the descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the solutions provided by the embodiments of this specification. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of this specification, which all belong to the protection scope of this specification. Therefore, the protection scope of the patent of this specification shall be based on the attached claims.

Claims

1. A testing method, characterized in that: include: In response to a test operation on the device under test (DUT), a test process is executed; The testing process includes: Based on a function to be tested of the DUT, determining a target instruction corresponding to the function to be tested, wherein the function to be tested includes an architectural function or a micro-architectural function of the DUT; Based on the target instruction, a test stimulus set is generated; the test stimulus set includes a target interface, and the target interface is used to add a test case to the test stimulus set when being called.

2. The method according to claim 1, characterized in that There are multiple test stimulus sets, and different test stimulus sets have different degrees of relevance to the function to be tested corresponding to the target instruction.

3. The method according to claim 1, characterized in that: The generating a test stimulus set based on the target instruction includes: Based on the target instruction, generating a first set, wherein the first set is used to verify the encoding and basic functions of the target instruction; Based on the target instruction, generating a second set, wherein the second set is used to verify the calculation and processing process of the target instruction on the data; Based on the target instruction, generating a third set, the third set is used to verify the calculation and processing process of the target instruction in a target scenario, the target scenario including a data bypass scenario; The first set, the second set and the third set have increasing relevance to the functions to be tested corresponding to the target instructions, and the first set, the second set and the third set respectively correspond to different target interfaces.

4. The method according to claim 3, characterized in that The generating a first set based on the target instruction comprises: Determining an operation code and an operand of the target instruction; The first set is generated based on an opcode and operands of the target instruction.

5. The method according to claim 3, characterized in that: The generating the second set based on the target instruction comprises: Determining data to be tested and test conditions, wherein the data to be tested includes different data types corresponding to the target instruction, and the test conditions include overflow conditions; The second set is generated based on the data to be tested and the test condition.

6. The method according to claim 3, characterized in that The generating a third set based on the target instruction comprises: In the target scenario, a third set corresponding to the target instruction is generated, and in the test cases in the third set, the operands of the target instruction are derived from the processing results of other instructions.

7. The method according to any one of claims 1 to 6, characterized in that: The process of determining the function to be tested of the DUT includes: Determining, according to the value of the target system register, the degree to which the DUT supports the function represented by the target system register; When the DUT supports the function represented by the target system register, the function represented by the target register is determined as the function to be tested, the target instruction corresponding to the function to be tested is in a target test state, and the target instruction representation in the target test state has been defined and can be executed by the DUT instruction.

8. The method according to any one of claims 1 to 6, characterized in that: When the target instruction includes a memory access instruction, the test stimulus set includes a fourth set, and the test process further includes: Determine the target memory address and the type of memory data to be accessed; Based on the target memory address and the memory access data type, the fourth set corresponding to the memory access instruction is generated, and the fourth set is used to verify whether the memory access instruction can correctly access the target memory address and / or whether it can correctly operate data of the memory access data type.

9. The method according to any one of claims 1 to 6, characterized in that: If there is a historical test stimulus set, the test process further includes: The target interface included in the historical test stimulus set is called to add the test cases in the currently generated test stimulus set to the historical test stimulus set.

10. A computing device, characterized in that The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the test method according to any one of claims 1 to 9 when executing the computer program.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the testing method according to any one of claims 1 to 9 is implemented.

Images