Chip testing method, device, equipment, storage medium and program product

Abstract

The application provides a chip test method, device, equipment, storage medium and program product. The method comprises the following steps: obtaining operator code of a target operator comprising a plurality of chip interface instructions, the target operator and a benchmark operator having the same calculation function, the chip interface instruction being used for implementing at least one hardware operation of a chip, and the hardware operation of the plurality of chip interface instructions being used for implementing the calculation function; analyzing the operator code to obtain a machine code file of the chip; obtaining test data and transmitting the test data and the machine code file to a chip simulation device, the chip simulation device being used for executing the machine code file on the test data to obtain a simulation calculation result of the target operator for the test data; obtaining the simulation calculation result from the chip simulation device and obtaining a benchmark calculation result of the benchmark operator for the test data; and generating a test result of the chip based on the simulation calculation result and the benchmark calculation result. Through the application, the chip test progress can be accelerated.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a chip testing method, apparatus, device, storage medium, and program product. Background Technology

[0002] Chip design involves millions of lines of hardware and software code. Functional verification is a crucial step in ensuring the correctness of the chip design, requiring multi-stage testing including software simulation, hardware simulation, and prototype verification. However, after the chip hardware design is completed, the development of the software stack often lags behind the hardware progress. The incompleteness of the software stack prevents timely prototype verification of the chip, thus slowing down the chip testing schedule. Summary of the Invention

[0003] This application provides a chip testing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can accelerate the chip testing process.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a chip testing method, including:

[0006] The target operator is obtained by acquiring operator code including multiple chip interface instructions, wherein the target operator and the reference operator have the same computational function, the chip interface instructions are used to implement at least one hardware operation of the chip, and the hardware operation combining the multiple chip interface instructions is used to implement the computational function.

[0007] The operator code is parsed to obtain the machine code file of the chip;

[0008] Acquire test data and transmit the test data and the machine code file to a chip simulation device, wherein the chip simulation device is used to execute the machine code file on the test data to obtain the simulation calculation result of the target operator for the test data;

[0009] The simulation calculation results are obtained from the chip simulation device, and the benchmark calculation results of the benchmark operator for the test data are obtained.

[0010] Based on the simulation results and the benchmark results, the test results of the chip are generated.

[0011] This application also provides a chip testing apparatus, including:

[0012] The first acquisition module is used to acquire the operator code of the target operator, which includes multiple chip interface instructions. The target operator and the reference operator have the same calculation function. The chip interface instructions are used to implement at least one hardware operation of the chip, and the hardware operation combining the multiple chip interface instructions is used to implement the calculation function.

[0013] The parsing module is used to parse the operator code to obtain the machine code file of the chip;

[0014] A transmission module is used to acquire test data and transmit the test data and the machine code file to a chip simulation device, wherein the chip simulation device is used to execute the machine code file on the test data to obtain the simulation calculation result of the target operator on the test data;

[0015] The second acquisition module is used to acquire the simulation calculation results from the chip simulation device and to acquire the benchmark calculation results of the benchmark operator for the test data;

[0016] The generation module is used to generate test results for the chip based on the simulation calculation results and the benchmark calculation results.

[0017] This application also provides an electronic device, including:

[0018] Memory is used to store executable instructions for a computer;

[0019] The processor, when executing computer-executable instructions stored in the memory, implements the chip testing method provided in the embodiments of this application.

[0020] This application also provides a computer-readable storage medium storing computer-executable instructions or computer programs, which, when executed by a processor, implement the chip testing method provided in this application.

[0021] This application also provides a computer program product, including computer-executable instructions or a computer program, which, when executed by a processor, implements the chip testing method provided in this application.

[0022] The embodiments of this application have the following beneficial effects: By combining chip interface instructions to implement the operator code of the target operator (which has the same computational function as the benchmark operator), and since the chip interface instructions can implement at least one hardware operation of the chip, parsing the operator code can obtain the chip's machine code file. Then, by using a chip simulation device to simulate the chip and run the machine code file, simulation calculation results are obtained. Finally, the simulation calculation results are combined with the benchmark calculation results to generate the chip's test results. Thus, by simply implementing the operator code based on the chip interface instructions and then combining it with a chip simulation device for operator simulation, chip testing can be achieved at the operator level in advance, without waiting for the software stack development to be complete, thus accelerating the chip testing process. Furthermore, combining the simulation calculation results with the benchmark calculation results to generate test results ensures the completeness of the chip testing. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the chip testing system provided in the embodiments of this application;

[0024] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0025] Figure 3 This is a schematic diagram of the first process of the chip testing method provided in the embodiments of this application;

[0026] Figure 4 This is a schematic diagram of the second process of the chip testing method provided in the embodiments of this application;

[0027] Figure 5 This is a schematic diagram of the third process of the chip testing method provided in the embodiments of this application;

[0028] Figure 6 This is a schematic diagram of the fourth process of the chip testing method provided in the embodiments of this application;

[0029] Figure 7 This is a schematic diagram of the fifth process of the chip testing method provided in the embodiments of this application.

[0030] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0033] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0034] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of a larger module or unit that includes the functionality of the module or unit.

[0035] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0036] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0037] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0038] 1) A target operator refers to a basic functional unit in a chip that implements specific computational functions (such as matrix multiplication, attention mechanism algorithms, activation functions, etc. in deep learning). Its core characteristics are: it is implemented by encoding chip interface instructions (API instructions) obtained from at least one chip instruction package. The interface and computational functions must be consistent with the benchmark operators in deep learning frameworks (such as Torch). It is the basic functional unit for accelerated computation of the chip, and its computational logic must be implemented through operator code. The target operator is the test object in the prototype verification stage. By running the execution logic of the target operator on a chip simulation device, the correctness and completeness of the chip hardware design can be verified, and its test scenario closely resembles the actual operation of the product.

[0039] 2) Chip interface instructions, or API instructions, are based on at least one pre-packaged application programming interface (API) of chip instructions (i.e., the upper-level interface of chip instructions). Each API instruction is used to implement one or a set of basic hardware operations of the chip (such as data loading, thread configuration, and operation execution), but it hides the details of the underlying binary instructions, presenting them in a more concise function or interface form. In this way, API instructions simplify direct manipulation of the chip's underlying hardware instructions, providing developers with a more user-friendly programming interface. Developers do not need to directly handle binary instructions; they only need to call the interface to control the hardware, facilitating software-level coding of operators. Using API instructions, operator code that implements specific computational functions (such as convolution and pooling in deep learning) can be written; that is, combining multiple API instructions can achieve specific computational functions.

[0040] 3) Chip instructions are a set of low-level operation rules that chip hardware can directly recognize and execute, serving as the fundamental specifications for chip design. They define the core elements of the basic hardware operations a chip can perform (such as data transmission, arithmetic, and logical judgments), opcodes (binary codes identifying specific operations), operand formats (such as register addresses and memory addresses), and addressing modes. They are the "low-level language" for hardware-software interaction. Chip instructions are the foundation for API instruction encapsulation and the basis for compilers or simulators to parse API instructions in operator code. In other words, API instructions must be parsed into binary instructions (sequences of 0s and 1s) conforming to the chip instruction set before they can be executed by the chip hardware.

[0041] 4) Chip simulation equipment, also known as emulator equipment, is a dedicated testing platform for the chip prototype verification stage. It is used to simulate the chip's operating environment after the chip hardware design is completed and before actual tape-out production, verifying the functionality and performance of the chip's hardware design. Specifically, the chip's hardware design (such as instruction set, operator execution logic, and memory format) needs to be tested using emulator equipment to verify the correctness, completeness, and performance of the chip's hardware design. In short, emulator equipment is the "simulation carrier" for chip prototype verification. The chip's hardware design is verified through simulation testing using emulator equipment, thereby accelerating the chip's progress from design to tape-out.

[0042] This application provides a chip testing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can accelerate the chip testing process. The following is a detailed description of the embodiments of this application based on the above explanation of the terms and concepts used.

[0043] The chip testing system provided in the embodiments of this application is described below. See also... Figure 1 , Figure 1 This is a schematic diagram of the architecture of a chip testing system provided in an embodiment of this application. To support an exemplary application, the chip testing system 100 includes a terminal 200 and a chip simulation device 300.

[0044] In this process, terminal 200, in response to a chip test command (such as one triggered by a user on terminal 200), acquires the operator code of the target operator, which includes multiple chip interface instructions. The target operator and the benchmark operator have the same computational function. The chip interface instructions are used to implement at least one hardware operation of the chip, and the combination of multiple chip interface instructions enables the computational function. The operator code is parsed to obtain the chip's machine code file. Test data is acquired, and the test data and machine code file are transmitted to chip simulation device 300. Chip simulation device 300 receives the test data and machine code file sent by terminal 200. The machine code file is executed on the test data to obtain the simulation calculation result of the target operator for the test data. Terminal 200 obtains the simulation calculation result from chip simulation device 300 and the benchmark calculation result of the benchmark operator for the test data. Based on the simulation calculation result and the benchmark calculation result, the chip's test result is generated.

[0045] In some embodiments, the chip testing method provided in this application is implemented by an electronic device. For example, it can be implemented by a terminal alone, by a server alone, or by a terminal and a server working together. The electronic device implementing the chip testing method provided in this application can be various types of terminals or servers. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal can be a laptop, tablet, desktop computer, etc., but is not limited to these. The terminal and server can be connected directly or indirectly through wired or wireless communication, and this application does not impose any restrictions on this.

[0046] The following describes an electronic device implementing the chip testing method provided in an embodiment of this application. See also... Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 500 provided in this embodiment can be a terminal or a server. Figure 2 As shown, electronic device 500 includes at least one processor 510, memory 550, at least one network interface 520, and user interface 530. The various components in electronic device 500 are coupled together via a bus system 540. It is understood that the bus system 540 is used to implement communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 2 The general labeled all buses as Bus System 540.

[0047] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0048] User interface 530 includes one or more output devices 531 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0049] Memory 550 may be removable, non-removable, or a combination thereof. Memory 550 may include one or more storage devices physically located away from processor 510. Memory 550 may include volatile memory or non-volatile memory, or both. Non-volatile memory may be read-only memory (ROM), and volatile memory may be random access memory (RAM). The memory 550 described in this application embodiment is intended to include any suitable type of memory.

[0050] In some embodiments, memory 550 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, as illustrated below. Operating system 551 includes system programs for handling various basic system services and performing hardware-related tasks, such as a framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks; network communication module 552 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including Bluetooth, Wi-Fi, and Universal Serial Bus (USB); presentation module 553 is used to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 531 associated with user interface 530 (e.g., a display screen, a speaker, etc.); input processing module 554 is used to detect and translate one or more user inputs or interactions from one or more input devices 532.

[0051] In some embodiments, the chip testing apparatus provided in this application can be implemented in software. Figure 2 A chip testing device 555 stored in memory 550 is shown. It can be software in the form of programs and plug-ins, including the following software modules: a first acquisition module 5551, a parsing module 5552, a transmission module 5553, a second acquisition module 5554, and a generation module 5555. These modules are logically related and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0052] The chip testing method provided in the embodiments of this application is described below. As mentioned above, the chip testing method provided in the embodiments of this application is implemented by an electronic device, such as a server or terminal alone, or a server and terminal working together. Therefore, the executing entity of each step will not be described again below. See Figure 3 , Figure 3 This is a first flowchart illustrating the chip testing method provided in this application embodiment. The chip testing method provided in this application embodiment includes:

[0053] Step 101: Obtain the operator code of the target operator, which includes multiple chip interface instructions.

[0054] Among them, the target operator and the reference operator have the same computational function, the chip interface instructions are used to implement at least one hardware operation of the chip, and the hardware operation of combining multiple chip interface instructions is used to implement the computational function.

[0055] For step 101, the operator code of the target operator is first obtained. This operator code can be obtained in advance through programming. The operator code includes multiple chip interface instructions. By combining the hardware operations of these multiple chip interface instructions, the operator code achieves the same computational function as the benchmark operator. It should be noted that the operator has a clear input / output format, functional definition, and behavioral logic. Here, the target operator is required to be completely matched with the benchmark operator in terms of interface (such as input data type, quantity, and output result format) and computational function. Chip interface instructions are application programming interfaces (APIs) (i.e., upper-level interfaces of chip instructions) that are pre-encapsulated based on at least one chip instruction. Each API instruction is used to implement one or a set of basic hardware operations of the chip (such as data loading, thread configuration, and operation execution). Chip instructions are a set of low-level operation rules that the chip hardware can directly recognize and execute, and are the basic specifications for chip design.

[0056] In some embodiments, the number of chip interface instructions is M, where M is an integer greater than 1; based on the fact that before executing step 101 "obtaining the operator code of the target operator including multiple chip interface instructions", the following steps may also be performed: obtaining N chip instructions, each chip instruction is used to implement a hardware operation of the chip, where N is an integer greater than or equal to M; encapsulating the N chip instructions to obtain M chip interface instructions, each chip interface instruction including at least one chip instruction.

[0057] Here, we first obtain N chip instructions, which constitute a chip instruction set. These chip instructions are a set of low-level operation rules that the chip hardware can directly recognize and execute. By encapsulating the N chip instructions, we obtain M chip interface instructions, each of which includes at least one chip instruction. Since the chip instructions are a set of low-level operation rules that the chip hardware can directly recognize and execute, the chip interface instructions are also hardware instructions that the chip hardware can directly recognize and execute. This ensures that the operator code obtained by programming based on the chip interface instructions also supports chip hardware recognition and execution. This provides a foundation for subsequent operator simulation based on the operator code using chip simulation equipment, thereby enabling chip verification and testing.

[0058] Applying the above embodiments, 1) it lowers the development threshold: without directly manipulating the underlying hardware instructions (i.e., chip instructions), developers can quickly implement operator functions through high-level API instructions, reducing dependence on chip instruction details and improving the development efficiency of operator code. 2) it enhances code compatibility: as an intermediate layer, API instructions can shield the differences between different chip instruction sets, enabling operator code to be reused on multiple hardware platforms and reducing cross-platform adaptation costs. 3) it ensures implementation correctness: API instructions encapsulate verified hardware operation logic, reducing errors that may occur when directly writing operator code and indirectly improving the correctness of operator functions. 4) it facilitates testing and iteration: operator code based on API instructions can be quickly self-tested in simulators, and modifications only require adjusting the API instruction call logic, without refactoring the underlying instructions, accelerating the iteration process. 5) it supports software-hardware collaboration: API instructions establish a standardized interface between software operators and chip instruction sets, enabling software development and hardware design to proceed in parallel, shortening the overall chip R&D cycle.

[0059] In some embodiments, see Figure 4 The process of "obtaining N chip instructions" can be achieved by performing the following steps: Step 201, obtain instruction description files for N chip instructions to be generated. The instruction description files include: the domain segment definition information of multiple instruction domain segments included in each chip instruction to be generated, and the combination format of multiple instruction domain segments; Step 202, for each chip instruction to be generated, generate the domain segment parameters of each instruction domain segment based on the domain segment definition information included in the chip instruction to be generated; Step 203, for each chip instruction to be generated, combine the domain segment parameters of multiple instruction domain segments included in the chip instruction to be generated according to the combination format of multiple instruction domain segments to obtain the chip instruction.

[0060] For step 201, each chip instruction to be generated includes multiple instruction field segments, and the instruction description file includes the field segment definition information and the combination format of each instruction field segment. For example, (1) Field segment definition information: clearly defines the specific rules of the multiple instruction field segments (such as opcode field, address code field, data field, etc.) contained in each chip instruction to be generated, such as the bit width of the opcode field (such as 8 bits), the value range (such as 00H-FFH corresponding to different operations), the addressing mode of the address code field (such as direct addressing, indirect addressing), etc., to ensure that the field segment parameters generated later conform to the hardware recognizable standard. (2) Field segment combination format: specifies the splicing order and overall structure of multiple instruction field segments, such as the fixed order of "opcode field (8 bits) + address code field (16 bits) + data field (32 bits)", to ensure that the final generated chip instruction conforms to the binary format specification of the chip instruction set, providing a prerequisite for subsequent hardware parsing and execution.

[0061] For step 202, based on the segment definition information, a valid specific parameter value is generated for each instruction segment. For example, if the segment definition information for the instruction segment "opcode field" defines "01H corresponds to addition operation", then the segment parameter "01H" is generated for the opcode field of the addition instruction; if the segment definition information for the instruction segment "address code field" defines "16-bit direct addressing, value range 0000H-FFFFH", then the segment parameter "1234H" is generated for the address code field. It should be noted that the generation process of segment parameters must strictly follow the rules of segment bit width, value constraints, etc., to avoid parameters exceeding the hardware's recognizable range and to ensure the independence and validity of the segment parameters for each instruction segment.

[0062] For step 203, the segment parameters of the multiple instruction fields included in the chip instruction to be generated are combined according to the combination format of multiple instruction fields to obtain the chip instruction. For example, the combination format of the chip instruction to be generated is "opcode (8 bits) + address code (16 bits) + data field (32 bits)". If the segment parameters of each instruction field are 01H (8 bits), 1234H (16 bits), and 56789ABC (32 bits), they are concatenated according to the combination format as: 01 12 34 56 78 9A BC (binary format). After combination, a complete chip instruction is formed. This chip instruction conforms to the instruction parsing logic of the chip hardware and can be directly used for subsequent simulation testing, realizing the transformation "from hardware rules to hardware instructions".

[0063] Applying the above embodiments, 1) ensuring the standardization of instruction generation: Step 201 clarifies the domain segment definition information and combination format, providing a unified standard for chip instruction generation and avoiding hardware inability to parse due to format chaos; Steps 202 and 203 generate and combine parameters according to rules, ensuring that chip instructions conform to the chip instruction set specification, laying a correct instruction foundation for subsequent simulation testing. 2) improving instruction generation efficiency: Through the standardized process of "defining according to the description file → generating parameters → combining instructions", N chip instructions can be generated in batches without defining the format and parameters one by one, reducing manual operation and repetitive work, and adapting to the needs of multi-instruction scenarios in chip testing. 3) enhancing testing flexibility and controllability: By modifying the instruction description file (such as adjusting the domain segment definition and combination format), different types and functions of chip instructions can be quickly generated, flexibly adapting to diverse testing needs (such as operator testing and anomaly testing), and facilitating the traceability of instruction generation logic, reducing the difficulty of troubleshooting.

[0064] Step 102: Parse the operator code to obtain the chip's machine code file.

[0065] For step 102, the operator code is parsable. By parsing the operator code through a simulator or compiler, the machine code file that the chip can run can be obtained, thereby ensuring that the chip simulation equipment can run the machine code file to simulate the target operator, and thus verify and test the correctness of the chip design based on the simulation calculation results.

[0066] In some embodiments, before executing step 102 "parse the operator code to obtain the chip's machine code file", the following steps may also be performed: obtain the test code of the target operator. Based on this, step 102 "parse the operator code to obtain the chip's machine code file" can be implemented by performing the following steps: parse the test code to obtain the control logic of the target operator; based on the control logic, parse the operator code to obtain the computation logic of the target operator; integrate the control logic and the computation logic to obtain the machine code file.

[0067] Here, test code was also written for the target operator. This test code can be parsed together with the operator code. Specifically, the test code is first parsed to obtain the control logic of the target operator. For example, this control logic may include: the call statement to the target operator, the result comparison logic, the operator name and parameters of the target operator (such as the address for obtaining input data and the address for obtaining output data when executing machine code). Then, based on the control logic, the operator code is parsed to obtain the computation logic of the target operator. For example, based on the operator call information in the control logic (such as the call statement, operator name and parameters, input data address, output data address, etc.), the operator code of the target operator to be parsed is located, and the operator code is then parsed to obtain the computation logic of the target operator. For example, this computation logic may include: the API instructions called (such as data reading API instructions, operation API instructions, result storage API instructions), the computation flow (such as the loop logic of matrix multiplication, the judgment logic of activation functions, etc.), and the resources required for operator computation (such as temporary storage space, register usage, etc.). Finally, the control logic and computation logic are integrated to obtain the machine code file of the target operator. This machine code file is the chip instruction stream. The chip instruction stream (all of which are machine code) includes, but is not limited to: the operator execution process, the storage address of the data required for operator calculation on the chip, the storage address of the chip instruction stream on the chip, and the storage address of the output data after the operator execution is completed.

[0068] Applying the above embodiments: 1) Adapting to hardware execution: High-level code (i.e., operator code and test code) is converted into a binary chip instruction stream that the chip can directly execute, realizing the mapping from software logic to hardware operation, and enabling the operator function to be executed concretely on the chip simulation platform. 2) Establishing a closed-loop test process: The chip instruction stream integrates the control logic of the test code and the computational logic of the operator code, ensuring that the entire process of test data loading, operator calling, and result output is completely reproduced at the hardware level, providing a foundation for subsequent result verification. 3) Supporting performance analysis: The execution process of the chip instruction stream is directly reflected in the hardware timing, providing accurate instruction-level granularity for extracting performance data such as computation time and resource utilization, and assisting in architecture optimization.

[0069] In some embodiments, the number of target operators is multiple; based on this, see [link to relevant documentation]. Figure 5 Step 102, "parse the operator code to obtain the chip's machine code file," can be achieved by executing the following steps 1021-1024: Step 1021, obtain the dependencies between multiple target operators, and construct a topology graph with the target operators as nodes and the dependencies as directed edges; Step 1022, determine the execution order of the multiple target operators according to the topology graph; Step 1023, parse the operator code to obtain the execution instructions of each target operator; Step 1024, based on the execution order of the multiple target operators, integrate the execution instructions of the multiple target operators to obtain the chip's machine code file.

[0070] For step 1021, the dependencies between multiple target operators are obtained. These multiple target operators, when combined, can achieve more complex operator functions than a single operator, such as specific machine learning models like feature extraction models and image recognition models composed of multiple target operators. These dependencies can include: 1) Data dependencies: If the input of operator B is the output of operator A (e.g., "the output feature map of the convolution operator is used as the input of the activation operator"), then a unidirectional dependency of "operator A → operator B" is formed; 2) Temporal dependencies: If operator C can only start after operators A and B have both been executed (e.g., "the pooling operator needs to wait for the convolution and activation operators to complete"), then a multi-source dependency of "operator A, operator B → operator C" is formed; 3) No dependencies: If the inputs and outputs of operators D and E are independent (e.g., two parallel convolution branches), then the two operators that can be executed in parallel are determined to have no dependency relationship. Dependencies can be extracted by combining the data transfer logic between the target operators in the operator code (e.g., the binding of output addresses and input addresses).

[0071] A topology graph is constructed using target operators as nodes and dependencies as directed edges (representing the direction of dependency). Each node corresponds to a target operator and may include the operator name, function (e.g., convolution, activation), input / output data addresses, and required hardware resources (e.g., computation units, registers). Directed edges represent the direction of dependency; for example, the edge "convolution operator → activation operator" is labeled "data transfer: convolution output address → activation input address". Nodes without dependencies (e.g., operators D and E) are marked as "parallelizable" in the topology graph to provide a basis for subsequent thread allocation.

[0072] For step 1022, the execution order of multiple target operators is determined according to the topology diagram. For example, 1) serial execution: For operator chains with unidirectional dependencies, they are executed in the order of the path in the topology diagram, and the output address of the previous operator is automatically used as the input address of the next operator; (2) parallel execution: For operators without dependencies, they are assigned to different threads for parallel execution. The number of threads and the fetch address are configured through the chip running configuration file to ensure that resources do not conflict; (3) For multi-source dependencies (such as operator A, operator B → operator C), operator A and operator B are executed in parallel first. After both are completed (triggered by the synchronization instruction), operator C is executed in series.

[0073] In step 1023, the execution instructions for each target operator are determined by parsing the operator code. In step 1024, the execution instructions of the multiple target operators are integrated according to their execution order to obtain the chip's machine code file. Thus, when executing this machine code file, the target operators can be executed according to their execution order.

[0074] Applying the above embodiments: 1) Ensuring the correctness of test logic: By clarifying operator dependencies and execution order through topology diagrams, test result distortion caused by disordered execution is avoided, ensuring that the test process on the chip simulation equipment conforms to the operator functional logic, and providing a reliable foundation for subsequent result verification. 2) Improving test efficiency: Sequential parsing and integration of instructions reduces redundant operations, accelerates the generation of machine code files, thereby enabling earlier operator testing, accelerating prototype verification, and shortening the test cycle. 3) Enhancing test completeness: Fully covering multi-operator collaborative scenarios, simulating the real operating environment through orderly integration of instructions, exposing dependency conflicts that are difficult to detect in single-operator testing, and assisting in the prototype verification of chip design.

[0075] Step 103: Obtain test data and transfer the test data and machine code file to the chip simulation device.

[0076] Among them, the chip simulation equipment is used to execute machine code files on the test data to obtain the simulation calculation results of the target operator on the test data.

[0077] For step 103, firstly, test data is acquired. This test data can be pre-generated and will be used synchronously with the benchmark operator to calculate the benchmark results, serving as a comparison benchmark. For example, test data can be generated in the following ways: 1) Generate basic test data: Based on the calculation functions and interfaces of the target operator, generate standardized input data (such as matrices, vectors, scalars, etc.), covering normal scenarios (such as typical numerical ranges) and basic boundary values ​​(such as zero, maximum, and minimum values). For example, generate positive and negative integer and decimal combinations for the addition operator to ensure basic function coverage. 2) Construct boundary and extreme test data: For extreme scenarios that the hardware may face, manually or through scripts generate extreme data, such as ultra-large dimension matrices, precision thresholds (such as the minimum positive value of a floating-point number), and abnormal format data (such as unaligned address data). This type of data is used to verify the hardware's ability to handle boundary conditions and expose potential overflows, precision loss, or address access errors. 3) Reusing Real-World Business Scenario Data: Extract real-world data (such as pixel matrices of actual images and feature vectors of speech signals) from the chip's target application scenarios (e.g., AI inference, image processing), and use this data as test data after format conversion. This type of data closely resembles actual usage scenarios, enabling more realistic verification of hardware performance in the product environment and improving the practicality of the verification. 4) Expanding Test Coverage with Random Data: Generate large amounts of unexpected data in batches using random number generation algorithms (e.g., pseudo-random number generators), increasing the randomness and coverage of the test. Random data can be combined with data generated through the above-mentioned targeted methods to discover occasional hardware logic vulnerabilities or data dependency issues in large-scale simulations.

[0078] After obtaining the test data, the test data and machine code file are transferred to the chip emulation device. Specifically, the chip emulation device has a corresponding data input method and provides a corresponding loading path for this method. The test data and machine code file are loaded (or imported) into the chip emulation device through the loading path corresponding to this data input method (i.e., the debugging interface or dedicated loading channel reserved by the chip emulation device). In this way, the chip emulation device obtains and stores the test data and machine code file.

[0079] Chip simulation equipment executes machine code files on test data to perform calculations on the target operators, thereby obtaining the simulation calculation results of the target operators on the test data. In practical applications, before executing the machine code file, the chip simulation equipment can be pre-reset (e.g., clear registers, memory address space, thread states) to remove interference from the environment before executing the machine code file, ensuring the purity of the environment for this simulation (i.e., executing the machine code file on the test data) and improving the accuracy of the simulation calculation results.

[0080] In some embodiments, the following steps may also be performed: obtaining the chip's chip runtime configuration file; based on this, "transferring test data and machine code files to the chip simulation device" can be achieved by performing the following steps: transferring the chip runtime configuration file, test data, and machine code files to the chip simulation device, wherein the chip runtime configuration file is used for the chip simulation device to execute the machine code file on the test data based on the chip runtime configuration file to obtain simulation calculation results.

[0081] Here, the chip runtime configuration file is a pre-written and compiled executable file used to configure chip runtime parameters (such as execution mode, number of threads, and instruction fetch address), add synchronization instructions, and control the start, run, and end of the test process. The chip runtime configuration file includes the following configurations: (1) configuring the chip execution mode; (2) configuring the chip instruction fetch mode; (3) configuring the thread instruction fetch start address; (4) setting the number of threads to start; (5) adding synchronization instructions, and the program exits after all threads have finished executing. Thus, the chip runtime configuration file, test data, and machine code file are transmitted to the chip simulation device. Based on this, the chip simulation device executes the machine code file on the test data according to the chip runtime configuration file to obtain the simulation calculation results.

[0082] Specifically, (1) Initialization loading: The chip simulation device first loads the chip running configuration file into the starting address of the instruction area, and loads the test data and machine code file into the data area and instruction area respectively (the machine code file follows the chip running configuration file); at the same time, the state of the chip simulation device is reset, the program counter (PC) points to the starting address of the chip simulation device, and the thread state and bus signals are reset.

[0083] (2) Environment configuration execution: The PC reads and executes the chip running configuration file in sequence to complete the configuration of the chip running parameters: set the execution and fetch mode (such as single / multi-threaded, sequential fetch); configure the number of threads, allocate the instruction stream start address to each thread and write it to the thread's private PC; write the base address of the test data in the data area and the storage address of the simulation calculation results into the corresponding registers to establish data access association.

[0084] (3) Thread startup and execution of machine code file: When the “thread startup” instruction is executed, the PC of each thread jumps to the starting address of the machine code file, reads the calculation instructions of the target operator from the instruction area, and the thread status is updated to “running”. The thread operates according to the machine code file: loads test data from the data area into the arithmetic unit, performs calculation (intermediate results are stored in the temporary address of the data area), and finally writes the simulation calculation result to the specified output address; if it is in synchronous mode, the thread sends a “ready signal” after completion, and global synchronization is triggered when all threads are ready.

[0085] (4) Closing and Result Export: After the synchronization signal is activated, the chip runs the configuration file and executes the closing instruction: the activation result is written back to lock the output data, the waveform capture parameters are configured, and the simulation end signal is triggered. The chip simulation device stops the clock drive, reads the simulation calculation results from the specified address in the data area and converts the format for subsequent comparison and verification with the benchmark calculation results.

[0086] Applying the above embodiments: 1) Improved test controllability: By uniformly configuring execution modes, thread parameters, etc., through the chip runtime configuration file, the simulation process is ensured to execute according to preset rules, avoiding hardware resource conflicts or disordered operations, and reducing test complexity. 2) Accelerated verification efficiency: The chip runtime configuration file directly associates instruction streams and data area addresses, eliminating the need for manual configuration, and allows for quick switching of test scenarios (such as single / multi-threaded, different operators) through parameters, shortening the verification cycle. 3) Enhanced test completeness: With the help of synchronous control and waveform capture configuration of the chip runtime configuration file, the hardware status can be comprehensively recorded. By comparing the output results, both functional correctness can be verified, and performance bottlenecks (such as thread collaboration efficiency) can be analyzed. 4) Closer to real-world scenarios: By simulating the program startup and execution logic of a real chip, the test results are more valuable and provide a reliable basis for design optimization before chip fabrication.

[0087] In some embodiments, "transferring the chip runtime configuration file, test data, and machine code file to the chip emulation device" can be achieved by performing the following steps: converting the storage format of the chip runtime configuration file from the current first storage format to the chip storage format, converting the storage format of the test data from the current second storage format to the chip storage format, and converting the machine code file from the current third storage format to the chip storage format; and transferring the chip runtime configuration file, test data, and machine code file in the chip storage format to the chip emulation device.

[0088] Here, the storage formats of the chip runtime configuration file, test data, and machine code file are first converted to the chip's storage format to ensure successful loading and storage of these files into the chip emulation device. Specifically, the storage format of the chip runtime configuration file is converted from the current first storage format to the chip's own storage format; the storage format of the test data is converted from the current second storage format to the chip's own storage format; and the machine code file is converted from the current third storage format to the chip's own storage format. This allows the chip runtime configuration file, test data, and machine code file, all in their respective chip storage formats, to be transferred to the chip emulation device.

[0089] In practical applications, when performing storage format conversion, the essence is to first ensure that the data format of the data to be transmitted (i.e., chip runtime configuration files, test data, and machine code files) conforms to the parsing requirements of the chip hardware, and then arrange its layout in the chip hardware according to the rules of the chip storage format. For example, the data format of the data to be transmitted is converted into a data format that the chip hardware can directly parse (such as 32-bit binary floating-point format); based on the correct data format, the data storage method is adjusted according to the physical layout rules of the chip hardware. It should be noted that the core of the chip storage format is the binary format adapted to the hardware architecture. Its specific rules (bit width, byte order, address mapping, etc.) are determined by the chip hardware design, and there is no unified "universal format," but all aim to "enable the chip hardware to read and process data efficiently and correctly." The chip storage format is the physical layout rule of data in the hardware storage medium (registers, memory, cache), defining "how data is placed in the hardware," and is directly related to the hardware architecture. For example: bit width alignment (e.g., data in a 32-bit chip must be aligned to 4 bytes, and padding with 0s if necessary); byte order (big-endian / little-endian, such as the storage order of multi-byte data); address mapping (the starting address and partition of data in memory, such as instructions being stored in 0x0000~0xFFFF and data being stored in 0x10000~0xFFFFF); memory unit reuse (e.g., data isolation layout when multiple threads share memory).

[0090] By applying the above embodiments, the chip runtime configuration file, test data, and machine code file are converted from their respective storage formats to the chip storage format, thereby ensuring that the chip runtime configuration file, test data, and machine code file can be successfully loaded and stored in the chip simulation device. This also ensures that the chip simulation device can correctly read and parse the chip runtime configuration file, test data, and machine code file, realize the simulation calculation of the target operator, and ensure the correct execution of the target operator.

[0091] In some embodiments, the machine code file includes: a first storage address of the chip runtime configuration file, test data, and machine code file respectively in the chip simulation device, and a second storage address of the simulation calculation result in the chip simulation device; based on this, "transferring the chip runtime configuration file, test data, and machine code file to the chip simulation device" can be achieved by performing the following steps: transferring the chip runtime configuration file, test data, and machine code file to their respective first storage addresses in the chip simulation device; based on this, obtaining the simulation calculation result from the chip simulation device includes: obtaining the simulation calculation result from the second storage address of the chip simulation device.

[0092] Here, the machine code file includes: the chip runtime configuration file, test data, and the first storage address of the machine code file in the chip simulation device. The first storage address of different data (i.e., the chip runtime configuration file, test data, and machine code file) can be different. For example, (1) the chip runtime configuration file belongs to the control program code and can be stored in the instruction area of ​​the chip simulation device. For example, it can be mapped to the low address segment preset by the chip simulation device (such as 0x0000_0000-0x000F_FFFF). When the simulation starts, it fetches instructions from the starting address of the chip runtime configuration file and triggers the subsequent process. (2) the test data belongs to the data information and can be stored in the data area of ​​the chip simulation device. For example, it can be mapped to the high address segment (such as 0x0100_0000-0xFFFF_FFFF) for access by the "data loading instruction" in the machine code file. (3) Machine code files belong to computational program code and can also be stored in the instruction area. They can also be adjacent to and contiguous with the address segment of the chip runtime configuration file. In practical applications, the chip runtime configuration file will explicitly configure the starting address of the machine code file. When the chip runtime configuration file executes the "start thread" instruction, it will load the starting address of the machine code file into the thread's program counter. The thread will then read and execute the specific computation instructions of the target operator (such as data loading, operation, result storage, etc.) starting from this address. Based on this, the chip runtime configuration file, test data, and machine code file are respectively transferred to their respective first storage addresses in the chip simulation device.

[0093] Similarly, the machine code file can also include the simulation calculation results at a second storage address in the chip simulation device. These simulation calculation results are also data-related information and can be stored in the data area of ​​the chip simulation device. Therefore, when retrieving simulation calculation results from the chip simulation device, the results are retrieved from the second storage address of the chip simulation device.

[0094] By applying the above embodiments, by setting separate storage addresses for different data (chip runtime configuration files, test data and machine code files, simulation calculation results), the hardware design safety principles are met, the risk of interference between modules is reduced, and the address association rules are clarified to ensure collaborative access to various types of data, thereby improving simulation execution efficiency and stability.

[0095] Step 104: Obtain the simulation calculation results from the chip simulation equipment, and obtain the benchmark calculation results of the benchmark operator for the test data.

[0096] For step 104, after obtaining the simulation calculation result from the chip simulation device, the simulation calculation result is retrieved from the chip simulation device, that is, exported from the second storage address used to store the simulation calculation result in the chip simulation device. Simultaneously, the benchmark calculation result of the benchmark operator for the test data is obtained. This benchmark operator can be a standard operator built into a deep learning framework, such as the benchmark operator built into the Torch deep learning framework. The benchmark operator is used to calculate the test data to obtain the benchmark calculation result. In practical applications, the benchmark operator is developed based on the hardware environment and software stack of a general-purpose CPU / GPU, relying on the underlying interface of the general-purpose computing framework. The calculation process of the benchmark operator can be implemented in the hardware environment of a general-purpose CPU / GPU. Therefore, since the simulation calculation result is exported from the chip simulation device, the storage format of the simulation calculation result is the chip storage format. At this time, it is necessary to convert the simulation calculation result from the chip storage format to a format adapted for CPU processing (including storage, reading, result comparison, etc.) to facilitate comparison with the benchmark calculation result.

[0097] Step 105: Generate chip test results based on simulation and benchmark calculation results.

[0098] For step 105, after obtaining the simulation calculation result calculated by the chip simulation device and the benchmark calculation result calculated based on the benchmark operator, the simulation calculation result and the benchmark calculation result are compared to obtain the chip test result.

[0099] In some embodiments, the test result is a first test result or a second test result; based on this, step 105, "generating the chip test result based on the simulation calculation result and the benchmark calculation result", can be implemented by performing the following steps: if the simulation calculation result and the benchmark calculation result are the same, a first test result is generated, which indicates that the chip verification is passed; if the simulation calculation result and the benchmark calculation result are different, a second test result is generated, which indicates that the chip verification is failed.

[0100] Here, after comparing the simulation calculation results with the benchmark calculation results, if the simulation calculation results and the benchmark calculation results are the same, a first test result is generated, which indicates that the chip verification has passed, specifically that the chip accuracy verification has passed; if the simulation calculation results and the benchmark calculation results are different, a second test result is generated, which indicates that the chip verification has failed, specifically that the chip accuracy verification has failed.

[0101] Applying the above embodiments, 1) anchoring functional correctness benchmarks: benchmark calculation results are theoretically correct values. By comparison, it is possible to directly determine whether the implementation of the target operator and the execution of the hardware in the chip conform to the functional design, and eliminate logical errors; 2) improving verification rigor: benchmark calculation results are independent of the standard results of the chip hardware, which can avoid the "self-proving correctness" deviation caused by inherent hardware defects and ensure the objectivity of verification; 3) accelerating problem localization: difference results can accurately point to the vulnerabilities in hardware design (such as arithmetic units, data paths) or operator implementation, reducing the cost of investigation; 4) ensuring adaptability to actual scenarios: benchmark calculation results are close to the needs of real applications. Passing the comparison means that the chip can output reliable results in actual business, laying the foundation for subsequent wafer fabrication and commercialization, and accelerating the chip testing pace.

[0102] In some embodiments, when the test result is the first test result, see [link to documentation]. Figure 6 The following steps 301-303 can also be performed: Step 301, injecting abnormal instructions into the machine code file to obtain the target machine code file; Step 302, transmitting the target machine code file to the chip simulation device, wherein the chip simulation device is used to execute the target machine code file on the test data to obtain the target simulation calculation result of the target operator on the test data; Step 303, analyzing the fault tolerance capability of the chip based on the target simulation calculation result and the simulation calculation result to obtain the fault tolerance capability analysis result.

[0103] For step 301, an abnormal instruction is first injected into the machine code file to obtain the target machine code file. In practical applications, correct instructions in the machine code file can be modified into incorrect instructions to obtain the target machine code file. For example, for a correct memory access instruction, the corresponding position can be modified to "out of bounds". For instance, if the highest address of the correct instruction is 18xx, modifying it to 19xx will result in an incorrect memory access instruction.

[0104] In step 302, the target machine code file is transferred to the chip simulation device; the chip simulation device executes the target machine code file on the test data to obtain the target simulation calculation result of the target operator for the test data. Thus, the chip simulation device executes the target machine code file, including exception instructions, to obtain the target simulation calculation result.

[0105] For step 303, based on the target simulation calculation results and the simulation calculation results, the fault tolerance capability of the chip is analyzed to obtain the fault tolerance capability analysis results. The target simulation calculation results and the simulation calculation results are compared. If the target simulation calculation results and the simulation calculation results are the same, a first fault tolerance capability analysis result is obtained, which indicates that the chip's fault tolerance capability is effective. If the target simulation calculation results and the simulation calculation results are different, a second fault tolerance capability analysis result is obtained, which indicates that the chip's fault tolerance capability is not effective and there is a problem with the fault tolerance capability. The fault tolerance capability analysis results include both the first and second fault tolerance capability analysis results.

[0106] Applying the above embodiments, 1) Early exposure of chip hardware defects: Injecting anomalies during the chip simulation stage, without the need for actual hardware, allows for early detection of vulnerabilities in the chip design's fault tolerance mechanisms (such as anomaly detection and error recovery), avoiding cost losses due to insufficient fault tolerance after later tape-out, aligning with the goal of early chip verification and optimization. 2) Accurate assessment of fault tolerance capabilities: By comparing the target simulation calculation results after anomaly injection with the normal simulation calculation results, the impact of anomalies on chip execution can be directly quantified, clarifying whether the fault tolerance mechanism effectively intercepts and repairs anomalies, avoiding the limitations of relying solely on theoretical analysis to judge fault tolerance performance. 3) Ensuring the baseline of functional correctness: Combining the comparison with the benchmark calculation results in the early stage, this process, while verifying fault tolerance capabilities, can also reconfirm that the chip can still maintain the correctness of core computing functions (or handle errors as expected) under abnormal scenarios, ensuring the reliability of the chip in complex environments and laying the foundation for subsequent actual hardware testing.

[0107] In some embodiments, the following steps may also be performed: obtaining a signal waveform diagram from a chip simulation device, the signal waveform diagram being generated based on hardware signals collected during the execution of a machine code file; analyzing the signal waveform diagram according to at least one operational state analysis dimension to obtain the chip's operational state; and analyzing the signal waveform diagram according to at least one performance analysis dimension to obtain the chip's performance data.

[0108] Here, signal waveforms are acquired from a chip simulation device. During the execution of the machine code file, hardware signals are captured based on pre-set waveform capture conditions, and the waveform is generated based on these captured hardware signals. For example, waveform capture conditions could be: 1) When the "input data load signal (load_en)" goes high, a hardware signal capture operation is triggered (recording begins) to capture the process of data being loaded from memory into the arithmetic unit. 2) When the "arithmetic unit start signal (calc_start)" goes high, a hardware signal capture operation is triggered to capture signal changes during the calculation process. 3) When the "output result write-back signal (write_back)" goes high, the hardware signal capture operation stops (recording stops) to avoid recording subsequent irrelevant operations. For example, hardware signals include, but are not limited to: 1) Control signals: such as data load enable (load_en), arithmetic unit start (calc_start), synchronization signal (sync), result write-back (write_back), etc., reflecting the triggering and termination of hardware operations. 2) Data signals: such as data bus and result bus, which record the specific data values ​​transmitted (such as matrix elements and intermediate calculation results). 3) Status signals: such as arithmetic unit status (alu_status, 0=idle / 1=in calculation) and thread status (thread_state, 0=not started / 1=running / 2=finished), which reflect the real-time working mode of the hardware components.

[0109] In some embodiments, analyzing the signal waveform diagram according to at least one operational state analysis dimension to obtain the chip's operational state includes performing at least one of the following operations: obtaining the triggering sequence of control signals from the signal waveform diagram, and determining the operational state by combining the triggering sequence with the expected triggering sequence; obtaining the value of the data signal based on the signal waveform diagram, and determining the operational state by combining the value with the expected value; and obtaining the duration and switching time point of the state signal based on the signal waveform diagram, and determining the operational state by combining the duration and switching time point.

[0110] Here, the dimensions of the running state analysis include: control signal analysis dimension, data signal analysis dimension and state signal analysis dimension. (1) Obtain the triggering order of the control signals from the signal waveform diagram, and combine the triggering order with the expected triggering order to determine the running state. For example, check whether the triggering order of the control signals conforms to the expected triggering order, such as whether the process of data loading → calculation start → result write-back is strictly executed in order. Specific operations: a) Observe whether load_en (data loading enable) becomes high level before calc_start (calculation start): if calc_start is triggered when load_en is not activated, it means that "calculation is earlier than data preparation", there is a timing error, that is, the running state is abnormal. b) Confirm whether sync (synchronization signal) is triggered after all threads' thread_state becomes "end": if the synchronization signal is triggered in advance, it means that the thread terminates before completing the calculation, there is a synchronization logic error, that is, the running state is abnormal.

[0111] (2) Based on the signal waveform diagram, obtain the value of the data signal, and combine the value with the expected value to determine the operating status. For example, check whether the value of the data signal is consistent with the expected input / output to ensure that there are no errors in the storage, operation, and transmission of data. Specific operation: Extract the value of data_bus during load_en=1 and compare it with the expected value (such as test data): If the value does not match, it means that there is a transmission error when the data is loaded from memory to the operation unit (such as bus failure, address mapping error), that is, the operating status is abnormal.

[0112] (3) Based on the signal waveform diagram, obtain the duration and switching time of the status signal, and determine the running status by combining the duration and switching time. For example, monitor whether the duration and switching time of the status signal meet the execution requirements of the target operator, and check for abnormalities such as deadlock and interruption. Specific operation: Observe whether alu_status immediately changes to "1 (calculating)" after calc_start=1, and remains "1" before result_bus starts transmitting. If alu_status suddenly jumps to "0 (idle)", it indicates that the operation unit exits abnormally in the middle, and there may be abnormal situations such as deadlock or resource conflict. b) Check the thread status thread_state: If the thread_state of a certain thread remains "1 (running)" for a long time without switching to "2 (finished)", it indicates that the thread is stuck and there is an abnormal running status.

[0113] In some embodiments, the signal waveform diagram is analyzed according to at least one performance analysis dimension to obtain the chip's performance data, including: performing at least one of the following operations: obtaining the execution duration of each execution stage of the machine code file from the signal waveform diagram, and determining the performance data based on the execution duration of each execution stage; determining the execution duration of the computation stage of the machine code file and the total execution duration of the machine code file based on the signal waveform diagram, and determining the performance data.

[0114] Here, the performance analysis dimensions include the stage duration dimension and the resource utilization dimension. (1) Stage duration dimension: From the signal waveform diagram, obtain the execution duration of each execution stage of the machine code file, and determine the performance data (such as whether the execution duration of each execution stage times out, execution efficiency, etc.) based on the execution duration of each execution stage. For example, a) Delay of the data loading stage: The time difference from load_en becoming high (data starts loading) to calc_start becoming high (calculation starts). b) Duration of the calculation stage: The time difference from calc_start activation (calculation starts) to result_bus first outputting data (calculation ends). c) Total execution time: The time difference from load_en activation (data loading starts) to write_back activation (result write back complete).

[0115] (2) Resource Utilization Dimension: Based on the signal waveform diagram, determine the execution time of the computation phase of the machine code file and the total execution time of the machine code file, and determine performance data (such as arithmetic unit utilization and thread parallel efficiency). For example, a) the proportion of time the arithmetic unit is in "computing (alu_status=1)" to the total execution time can be considered as arithmetic unit utilization. b) In a multi-threaded scenario, the ratio of the sum of the "running (thread_state=1)" times of all threads to the total execution time can be considered as thread parallel efficiency. Resource utilization includes arithmetic unit utilization and thread parallel efficiency.

[0116] By applying the embodiments described above, the operator code of the target operator (which has the same computational function as the benchmark operator) is implemented by combining chip interface instructions. Since the chip interface instructions can implement at least one hardware operation of the chip, parsing the operator code can yield the chip's machine code file. The machine code file is then run using a chip simulation device to simulate the chip and obtain simulation calculation results. Finally, the simulation calculation results are combined with the benchmark calculation results to generate the chip's test results. Thus, by implementing the operator code based on the chip interface instructions and then performing operator simulation using a chip simulation device, chip testing can be achieved at the operator level in advance, without waiting for the software stack to be fully developed, thus accelerating the chip testing process. Furthermore, generating test results by combining simulation calculation results with benchmark calculation results ensures the completeness of the chip testing.

[0117] The following describes an exemplary application of the embodiments of this application in a real-world scenario. Chip design involves millions of lines of hardware and software code. Functional verification is a crucial step in ensuring the correctness of the chip design. In related technologies, this requires multi-stage testing, including software simulation, hardware simulation, and prototype verification. However, after the chip hardware design is completed, the development of the software stack often lags behind the hardware progress. The incompleteness of the software stack prevents timely prototype verification of the chip, slowing down the chip testing progress and delaying the chip's tape-out and market launch cycle.

[0118] Based on this, this application provides a chip testing method that allows for chip testing at the operator level without waiting for the chip software stack to be fully developed, thus testing the completeness of the chip hardware and accelerating chip prototype verification. This results in: 1) accelerating the chip testing pace by enabling prototype verification testing at the operator level well in advance; 2) operator-level testing closely approximating the actual operating scenarios of the product, increasing the completeness of chip prototype verification; and 3) performance data from operator-level prototype verification, which can be used to assess the chip's actual operating performance and optimize the operator implementation scheme. The following is a detailed explanation.

[0119] In this embodiment of the application, the implementation basis of chip testing is: a) API instructions obtained by encapsulating chip instruction sets; b) providing a compiler or simulator that can parse the encapsulated API instructions into chip hardware instructions represented by 0 and 1.

[0120] (1) Based on the encapsulated API instructions, software operator programming is performed to obtain the operator code of the target operator. The target operator and the benchmark operator (such as the Torch operator) have the same interface and calculation function. At the same time, the operator self-test code of the target operator (i.e. the above test code) is written.

[0121] (2) Using a chip simulator or compiler, the operator self-test code and operator code are parsed into a chip instruction stream file that can run on the chip hardware. The chip instruction stream file includes: the operator execution process, the storage address of the data required for operator calculation, the storage address of the chip instruction stream, and the storage address of the output data (such as the simulation calculation results mentioned above) after the operator execution is completed.

[0122] (3) Write the main function to configure the startup chip and compile it to obtain a binary file (i.e., the chip running configuration file mentioned above). The binary file includes the following settings: a) set the chip execution mode; b) configure the chip instruction fetch mode; c) configure the thread instruction fetch start address; d) set the number of startup threads; e) add synchronization instructions, and the program exits after all threads have finished executing.

[0123] (4) Data preparation: Convert the data format of the target operator input data (i.e. the test data mentioned above), the chip instruction stream file and the binary file obtained by compiling the chip configuration main function into a data format that is compatible with the chip storage format; load the input data obtained after data format conversion, the chip instruction stream file and the binary file obtained by compiling the chip configuration main function into the Emulator simulation device through the data injection method provided by the Emulator simulation device.

[0124] (5) Run the Emulator simulation device to perform simulation, and capture waveforms according to the pre-set waveform capture conditions during the simulation process. Then, by analyzing the waveforms, obtain the hardware operating status and performance data.

[0125] (6) Obtaining results: The storage address of the simulation calculation results is specified in the chip instruction stream file. Therefore, the simulation calculation results are exported from the storage address of the Emulator simulation device in the same way. Then, the data format of the simulation calculation results is converted so that the converted data format can be read and compared by the CPU.

[0126] (7) Result Comparison: Run benchmark operators (such as Torch operators) on the test data using a CPU or GPU platform to obtain benchmark calculation results. Compare the benchmark calculation results with the simulation calculation results to obtain the chip verification results (i.e., the test results mentioned above). Specifically, if the benchmark calculation results and the simulation calculation results are the same, the chip verification result indicates that the correctness verification of the chip design has passed; if the benchmark calculation results and the simulation calculation results are different, the chip verification result indicates that the correctness verification of the chip design has failed.

[0127] See Figure 7 , Figure 7 This is a schematic diagram of the fifth process of the chip testing method provided in this application embodiment. Here, the chip testing method includes: 1) obtaining operator code and operator self-test code; 2) parsing the operator code and operator self-test code through a simulator or compiler to obtain a chip instruction stream file; 3) obtaining the chip configuration main function; 4) compiling the chip configuration main function to obtain a binary file; 5) obtaining the input data required for chip testing; 6) converting the chip instruction stream file, binary file, and input data to obtain a target file adapted to the chip storage format; 7) loading the target file into the Emulator simulation device; 8) running the Emulator simulation device; 9) triggering a synchronization signal, completing the execution of all threads, and ending the simulation; 10) storing the simulation calculation results; 11) capturing waveforms and analyzing the waveforms to obtain hardware operating status and performance data; 12) running a benchmark operator on the test data to obtain benchmark calculation results, and comparing the simulation calculation results with the benchmark calculation results to obtain the chip verification results.

[0128] The specific implementation process of this application can be as follows: Based on the encapsulated API instructions, develop the chip's operator library operators, and write operator self-test code on a simulator to verify and ensure the correctness of the operator writing. Using this as a benchmark, without waiting for the subsequent software stack to be perfected, perform the following processing to conduct prototype verification of the operator on the Emulator simulation platform. Specifically, execute the operator code and operator self-test code on the simulator. The simulator will parse the operator code and operator self-test code into instruction streams and redirect the instruction streams to an instruction stream file, which serves as one of the input files for the Emulator simulation platform. After obtaining the instruction stream file, write the startup program (i.e., the chip configuration main function), configuring: starting threads, configuring the number of threads, configuring the thread execution start address, etc. These are also configured through the encapsulated API instructions. After completion, compile it into a binary file, which also serves as one of the input files for the Emulator simulation platform. Finally, for the data file, during operator self-testing, randomly generate input data, and use the same input data to input both the developed operator and the torch operator to verify the correctness of the operator. Here, the input and output data that pass the self-test are saved in binary format. Data format conversion scripts are developed. Two conversion scripts will be written specifically for the chip's storage format: one for converting the instruction stream file to match the chip's storage format, and another for converting the binary file to match the chip's storage format. The Emulator simulation platform has a backdoor loading operation. Through the backdoor loading path provided by the Emulator machine, the three converted files are loaded into the corresponding storage space of the Emulator simulation platform. Next, the Emulator simulation platform test is started. After the data loading is complete, the Emulator simulation platform's state is reset, clearing any interference from before the run to ensure the purity of the simulation. Before this, waveform capture signals and capture times can be added. After execution, the simulation calculation result address space data can be exported through the backdoor export method provided by the Emulator platform. This allows for verification of the simulation calculation results' correctness by comparing them with the benchmark calculation results.

[0129] By applying the above embodiments of this application, 1) the chip testing pace is accelerated, and prototype verification testing at the operator level can be carried out much earlier; 2) the operator-level testing is close to the actual operating scenario of the product, increasing the completeness of chip prototype verification; 3) the performance data obtained from the prototype verification at the operator level can be used to understand the actual operating performance of the chip and to optimize the operator implementation scheme.

[0130] The following description continues to illustrate the exemplary structure of the chip testing device 555 provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 2As shown, the software modules stored in the chip testing device 555 in the memory 550 may include: a first acquisition module 5551, used to acquire operator code of a target operator including multiple chip interface instructions, wherein the target operator and the benchmark operator have the same calculation function, the chip interface instructions are used to implement at least one hardware operation of the chip, and the hardware operation combining the multiple chip interface instructions is used to implement the calculation function; a parsing module 5552, used to parse the operator code to obtain the machine code file of the chip; a transmission module 5553, used to acquire test data and transmit the test data and the machine code file to a chip simulation device, wherein the chip simulation device is used to execute the machine code file on the test data to obtain the simulation calculation result of the target operator for the test data; a second acquisition module 5554, used to acquire the simulation calculation result from the chip simulation device and acquire the benchmark calculation result of the benchmark operator for the test data; and a generation module 5555, used to generate the test result of the chip based on the simulation calculation result and the benchmark calculation result.

[0131] In some embodiments, the first acquisition module 5551 is further configured to acquire the test code of the target operator before parsing the operator code to obtain the machine code file of the chip; the parsing module 5552 is further configured to parse the test code to obtain the control logic of the target operator; based on the control logic, parse the operator code to obtain the calculation logic of the target operator; and integrate the control logic and the calculation logic to obtain the machine code file.

[0132] In some embodiments, the first acquisition module 5551 is further configured to acquire the chip runtime configuration file of the chip; the transmission module 5553 is further configured to transmit the chip runtime configuration file, the test data and the machine code file to the chip simulation device, wherein the chip runtime configuration file is used by the chip simulation device to execute the machine code file on the test data based on the chip runtime configuration file to obtain the simulation calculation result.

[0133] In some embodiments, the transmission module 5553 is further configured to convert the storage format of the chip runtime configuration file from the current first storage format to the chip storage format of the chip, convert the storage format of the test data from the current second storage format to the chip storage format, and convert the machine code file from the current third storage format to the chip storage format; and transmit the chip runtime configuration file, the test data, and the machine code file in the chip storage format to the chip simulation device.

[0134] In some embodiments, the machine code file includes: the chip runtime configuration file, the test data, and the machine code file, each stored at a first storage address in the chip simulation device; and the simulation calculation result is stored at a second storage address in the chip simulation device. The transmission module 5553 is further configured to transmit the chip runtime configuration file, the test data, and the machine code file to their respective first storage addresses in the chip simulation device. The second acquisition module 5554 is further configured to acquire the simulation calculation result from the second storage address in the chip simulation device.

[0135] In some embodiments, the number of chip interface instructions is M, where M is an integer greater than 1; the first acquisition module 5551 is further configured to acquire N chip instructions before acquiring the operator code including multiple chip interface instructions of the target operator, each chip instruction being used to implement one of the hardware operations of the chip, where N is an integer greater than or equal to M; and to encapsulate the N chip instructions to obtain M chip interface instructions, each chip interface instruction including at least one chip instruction.

[0136] In some embodiments, the first acquisition module 5551 is further configured to acquire instruction description files for N chip instructions to be generated, the instruction description files including: domain segment definition information of multiple instruction domain segments included in each chip instruction to be generated, and the combination format of the multiple instruction domain segments; for each chip instruction to be generated, based on the domain segment definition information of each instruction domain segment included in the chip instruction to be generated, generating domain segment parameters for each instruction domain segment; for each chip instruction to be generated, combining the domain segment parameters of the multiple instruction domain segments included in the chip instruction to be generated according to the combination format of the multiple instruction domain segments to obtain the chip instruction.

[0137] In some embodiments, the test result is a first test result or a second test result; the generation module 5555 is further configured to generate the first test result when the simulation calculation result and the benchmark calculation result are the same, wherein the first test result indicates that the chip verification is passed; and to generate the second test result when the simulation calculation result and the benchmark calculation result are different, wherein the second test result indicates that the chip verification is failed.

[0138] In some embodiments, when the test result is the first test result, the generation module 5555 is further configured to inject abnormal instructions into the machine code file to obtain a target machine code file; transmit the target machine code file to the chip simulation device, wherein the chip simulation device is configured to execute the target machine code file on the test data to obtain the target simulation calculation result of the target operator for the test data; and analyze the fault tolerance capability of the chip based on the target simulation calculation result and the simulation calculation result to obtain the fault tolerance capability analysis result.

[0139] In some embodiments, the generation module 5555 is further configured to acquire a signal waveform diagram from the chip simulation device, the signal waveform diagram being generated based on hardware signals collected during the execution of the machine code file; analyze the signal waveform diagram according to at least one operating state analysis dimension to obtain the operating state of the chip; and analyze the signal waveform diagram according to at least one performance analysis dimension to obtain the performance data of the chip.

[0140] In some embodiments, the generation module 5555 is further configured to perform at least one of the following operations: obtain the triggering order of control signals from the signal waveform diagram, and determine the operating state by combining the triggering order and the expected triggering order; obtain the value of data signals based on the signal waveform diagram, and determine the operating state by combining the value and the expected value; obtain the duration and switching time point of state signals based on the signal waveform diagram, and determine the operating state by combining the duration and switching time point; the generation module 5555 is further configured to perform at least one of the following operations: obtain the execution duration of each execution stage of the machine code file from the signal waveform diagram, and determine the performance data based on the execution duration of each execution stage; determine the execution duration of the calculation stage of the machine code file and the total execution duration of the machine code file based on the signal waveform diagram, and determine the performance data.

[0141] In some embodiments, the number of target operators is multiple. The parsing module 5552 is further configured to obtain the dependencies between the multiple target operators, and construct a topology graph with the target operators as nodes and the dependencies as directed edges; determine the execution order of the multiple target operators according to the topology graph; parse the operator code to obtain the execution instructions of each target operator; and integrate the execution instructions of the multiple target operators based on the execution order of the multiple target operators to obtain the machine code file of the chip.

[0142] It should be noted that the description of the device embodiments in this application is similar to the description of the method embodiments described above, and has similar beneficial effects as the method embodiments, so it will not be repeated here. Any technical details not covered in the chip testing device provided in the embodiments of this application can be understood based on the description of the technical details in the above method embodiments.

[0143] This application also provides a computer program product, which includes computer-executable instructions or a computer program stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions or computer program from the computer-readable storage medium and executes the computer-executable instructions or computer program, causing the electronic device to perform the chip testing method provided in this application.

[0144] This application also provides a computer-readable storage medium storing computer-executable instructions or computer programs. When the computer-executable instructions or computer programs are executed by a processor, the processor will execute the chip testing method provided in this application.

[0145] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0146] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0147] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0148] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0149] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A chip testing method, characterized in that, The method includes: The target operator is obtained by acquiring operator code including multiple chip interface instructions, wherein the target operator and the reference operator have the same computational function, the chip interface instructions are used to implement at least one hardware operation of the chip, and the hardware operation combining the multiple chip interface instructions is used to implement the computational function. The operator code is parsed to obtain the machine code file of the chip; Acquire test data and transmit the test data and the machine code file to a chip simulation device, wherein the chip simulation device is used to execute the machine code file on the test data to obtain the simulation calculation result of the target operator for the test data; The simulation calculation results are obtained from the chip simulation device, and the benchmark calculation results of the benchmark operator for the test data are obtained. Based on the simulation results and the benchmark results, test results for the chip are generated. When the test result is the first test result indicating that the chip has passed verification, an abnormal instruction is injected into the machine code file to obtain the target machine code file. The target machine code file is transmitted to the chip simulation device, wherein the chip simulation device is used to execute the target machine code file on the test data to obtain the target simulation calculation result of the target operator for the test data; Based on the target simulation calculation results and the simulation calculation results, the fault tolerance capability of the chip is analyzed, and the fault tolerance capability analysis results are obtained.

2. The method as described in claim 1, characterized in that, Before parsing the operator code to obtain the machine code file of the chip, the method further includes: Obtain the test code for the target operator; The process of parsing the operator code to obtain the machine code file of the chip includes: The control logic of the target operator is obtained by parsing the test code. Based on the control logic, the operator code is parsed to obtain the computation logic of the target operator; The control logic and the calculation logic are integrated to obtain the machine code file.

3. The method as described in claim 1 or 2, characterized in that, The method further includes: Obtain the chip's operating configuration file; The step of transmitting the test data and the machine code file to the chip emulation device includes: The chip runtime configuration file, the test data, and the machine code file are transmitted to the chip simulation device. The chip runtime configuration file is used by the chip simulation device to execute the machine code file on the test data based on the chip runtime configuration file to obtain the simulation calculation results.

4. The method as described in claim 3, characterized in that, The step of transmitting the chip runtime configuration file, the test data, and the machine code file to the chip simulation device includes: The storage format of the chip's runtime configuration file is converted from the current first storage format to the chip's chip storage format; the storage format of the test data is converted from the current second storage format to the chip storage format; and the machine code file is converted from the current third storage format to the chip storage format. The chip runtime configuration file, the test data, and the machine code file in the chip storage format are transmitted to the chip simulation device.

5. The method as described in claim 3, characterized in that, The machine code file includes: the chip runtime configuration file, the test data, and the machine code file, each stored at a first storage address in the chip simulation device; and the simulation calculation results stored at a second storage address in the chip simulation device. The step of transmitting the chip runtime configuration file, the test data, and the machine code file to the chip simulation device includes: The chip runtime configuration file, the test data, and the machine code file are respectively transferred to their respective first storage addresses in the chip simulation device; The step of obtaining the simulation calculation results from the chip simulation device includes: The simulation calculation result is obtained from the second storage address of the chip simulation device.

6. The method as described in claim 1, characterized in that, The number of chip interface instructions is M, where M is an integer greater than 1; before obtaining the operator code of the target operator, which includes multiple chip interface instructions, the method further includes: Obtain N chip instructions, each of which is used to implement one of the hardware operations of the chip, where N is an integer greater than or equal to M; The N chip instructions are encapsulated to obtain M chip interface instructions, each of which includes at least one of the chip instructions.

7. The method as described in claim 6, characterized in that, The acquisition of N chip instructions includes: Obtain instruction description files for N chip instructions to be generated. The instruction description files include: the domain segment definition information of multiple instruction domain segments included in each chip instruction to be generated, and the combination format of the multiple instruction domain segments; For each chip instruction to be generated, based on the domain segment definition information included in each chip instruction to be generated, domain segment parameters for each instruction domain segment are generated; For each chip instruction to be generated, the segment parameters of the multiple instruction segments included in the chip instruction to be generated are combined according to the combination format of the multiple instruction segments to obtain the chip instruction.

8. The method as described in claim 1, characterized in that, The test result is either the first test result or the second test result; The step of generating test results for the chip based on the simulation results and the benchmark results includes: If the simulation calculation result and the benchmark calculation result are the same, the first test result is generated; If the simulation calculation result and the benchmark calculation result are different, a second test result is generated, which indicates that the chip verification fails.

9. The method as described in claim 1, characterized in that, The method further includes: The signal waveform diagram is obtained from the chip simulation device, and the signal waveform diagram is generated based on the hardware signals collected during the execution of the machine code file; The signal waveform is analyzed according to at least one operational status analysis dimension to obtain the operational status of the chip; The signal waveform is analyzed according to at least one performance analysis dimension to obtain the chip's performance data.

10. The method as described in claim 9, characterized in that, The step of analyzing the signal waveform according to at least one operational state analysis dimension to obtain the chip's operational state includes: Do at least one of the following: The triggering sequence of the control signals is obtained from the signal waveform diagram, and the operating state is determined by combining the triggering sequence with the expected triggering sequence. Based on the signal waveform diagram, the value of the data signal is obtained, and the operating state is determined by combining the value and the expected value. Based on the signal waveform, the duration and switching time of the status signal are obtained, and the operating status is determined by combining the duration and switching time. The step of analyzing the signal waveform according to at least one performance analysis dimension to obtain the chip's performance data includes: Do at least one of the following: From the signal waveform diagram, the execution time of each execution stage of the machine code file is obtained, and the performance data is determined based on the execution time of each execution stage; Based on the signal waveform diagram, the execution time of the computation phase of the machine code file and the total execution time of the machine code file are determined, and the performance data is determined.

11. The method as described in claim 1, characterized in that, The number of target operators is multiple, and parsing the operator code to obtain the machine code file of the chip includes: Obtain the dependencies between multiple target operators, and construct a topology graph with the target operators as nodes and the dependencies as directed edges; According to the topology diagram, the execution order of the multiple target operators is determined; Parse the operator code to obtain the execution instructions for each target operator; Based on the execution order of the multiple target operators, the execution instructions of the multiple target operators are integrated to obtain the machine code file of the chip.

12. A chip testing device, characterized in that, The device includes: The first acquisition module is used to acquire the operator code of the target operator, which includes multiple chip interface instructions. The target operator and the reference operator have the same calculation function. The chip interface instructions are used to implement at least one hardware operation of the chip, and the hardware operation combining the multiple chip interface instructions is used to implement the calculation function. The parsing module is used to parse the operator code to obtain the machine code file of the chip; A transmission module is used to acquire test data and transmit the test data and the machine code file to a chip simulation device, wherein the chip simulation device is used to execute the machine code file on the test data to obtain the simulation calculation result of the target operator on the test data; The second acquisition module is used to acquire the simulation calculation results from the chip simulation device and to acquire the benchmark calculation results of the benchmark operator for the test data; A generation module is used to generate test results for the chip based on the simulation calculation results and the benchmark calculation results; The generation module is further configured to, when the test result is a first test result indicating that the chip has passed verification, inject abnormal instructions into the machine code file to obtain a target machine code file; transmit the target machine code file to the chip simulation device, wherein the chip simulation device is configured to execute the target machine code file on the test data to obtain the target simulation calculation result of the target operator for the test data; and analyze the fault tolerance capability of the chip based on the target simulation calculation result and the simulation calculation result to obtain the fault tolerance capability analysis result.

13. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions for a computer; A processor, when executing computer-executable instructions stored in the memory, implements the chip testing method according to any one of claims 1 to 11.

14. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by the processor, the chip testing method according to any one of claims 1 to 11 is implemented.

15. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by the processor, the chip testing method according to any one of claims 1 to 11 is implemented.

Images