An Automated Testing Method for Integrated Circuits Based on Generative AI

Abstract

This invention discloses an automated testing method for integrated circuits based on generative AI. The method includes: a first generative AI model generating a test plan based on an acquired target document; decomposing the test plan into several test sub-tasks; an agent generating test sub-task documents based on these sub-tasks; integrating the generated test sub-task documents to generate a test program for a target test device; and running the test program on the target test device to test the integrated circuit. This automated testing method for integrated circuits based on generative AI efficiently and reliably generates the required test programs based on product documentation, test documentation, and automated test device documentation. It can replace or assist test engineers in better completing testing tasks, improving the efficiency of integrated circuit test scheme development and debugging optimization, and accelerating the chip product development cycle.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit testing, and more specifically to an automated integrated circuit testing method based on generative AI. Background Technology

[0002] Integrated circuits (ICs), also commonly known as chips, are circuits with specific functions formed by processing semiconductor substrates with extremely high integration using semiconductor manufacturing processes. The core components of integrated circuits include chip design, manufacturing, packaging, and testing. Among these, integrated circuit testing is integral to the entire process of chip design, manufacturing, and packaging, and is a crucial step in ensuring chip yield and cost control. Due to the continuous improvement of semiconductor manufacturing processes and the increasing complexity of integrated circuits, the testing stage plays an increasingly important role in the semiconductor manufacturing process.

[0003] Integrated circuit testing is typically performed using automated test equipment (ATE). An ATE can integrate various test resources within a single mainframe, including but not limited to chip power supplies, voltage sources, current sources, voltmeters, ammeters, timing measurement units, RF units, and switch matrices. These test resources can be controlled via software programming according to the testing requirements of the IC product. By combining ATE with probe stations or sorting machines, chips can be automatically loaded, tested, and sorted, improving production efficiency, reducing testing costs, and obtaining efficient, reliable, and repeatable test results, ensuring control over chip quality.

[0004] While the widespread adoption of ATE (Automatic Test Equipment) has significantly improved the automation level of integrated circuit testing, the development of ATE test solutions still relies heavily on test engineers with extensive professional knowledge and development experience. Test engineers must understand the IC's design, function, and expected performance; select the appropriate ATE and its resource configuration for the IC product; and design the corresponding test hardware, such as test carriers, probe cards, and test sockets. Furthermore, test engineers need to develop test procedures and code, control test resources to acquire data, and repeatedly verify, debug, and optimize the software and hardware based on test results to ensure the effectiveness and accuracy of the test solution. Depending on the complexity of the chip and the engineer's experience and skills, developing a complete ATE test solution often takes months or even longer, and the total cost can range from tens of thousands to millions of dollars.

[0005] Generative AI is a subfield of artificial intelligence. Unlike traditional discriminative models, generative AI can not only identify or classify data, but also understand instructions and create new, unseen data samples. Generative AI technology has developed rapidly in recent years and has demonstrated powerful potential. The emergence of large language models and stable diffusion models has brought revolutionary changes to content generation.

[0006] Current generative AI models are trained for specific application scenarios, such as text generation, machine translation, and image creation, lacking an understanding of ATE (Automatic Test Equipment) domain knowledge. Training effective generative AI models requires a large amount of data. In the ATE testing field, especially for new IC product designs, obtaining sufficient training data is difficult, limiting the training effectiveness of the models. Furthermore, generative AI typically requires significant computing resources for training and generation, resulting in high computational costs and time latency. Summary of the Invention

[0007] To overcome the shortcomings of the above technologies, this invention provides an automated testing method for integrated circuits based on generative AI. It utilizes a generative AI model to generate test plans, decomposes them into different sub-tasks, and, based on the characteristics of each sub-task, calls an intelligent agent to generate test programs. This method efficiently and reliably generates the required test programs based on product documentation, test documentation, and automated test equipment documentation. It can replace or assist test engineers in better completing testing tasks, improving the efficiency of integrated circuit test solution development and debugging optimization, and accelerating the chip product development cycle.

[0008] The technical solution adopted by this invention to overcome its technical problems is as follows: This invention proposes an automatic testing method for integrated circuits based on generative AI, comprising: S1, a first generative AI model generating a test plan based on an acquired target document; S2, decomposing the test plan into several test sub-tasks; S3, an agent generating test sub-task documents based on the test sub-tasks, wherein the agent includes at least a second generative AI model and several tools; S4, integrating the generated test sub-task documents to generate a test program for a target test device; S5, running the test program on the target test device to test the integrated circuit.

[0009] Furthermore, the decomposition of the test plan into several test sub-tasks specifically includes: decomposing the test plan into at least test resource allocation, test vector generation, open and short circuit testing, built-in self-testing, and tuning test sub-tasks.

[0010] Furthermore, the intelligent agent includes at least a second generative AI model.

[0011] Furthermore, the intelligent agent generates test subtask documents based on test subtasks, specifically including: the intelligent agent determines the type of test subtask, and calls a tool to generate test subtask documents based on the test subtask type. The tool is a database retrieval tool built based on the test equipment, a program running according to preset rules, or a generative AI model. At least one test subtask is used to input the retrieval results of the database retrieval tool into the generative AI model, thereby generating test subtask documents suitable for the test equipment.

[0012] Furthermore, the agent determines the type of the test subtask and, based on the test subtask type, calls a tool to generate a test subtask document. Specifically, this includes: if the agent determines that the test subtask is related to test vector generation or test resource allocation, the agent calls the test vector generation program and the test resource allocation program, and inputs the test subtask and the outputs of the called test vector generation program and test resource allocation program into the second generative AI model to generate a test subtask document; if the agent determines that the test subtask uses a test instruction set, the agent calls a database retrieval tool, and outputs the test subtask and the retrieval results of the database retrieval tool into the second generative AI model to generate a test subtask document.

[0013] Furthermore, the agent generates test subtask documents corresponding to the training data, based on retrieval enhancement generation technology, combined with prompting engineering and context learning technology.

[0014] Furthermore, the database retrieval tool may be one or more, each corresponding to a test subtask.

[0015] Furthermore, the second generative model invoked by the agent based on different test subtasks is one or more generative AI models.

[0016] Furthermore, the second generative AI model is constructed at least through model compression techniques.

[0017] Furthermore, the model compression technique is selected from at least one or a combination of several of the following: pruning, knowledge distillation, parameter quantization, and low-rank decomposition.

[0018] Furthermore, it also includes testing integrated circuits to obtain test documents, establishing a debugging database based on the test documents, and having the debugging agent call a third generative AI model based on the query of the debugging database to modify and optimize the test program.

[0019] The beneficial effects of this invention are:

[0020] 1. This invention utilizes a generative AI model to generate test plans and decomposes them into different sub-tasks. Based on the characteristics of the sub-tasks, appropriate AI agents are used to establish a test instruction set database. Furthermore, retrieval enhancement techniques are employed to enable the model to generate test programs outside the training data range, thereby avoiding the occurrence of hallucination phenomena.

[0021] 2. During the program debugging and optimization phase, the debugging agent is used to automatically resolve errors in the program and optimize it, providing test program iterative modification and optimization solutions.

[0022] 3. It can efficiently and reliably generate the required test programs based on product documents, test documents, and automatic test equipment documents, which can replace or assist test engineers to better complete the test work, improve the efficiency of integrated circuit test solution development and debugging optimization, and accelerate the development cycle of chip products. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating an automatic testing method for integrated circuits based on generative AI, according to an embodiment of the present invention.

[0024] Figure 2 This is a test plan generated by an automatic testing method for integrated circuits based on generative AI, according to an embodiment of the present invention.

[0025] Figure 3 This is a schematic diagram of the intelligent agent's workflow according to an embodiment of the present invention. Detailed Implementation

[0026] To further understand this invention, some of the terms mentioned in this invention will first be explained:

[0027] IC: integrated circuit;

[0028] ATE: Automatic Test Equipment;

[0029] To facilitate a better understanding of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The following are merely exemplary and do not limit the scope of protection of the present invention.

[0030] like Figure 1 The diagram shown is a schematic representation of the process described in this embodiment.

[0031] S1, the first generative AI model generates a test plan based on the acquired target document.

[0032] In one embodiment of the present invention, the generative AI model may be one or more of the large language models such as ChatGPT, LaMDA, PEER, and LLAMA2, or a generative AI model built on a similar framework.

[0033] The target documents include technical documents for integrated circuit products, test requirements information, and target test equipment information.

[0034] The technical documentation for integrated circuit products mainly refers to chip datasheets and chip test instruction documents, including basic product information, chip performance indicators, key parameter lists, test pins, technical specifications, communication protocols, clock frequencies, etc. Test requirement information provides a summary of the tests required for the chip. Target test equipment information includes equipment operation manuals, program documents, etc. Target documents are not limited to PDF, TXT, Excel, or Word formats.

[0035] After acquiring information from this type of target document, the first generative AI model automatically generates a document like this. Figure 2 The test plan shown describes the specific test steps for the chip and the parameters required for each test step, such as... Figure 2 As shown, the process includes three steps: open / short circuit test, leakage current test, and communication test. For example, applying a -100μA current to the IO pin corresponds to the input current parameter required for the test in the first step.

[0036] S2 breaks down the test plan into several test subtasks.

[0037] Each step of the test plan is a test subtask.

[0038] In one embodiment of the present invention, the test plan is decomposed into test subtasks of different task types, such as test resource allocation, test vector generation, open / short circuit testing, built-in self-testing, and debugging testing. These task types can be categorized as test set instruction-related tasks, communication protocol-related tasks, and test set resource allocation tasks.

[0039] like Figure 2 Each description in the test plan shown can be considered a test subtask.

[0040] S3, the agent generates test subtask documents based on the test subtasks.

[0041] The agent determines the type of test subtask and calls a tool to generate a test subtask document based on the test subtask type. The tool is a database retrieval tool built based on the test equipment document, a program running according to preset rules, or a generative AI model. At least one test subtask is used to input the retrieval results of the database retrieval tool into the generative AI model, thereby generating a test subtask document suitable for the test equipment.

[0042] In one embodiment of the present invention, the intelligent agent is a chain-like fusion of one or more generative AI models and other software or tools, possessing retrieval and reasoning capabilities, and able to select the most appropriate tools and actions according to the requirements of a specific task. The intelligent agent can select different implementation processes based on specific test sub-tasks.

[0043] In some implementations, the intelligent agent includes a second generative AI model, which can be one or more AI models, and can be AI models from different domains, including machine language learning models with different parameter scales and different application domains, such as generative AI models applied to automatic programming or text generation.

[0044] like Figure 3 As shown, the test subtask description is input into the agent. The agent determines whether the task involves test vector generation or test resource allocation. If so, it calls the corresponding tools to generate test vectors and test resource allocations that conform to the test machine specifications. The output of the test vector generation program or the test resource allocation program, together with the test subtask description, is used as the input information of the second generative AI model to generate the corresponding test subtask document.

[0045] If the agent determines that the test subtask requires the use of the test machine instruction set, the agent calls the database retrieval tool and uses the test subtask and the retrieval results of the database retrieval tool together as input information for the second generative AI model to generate the corresponding test subtask document.

[0046] In one embodiment of the present invention, a database retrieval tool is constructed based on the target test equipment documentation, specifically including reading relevant information from technical documents and ATE equipment manuals, and establishing a test instruction database using vector storage technology.

[0047] In one embodiment of the present invention, the intelligent agent retrieves a test instruction database based on a test subtask. If the test subtask involves setting up and operating a test machine, the agent retrieves the test instruction database and inputs the retrieved test machine instructions as background knowledge along with the test subtask description into a second generative AI model. The second generative AI model then generates a test subtask document, i.e., test program code, that conforms to the test machine format and syntax.

[0048] It should be noted that different testing machines have different instruction sets and formats. Some testing machines use C language as the programming language, while others use VB. The program framework of different devices is also different.

[0049] In some implementations, for tasks related to communication protocols, pre-written programs can be invoked to generate test vectors, which then need to be used to communicate with the chip within the test program. For test machine resource allocation tasks, specific templates can be used as input to the AI ​​model.

[0050] In some implementations, the agent invokes a second generative AI model to generate test subtask documents, specifically including: generating test subtask documents corresponding to the agent's training data through retrieval enhancement techniques, combined with prompt engineering and in context learning techniques.

[0051] Retrieval enhancement techniques retrieve the necessary information for the task and input this information, along with the task description, into the agent. During this process, prompts need to be adjusted to allow the model to generate better responses. Prompt engineering involves adjusting the language used to describe the task requirements when inputting them into the AI ​​model to avoid ambiguity and enable the AI ​​to more accurately understand the input information. Context learning is an inherent capability of generative AI models; for knowledge beyond their training data, they can acquire it from the input information through context learning. Retrieval enhancement techniques provide contextual information to the AI ​​model through retrieval, leveraging the AI ​​model's inherent contextual learning capabilities to better generate model output information. In this embodiment of the invention, the contextual information is the test machine operation instruction template corresponding to the test sub-task, and the task description is the test sub-task.

[0052] In some implementations, the second generative AI model invoked by the agent utilizes model compression techniques, such as pruning, knowledge distillation, parameter quantization, and low-rank decomposition, to reduce the model's computational resource requirements, improve inference speed, and lower deployment costs. Since AI models typically have many parameters, require significant storage space, and have high resource demands, commonly used model compression techniques can be chosen to reduce resource requirements during actual model deployment.

[0053] S4, integrate the generated test subtask documents to generate the test program for the target test equipment.

[0054] The integration process can be accomplished by selecting appropriate software or machine learning models.

[0055] In one embodiment of the present invention, each sub-task document is filled into the corresponding template position of the test program by pre-setting a test program template.

[0056] S5, run the test program in the target test equipment to test the integrated circuit.

[0057] The test program is loaded onto the target test device and run, mobilizing the test machine resources to test the integrated circuit product and generate test documentation.

[0058] S6. The integrated circuit is tested to obtain test documents, and a debugging database is established based on the test documents. The debugging agent calls the third generative AI model based on the query of the debugging database to modify and optimize the test program.

[0059] Test documentation includes test logs, test error messages, and test reports. A debugging database is built using these test logs, test error messages, and test reports.

[0060] By leveraging debug database retrieval to assist AI in generating effective information, the debug agent receives feedback on error messages, test logs, and solutions to specific problem types during the debugging process. This helps the debug agent improve the program. The debug agent can utilize its reasoning capabilities and document retrieval information to invoke appropriate generative AI models, providing iterative modification and optimization schemes for the test program.

[0061] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

Claims

1. An automated testing method for integrated circuits based on generative AI, characterized in that, include: S1, The first generative AI model generates a test plan based on the acquired target document; S2, decompose the test plan into several test sub-tasks; S3, the intelligent agent generates test subtask documents based on the test subtasks, wherein the intelligent agent is a chain-like fusion of several generative AI models and several tools; S4, integrate the generated test subtask documents to generate the test program for the target test equipment; S5, run the test program in the target test equipment to test the integrated circuit; The process of decomposing the test plan into several test sub-tasks includes: decomposing the test plan into at least test resource allocation, test vector generation, open and short circuit testing, built-in self-test, and debugging test sub-tasks; the test sub-task types are divided into test machine instruction-related tasks, communication protocol-related tasks, and test machine resource allocation tasks. The agent generates a test subtask document based on the test subtask, specifically including: if the agent determines that the test subtask is related to test vector generation or test resource allocation, the agent calls the test vector generation program and the test resource allocation program, and inputs the test subtask and the output of the called test vector generation program and the output of the test resource allocation program into the second generative AI model, thereby generating the test subtask document; If the agent determines that the test subtask uses the test instruction set, the agent calls the database retrieval tool and outputs the test subtask and the retrieval results of the database retrieval tool to the second generative AI model, thereby generating the test subtask document; The agent invokes a second generative AI model to generate test subtask documents, including: generating test subtask documents corresponding to training data of the agent, not limited to the agent, through retrieval enhancement techniques; the second generative AI model is a generative AI model applied to text generation. It also includes testing integrated circuits to obtain test documents, establishing a debugging database based on the test documents, and using the debugging agent to retrieve the debugging database and call a third generative AI model to modify and optimize the test program.

2. The automatic testing method for integrated circuits based on generative AI according to claim 1, characterized in that, The agent generates test subtask documents corresponding to the training data, based on retrieval enhancement generation technology, combined with prompting engineering and context learning technology.

3. In the automatic testing method for integrated circuits based on generative AI according to claim 2, the retrieval results of the database retrieval tool are one or more, each corresponding to a test subtask.

4. The automatic testing method for integrated circuits based on generative AI according to claim 1, characterized in that, The second generative model invoked by the agent based on different test subtasks is one or more generative AI models.

5. The automatic testing method for integrated circuits based on generative AI according to claim 1, characterized in that, The second generative AI model is constructed at least through model compression techniques.

6. The automatic testing method for integrated circuits based on generative AI according to claim 5, wherein the model compression technique is selected from at least one or a combination of pruning, knowledge distillation, parameter quantization and low-rank decomposition.

Images