Verification environment generation system for integrated circuit, method for generating verification environment for integrated circuit, and program for generating verification environment for integrated circuit

Abstract

Provided is a technology for supporting the functional and logic verification processes of an integrated circuit. This verification environment generation system comprises an input unit for acquiring specification information (112) regarding an integrated circuit. The specification information (112) is information written in a natural language and includes circuit specifications and operational specifications of the integrated circuit. The verification environment generation system further comprises: a control unit that enters the specification information (112) into an artificial intelligence (122) and acquires a verification environment (300) for the integrated circuit, verification items (311), and an assertion (309) generated by the artificial intelligence (122) on the basis of the specification information (112); and an output unit that outputs the verification environment (300), the verification items (311), and the assertion (309).

Description

Integrated circuit verification environment generation system, method for generating an integrated circuit verification environment, and program for generating an integrated circuit verification environment

[0001] The present disclosure relates to integrated circuit verification, and more particularly to techniques for generating a verification environment for an integrated circuit.

[0002] In recent years, the performance of integrated circuits has improved, resulting in an increase in the amount of digital signals processed by the integrated circuits. To realize such high-performance electronic products, the miniaturization of manufacturing processes for integrated circuits, such as LSI (Large Scale Integration), has also progressed exponentially. Accordingly, the development costs of integrated circuits have been increasing. Therefore, there is a need for technology to support the development, manufacturing, or testing of integrated circuits.

[0003] Regarding technology for supporting the development, manufacturing, or testing of integrated circuits, for example, Japanese Patent Application Laid-Open No. 2020-149270 (Patent Document 1) discloses a circuit optimization device that "is provided with circuit information created based on specification information of a circuit to be designed, constraint information used for functional verification of the circuit, and customization directionality information indicating the direction for customizing the circuit, and is equipped with a reinforcement learning unit that generates circuit information as a conclusion that best matches the direction of the customization directionality information based on this information" (see [Abstract]).

[0004] Japanese Patent Application Laid-Open No. 2020-149270

[0005] According to the technology disclosed in Patent Document 1, machine learning is utilized to optimize circuit configuration information of an integrated circuit. However, the technology disclosed in Patent Document 1 cannot support the process of functional and logical verification of an integrated circuit. Therefore, there is a need for a technology to support the process of functional and logical verification of an integrated circuit.

[0006] The present disclosure has been made in view of the above-described background, and an object of one aspect is to provide a technique for supporting the process of verifying the function and logic of an integrated circuit.

[0007] According to one embodiment, there is provided a verification environment generation system for an integrated circuit. The verification environment generation system includes an input unit for acquiring specification information related to the integrated circuit. The specification information is information written in natural language and includes circuit specifications and operational specifications of the integrated circuit. The verification environment generation system further includes a control unit that inputs the specification information to an artificial intelligence (AI) and acquires a verification environment, verification items, and assertions for the integrated circuit generated by the AI ​​based on the specification information, and an output unit that outputs the verification environment, verification items, and assertions.

[0008] According to an embodiment, the process of functional and logical verification of an integrated circuit can be supported.

[0009] The above and other objects, features, aspects and advantages of the present disclosure will become apparent from the following detailed description of the disclosure taken in conjunction with the accompanying drawings.

[0010] 1 is a diagram showing an example of the configuration of the verification environment generation system 10. FIG. 2 is a diagram showing an example of an apparatus 200 used as the verification environment generation apparatus 120. FIG. 3 is a diagram showing an example of elements included in verification environment data. FIG. 4 is a diagram showing an example of an outline of the operation of the verification environment generation apparatus 120. FIG. 5 is a diagram showing an example of a task generation operation of the verification environment generation apparatus 120. FIG. 6 is a diagram showing an example of a verification item generation operation of the verification environment generation apparatus 120. FIG. 7 is a diagram showing an example of a test pattern and assertion generation operation of the verification environment generation apparatus 120. FIG. 8 is a diagram showing an example of a confirmation item 800 for output data of the artificial intelligence 122. FIG. 9 is a diagram showing an example of a processing procedure of the verification unit 128. FIG. 10 is a diagram showing an example of pre-learning or additional learning of the artificial intelligence 122. FIG. 11 is a diagram showing an example of an operation in the verification environment generation apparatus 120 to generate a model of a modified circuit with some functions changed in order to respond to a request to change the function or circuit of an integrated circuit. FIG. 12 is a diagram showing an example of an operation in the verification environment generation apparatus 120 to generate a modified reference model 1235 required when verifying the modified integrated circuit model 1136 in the verification environment 300. FIG. 10 is a diagram showing an example of an operation of generating, by the verification environment generating device 120, components of a task group for input signal testing and a task group for output signal testing, which are required when verifying a modified integrated circuit model 1136, in the verification environment 300. FIG. 11 is a diagram showing an example of an operation of generating, by the verification environment generating device 120, components of a modified assertion 309, which are required when verifying a modified integrated circuit model 1136, in the verification environment 300. FIG. 12 is a diagram showing an example of an operation of generating, by the verification environment generating device 120, a modified existing test pattern, which is required when verifying an existing function in a modified integrated circuit model 1136, in the verification environment 300. FIG. 13 is a diagram showing another example of confirmation items 1600 for output data of the artificial intelligence 122.

[0011] Hereinafter, embodiments of the technical concept of the present disclosure will be described with reference to the drawings. In the following description, identical components are assigned the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated. Furthermore, each embodiment, each modification, each software configuration, each hardware configuration, each function, each process, etc. may be selectively combined as appropriate.

[0012] <A. Verification Environment Generation System> First, a verification environment generation system 10 for an integrated circuit according to the present embodiment will be described with reference to FIGS. 1 and 2. The verification environment generation system 10 supports functional and logical verification of a newly developed integrated circuit. More specifically, the verification environment generation system 10 accepts input of integrated circuit specification information written in natural language and analyzes the specification information. The verification environment generation system 10 then extracts necessary information from the specification information and generates various data necessary for functional and logical verification of the integrated circuit. By automating the construction of a verification environment, the verification environment generation system 10 can reduce the number of development steps for the integrated circuit and suppress the occurrence of human error.

[0013] 1 is a diagram showing an example of the configuration of a verification environment generation system 10. The verification environment generation system 10 includes a user terminal 110 and a verification environment generation device 120. The user terminal 110 and the verification environment generation device 120 are configured to be able to communicate with each other via a network 150. The user terminal 110 stores integrated circuit specification information 112 and an interaction program 114. The user terminal 110 is configured to be connectable to an input device 130 and an output device 140. The verification environment generation device 120 includes an artificial intelligence 122, an input unit 124, an output unit 126, a control unit 127, and a verification unit 128.

[0014] In one aspect, the verification environment generation system 10 may be realized as a cloud service or a web application. In this case, the verification environment generation system 10 may include the verification environment generation device 120, but may not include the user terminal 110. The verification environment generation device 120 may provide various functions to any terminal via a browser. In another aspect, the verification environment generation system 10 may be realized as a server-client system. In this case, the interaction program 114 and the like are implemented as client applications. Furthermore, each component of the verification environment generation device 120 is implemented as a server application. In yet another aspect, the verification environment generation system 10 may be realized as a standalone device. In this case, the verification environment generation system 10 is a single device that has the functions of both the user terminal 110 and the verification environment generation device 120, and may be referred to as a verification environment generation device.

[0015] The user terminal 110 is used by an integrated circuit developer, a person in charge of functional and logical verification of the integrated circuit, etc. The user terminal 110 transmits specification information 112 of the integrated circuit to the verification environment generating device 120. The user terminal 110 also receives data of the verification environment generated by the verification environment generating device 120. In one aspect, the user terminal 110 includes a personal computer, a tablet, a smartphone, a workstation, and any other information processing device.

[0016] The integrated circuit specification information 112 is information written in natural language. The integrated circuit specification information 112 includes circuit specifications and operational specifications of the integrated circuit. The circuit specifications include, for example, the interface (IF) specifications of the integrated circuit and the functional specifications of the integrated circuit. The operational specifications include, for example, the clock signal and reset signal specifications of the integrated circuit and input / output signal information (which may be referred to as input / output signal specifications) other than the clock signal and the reset signal of the integrated circuit. The functional specifications of the integrated circuit may also include a description of the operation of the modified circuit, and therefore may also be considered operational specifications. In one aspect, the integrated circuit specification information 112 may include information about another integrated circuit that served as the basis for the integrated circuit under development.

[0017] The interaction program 114 is an interface for utilizing the functions of the verification environment generating device 120. The interaction program 114 has a function for uploading the specification information 112 to the verification environment generating device 120 and a function for downloading verification environment data from the verification environment generating device 120. The interaction program 114 can also provide the user with functions for utilizing the verification environment generating device 120 through natural language interaction. The user can utilize any function of the verification environment generating device 120 via the interaction program 114.

[0018] The input device 130 is connected to the user terminal 110. A user may input any command to the user terminal 110 via the input device 130. As an example, a user may input a command in natural language to the dialogue program 114 via the input device 130. In one aspect, the input device 130 includes a keyboard, a mouse, a touch panel, a microphone, and any other input device. In another aspect, the input device 130 may be configured as an integral part of the user terminal 110.

[0019] The output device 140 is connected to the user terminal 110. The user can view any information stored in the user terminal 110 via the output device 140. For example, the user can view responses from the dialogue program 114 and downloaded data of the verification environment via the output device 140. In one aspect, the output device 140 includes a liquid crystal display, an organic electroluminescence (EL) display, or any other output device. In another aspect, the output device 140 may be configured as an integral part of the user terminal 110.

[0020] The verification environment generating device 120 is configured to receive the specification information 112 and to output data of a verification environment for the integrated circuit generated from the specification information 112. In one aspect, the verification environment generating device 120 may include an artificial intelligence 122. In this case, the verification environment generating device 120 may generate data of a verification environment for the integrated circuit from the specification information 112 using the artificial intelligence 122 and output the data of the verification environment for the integrated circuit. In another aspect, the verification environment generating device 120 may be configured to communicate with an external artificial intelligence 122. In this case, the verification environment generating device 120 transmits or inputs the specification information 112 to the artificial intelligence 122. Furthermore, the verification environment generating device 120 acquires data of the verification environment for the integrated circuit generated by the artificial intelligence 122 from the specification information 112 from the artificial intelligence 122. Furthermore, in another aspect, the verification environment generating device 120 may be configured to communicate with multiple artificial intelligences 122. As an example, the verification environment generating device 120 may be configured to input the specification information 112 to a first artificial intelligence (AI) and acquire a portion of the data of the verification environment for the integrated circuit generated by the first AI, and further be configured to input the specification information 112 to a second AI and acquire another portion of the data of the verification environment for the integrated circuit generated by the second AI. By dividing the roles among multiple AIs 122 in this manner, the functions of each AI 122 can be reduced. This reduces the maintenance and training costs of each AI 122. Furthermore, in another aspect, the verification environment generating device 120 may be configured to switch the AI ​​122 with which it communicates. This allows the verification environment generating device 120 to select an AI 122 appropriate for the characteristics of the integrated circuit, etc.

[0021] The verification environment generating device 120 also transmits data of the generated verification environment to the user terminal 110. In one aspect, the verification environment generating device 120 may be realized as a server device. In another aspect, the verification environment generating device 120 may be realized as a cloud environment. In yet another aspect, the verification environment generating device 120 may be realized as an application, a virtual machine, an instance, or a container that runs on the server device or the cloud environment.

[0022] The artificial intelligence 122 analyzes the specification information 112 and automatically generates data for a verification environment for the integrated circuit. The artificial intelligence 122 includes a trained model based on machine learning, a generative artificial intelligence (AI), and a large language model (LLM). In one aspect, the verification environment generation device 120 may include the artificial intelligence 122. In another aspect, the artificial intelligence 122 may be external to the verification environment generation device 120 and configured to be able to communicate with the verification environment generation device 120.

[0023] The input unit 124 is an interface for accepting various data from the user terminal 110. In one aspect, the input unit 124 can receive any data, such as the integrated circuit specification information 112, from the user terminal 110 via the network 150. In another aspect, the input unit 124 can accept input of any data, such as the integrated circuit specification information 112, from an input device directly connected to the verification environment generating device 120.

[0024] The output unit 126 is an interface for outputting various data to the user terminal 110. In one aspect, the output unit 126 may transmit any information, such as data on the verification environment, to the user terminal 110 via the network 150. In another aspect, the output unit 126 may output any information, such as data on the verification environment, to an output device directly connected to the verification environment generating device 120.

[0025] The verification unit 128 supports the user in verifying the data in the verification environment. For example, the verification unit 128 analyzes the data in the verification environment and outputs errors and warnings according to predetermined rules. Furthermore, the verification unit 128 may prompt the user to correct the specification information 112, which is input data to the artificial intelligence 122, along with the errors and warnings.

[0026] The control unit 127 controls the entire verification environment generating device 120. The subject of each process in the following description may be replaced with the control unit 127 rather than the verification environment generating device 120. The control unit 127 is configured to input the specification information 112 to the artificial intelligence 122 and acquire data on the verification environment for the integrated circuit generated by the artificial intelligence 122 based on the specification information 112. If the artificial intelligence 122 is located outside the verification environment generating device 120, the verification environment generating device 120 further includes a communication unit (not shown) for communicating with the artificial intelligence 122. The control unit 127 may communicate with the artificial intelligence 122 via the communication unit. The control unit 127 may also be configured to mediate or control data exchange between the artificial intelligence 122, the input unit 124, the output unit 126, and the verification unit 128. Alternatively, the artificial intelligence 122, the input unit 124, the output unit 126, and the verification unit 128 may be configured to exchange data directly without going through the control unit 127.

[0027] As described above, the verification environment generation system 10 inputs the specification information 112 of an integrated circuit into the artificial intelligence 122, and automatically generates and provides data of the verification environment for the integrated circuit. By using the verification environment generation system 10, an integrated circuit designer can obtain various benefits related to the following issues.

[0028] In conventional functional and logical verification of integrated circuits, designers had to manually create verification environment data, such as verification items, verification environment, reference models, test patterns, and assertions. Furthermore, when changes occurred to the circuit configuration, the verification items, verification environment, test patterns, and assertions could also be changed. In such cases, designers had to manually modify the verification environment data that needed to be changed. Creating and modifying such verification environment data was a significant burden on designers. Furthermore, these tasks were highly dependent on the designer's level of expertise, and manually creating and modifying verification environment data could lead to human error and increased logic verification workload.

[0029] By using the verification environment generation system 10, users such as integrated circuit designers can automatically generate verification environment data. As a result, the verification environment generation system 10 can significantly reduce the amount of work required for functional and logical verification of an integrated circuit. Furthermore, the verification environment generation system 10 can reduce the occurrence of human error and an increase in the amount of work required for logical verification, thereby improving the quality of the work required for functional and logical verification of an integrated circuit.

[0030] Next, some terms used in this specification will be explained. In this specification, a "system" refers to a single device or a combination of multiple devices. A device may include a personal computer, a workstation, a server device, a tablet, or a smartphone. A device may also include a system-on-a-chip (SoC) or a system-on-module (SoM). A device may also include any peripheral device, such as a switch, a router, a display, a keyboard, or a mouse. A device may also include a virtual machine or instance built in a cloud environment. In one aspect, a system may be connected to input / output devices, such as a display and a keyboard, and used as a standalone device. In another aspect, a system may provide various functions as a service or web application via a network. In this case, a user may use the system's functions via a browser or client software installed on their device.

[0031] As used herein, "artificial intelligence" can refer to AI models, neural networks, machine learning, deep learning, generative AI, or large language models (LLMs), depending on the context, or it can refer to a combination of these technologies.

[0032] In this specification, "specification information" includes any specifications, such as the circuit configuration, operation, or function, of an integrated circuit. The specification information may be defined in any format and may include a description of the specification in natural language. The specification information may also include diagrams, tables, graphs, and the like, in addition to a description of the specification in natural language. Furthermore, the specification information may include the interface, functions, clock and reset signals, and other signals of the integrated circuit, information about other integrated circuits on which the integrated circuit under development is based, and any other specifications.

[0033] In this specification, the term "verification environment data" includes any elements necessary for functional and logical verification of an integrated circuit. For example, the verification environment data includes the verification environment, verification items, tasks, reference models, models of the integrated circuit to be verified, assertions, and assertion properties.

[0034] In this specification, a "user" refers to a person who uses the verification environment generation system 10 or the verification environment generation device 120. The user may include an integrated circuit designer, an integrated circuit verification engineer, or any other person.

[0035] FIG. 2 is a diagram illustrating an example of a device 200 used as the verification environment generating device 120. The device 200 may be used as the user terminal 110. Furthermore, the user terminal 110 and the verification environment generating device 120 may not have some of the components shown in FIG. 2. Furthermore, the user terminal 110 and the verification environment generating device 120 may have components not shown in FIG. 2. Furthermore, the user terminal 110 and the verification environment generating device 120 may be realized by two or more devices 200. The various functions of the verification environment generating system 10 described in this specification may be realized by executing a program on the hardware shown in FIG. 2.

[0036] The device 200 includes a processor 201, a memory 202, a storage 203, an external device IF 204, an input IF 205, an output IF 206, and a communication IF 207. In one aspect, the device 200 may include two or more of each component, or may not include some of the components.

[0037] The processor 201 may execute programs for implementing various functions of the device 200. The processor 201 may be configured, for example, with at least one integrated circuit. According to an embodiment, the integrated circuit may include at least one central processing unit (CPU), at least one graphics processing unit (GPU), at least one field programmable gate array (FPGA), at least one application specific integrated circuit (ASIC), at least one artificial intelligence (AI) chip, or a combination thereof.

[0038] The memory 202 functions as a workspace for the processor 201. The memory 202 stores programs executed by the processor 201 and data referenced by the processor 201. In one aspect, the memory 202 can be realized by a dynamic random access memory (DRAM), a static random access memory (SRAM), or the like.

[0039] Storage 203 is a non-volatile memory that stores programs executed by processor 201 and data referenced by processor 201. Processor 201 executes programs read from storage 203 to memory 202 and references data read from storage 203 to memory 202. In one aspect, storage 203 can be realized by a hard disk drive (HDD), a solid state drive (SSD), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a flash memory, or the like.

[0040] The external device IF 204 can be connected to any external device such as a printer, a scanner, an external HDD, etc. In one aspect, the external device IF 204 can be realized by a USB (Universal Serial Bus) terminal or the like.

[0041] The input IF 205 can be connected to any input device such as a keyboard, a mouse, a touchpad, a gamepad, etc. In one aspect, the input IF 205 can be realized by a USB terminal, a PS / 2 terminal, a Bluetooth (registered trademark) module, etc.

[0042] Output IF 206 can be connected to any output device such as a cathode ray tube display, a liquid crystal display, an organic EL display, etc. In one aspect, output IF 206 can be realized by a USB terminal, a D-sub terminal, a DVI (Digital Visual Interface) terminal, an HDMI (High-Definition Multimedia Interface) terminal, a DisplayPort terminal, etc.

[0043] Communication IF 207 is connected to other devices via a wired network or a wireless network. In one aspect, communication IF 207 may be implemented by a wired local area network (LAN) port, a Wi-Fi (Wireless Fidelity) (registered trademark) module, or the like. In another aspect, communication IF 207 may transmit and receive data using a communication protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol), UDP (User Datagram Protocol), or the like.

[0044] <B. Data Generated by Artificial Intelligence> Figure 3 shows an example of elements included in the data of the verification environment. Functional and logical verification of an integrated circuit is performed on logic simulation software using the data of the verification environment. Therefore, each element shown in Figure 3 is realized as digital data.

[0045] The verification environment data includes a verification environment 300 and a verification item 311. The verification environment 300 also includes a task group 301, a reference model 305, an integrated circuit model 306, an expected value comparison unit 307, an assertion 309, and a test pattern 310.

[0046] The task group 301 provides input signals (also called stimuli) to a reference model 305 and an integrated circuit model 306. The task group 301 includes an interface test task 302, a clock signal and reset signal generation task 303, and an input signal test task 304.

[0047] The reference model 305 is a model that performs correct operation during logic verification. In other words, the reference model 305 operates according to specifications. The integrated circuit model 306 is a model that simulates the operation of a designed integrated circuit, and is the subject of functional and logical verification.

[0048] The expected value comparison unit 307 executes an output signal expected value comparison task 308. The output signal expected value comparison task 308 is a task for comparing the output signal of the integrated circuit model 306 with the output signal of the reference model 305 to determine whether the logic verification passes or fails. If the output signal of the integrated circuit model 306 matches the output signal of the reference model 305, the logic verification for the output signal passes. The assertion 309 determines whether the output signal of the integrated circuit model 306 satisfies protocol specifications such as output timing and handshake.

[0049] The test pattern 310 calls the task group 301 in accordance with the verification items 311. In response to the call, the task group 301 provides input signals to the reference model 305 and the integrated circuit model 306. The test pattern 310 can also start and stop the reference model 305 and the integrated circuit model 306. Furthermore, the test pattern 310 calls the expected value comparison unit 307 and the assertion 309 to perform logic verification and pass / fail judgment of the protocol specifications, etc.

[0050] The verification items 311 include a list of items to be verified in the function and logic verification of the integrated circuit model 306. The number of test patterns 310 created corresponds to the number of verification items included in the verification items 311. The verification environment 300 executes logic verification for each test pattern 310. If all logic verifications pass, the logic verification is completed.

[0051] Conventionally, each element of the verification environment data shown in FIG. 3 has been created manually. Therefore, creating the verification environment data has been a significant burden on logic verifiers. Furthermore, the accuracy of the verification environment data has depended on the proficiency of the logic verifier, resulting in variations among individuals. Furthermore, manually creating the verification environment data can lead to human error and an increase in the number of steps required for logic verification. The verification environment generation system 10 and verification environment generation device 120 according to this embodiment can solve various conventional problems by automatically generating each element of the verification environment data.

[0052] 4 is a diagram showing an example of an outline of the operation of the verification environment generating device 120. The verification environment generating device 120 receives integrated circuit specification information 112 from the user terminal 110 via the input unit 124, and inputs the specification information 112 as input data to the artificial intelligence 122. The verification environment generating device 120 also transmits verification environment data generated by the artificial intelligence 122 to the user terminal 110 via the output unit 126.

[0053] The specification information 112 includes at least a specification 402 of the developed integrated circuit. The specification 402 describes the circuit specifications and operation specifications of the integrated circuit. Assume that the integrated circuit has been developed based on another integrated circuit (such as a previous product model). In this case, the user may further include, in the specification information 112, information 404 about changes in the integrated circuit from the other integrated circuit and defect information 406 of the other integrated circuit. Furthermore, the user may also include, in the specification information 112, verification items used in verifying the other integrated circuit. Each piece of information included in the specification information 112 is written in natural language.

[0054] If the developed integrated circuit is a completely new design and there is no other integrated circuit on which it is based, the specification information 112 includes only the specification 402 of the developed integrated circuit. Even if there is another integrated circuit on which it is based, the user may choose not to include in the specification information 112 information 404 about changes in the integrated circuit from the other integrated circuit and defect information 406 of the other integrated circuit. If this information is included in the specification information 112, the artificial intelligence 122 may analyze the information 404 about changes in the integrated circuit from the other integrated circuit and generate verification items for testing the changes in the integrated circuit. Similarly, the artificial intelligence 122 is configured to analyze defect information 406 of the other integrated circuit and generate verification items for testing whether defects in the other integrated circuit have been corrected in the integrated circuit.

[0055] The artificial intelligence 122 analyzes the specification information 112 to generate data of the verification environment, such as the verification items 311, the verification environment 300, and the assertions 309. The artificial intelligence 122 may interpret the specification information 112, extract words related to the data of the verification environment, and further generate the data of the verification environment based on these words. Since the assertions 309 include properties, etc., in this specification, the generation of assertions includes the generation of assertions and properties, etc.

[0056] In one aspect, the artificial intelligence 122 may be configured to analyze the specification information 112 and generate at least a portion of the integrated circuit verification environment 300, the verification items 311, and the assertions 309. This allows the artificial intelligence 122 to generate a portion of the verification environment data even if the specification information 112 is incomplete.

[0057] In one aspect, the artificial intelligence 122 may be a trained model through deep learning in which features and network connection weight coefficients are generated using a neural network. In another aspect, the artificial intelligence 122 may be a large-scale language model. In this case, the artificial intelligence 122 may be a model obtained by additionally training an integrated circuit specification, verification environment data, and the like, on an existing large-scale language model. Furthermore, the artificial intelligence 122 may be a generation AI for generating verification environment data.

[0058] As described above with reference to FIG. 4 , the verification environment generation system 10 includes an input unit 124 for acquiring specification information 112 related to an integrated circuit. The specification information 112 is information written in natural language and includes circuit specifications and operational specifications of the integrated circuit. The verification environment generation system 10 also includes a control unit 127 that inputs the specification information 112 to the artificial intelligence 122 and acquires the verification environment 300, verification items 311, and assertions 309 of the integrated circuit generated by the artificial intelligence 122 based on the specification information 112, and an output unit 126 that outputs the verification environment 300, verification items 311, and assertions 309. Furthermore, if the integrated circuit is based on another integrated circuit (such as a previous product model), the verification environment generation system 10 may use information about the other integrated circuit to generate data for the verification environment.

[0059] The verification environment generation system 10 may implement various processes or methods disclosed herein by executing a program on the hardware shown in FIG. 2 . That is, a program for generating a verification environment for an integrated circuit may be provided to a computer or system. The program causes the computer to acquire specification information 112 related to the integrated circuit. The specification information 112 is information written in natural language and includes circuit specifications and operational specifications of the integrated circuit. The program also causes the computer to input the specification information 112 to an artificial intelligence 122, acquire a verification environment 300, verification items 311, and assertions 309 for the integrated circuit generated by the artificial intelligence 122 based on the specification information 112, and output the verification environment 300, verification items 311, and assertions 309.

[0060] 5 is a diagram showing an example of the operation of generating tasks and reference models by the verification environment generation device 120. The specification 402 includes descriptions of an integrated circuit interface specification 502, clock signal and reset signal specifications 504, input / output signal information 506, and integrated circuit function specification 508. By analyzing these descriptions, the artificial intelligence 122 can generate an interface test task 302, a clock signal and reset signal generation task 303, an input signal test task 304, an output signal expected value comparison task 308, and a reference model 305.

[0061] An example of a procedure for the artificial intelligence 122 to generate the interface test task 302 will be described. First, the artificial intelligence 122 extracts interface-related signal names, keywords related to the signals, and descriptions of the operation of these signals from the integrated circuit interface specification 502. The keywords here are, for example, data, address, request, acknowledgment, read / write selection, etc.

[0062] The artificial intelligence 122 may also generate source code for implementing the function or operation of each signal from the extracted keywords and descriptions. The artificial intelligence 122 may also generate source code for comparing signals output from the reference model 305 and the integrated circuit model 306. Furthermore, the artificial intelligence 122 may generate an interface testing task 302 for implementing at least a portion of the processing of these source codes. In one aspect, the source code generated by the artificial intelligence 122 may be written in a hardware description language (HDL).

[0063] Similarly, the artificial intelligence 122 extracts each signal name, keywords related to the signal, and an operational description of each signal from the clock signal and reset signal specifications 504. In this case, the signal name and keywords include, for example, the clock operating frequency, duty ratio, presence or absence of gating, jitter, and positive / negative significance of reset timing. Furthermore, the artificial intelligence 122 can generate source code for realizing the function or operation of each signal from the extracted keywords and descriptions. Furthermore, the artificial intelligence 122 can generate a clock signal and reset signal generation task 303 for realizing at least a portion of the processing of this source code.

[0064] Similarly, the artificial intelligence 122 extracts each signal name, keywords related to the signal, an operation description of each signal, and each operation description from the input / output signal information 506. The artificial intelligence 122 can also generate source code for realizing the function or operation of each signal from the extracted keywords and descriptions. Furthermore, the artificial intelligence 122 can generate an input signal testing task 304 and an output signal expected value comparison task 308 for realizing at least a portion of the processing of this source code.

[0065] The artificial intelligence 122 may generate the reference model 305 by the following procedure. The artificial intelligence 122 extracts, from the functional specification 508, keywords and descriptions related to operations related to signal processing (arithmetic operations, logic, etc.), data movement, and data retention. The artificial intelligence 122 may also generate source code (C / C++ or MATLAB (registered trademark) code, etc.) for realizing the extracted descriptions from the extracted keywords and descriptions. The artificial intelligence 122 may then generate the reference model 305 that realizes the processing of the generated source code.

[0066] 5 , the specification information 112 includes an interface specification 502 of the integrated circuit, clock signal and reset signal specifications 504 of the integrated circuit, input / output signal information 506 other than the clock signal and reset signal of the integrated circuit, and a functional specification 508 of the integrated circuit. The control unit 127 is configured to be able to acquire an interface test task 302 generated based on the interface specification 502, a clock signal and reset signal generation task 303 generated based on the clock signal and reset signal specifications 504, an input signal test task 304 and an output signal expected value comparison task 308 generated based on the input / output signal information 506, and a reference model 305 that simulates the operation of the integrated circuit generated based on the functional specification 508.

[0067] 6 is a diagram showing an example of the operation of the verification environment generation device 120 to generate verification items. In addition to the elements shown in FIG. 5, the specification 402 may include descriptions such as a list of integrated circuit functions 601, information 404 on changes in the integrated circuit from other integrated circuits, information 603 on notable design considerations, information 604 on critical points, and information 605 on protocol specifications. The artificial intelligence 122 is configured to be able to input this information included in the specification 402 and defect information 406 on the other integrated circuits that serve as the basis for the integrated circuit.

[0068] The artificial intelligence 122 analyzes the specifications 402 and the descriptions of the defect information 406 of other integrated circuits to generate verification items 611 for testing each function of the integrated circuit, verification items 612 for testing changes to the integrated circuit, verification items 613 for testing notable design considerations in the integrated circuit, verification items 614 for testing critical points, verification items 615 for testing protocol specifications, and verification items 616 for testing whether defects in other integrated circuits have been corrected. These verification items are part of the verification items 311.

[0069] In one aspect, the artificial intelligence 122 may receive input of words or phrases contained in the specification 402 and other integrated circuit defect information 406. In this case, the artificial intelligence 122 may generate each verification item written in natural language based on the input words or phrases.

[0070] The artificial intelligence 122 can generate verification items 611 for testing each function of the integrated circuit by the following procedure: The artificial intelligence 122 extracts keywords and descriptions of operations related to signal processing (arithmetic operations, logic, etc.), data movement, and data storage processes from the integrated circuit function list 601. The artificial intelligence 122 also generates sentences describing test contents associated with the extracted operations and data processing from the extracted keywords and descriptions, and generates verification items 611 including the sentences.

[0071] The artificial intelligence 122 can also generate verification items 612 for testing changes to an integrated circuit, verification items 613 for testing notable design considerations in an integrated circuit, verification items 614 for testing critical points, verification items 615 for testing protocol specifications, and verification items 616 for testing that defects in other integrated circuits have been corrected using similar logic, although the extracted keywords and descriptions are different.

[0072] 6 , the input unit 124 is configured to be able to input a list of integrated circuit functions 601, information on notable design considerations 603, information on critical points of conflicts or arbitration between multiple operations 604, and protocol specification information 605. The control unit 127 is configured to be able to acquire verification items 611 for testing each function of the integrated circuit generated based on the list of integrated circuit functions 601, verification items 613 for testing notable design considerations generated based on the information on notable design considerations 603, verification items 614 for testing critical points generated based on the information on critical points 604, and verification items 615 for testing protocol specifications generated based on the information on protocol specification 605.

[0073] Furthermore, if the integrated circuit is based on another integrated circuit (such as a previous product model), the verification environment generation system 10 may use information about the other integrated circuit to generate data for the verification environment. That is, the input unit 124 is configured to be able to input information 404 about changes in the integrated circuit from the other integrated circuit and information 406 about defects in the other integrated circuit, which are included in the specification information 112. The control unit 127 is configured to be able to acquire verification items 612 for testing the changes in the integrated circuit generated based on the information 404 about changes in the integrated circuit from the other integrated circuit, and verification items 616 for testing whether the defects in the other integrated circuit have been corrected in the integrated circuit generated based on the information 406 about defects in the other integrated circuit.

[0074] 7 is a diagram showing an example of the operation of generating test patterns and assertions by the verification environment generation device 120. The artificial intelligence 122 is configured to be able to input to itself verification items 611 for testing each function of the integrated circuit generated by the artificial intelligence 122, verification items 612 for testing changes to the integrated circuit, verification items 613 for testing notable design considerations in the integrated circuit, verification items 614 for testing critical points, verification items 615 for testing protocol specifications, and verification items 616 for testing whether defects in other integrated circuits have been corrected. Based on these input verification items, the artificial intelligence 122 generates test patterns and assertions corresponding to each verification item.

[0075] The artificial intelligence 122 extracts explanations of signal generation methods, circuit setting methods, etc. from the verification items 611 for testing each function of the integrated circuit. The artificial intelligence 122 generates test patterns 711 for testing each function from the extracted explanations. Furthermore, the artificial intelligence 122 generates assertions 721 for testing each function of the integrated circuit corresponding to these tasks from the extracted explanations.

[0076] The artificial intelligence 122 can also generate test patterns 712 for testing changes to an integrated circuit, test patterns 713 for testing notable design considerations in an integrated circuit, test patterns 714 for testing critical points, test patterns 715 for testing protocol specifications, and test patterns 716 for testing whether defects in other integrated circuits have been corrected using logic similar to that for generating test patterns 711 for testing each function, although the extracted keywords and descriptions are different.

[0077] In addition, the artificial intelligence 122 can generate assertions 722 for testing changes to an integrated circuit, assertions 723 for testing notable design considerations in an integrated circuit, assertions 724 for testing critical points, assertions 725 for testing protocol specifications, and assertions 726 for testing that defects in other integrated circuits have been corrected using logic similar to that used to generate assertions 721 for testing each function, although the extracted keywords and descriptions are different.

[0078] 5 to 7, the logic by which the artificial intelligence 122 can generate data for the verification environment from the specification 402, etc. has been described. However, in reality, the artificial intelligence 122 does not have to operate according to the above logic. By learning from a huge amount of training data, the artificial intelligence 122 can output, in a black box manner, results similar to those obtained by processing according to the above logic.

[0079] 7 , the input unit 124 is configured to be able to input the verification items 611 for testing each function of the integrated circuit, the verification items 613 for testing notable design considerations, the verification items 614 for testing critical points, and the verification items 615 for testing protocol specifications. The control unit 127 is configured to be able to acquire the test patterns 711 and assertions 721 for testing each function of the integrated circuit generated based on the verification items 611 for testing each function of the integrated circuit, the test patterns 713 and assertions 723 for testing notable design considerations generated based on the verification items 613 for testing notable design considerations, the test patterns 714 and assertions 724 for testing critical points generated based on the verification items 614 for testing critical points, and the test patterns 715 and assertions 725 for testing protocol specifications generated based on the verification items 615 for testing protocol specifications.

[0080] Furthermore, if the integrated circuit is a circuit based on another integrated circuit (such as a previous product model), the input unit 124 is configured to be able to input a verification item 612 for testing the changes and a verification item 616 for testing whether a defect in the other integrated circuit has been corrected in the integrated circuit. The control unit 127 is configured to be able to acquire a test pattern 712 and assertions 722 for testing the changes from the other integrated circuit in the integrated circuit generated based on the verification item 612 for testing the changes, and a test pattern 716 and assertions 726 for testing whether a defect in the other integrated circuit has been corrected in the integrated circuit generated based on the verification item 616 for testing whether a defect in the other integrated circuit has been corrected.

[0081] 11 is a diagram showing an example of an operation of generating a model of a modified circuit in which some functions have been changed in the verification environment generation device 120 in order to respond to a request to change the function or circuit of the integrated circuit. The artificial intelligence 122 is configured to be able to input a list of functions 601 of the original integrated circuit, a model 1101 of the original integrated circuit, and information 1134 on changes in the function of the integrated circuit.

[0082] The artificial intelligence 122 can generate a model 1136 of the modified integrated circuit by analyzing the description of the information 1134 on changes to the functions of the integrated circuit based on the function list 601 of the original integrated circuit and the model 1101 of the original integrated circuit.

[0083] 12 is a diagram showing an example of an operation in which the verification environment generation device 120 generates a modified reference model 1235 that is required when verifying a modified integrated circuit model 1136 in the verification environment 300. The artificial intelligence 122 is configured to be able to input the function list 601 of the original integrated circuit, the original reference model 1201, and information 1134 on changes in the functions of the integrated circuit.

[0084] The artificial intelligence 122 can generate a modified reference model 1235 by analyzing the description of the information 1134 on changes to the functions of the integrated circuit based on the original integrated circuit function list 601 and the original reference model 1201.

[0085] 13 is a diagram showing an example of an operation in which the verification environment generating device 120 generates components of a task group for input signal testing and a task group for output signal testing, which are required to verify the modified integrated circuit model 1136 in the verification environment 300. The artificial intelligence 122 is configured to be able to input the original interface specification 502, the original interface testing task 302, and the interface changes 1301. The artificial intelligence 122 is also configured to be able to input the original clock signal and reset signal specification 504, the original clock signal and reset signal generation task 303, and the clock signal and reset signal changes 1302. The artificial intelligence 122 is also configured to be able to input the original input / output signal information 506, the original input signal testing task 304, the original output signal expected value comparison task 308, and the input / output signal changes 1303.

[0086] The artificial intelligence 122 may generate a modified interface testing task 1332 by analyzing the description of the interface change 1301 based on the original interface specification 502 and the original interface testing task 302. The artificial intelligence 122 may also generate a modified clock signal and reset signal generation task 1333 by analyzing the description of the clock signal and reset signal change 1302 based on the original clock signal and reset signal specification 504 and the original clock signal and reset signal generation task 303. The artificial intelligence 122 may also generate a modified input signal testing task 1334 and a modified output signal expected value comparison task 1338 by analyzing the description of the input / output signal change 1303 based on the original input / output signal information 506, the original input signal testing task 304, and the original output signal expected value comparison task 308.

[0087] 14 is a diagram showing an example of an operation in which the verification environment generation device 120 generates components of the modified assertions 309 required to verify the modified integrated circuit model 1136 in the verification environment 300. The artificial intelligence 122 is configured to be able to input verification items 611 for testing each original function, assertions 721 for testing each original function, and changes 1401 related to each function. The artificial intelligence 122 is also configured to be able to input verification items 613 for testing original notable design considerations, assertions 723 for testing original notable design considerations, and changes 1403 related to the notable design considerations. The artificial intelligence 122 is also configured to be able to input verification items 614 for testing original critical points, assertions 724 for testing original critical points, and changes 1404 related to the critical points. The artificial intelligence 122 is also configured to be able to input verification items 615 for testing the original protocol specification, assertions 725 for testing the original protocol specification, and changes related to the protocol specification 1405. The artificial intelligence 122 is also configured to be able to input verification items 616 for testing that a defect in the original integrated circuit has been corrected, assertions 726 for testing that a defect in the original integrated circuit has been corrected, and changes 1406 that affect the defect correction in the integrated circuit.

[0088] The artificial intelligence 122 may generate assertions 1421 for testing each function after the change by analyzing the description of change 1401 related to each function based on the verification items 611 for testing each original function and the assertions 721 for testing each original function. The artificial intelligence 122 may also generate assertions 1423 for testing notable design considerations after the change by analyzing the description of change 1403 related to notable design considerations based on the verification items 613 for testing original notable design considerations and the assertions 723 for testing the original notable design considerations. The artificial intelligence 122 may also generate assertions 1424 for testing critical points after the change by analyzing the description of change 1404 related to critical points based on the verification items 614 for testing original critical points and the assertions 724 for testing the original critical points. The artificial intelligence 122 may also generate assertions 1425 for testing the modified protocol specification by analyzing the description of changes 1405 related to the protocol specification based on the verification items 615 for testing the original protocol specification and the assertions 725 for testing the original protocol specification. Furthermore, the artificial intelligence 122 may generate assertions 1426 for testing that defects in the modified integrated circuit have been corrected by analyzing the description of changes 1406 that affect the correction of defects in the integrated circuit based on the verification items 616 for testing that defects in the original integrated circuit have been corrected and the assertions 726 for testing that defects in the original integrated circuit have been corrected.

[0089] 15 is a diagram showing an example of the operation of generating, by the verification environment generation device 120, modified existing test patterns that are required when verifying existing functions in the modified integrated circuit model 1136 in the verification environment 300. The modified existing test patterns refer to existing test patterns for testing existing functions of the integrated circuit, but which require some modification because the timing, settings, etc., change in the circuit of the integrated circuit changes the existing test patterns, and therefore the existing test patterns will not be verified correctly if they are input as is. Therefore, the functions verified by the modified existing test patterns correspond to the existing functions of the integrated circuit before the modification.

[0090] The artificial intelligence 122 can generate a modified existing test pattern 1501 by analyzing the description of the information 1134 on changes in the functions of the integrated circuit based on existing verification items 1511 for testing each function and existing test patterns 1521 for testing each function.

[0091] Although not shown, an example of the operation of the verification environment generating device 120 to generate additional verification items and verification patterns required due to circuit changes in the integrated circuit is shown. First, the verification environment generating device 120 obtains verification items 612 for testing the changes in the integrated circuit by inputting information 404 of the changes in the integrated circuit's functions and a generation instruction shown in Figure 6 to the artificial intelligence 122. Next, the verification environment generating device 120 obtains test patterns 712 for testing the changes in the integrated circuit by inputting verification items 612 for testing the changes in the integrated circuit and a generation instruction shown in Figure 7 to the artificial intelligence 122.

[0092] <C. Verification and Learning of Generated Data> Figure 8 is a diagram showing an example of check items 800 for the output data of the artificial intelligence 122. After generating the data of the verification environment, the verification environment generation system 10 may execute a process of presenting the check items 800 and a process of prompting the user to correct the input data to the artificial intelligence 122. In this case, the verification unit 128 executes the process.

[0093] The confirmation items 800 may include at least one of an item 802 for checking whether the verification environment is constructed correctly, an item 804 for checking whether the verification items are extracted correctly, an item 806 for checking whether the test patterns are generated correctly, an item 808 for checking whether the reference model is constructed correctly, and an item 810 for checking whether the assertions are generated correctly.

[0094] 16 is a diagram showing another example of check items 1600 for the output data of the artificial intelligence 122. After generating the data of the verification environment, the verification environment generation system 10 may further execute a process of presenting the check items 1600 and a process of prompting the user to correct the input data to the artificial intelligence 122. In this case, the verification unit 128 executes the process.

[0095] The confirmation items 1600 may include at least one of the following: item 1601 whether the circuit changes to the integrated circuit have been made correctly; item 1602 whether the reference model has been changed correctly; item 1603 whether the input / output signal generation task has been changed correctly; and item 1604 whether the assertions have been changed correctly.

[0096] 9 is a diagram showing an example of a processing procedure of the verification unit 128. The processor 201 may load a program for performing the processing of FIG. 9 from the storage 203 into the memory 202 and execute the program. In another aspect, part or all of the processing may be realized as a combination of circuit elements configured to perform the processing. Furthermore, in another aspect, the order of the following steps may be reversed. Furthermore, the processing entity of each of the following steps may be replaced by the verification environment generation system 10 or the verification environment generation device 120.

[0097] In step S901, the verification unit 128 analyzes the generated verification environment data and determines whether there is a problem with the check item 800. More specifically, the verification unit 128 analyzes the elements included in the verification environment data, such as the verification environment 300, task group 301, reference model 305, integrated circuit model 306, expected value comparison unit 307, assertion 309, test pattern 310, and verification item 311, individually or in combination. The verification unit 128 can then output verification results for each of the items 802, 804, 806, 808, and 810. If the verification unit 128 determines that there is a problem with the check item 800 (YES in step S901), it transfers control to step S902. If not (NO in step S901), the verification process ends.

[0098] In step S902, the verification unit 128 identifies the cause. As an example, the verification unit 128 calculates the similarity (e.g., cosine similarity) between each generated verification item and each function described in the specification 402. Then, if the specification 402 contains a "function X with low similarity to any verification item" (i.e., if there is a possibility that a verification item for verifying the operation of function X has not been generated), the verification unit 128 may identify the presence of the "function X with low similarity to any verification item" as one of the causes. As another example, the verification unit 128 may run data of the generated verification environment on a simulator to acquire an "error message, a warning message, or both." Alternatively, the verification unit 128 may acquire an "error message, a warning message, or both" from the user terminal 110. If the verification unit 128 acquires an error message, a warning message, or both, the verification unit 128 may identify a function, a signal, or the like associated with these messages as one of the causes.

[0099] In step S903, the verification unit 128 generates and presents a proposed correction for the input data to the artificial intelligence 122 to solve the problem. The proposed correction may be generated by the artificial intelligence 122 or the verification unit 128. The proposed correction may include additional sentences to be included in the input data. As an example, the verification unit 128 may present additional sentences to generate a verification item for the "low-similarity function X" identified in step S902. As another example, the verification unit 128 may present additional sentences to resolve the "error message, warning message, or both" identified in step S902, or sentences to be deleted. The verification unit 128 sends the proposed corrections to the dialogue program 114 via the output unit 126. The user may view the proposed corrections via the dialogue program 114.

[0100] In step S904, the verification unit 128 receives a suggested correction for the input data of the artificial intelligence 122 from the user terminal 110. Upon receiving the suggested correction, the verification unit 128 transfers control to step S901. In this way, the verification unit 128 can prompt the user to correct the input data until there are no problems with the confirmation items 800. In some aspects, the artificial intelligence 122 or its learning engine may incorporate the suggested correction obtained in step S904 as learning data and perform additional learning (also called fine tuning).

[0101] FIG. 10 is a diagram illustrating an example of pre-learning or additional learning of the artificial intelligence 122. The artificial intelligence 122 or its learning engine may have an additional learning function. Furthermore, the control unit 127 may be configured to input data of previously developed integrated circuits as training data to the artificial intelligence 122 to allow the artificial intelligence 122 to perform additional learning. As an example, the artificial intelligence 122 or its learning engine is configured to input data related to previously developed integrated circuits as training data. The training data includes, for example, a specification 1010 of a previously developed integrated circuit, a verification environment 1020 of a previously developed integrated circuit, verification items 1030 of a previously developed integrated circuit, a test pattern 1040 of a previously developed integrated circuit, and assertions 1050 of a previously developed integrated circuit. The assertions 1050 also include properties, etc.

[0102] In one aspect, the learning of the artificial intelligence 122 may be performed according to the following policy, for example: In the following description, the reward is a reward given to the artificial intelligence 122 during learning.

[0103] (i) Calculate the similarity in natural language processing between the generated verification items and the verification items for testing (such as cosine similarity in Word2Vec). Then, increase the reward if the similarity is high, and decrease the reward if the similarity is low. (ii) Use the generated test patterns to execute tests in units of a predetermined number of patterns. As a result, increase the reward if test coverage improves significantly. Leave the reward unchanged if test coverage improves moderately. Decrease the reward if test coverage does not improve much. (iii) Increase the reward if the generated assertions match the test protocol. Decrease the reward if they do not match.

[0104] (iv) After repeatedly performing the learning steps (i) to (iii) above, a test data set (such as a specification) is input into the artificial intelligence 122, causing the artificial intelligence 122 to generate verification environment data. The verification environment data output by the artificial intelligence 122 is then evaluated. The verification environment data may be evaluated more highly if it is generated based on fewer generation instructions (input data). Furthermore, the closer the verification environment data is to the desired outcome (such as an example answer for the verification environment data), the more highly the verification environment data may be evaluated. For example, the evaluation index may be the percentage of items with a cosine similarity between the verification items automatically generated by the artificial intelligence 122 and a predetermined value or greater, out of the total number of functions included in the test data set. If the values ​​of these evaluation indexes are low, the user may consider whether the training data used for additional training contains errors or bias, whether there is additional training data that should be added, etc. The verification environment generation system 10 may be equipped with the above-described series of learning functions and a function for evaluating the learning results. The learning shown in FIG. 10 may be performed not only as additional training but also as pre-training.

[0105] 10, the artificial intelligence 122 is configured to be able to receive data of previously developed integrated circuits as training data and perform additional learning. In addition, the control unit 127 is configured to be able to input data of previously developed integrated circuits as training data to the artificial intelligence 122 in order to cause the artificial intelligence 122 to perform additional learning.

[0106] D. Summary As described above, the verification environment generation system 10 can analyze or interpret the integrated circuit specification information 112 written in natural language and generate verification environment data for verifying the function and logic of the integrated circuit. By using the verification environment generation system 10, users such as integrated circuit designers can automatically generate verification environment data. As a result, the verification environment generation system 10 can significantly reduce the amount of work required to verify the function and logic of an integrated circuit. Furthermore, the verification environment generation system 10 can reduce the occurrence of human error and an increase in the amount of work required for logic verification, thereby improving the quality of the work required to verify the function and logic of the integrated circuit.

[0107] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present disclosure is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope equivalent to the claims. Furthermore, the disclosures described in the embodiments and each modification are intended to be implemented, as far as possible, either alone or in combination.

[0108] 10 Verification environment generation system, 110 User terminal, 112 Specification information, 114 Interactive program, 120 Verification environment generation device, 122 Artificial intelligence, 124 Input unit, 126 Output unit, 127 Control unit, 128 Verification unit, 130 Input device, 140 Output device, 150 Network, 200 Device, 201 Processor, 202 Memory, 203 Storage, 204 External device IF, 205 Input IF, 206 Output IF, 207 Communication IF, 300 Verification environment, 301 Task group, 302, 1332 Interface test task, 303, 1333 Reset signal generation task, 304 Input signal test task, 305, 1235 Reference model, 306, 1136 Integrated circuit model, 307 Expected value comparison unit, 308, 1338 Task for comparing output signal expected values, 309: Assertion, 310: Test pattern, 311: Verification item, 402: Specifications of developed integrated circuit, 404, 1134: Information on changes in the integrated circuit from other integrated circuits, 406: Information on defects in other integrated circuits, 502: Interface specifications, 504: Clock signal and reset signal specifications, 506: Input / output signal information, 508: Functional specifications, 601: List of integrated circuit functions, 603: Information on notable design considerations, 604: Information on critical points, 605: Protocol specification information, 611, 1511: Verification items for testing each function of the integrated circuit, 612: Verification items for testing changes in the integrated circuit, 613: Verification items for testing notable design considerations in the integrated circuit, 614: Verification items for testing critical points, 615: Verification items for testing protocol specifications, 616: Verification items for testing that defects in other integrated circuits have been corrected, 711, 1521: Test patterns for testing each function, 712 Test patterns for testing changes in integrated circuits, 713 test patterns for testing notable design considerations in integrated circuits, 714 test patterns for testing critical points, 715 test patterns for testing protocol specifications, 716 test patterns for testing that defects in other integrated circuits have been corrected, 721,1421 Assertions for testing each function, 722 Assertions for testing changes to an integrated circuit, 723, 1423 Assertions for testing notable design considerations in an integrated circuit, 724, 1424 Assertions for testing critical points, 725, 1425 Assertions for testing protocol specifications, 726, 1426 Assertions for testing that defects in other integrated circuits have been corrected, 800, 1600 Confirmation items, 802 Items for checking whether the verification environment is correctly constructed, 804 Items for checking whether verification items are correctly extracted, 806 Items for checking whether test patterns are correctly generated, 808 Items for checking whether the reference model is correctly constructed, 810 Items for checking whether assertions are correctly generated, 1010 Specifications of integrated circuits developed in the past, 1020 Verification environments for integrated circuits developed in the past, 1030 Verification items for integrated circuits developed in the past, 1040 Test patterns for integrated circuits developed in the past, 1050 Assertions of integrated circuits developed in the past, 1101 original integrated circuit model, 1201 original reference model, 1301 interface changes, 1302 clock signal and reset signal changes, 1303 input / output signal changes, 1401 changes related to each function, 1403 changes related to notable design considerations, 1404 changes related to critical points, 1405 changes related to protocol specifications, 1406 changes affecting integrated circuit defect correction, 1501 existing test patterns after changes, 1601 item on whether the circuit changes of the integrated circuit have been made correctly, 1602 item on whether the reference model has been changed correctly, 1603 item on whether the input / output signal generation task has been changed correctly, 1604 item on whether the assertions have been changed correctly.

Claims

1. A verification environment generation system comprising: an input unit for acquiring specification information relating to an integrated circuit, the specification information being information written in natural language and including circuit specifications and operational specifications of the integrated circuit; a control unit for inputting the specification information into artificial intelligence and acquiring a verification environment, verification items and assertions for the integrated circuit generated by the artificial intelligence based on the specification information; and an output unit for outputting the verification environment, the verification items and the assertions.

2. The verification environment generation system of claim 1, wherein the specification information includes interface specifications of the integrated circuit, clock signal and reset signal specifications of the integrated circuit, input / output signal information other than the clock signal and the reset signal of the integrated circuit, and functional specifications of the integrated circuit, and the control unit is configured to be able to acquire interface testing tasks generated based on the interface specifications, clock signal and reset signal generation tasks generated based on the specifications of the clock signal and the reset signal, input signal testing tasks and output signal expected value comparison tasks generated based on the input / output signal information, and a reference model that simulates the operation of the integrated circuit generated based on the functional specifications.

3. The verification environment generation system of claim 1 or 2, wherein the input unit is configured to be able to input a list of functions in the integrated circuit, information on notable design considerations, information on critical points of conflicts or arbitration between multiple operations, and information on protocol specifications, and the control unit is configured to be able to acquire verification items for testing each function of the integrated circuit generated based on the list of functions, verification items for testing the notable design considerations generated based on the information on the notable design considerations, verification items for testing the critical points generated based on the information on the critical points, and verification items for testing the protocol specifications generated based on the information on the protocol specifications.

4. The verification environment generation system according to claim 3, wherein the input unit is configured to be able to input verification items for testing each function of the integrated circuit, verification items for testing the notable design considerations, verification items for testing the critical points, and verification items for testing the protocol specification, and the control unit is configured to be able to acquire test patterns and assertions for testing each function of the integrated circuit generated based on the verification items for testing each function of the integrated circuit, test patterns and assertions for testing the notable design considerations generated based on the verification items for testing the notable design considerations, test patterns and assertions for testing the critical points generated based on the verification items for testing the critical points, and test patterns and assertions for testing the protocol specification generated based on the verification items for testing the protocol specification.

5. The verification environment generation system of claim 1 or 2, wherein the integrated circuit is a circuit based on another integrated circuit, the input unit is configured to be able to input information on changes in the integrated circuit from the other integrated circuit and defect information of the other integrated circuit together with the specification information, and the control unit is configured to be able to acquire verification items for testing the changes in the integrated circuit generated based on the information on changes in the integrated circuit from the other integrated circuit, and verification items for testing that defects in the other integrated circuit have been corrected in the integrated circuit generated based on the defect information of the other integrated circuit.

6. The verification environment generation system described in claim 5, wherein the input unit is configured to be able to input verification items for testing the changes and verification items for testing whether defects in the other integrated circuits have been corrected in the integrated circuit, and the control unit is configured to be able to acquire test patterns and assertions for testing changes in the integrated circuit from the other integrated circuits, which are generated based on the verification items for testing the changes, and test patterns and assertions for testing whether defects in the other integrated circuits have been corrected in the integrated circuit, which are generated based on the verification items for testing whether defects in the other integrated circuits have been corrected.

7. A verification environment generation system as described in claim 1 or 2, wherein the control unit is configured to be able to input data from previously developed integrated circuits as training data into the artificial intelligence in order to allow the artificial intelligence to undergo additional learning.

8. The verification environment generation system described in claim 1, wherein the control unit inputs a list of functions of the original integrated circuit, a model of the original integrated circuit, and changes to the functions of the integrated circuit into the artificial intelligence, and obtains a model of the modified integrated circuit generated by the artificial intelligence.

9. The verification environment generation system described in claim 8, wherein the control unit inputs a list of functions of the original integrated circuit, the original reference model, and changes to the functions of the integrated circuit into the artificial intelligence, and obtains a modified reference model generated by the artificial intelligence.

10. The verification environment generation system described in claim 8, wherein the control unit: inputs the original interface specifications, the original interface testing task, and interface changes resulting from changes in the function of the integrated circuit to the artificial intelligence, and obtains the modified interface testing task generated by the artificial intelligence; inputs the original clock signal and reset signal specifications, the original clock signal and reset signal generation task, and changes in the clock signals and reset signals resulting from changes in the function of the integrated circuit to the artificial intelligence, and obtains the modified clock signal and reset signal generation task generated by the artificial intelligence; inputs the original input / output signal information, the original input signal testing task, the original output signal expected value comparison task, and changes in the input / output signals resulting from changes in the function of the integrated circuit to the artificial intelligence, and obtains the modified input signal testing task and the modified output signal expected value comparison task generated by the artificial intelligence.

11. The control unit generates assertions for testing each of the modified functions by inputting verification items for testing each of the original functions, assertions for testing each of the original functions, and changes to each of the functions among the changes to the functions of the integrated circuit into the artificial intelligence; inputs verification items for testing original notable design considerations, assertions for testing the original notable design considerations, and changes to the notable design considerations due to changes to the functions of the integrated circuit into the artificial intelligence, and obtains assertions for testing the notable design considerations after the change generated by the artificial intelligence; inputs verification items for testing original critical points, assertions for testing the original critical points, and changes to the critical points due to changes to the functions of the integrated circuit into the artificial intelligence, and obtains assertions for testing the modified critical points generated by the artificial intelligence; 9. The verification environment generation system of claim 8, wherein verification items for testing an original protocol specification, assertions for testing the original protocol specification, and changes to the protocol specification due to changes in the function of the integrated circuit are input to the artificial intelligence, and assertions for testing the modified protocol specification generated by the artificial intelligence are obtained; and verification items for testing that defects in the original integrated circuit have been corrected, assertions for testing that defects in the original integrated circuit have been corrected, and changes that affect the defect correction in the integrated circuit due to changes in the function of the integrated circuit are input to the artificial intelligence, and assertions for testing that defects in the modified integrated circuit generated by the artificial intelligence have been corrected are obtained.

12. The verification environment generation system described in claim 8, wherein the control unit inputs existing verification items for testing each function, existing test patterns for testing each function, and changes to the functions of the integrated circuit into the artificial intelligence, and obtains the existing test patterns after the changes generated by the artificial intelligence.

13. A computer-implemented method for generating a verification environment for an integrated circuit, comprising: obtaining specification information regarding the integrated circuit, the specification information being information written in natural language and including circuit specifications and operational specifications of the integrated circuit; the method further comprising: inputting the specification information into artificial intelligence, and obtaining a verification environment, verification items, and assertions for the integrated circuit generated by the artificial intelligence based on the specification information; and outputting the verification environment, the verification items, and the assertions.

14. A program for causing a computer to generate a verification environment for an integrated circuit, the program causing the computer to acquire specification information regarding the integrated circuit, the specification information being information written in natural language and including circuit specifications and operational specifications of the integrated circuit, the program further causing the computer to input the specification information into artificial intelligence, acquire a verification environment, verification items, and assertions for the integrated circuit generated by the artificial intelligence based on the specification information, and output the verification environment, the verification items, and the assertions.

Images