Fpga verification method and system based on large language model and coverage feedback

Abstract

The application provides an FPGA verification method and system based on a large language model and coverage feedback, acquires RTL source code of a design under test, extracts design under test information, combines a preset prompt word template and a black box test constraint protocol, constructs initial generated instructions, drives the large language model to generate initial test code, compiles the test code, simulates based on the compiled code, generates a coverage database file, analyzes the coverage database file, obtains verification blind spots, constructs directional enhancement prompt words for specific blind spots, combines the black box constraint mechanism again, generates directional test tasks for uncovered logic, dynamically inserts the directional test tasks into the test code, recompiles and simulates until a preset target coverage rate or an iteration upper limit is reached, and finally outputs optimal test code. The application forms a closed loop feedback data path, so that the verification process can continuously evolve based on real-time data feedback.

Description

Technical Field

[0001] This invention relates to the field of electronic design automation (EDA) assisted verification technology, specifically to an FPGA verification method and system based on large language models and coverage feedback. It also relates to a corresponding computer device. Background Technology

[0002] Currently, mainstream commercial EDA verification processes primarily rely on Constrained-Random Verification (CRV) within the Universal Verification Methodology (UVM) framework. While this method reduces the workload of manually writing stimuli to some extent, it still essentially falls under the category of "blind testing." Verification engineers need to expend significant effort manually writing complex constraint rules and coverage collection models. When facing large-scale designs containing deep state machines or complex temporal logic, random stimuli often struggle to penetrate shallow logic, resulting in extremely low efficiency in exploring the state space. The verification process typically exhibits the characteristic of "diminishing marginal returns," meaning that after reaching a certain coverage level (e.g., 80%), the remaining corner cases are extremely difficult to discover through random seed collisions, leading to extremely slow coverage convergence and often requiring tedious manual intervention to write targeted test cases.

[0003] In recent years, attempts to directly generate Verilog / SystemVerilog test code using Large Language Models (LLMs) have generally suffered from a serious "model illusion" problem. Due to a lack of deep understanding of Hardware Description Language (HDL) simulation mechanisms, general-purpose LLMs tend to adopt a software programming mindset when generating test code, frequently and incorrectly attempting to directly read or assign values ​​to registers, nets, or private state variables within the Module Under Test (DUT) (i.e., "white-box" operations). This behavior of directly manipulating internal signals by bypassing top-level ports violates SystemVerilog's syntactic scoping rules and module encapsulation principles, directly causing the generated test platform to fail to compile in simulation tools such as Verilator or VCS. Furthermore, existing solutions lack syntactic constraints on the generated code, and the generated stimulus timing often does not meet the synchronization requirements of the hardware clock edges, resulting in test vectors that are unsynthesizable or unable to trigger effective logic flips.

[0004] Most existing AI-assisted verification tools are still in the "open-loop generation" stage, meaning the model generates test cases only once based on static design code, and the task ends after generation, without sensing the actual effect after simulation. These solutions fail to establish a data path with the coverage database generated by EDA tools, and large language models cannot know which lines of code have been covered and which branches have not been triggered. Therefore, when the initially generated test case coverage is insufficient, the system cannot perform targeted supplementation and can only rely on manual analysis of the coverage report and manual prompting of the model. This working mode without a feedback loop prevents the model from improving verification quality through an iterative process of "trial and error-analysis-correction" like human engineers, making it difficult to achieve true end-to-end automation, thus greatly reducing its practicality when handling verification tasks with high reliability requirements. Summary of the Invention

[0005] To address the aforementioned shortcomings in existing technologies, this invention provides an FPGA verification method and system based on a large language model and coverage feedback. A corresponding computer device is also provided.

[0006] According to one aspect of the present invention, an FPGA verification method based on large language models and coverage feedback is provided, comprising: Obtain the RTL source code of the design under test and extract the design information; Based on the design information to be tested, and combined with the preset prompt word template and black-box testing constraint protocol, an initial generation instruction is constructed; The initial generation instructions are used to drive the large language model to generate initial test code; The test code is compiled, and error logs are automatically extracted and fed back to the large language model for self-correction until compilation succeeds. Simulation is performed based on the compiled code to generate a coverage database file; The coverage database file is parsed to obtain verification blind spots; Based on the aforementioned verification blind spots, targeted enhanced prompts for specific blind spots are constructed, and combined with the black-box constraint mechanism, targeted test tasks for uncovered logic are generated through a large language model. The targeted test tasks are dynamically inserted into the test code, and then recompiled and simulated until the preset target coverage or iteration limit is reached. Finally, the best test code is output as a complete verification environment that meets the coverage requirements.

[0007] Preferably, the step of constructing initial generation instructions based on the design information to be tested, combined with a preset prompt word template and a black-box testing constraint protocol, includes: Based on the design information to be tested, and in conjunction with the preset prompt word template, an initial prompt word containing the design information to be tested is constructed; A mandatory black-box testing constraint protocol is implemented; wherein, the black-box testing constraint protocol explicitly requires that the large language model can only control the design through the input port of the top-level module, and strictly prohibits the use of hierarchical references to access the internal signals of the module or the use of forced assignment statements; By combining the initial prompt words and the black-box testing constraint protocol, an initial generation instruction is constructed.

[0008] Preferably, the step of compiling the test code and automatically extracting error logs to feed back to the large language model for correction until compilation succeeds includes: The system automatically calls EDA tools to compile the generated code; when a compilation syntax error occurs, it automatically captures the error log output by the compiler and feeds it back to the large language model for self-correction until compilation succeeds.

[0009] Preferably, the step of performing coverage parsing on the database file to obtain verification blind spots includes: Read the database file, which is a commented Verilog code file, in which coverage markers are used to identify the number of times each line is executed; The missing coverage data is obtained by identifying lines of code with a count of zero, conditional branch markers, and uncovered branch markers in the coverage markers using a pattern matching algorithm. By employing a mapping method that associates missing coverage data with source code context, the missing coverage data is associated with its corresponding source code lines. Combined with pre-established semantic analysis results (such as FSM state transition diagrams and function point lists), the missing coverage data is transformed into a natural language description that can be understood by a large language model, thereby clarifying the verification blind spots.

[0010] Preferably, generating targeted test tasks for uncovered logic includes: Based on the context information of the uncovered logic, a targeted enhancement prompt word is constructed, and the black-box test constraint protocol is loaded as the generation constraint condition; The targeted enhancement prompt is sent to the large language model, which generates test task code for the uncovered logic. The test task code triggers the execution of the uncovered logic by applying a specific timing sequence to the input port of the FPGA device within a preset simulation cycle.

[0011] According to another aspect of the present invention, an FPGA verification system based on a large language model and coverage feedback is provided, comprising: The semantic parsing and initial generation module is used to obtain the RTL source code of the design under test, extract the design under test information, and use the initial generation instructions to drive the large language model to generate initial test code. The simulation execution and data acquisition module compiles the test code and automatically extracts error logs to feed back to the large language model for self-correction until compilation is successful; it then performs simulation based on the compiled code and generates a coverage database file. A coverage blind spot analysis module is used to parse the coverage database file to obtain verification blind spots; The targeted enhancement generation module constructs targeted enhancement prompts for specific blind spots based on the aforementioned verification blind spots. It then combines black-box constraint mechanisms to generate targeted test tasks for uncovered logic through a large language model. The targeted test tasks are dynamically inserted into the test code and recompiled and simulated until the preset target coverage or iteration limit is reached. Finally, the best test code is output as a complete verification environment that meets the coverage requirements.

[0012] Preferably, the input end of the semantic parsing and initial generation module is connected to the source code storage area of ​​the design under test through a file reading interface, and simultaneously establishes a bidirectional communication link with an external large language model service through a network socket or HTTP protocol interface; the output end of the semantic parsing and initial generation module is logically connected to the input end of the simulation execution and data acquisition module, transmitting the generated initial code stream to the simulation environment; the data output end of the simulation execution and data acquisition module is divided into two paths, one of which writes the compilation log to a temporary storage area through a file system interface. When the semantic parsing and initial generation module detects a compilation failure, it reads the compilation log and extracts the error information to trigger the large language model to perform syntax correction; the other path is connected through a file system interface... The coverage database storage area is connected to the coverage database storage area for real-time writing of the coverage database file generated by the simulation. The input end of the coverage blind spot analysis module is connected to the coverage database storage area and the source code storage area respectively, establishing a bidirectional index mapping relationship between coverage data and source code. The output end of the coverage blind spot analysis module is connected to the input end of the targeted enhancement generation module, transmitting natural language description data after semantic conversion. The targeted enhancement generation module also includes an energy allocation unit, used to track the coverage status of each function point and dynamically select the target function point to be covered in the next round. The code output end of the large language model is finally looped back to the code merging interface of the simulation execution and data acquisition module, forming a closed-loop feedback data path.

[0013] According to a third aspect of the present invention, a computer device for FPGA verification is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, can be used to perform the method described in any one of the above inventions.

[0014] By adopting the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art: 1. Achieved "precision-guided" verification based on a coverage-based closed loop, significantly improving convergence efficiency. This invention not only automates code generation but also establishes a data closed loop from "coverage analysis" to "stimulus generation." By transforming the abstract coverage data (such as uncovered branches and unjudged states) fed back by EDA tools into semantic descriptions understandable by a large language model, the system can accurately locate verification blind spots. The Coverage-Guided Proxy (CGA) can generate targeted test tasks for these blind spots, like a "precision-guided weapon" to precisely eliminate uncovered logical dead zones. This iterative approximation method greatly reduces the generation of invalid random stimuli, enabling code coverage to converge to 100% quickly with fewer iterations.

[0015] 2. By utilizing black-box constraint mechanisms, the high quality and compilability of the generated code are ensured. This invention embeds strict black-box testing constraints into the prompt word project, forcing the large language model to drive the design solely through timing combinations of top-level input ports (clock, reset, and data lines), fundamentally eliminating violations such as the model directly manipulating internal signals. This mechanism guarantees that the generated SystemVerilog test code strictly conforms to IEEE standard syntax and hardware simulation specifications, significantly improving the compilation pass rate of the generated code in mainstream simulators such as Verilator and VCS, preventing frequent interruptions to the automated process due to syntax errors.

[0016] 3. Significantly reduces the verification threshold and labor costs, achieving automation with "zero human intervention." This invention encapsulates the complex verification process (environment setup, stimulus writing, result analysis, and targeted enhancement) into a fully automated pipeline. Verification engineers do not need to be proficient in complex UVM methodologies or manually write tedious targeted test cases; they only need to provide RTL source code, and the system can automatically evolve a high-coverage test platform. This not only frees verification engineers from repetitive tasks, allowing them to focus on architecture-level system verification, but also lowers the technical threshold for junior engineers to perform high-quality verification.

[0017] 4. Possesses deep logic exploration capabilities, enhancing the security and robustness of chip design. This invention leverages the powerful semantic understanding capabilities of a large language model to infer complex temporal dependencies that are difficult for traditional algorithms to detect. The system-generated targeted stimuli often include multi-stage state transition sequences and extreme boundary condition combinations, effectively triggering deep-seated bugs hidden in the design (such as state machine deadlock, counter overflow, FIFO read / write conflicts, etc.). This deep exploration capability of corner cases significantly improves the reliability and functional safety of the final delivered chip. Attached Figure Description

[0018] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating the FPGA verification method based on a large language model and coverage feedback in a preferred embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the components of an FPGA verification system based on a large language model and coverage feedback in a preferred embodiment of the present invention.

[0020] Figure 3 This is a flowchart of the FPGA verification system based on a large language model and coverage feedback in a preferred embodiment of the present invention. Detailed Implementation

[0021] The embodiments of the present invention are described in detail below: These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

[0022] In the current field of integrated circuit front-end verification, traditional verification methods often suffer from inefficiency and difficulty in achieving coverage convergence, while existing AI-assisted tools generate code with low quality, compile failures, and a lack of feedback mechanisms. Specifically: Traditional constrained-random verification suffers from the "coverage wall" problem. In traditional processes, as verification progresses, the efficiency of random stimuli in triggering deep state machines and edge logic decreases exponentially, leading to stagnant coverage in the later stages of verification. Engineers are forced to invest significant time manually writing targeted test cases to fill the final blind spots. General-purpose large language models generate hardware code with "internal signal illusions" and non-synthesizable issues. Existing large language models lack understanding of hardware black-box testing principles, and the generated test code often attempts to illegally access private signals or state variables within the module under test. This results in the generated code failing to compile in standard EDA simulators or failing to run due to timing logic errors, severely limiting the practical application of AI technology in hardware verification. Existing automation tools lack a data-driven closed-loop mechanism. Current generative verification tools are mostly "one-time generation" tools, unable to perceive simulation results or dynamically adjust generation strategies based on the current coverage status. This "open-loop" mode prevents the model from optimizing subsequent test stimuli by analyzing missed test points, unlike human engineers, resulting in wasted computing power and incomplete verification.

[0023] To address the aforementioned issues, one embodiment of the present invention provides an FPGA verification method based on a large language model and coverage feedback. This method designs the RTL (Register Transfer Level) code function verification and coverage convergence stage in the front-end design of digital integrated circuits. It aims to use artificial intelligence technology to solve the problems of long verification cycles and insufficient completeness in large-scale digital logic circuits, and can be widely used in semiconductor chip design, FPGA application development, and high-reliability embedded system integration.

[0024] Specifically, such as Figure 1 As shown, the FPGA verification method based on large language models and coverage feedback provided in this embodiment may include: S1, Obtain the RTL source code of the design under test (DUT) and extract the design under test information; S2, based on the design information to be tested, combined with the preset prompt word template and black-box test constraint protocol, constructs the initial generation instructions; S3 uses initial generation instructions to drive a pre-trained large language model to generate initial test code; S4 compiles the test code and automatically extracts error logs, feeding them back to the large language model for self-correction until compilation succeeds. S5 performs simulation based on the compiled code, generating a database file containing row coverage, branch coverage, and state machine coverage. S6, perform coverage parsing on the database file to obtain verification blind spots; S7, based on the verification blind spot, construct targeted enhanced prompt words for specific blind spots, and combine them again with the black box constraint mechanism to generate targeted test tasks for the uncovered logic through the above large language model; S8 dynamically inserts targeted test tasks into the test code, recompiles and simulates until the preset target coverage or iteration limit is reached, and finally outputs the best test code as a complete verification environment that meets the coverage requirements.

[0025] In some preferred embodiments, S1, which involves obtaining the RTL source code of the design under test (DUT) and extracting the DUT information, may further include: Using a parser, the port list and parameter definitions of the top-level module are parsed out through abstract syntax tree technology to obtain the design information under test.

[0026] In some preferred embodiments, the above-mentioned S2, based on the design information under test and combined with a preset prompt word template and black-box testing constraint protocol, constructs an initial generation instruction, which may further include: S21, Based on the design information to be tested and combined with the preset prompt word template, construct an initial prompt word containing the design information to be tested; S22, mandatory black-box test constraint protocol; the black-box test constraint protocol explicitly requires that the large language model can only control the design through the input ports of the top-level module (such as clock, reset and data signals), and strictly prohibits the use of hierarchical references to access the internal signals of the module or the use of forced assignment statements; S23, combining the initial prompt words and the black-box testing constraint protocol, constructs the initial generation instructions.

[0027] In some preferred embodiments, S3 above, which uses initial generation instructions to drive a pre-trained large language model to generate initial test code, may further include: The initial generation instructions are sent to the large language model, which drives the large language model to generate the initial SystemVerilog test platform and obtain the initial SystemVerilog test code. The initial SystemVerilog test code includes clock drive, reset logic and basic random stimulus.

[0028] In some preferred embodiments, S4 above, which involves compiling the test code and automatically extracting error logs to feed back to the large language model for correction until compilation succeeds, may further include: The system automatically calls EDA tools to compile the generated code; when a compilation syntax error occurs, it automatically captures the error log output by the compiler and feeds it back to the large language model for self-correction until compilation succeeds.

[0029] In some preferred embodiments, S6 above, which involves performing coverage parsing on the database file to obtain verification blind spots, may further include: S61, Read the database file, which is a commented Verilog code file, in which coverage markers are used to indicate the number of times each line is executed; S62, using a pattern matching algorithm to identify lines of code with a count of zero, conditional branch markers, and uncovered branch markers in the coverage markers, to obtain missing coverage data; S63 employs a mapping method that associates missing coverage data with source code context, linking missing coverage data with its corresponding source code lines. Combined with pre-established semantic analysis results (such as FSM state transition diagrams and function point lists), the missing coverage data is transformed into a natural language description that the large language model can understand, thereby clearly identifying verification blind spots.

[0030] In some preferred embodiments, the above-mentioned S7, generating targeted test tasks for uncovered logic, may further include: S71, construct targeted enhancement prompts based on the context information of the uncovered logic, and load the black-box test constraint protocol as the generated constraint conditions; S72 sends the targeted enhancement prompt to the large language model, which generates test task code for the uncovered logic. The test task code triggers the execution of the uncovered logic by applying a specific timing sequence to the input ports of the FPGA device within a preset simulation cycle.

[0031] The FPGA verification method based on a large language model and coverage feedback provided in the above embodiments of the present invention begins its execution flow with the design-under-test (DUT) parsing and environment initialization phase. First, the user-provided Verilog or SystemVerilog RTL source code is read in, and abstract syntax tree (AST) technology is used to parse the port list and parameter definitions of the top-level module. Then, the initial test platform generation step based on black-box constraints is initiated. In this step, initial prompts containing DUT information are constructed, and a black-box test constraint protocol is forcibly implanted. This protocol explicitly requires that the large language model can only control the design by driving top-level input ports (such as clock, reset, and data signals), and strictly prohibits accessing internal module signals using hierarchical references or using forced assignment statements. Based on this constraint, the model generates an initial SystemVerilog test platform.

[0032] Next, the automated compilation simulation and syntax correction steps begin. EDA tools are used to compile the generated code. If compilation syntax errors are encountered, error logs are automatically extracted and fed back to the model for self-correction until compilation succeeds. Simulation is then executed, generating a database file containing line coverage, branch coverage, and state machine coverage. At this point, the process enters the core semantic mapping analysis step of the coverage data. The system parses the coverage database, extracts uncovered code regions, and reads the context information of the corresponding locations in the source code. Through semantic analysis, the "uncovered lines of code" are transformed into specific logical meanings, such as "the state machine did not transition from the idle state to the transmission state" or "the FIFO full flag was not triggered," thereby clearly identifying verification blind spots.

[0033] Based on the above analysis results, the targeted enhancement stimulus generation step is executed. Targeted enhancement cue words for specific blind spots are constructed, and black-box constraint mechanisms are applied again. Based on the cue words, the model infers how to trigger specific internal logic through timing combinations of external ports and generates dedicated test tasks. Finally, iterative closure and code merging steps are executed, dynamically inserting the newly generated targeted tasks into the SystemVerilog test platform, and recompiling and simulating. It is determined whether the current synthesis coverage has reached a preset threshold (e.g., 100%). If not, the analysis and enhancement steps are repeated until coverage converges, ultimately outputting a high-coverage complete verification environment.

[0034] The FPGA verification method based on a large language model and coverage feedback provided in the above embodiments of the present invention has two main improvements: "coverage feedback based on semantic mapping" and "causal reasoning under black-box testing constraints". The technical principles that lead to these improvements are as follows: First, regarding the coverage blind spot analysis and targeted enhancement steps, the core principles lie in "targeted reduction of information entropy" and "cross-modal semantic alignment." Traditional random testing is essentially a high-entropy blind search. As coverage increases, the remaining uncovered logic is usually located deep in the state space, and the probability of simply relying on random seed collisions is extremely low. This invention obtains explicit "missing information" by parsing the coverage database. However, EDA tools only output binary statistical data (such as "Line 50 not hit"), which large language models cannot directly understand. Based on this, this invention introduces a semantic mapping mechanism, utilizing the pre-trained knowledge of the large language model to "translate" the tedious line number data into natural language descriptions with logical context (such as "FIFO write enable signal not set in full state"). This transformation converts an inefficient "random mathematical collision problem" into a "logical reasoning and completion problem" that large language models excel at. Based on the understanding of the code intent, the model can reverse-engineer the specific preconditions required to trigger the logic, thereby generating highly targeted test stimuli and breaking the convergence bottleneck of random testing.

[0035] Secondly, regarding the black-box testing constraint mechanism, its core principle lies in "forced alignment of simulation physical laws" and "temporal transformation of assignment problems." Large language models, when generating code, tend to find the shortest path by default, that is, directly modifying internal variables through assignment statements to achieve the target state. This violates the physical law that signals must be transmitted step-by-step in hardware simulation (i.e., the black-box principle), leading to compilation failure. This invention employs a black-box constraint mechanism, which is essentially a forced guidance of a "chain-of-thought." It forces the model to abandon the "God's-eye view" of direct assignment and instead perform causal reasoning. That is, the model must consider: "If I want the internal counter to equal 10 (result), how many cycles of pulses (cause) must I apply to the external clock port?" Through this constraint, the model no longer generates simple state assignment code, but rather an excitation sequence containing precise timing control. This code, generated based on causal logic, naturally conforms to the various timing requirements and module encapsulation rules of hardware description languages, thus fundamentally eliminating the code non-synthesizability problem caused by "model illusion" and ensuring the robustness of the automated process.

[0036] Based on the same inventive concept, one embodiment of the present invention also provides an FPGA verification system based on a large language model and coverage feedback.

[0037] Specifically, such as Figure 2 As shown, the FPGA verification system based on large language models and coverage feedback provided in this embodiment may include: The semantic parsing and initial generation module is used to obtain the RTL source code of the design under test (DUT), extract the DUT information, and use the initial generation instructions to drive the large language model to generate initial test code. The simulation execution and data acquisition module compiles the test code and automatically extracts error logs to feed back to the large language model for self-correction until compilation is successful. Based on the compiled code, simulation is performed to generate a database file containing line coverage, branch coverage, and state machine coverage. The coverage blind spot analysis module is used to parse the database file to obtain verification blind spots. The targeted enhancement generation module constructs targeted enhancement prompts for specific blind spots based on the verification blind spots. It then combines black-box constraint mechanisms to generate targeted test tasks for uncovered logic through a large language model. The targeted test tasks are dynamically inserted into the test code, and the code is recompiled and simulated. If the coverage of the newly generated test code is better than the historical best, the best test code is updated. The above process is repeated until the preset target coverage or the maximum number of iterations is reached. Finally, the best test code is output as a complete verification environment that meets the coverage requirements.

[0038] The FPGA verification system based on a large language model and coverage feedback provided in this embodiment consists of four core parts in terms of logical structure: semantic parsing and initial generation module, simulation execution and data acquisition module, coverage blind spot analysis module, and targeted enhancement generation module.

[0039] In some preferred embodiments, the semantic parsing and initial generation module serves as the system's input interface, responsible for reading the RTL source code of the Design Under Test (DUT). This module integrates a parser to extract the input / output port definitions, parameter configurations, and module hierarchy of the top-level modules. Its core function is to construct initial generation instructions by combining preset prompt word templates with black-box testing constraint protocols. These instructions are sent to the large language model, driving it to generate an initial SystemVerilog test platform containing clock-driven, reset logic, and basic random stimuli.

[0040] In some preferred embodiments, the simulation execution and data acquisition module acts as a bridge between the software control logic and the EDA simulation tool. This module is responsible for receiving the generated test code and automatically calling simulators such as Verilator or Iverilog for compilation and execution. During the compilation phase, if a syntax error occurs, this module automatically captures the error log output by the compiler and feeds it back to the preceding module for correction. During the simulation phase, this module is responsible for collecting waveform files and a code coverage database to provide raw data support for subsequent analysis.

[0041] In some preferred embodiments, the coverage blind spot analysis module is the core decision-making unit of this system. This module reads the coverage database and uses algorithms to identify uncovered lines of code, conditional branches, and state machine states that have not transitioned. The key innovation of this module lies in the integration of a semantic analysis-based mapping algorithm (hereinafter referred to as the semantic mapping algorithm), which can map abstract coverage missing data (such as line numbers or state codes) back to the source code context, transforming it into a natural language description that the large language model can understand, such as pointing out specific missing logical functions, rather than simply reporting data statistics.

[0042] In some preferred embodiments, a targeted enhancement generation module (also known as a coverage-oriented proxy) is responsible for implementing closed-loop feedback. This module receives semantic descriptions from the analysis module and constructs targeted cue words containing specific verification objectives. During this process, the module again applies black-box constraint mechanisms to guide the large language model to generate targeted test tasks specifically for uncovered logic. The generated task code is dynamically injected into the main testing platform, thus forming an automated closed loop of "analysis-generation-verification".

[0043] The FPGA verification system based on a large language model and coverage feedback provided in the above embodiments of the present invention is constructed based on a modular, closed-loop feedback data flow structure. From a connectivity perspective, the semantic parsing and initial generation module serves as the system's entry point, connected to the source code storage area of ​​the design under test (DUT) via a file reading interface, and simultaneously establishing a bidirectional communication link with an external large language model service via a network socket or HTTP protocol interface. The output of the semantic parsing and initial generation module is logically connected to the input of the simulation execution and data acquisition module, transmitting the generated initial SystemVerilog code stream to the simulation environment. The simulation execution and data acquisition module integrates an EDA toolchain calling interface, capable of starting and controlling the underlying compiler and simulator kernel. The data output of the simulation execution and data acquisition module is divided into two paths: one path writes the compilation log to a temporary storage area via a file system interface; when the semantic parsing and initial generation module detects a compilation failure, it reads the compilation log and extracts error information to trigger the large language model for syntax correction; the other path connects to the coverage database storage area via a file system interface, used for real-time writing of the coverage database file generated by the simulation.

[0044] The coverage blind spot analysis module is structurally a key node in the feedback loop. Its inputs are connected to both the coverage database storage area and the source code storage area, establishing a bidirectional index mapping between coverage data and source code. This module uses an Abstract Syntax Tree (AST) parser to locate binary missing markers (such as MissedBin) in the coverage database to specific line numbers and syntax blocks in the source code. Its output is connected to the input of the targeted enhancement generation module, transmitting semantically transformed natural language description data. The targeted enhancement generation module also includes an energy allocation unit, used to track the coverage status of each function point and dynamically select the target function point to be covered in the next round. The code output of the large language model ultimately loops back to the code merging interface of the simulation execution and data acquisition module. Through this interconnected approach, the four modules together form a closed-loop feedback data path of "code generation - simulation verification - data analysis - targeted enhancement," ensuring that the verification process can continuously evolve based on real-time data feedback.

[0045] It should be noted that the steps in the method provided by the present invention can be implemented using the corresponding components in the system. Those skilled in the art can refer to the technical solution of the system to implement the steps of the method, and can also refer to the technical solution of the method to implement the composition of the system. That is, the embodiments in the system and the embodiments in the method can be understood as preferred examples of each other, which will not be elaborated here.

[0046] The technical solutions provided by the above embodiments of the present invention will be further described in detail below with reference to specific application examples.

[0047] Application Example 1: In this specific application example, the implementation effect of the above-mentioned technical solution of the present invention is illustrated by taking the verification process of the deep state machine logic in a complex communication protocol control unit as an example.

[0048] This specific application example verifies the deep state machine logic in the design under test (DUT), which is a complex communication protocol control unit containing a finite state machine with ten states. The state transition logic of this state machine has complex timing dependencies. For example, transitioning from the "IDLE" state to the "PROCESS" state requires first receiving a "Handshake" signal, then waiting for three clock cycles, and finally receiving a "Valid" signal. In traditional random testing, because the input stimuli are randomly generated, it is difficult to coincidentally satisfy the above strict timing combination, causing simulations to often be limited to shallow initial states and unable to cover the deep core processing logic.

[0049] Using the technical solution provided in the above embodiments of the present invention, the deep state machine logic of the communication protocol control unit is verified as follows: Obtain the RTL source code of the communication protocol control unit and extract the design information under test; use the initial generation instructions to drive the large language model to generate the initial SystemVerilog test code; The test code is compiled, and error logs are automatically extracted and fed back to the large language model for self-correction until compilation is successful. Simulation is performed based on the compiled code to generate a coverage database file containing line coverage, branch coverage, and state machine coverage. Basic testing revealed that the state machine failed to transition to the "processing" state. At this point, coverage database files were analyzed using coverage blind spot analysis, and semantic understanding of the source code was combined to identify the conditional code for the state transition. Subsequently, targeted enhancement prompts were constructed, and black-box testing constraints were applied to guide the large language model in causal reasoning. The model no longer attempted to illegally modify internal state variables, but instead generated a driving task that conformed to the physical timing sequence: after pulling a handshake signal high on an external port, it precisely drove the clock signal to toggle three times, followed by pulling a data valid signal high. By injecting this targeted task into the simulation, the system successfully triggered the state transition. After several rounds of similar iterations, all state branch paths of the state machine were completely covered, achieving complete verification of deeply complex logic.

[0050] Application Example 2: In this specific application example, the implementation effect of the above-mentioned technical solution of the present invention is illustrated by taking the fully automated cold start verification process for legacy code or third-party IP cores that lack design documents as an example.

[0051] This specific application example demonstrates a fully automated cold-start verification of legacy code or third-party IP cores lacking design documentation. In engineering practice, verification engineers often encounter situations where they receive modules (such as a complex SPI controller) that only have RTL source code but lack a supporting test platform or documentation. The following uses the technical solution provided in the above embodiments of the present invention to perform a fully automated cold-start verification of legacy code or third-party IP cores lacking design documentation. Engineers do not need to manually read the code or manually set up the environment; they only need to input the RTL file path into the system to start the verification process.

[0052] First, the top-level port definition is automatically parsed to generate a standard test platform architecture including clock and reset logic. After establishing basic coverage in the first round of simulation, uncovered functionalities such as interrupt triggering logic and different operating mode configurations are automatically identified. Subsequently, it enters an unattended self-evolution mode, automatically trying all four operating mode configuration combinations of SPI through multiple iterations and generating corresponding data transmission test vectors. The entire process requires no manual intervention, evolving from scratch into a complete verification environment and waveform report with high coverage (e.g., above 95%), significantly reducing the manpower cost and technical threshold for regression testing of legacy assets.

[0053] In specific hardware development scenarios, the embodiments of the present invention can also be applied to the prototyping and development stages of complex FPGAs (Field-Programmable Gate Arrays) and ASICs (Application-Specific Integrated Circuits). They are particularly suitable for verifying digital IP cores (Intellectual Property Cores) that include complex finite state machines (FSMs), multi-stage pipelined processing logic, and proprietary bus protocols. With the exponential growth of logic resources in modern FPGA chips (such as Xilinx Versal and Intel Agilex series), traditional manual test stimulus writing can no longer meet time-to-market requirements. This invention can automatically generate a high-coverage SystemVerilog test platform, helping design teams quickly identify deep logic defects before tape-out or board-based debugging.

[0054] The FPGA verification method and system based on large language models and coverage feedback provided in the above embodiments of the present invention have the following significant advantages compared with the prior art: 1. It breaks through the convergence bottleneck of traditional random testing.

[0055] In terms of functionality and effectiveness, this invention solves the problem of diminishing marginal returns that occurs in the later stages of traditional restricted randomized verification. By accurately locating verification blind spots and generating targeted stimuli through semantic analysis, this invention can efficiently fill the last 1% to 5% coverage gap, significantly reducing ineffective simulation time.

[0056] In terms of structural principles: This effect is achieved based on a closed-loop feedback topology constructed by the system. The output of the coverage blind spot analysis module is not the end point of the system, but is connected to the input of the directional enhancement generation module via a data link. The output of the directional enhancement generation module is then looped back to the simulation execution and data acquisition module. This interconnected feedback loop design breaks the limitation of the unidirectional "open-loop" execution of traditional verification tools, forming a control loop with self-awareness and error correction capabilities. The system uses this physical link to convert the simulation results (coverage data) of the previous round into control signals (directional prompts) generated in the next round in real time, thus structurally ensuring that each iteration is a precise repair of existing defects.

[0057] 2. It fundamentally eliminates the "illusion" and non-synthesizable problem of generating hardware code from large language models, ensuring the continuity of the automated process.

[0058] In terms of functionality and effectiveness: This invention addresses the pain point that existing AI-assisted tools often cause compilation failures due to illegal access to internal signals (i.e., "white-box operations"). The generated SystemVerilog code strictly adheres to hardware module encapsulation principles and timing logic specifications, eliminating simulation interruptions caused by hierarchical reference errors or non-physical timing assignments, and ensuring a high compilation pass rate for the generated test platform in mainstream EDA simulators.

[0059] In terms of structural principles: This effect is achieved based on a three-layer protective black-box constraint filtering architecture deployed in the system. For example... Figure 2 As shown, the architecture consists of the following three layers: The first layer is the Prompt constraint layer. A rule-based validation interface is set up at the front end of the semantic parsing and initial generation modules, as well as the targeted enhancement generation module. Before sending a request to the API, this interface dynamically loads a pre-defined negative constraint protocol, injecting a whitelist of the DUT's legal operation signals (InputPorts, OutputPorts) into the prompt words. It also explicitly prohibits non-synthesized statements such as `force`, `assign`, `deassign`, and `release`, as well as key internal signal patterns such as the FSM state registers (`state`, `next_state`) and internal counters. The second layer is the intelligent conversion layer. For implicit hierarchical references that may arise from large language models (such as the `DUT.state` form), the system automatically performs hierarchical access detection after the API returns the result, converting illegal internal signal operations into equivalent implementations driven by the top-level input ports. The third layer is the quality assessment layer. Before the final output, the code is scanned for illegal patterns. If the proportion of internal signal accesses exceeds a threshold, a quality score is returned and a regeneration process is triggered.

[0060] The three-layer collaborative process described above, encompassing prompt word construction, return result conversion, and output quality control, jointly ensures the legality and composability of the generated code.

[0061] In terms of structural principles: This effect is achieved based on a "black-box constraint filtering" logic unit embedded in the instruction construction layer. For example... Figure 2 As shown, rule verification interfaces are set up at the front end of the semantic parsing and initial generation module and the targeted enhancement generation module. Before sending a request to the API, this interface dynamically loads a preset negative constraint protocol, injecting the DUT's legitimate operation signals into the prompt words in the form of a whitelist, explicitly prohibiting the model from operating on internal signals. After the API returns the result, the generated code is verified a second time through an intelligent code conversion layer, converting illegal internal signal accesses into top-level port operations that conform to the encapsulation principle. The above two layers of protection work together to effectively ensure the legitimacy and synthesizability of the generated code.

[0062] 3. It achieves "cross-modal fusion" of coverage statistics and source code semantics, which significantly improves the logical accuracy of automatically generated incentives.

[0063] In terms of functionality and effectiveness: This invention surpasses the low-dimensional information level of traditional tools that can only report "where is not covered," and can provide the generative model with high-dimensional semantic information such as "what logical functions have not been tested." This semantic-level feedback enables large language models to deeply understand the code intent, thereby generating effective stimuli containing complex temporal dependencies (such as handshake protocols and state transitions), rather than blind random waveforms.

[0064] Structural Principle: This effect is achieved based on a "Y"-shaped dual-channel data aggregation structure used in the coverage blind spot analysis module. This module has dual input interfaces: the first input interface connects to the coverage database (binary statistical data) generated by the simulation execution and data acquisition module, and the second input interface connects to the RTL source code (textual semantic data) obtained by the semantic parsing module. As the system's data fusion node, this module utilizes a line number indexing mechanism to physically align and fuse abstract statistical features with specific code logic context. This structure enables the system to extract high-dimensional feature vectors containing logical meaning, providing accurate decision-making basis for subsequent targeted enhancement generation.

[0065] 4. It achieves full-process automation with "zero human intervention", which greatly reduces the technical threshold and workload of verification engineers.

[0066] In terms of functionality and effectiveness: This invention encapsulates the tedious steps of environment setup, test case writing, result analysis, and code correction into an automated pipeline, achieving "one-click" verification. Verification engineers do not need to be proficient in complex UVM methodologies or manually write targeted test cases; they only need to provide RTL design files to obtain a high-quality verification environment, greatly improving R&D efficiency.

[0067] In terms of structural principles: This effect is achieved based on the modular pipeline cascade architecture adopted in this invention. For example... Figure 3 As shown, the processing steps from the first to the seventh are tightly coupled through standardized data interfaces (such as JSON data streams and file I / O streams), forming an automated workflow. Specifically, the simulation execution and data acquisition module integrates an exception capture and automatic retry subunit. This unit handles compilation errors through local logic judgment circuits or automatically retries exceptions such as connection timeouts, read timeouts, and network timeouts due to server busy, through timeout configuration, achieving automatic recovery from abnormal states. This highly integrated system architecture eliminates the need for manual intervention, ensuring that the verification task can operate unattended 24 / 7, like an industrial assembly line.

[0068] An embodiment of the present invention also provides a computer device for FPGA verification, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can be used to perform any of the methods described in the above embodiments of the present invention.

[0069] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs and functional modules that implement the above methods), computer instructions, etc., and the aforementioned computer programs and computer instructions can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0070] A processor is used to execute computer programs stored in memory to implement the various steps of the methods or various modules of the systems involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method and system embodiments.

[0071] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0072] This embodiment also provides a computer-readable storage medium for storing a computer program that, when executed by a processor, can be used to perform the method of any of the above embodiments of the present invention.

[0073] Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of computer programs from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a user device. Of course, the processor and storage medium can also exist as discrete components in a communication device.

[0074] Any matters not covered in the above embodiments of the present invention are well-known in the art.

[0075] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. An FPGA verification method based on large language models and coverage feedback, characterized in that, include: Obtain the RTL source code of the design under test and extract the design information; Based on the design information to be tested, and combined with the preset prompt word template and black-box testing constraint protocol, an initial generation instruction is constructed; The initial generation instructions are used to drive the large language model to generate initial test code; The test code is compiled, and error logs are automatically extracted and fed back to the large language model for self-correction until compilation succeeds. Simulation is performed based on the compiled code to generate a coverage database file; The coverage database file is parsed to obtain verification blind spots; Based on the aforementioned verification blind spots, targeted enhanced prompts for specific blind spots are constructed, and combined with the black-box constraint mechanism, targeted test tasks for uncovered logic are generated through a large language model. The targeted test tasks are dynamically inserted into the test code, and then recompiled and simulated until the preset target coverage or iteration limit is reached. Finally, the best test code is output as a complete verification environment.

2. The FPGA verification method based on large language model and coverage feedback according to claim 1, characterized in that, The process of obtaining the RTL source code of the design under test and extracting the design under test information includes: Using a parser, the port list and parameter definitions of the top-level module are parsed out through abstract syntax tree technology to obtain the design information under test.

3. The FPGA verification method based on large language model and coverage feedback according to claim 1, characterized in that, Based on the design information under test, and combined with a preset prompt word template and black-box testing constraint protocol, the initial generation instructions are constructed, including: Based on the design information to be tested, and in conjunction with the preset prompt word template, an initial prompt word containing the design information to be tested is constructed; A mandatory black-box testing constraint protocol is implemented; wherein, the black-box testing constraint protocol explicitly requires that the large language model can only control the design through the input port of the top-level module, and strictly prohibits the use of hierarchical references to access the internal signals of the module or the use of forced assignment statements; By combining the initial prompt words and the black-box testing constraint protocol, an initial generation instruction is constructed.

4. The FPGA verification method based on large language model and coverage feedback according to claim 1, characterized in that, The process of using the initial generation instructions to drive the large language model to generate initial test code includes: The initial generation instruction is sent to the large language model to drive the large language model to generate an initial test platform and obtain initial test code; wherein, the initial test code includes clock drive, reset logic and basic random stimulus.

5. The FPGA verification method based on large language model and coverage feedback according to claim 1, characterized in that, The process of compiling the test code and automatically extracting error logs to feed back to the large language model for correction until compilation succeeds includes: The system automatically calls EDA tools to compile the generated code; when a compilation syntax error occurs, it automatically captures the error log output by the compiler and feeds it back to the large language model for self-correction until compilation succeeds.

6. The FPGA verification method based on large language model and coverage feedback according to claim 1, characterized in that, The step of performing coverage parsing on the database file to obtain verification blind spots includes: Read the database file, which is a commented Verilog code file, in which coverage markers are used to identify the number of times each line is executed; The missing coverage data is obtained by identifying lines of code with a count of zero, conditional branch markers, and uncovered branch markers in the coverage markers using a pattern matching algorithm. By employing a mapping method that associates missing coverage data with source code context, the missing coverage data is associated with its corresponding source code lines. Combined with pre-established semantic analysis results, the missing coverage data is transformed into a natural language description that can be understood by a large language model, thereby clarifying the verification blind spots.

7. The FPGA verification method based on large language model and coverage feedback according to claim 1, characterized in that, The generation of targeted test tasks for uncovered logic includes: Based on the context information of the uncovered logic, a targeted enhancement prompt word is constructed, and the black-box test constraint protocol is loaded as the generation constraint condition; The targeted enhancement prompt is sent to the large language model, which generates test task code for the uncovered logic. The test task code triggers the execution of the uncovered logic by applying a specific timing sequence to the input port of the FPGA device within a preset simulation cycle.

8. An FPGA verification system based on a large language model and coverage feedback, characterized in that, include: The semantic parsing and initial generation module is used to obtain the RTL source code of the design under test, extract the design under test information, and use the initial generation instructions to drive the large language model to generate initial test code. The simulation execution and data acquisition module compiles the test code and automatically extracts error logs to feed back to the large language model for self-correction until compilation is successful; it then performs simulation based on the compiled code and generates a coverage database file. A coverage blind spot analysis module is used to parse the coverage database file to obtain verification blind spots; The targeted enhancement generation module constructs targeted enhancement prompts for specific blind spots based on the aforementioned verification blind spots. It then combines black-box constraint mechanisms to generate targeted test tasks for uncovered logic through a large language model. The targeted test tasks are dynamically inserted into the test code and recompiled and simulated until the preset target coverage or iteration limit is reached. Finally, the optimal test code is output as the complete verification environment.

9. The FPGA verification system based on large language model and coverage feedback according to claim 8, characterized in that, The input of the semantic parsing and initial generation module is connected to the source code storage area of ​​the design under test through a file reading interface, and simultaneously establishes a bidirectional communication link with an external large language model service through a network socket or HTTP protocol interface. The output of the semantic parsing and initial generation module is logically connected to the input of the simulation execution and data acquisition module, transmitting the generated initial code stream to the simulation environment. The data output of the simulation execution and data acquisition module is divided into two paths. One path writes the compilation log to a temporary storage area through a file system interface. When the semantic parsing and initial generation module detects a compilation failure, it reads the compilation log and extracts the error information to trigger the large language model to perform syntax correction. The other path connects to the coverage database storage area through a file system interface for real-time writing of the coverage database file generated by the simulation. The input of the coverage blind spot analysis module is connected to the coverage database storage area and the source code storage area respectively, establishing a bidirectional index mapping relationship between coverage data and source code; the output of the coverage blind spot analysis module is connected to the input of the targeted enhancement generation module, transmitting natural language description data after semantic conversion; the targeted enhancement generation module also includes an energy allocation unit, used to track the coverage status of each functional point and dynamically select the target functional point to be covered in the next round; the code output of the large language model is finally looped back to the code merging interface of the simulation execution and data acquisition module, forming a closed-loop feedback data path.

10. A computer device for FPGA verification, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it can be used to perform the method of any one of claims 1-8.

Images