A cpu instruction random generator

Abstract

The present application relates to a kind of cpu instruction random generator, running flow: first instruction package generation and sampling module is used to generate multiple random instruction packages, random instruction package is executed cross-compiled tool chain, is compiled into hexadecimal file, coverage test collection module will call CPU simulation verification environment, hexadecimal file is loaded and run simulation, export the coverage file of the instruction package corresponding to each hexadecimal file;While instruction package reversible fingerprint calculation module will each random instruction package encode, generate fingerprint information;Then instruction package fingerprint code sequence and the coverage file corresponding to its instruction package are sent into instruction package and its coverage self-training module, to dig the influence of instruction package on coverage and reorganize instruction package according to result and obtain coverage as the feedback of reinforcement learning.The present application can reduce the input quantity required for verification;While using python automation management process, no human intervention is required to stimulate selection.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit CPU processor design verification technology, and in particular to the application method of artificial intelligence technology in random simulation verification of CPU instruction sets, referring to a CPU instruction random generator. Background Technology

[0002] Hardware simulation verification, performed after each stage of the Electronic Design Automation (EDA) process to ensure functional correctness, is often the most time-consuming part, accounting for over 70% of the development cycle. Traditional simulation verification methods involve inputting a series of randomized test cases to compare the performance of the device under test (DUT) against a reference model. This often requires a large number of input test cases to achieve acceptable high coverage. Another approach relies on domain experts to generate test cases reflecting specific constraints. However, this method is typically more expensive and cannot scale to very complex designs.

[0003] The use of learning-based methods in optimization problems has become a growing trend in systems engineering. Specifically, the use of reinforcement learning (RL) enables many optimization problems. Digital validation, particularly optimizing the set of input signals provided to a DUT, can be framed as an RL optimization problem. This is because there is a potential similarity between the RL environment and the FSM (Fixed Streaming Method) DUT. The hardware implementation of an FSM consists of only two types of components: combinatorial component elements and memory elements. The contents of the memory elements completely define the state of the DUT. This inherent stateful property is precisely what MDP can model. Furthermore, the optimization objective of high coverage is implicit and best expressed as the reward signal of the RL agent. Therefore, the DUT becomes the RL environment, and the RL agent, instead of artificial intelligence, guides the validation process towards higher coverage through the selection of input stimuli. Summary of the Invention

[0004] Therefore, the technical problem to be solved by this invention is to overcome the problems of slow time consumption and low efficiency of instruction randomization in existing random instruction generators for improving design coverage. In this way, a reinforcement learning-assisted simulation verification method is provided, especially for CPU random instruction verification method. By combining random instruction generator and reinforcement learning, high-quality test instructions are selected through reinforcement learning, thereby improving the efficiency of CPU verification environment coverage. It can achieve the verification goal, namely coverage index, more efficiently.

[0005] To address the aforementioned technical problems, this invention provides a CPU instruction random generator, wherein the generator is designed based on reversible encoding, combined with recurrent neural network algorithms and reinforcement learning algorithms, and includes:

[0006] The instruction package generation and sampling module extracts a fixed number of instructions from a massive pool of existing random instructions, adds an instruction execution environment header and instruction execution exception handling logic tail, executes a cross-compilation toolchain, and generates a massive number of hexadecimal HEX files. This module includes a random instruction generation and sampling module. The random instruction generation and sampling module constructs instructions in segments based on a proprietary CPU instruction architecture template. The instruction template includes opcode, Ra register, Rb register, Rc register, function, disp, and imm fields, and adds basic constraints. Different combinations of these fields form different instructions.

[0007] The instruction packet reversible fingerprint calculation module is used to encode a fixed-length random instruction packet to generate fingerprint information for subsequent backpropagation (BP) training. The instruction packet reversible fingerprint calculation module includes a function function, which contains two parts of functional logic. The first part extracts the opcode field from each instruction in the instruction packet, calculates the probability distribution of various opcode fields, and inputs them into the Huffman encoder to obtain the fingerprint encoding that represents the instruction packet, which serves as a sample for supervised training.

[0008] The coverage test collection module calls the CPU simulation verification environment, automatically packages random instructions into the package one by one, runs the simulation one by one, and outputs the coverage file for each instruction package.

[0009] The instruction packet and its coverage self-training module uses a BP network to explore the impact of test instruction packet sequences on coverage. The fingerprint encoding sequence of the instruction packet is used as a sample, and the coverage is used as a label to carry out model training. The predicted instruction sequence is reverse-engineered from Huffman encoding to obtain the instruction type operation distribution. The instruction packet is reorganized and fed into a simulation verification environment to obtain the real coverage value as reinforcement learning feedback.

[0010] The random generator also includes the following steps:

[0011] Step S1: First, the instruction package generation and sampling module generates multiple random instruction packages. The random instruction packages are then compiled into hexadecimal files using the cross-compilation toolchain. The coverage test collection module calls the CPU simulation verification environment, loads the hexadecimal files, runs the simulation, and outputs the coverage file of the instruction package corresponding to each hexadecimal file.

[0012] Step S2: Simultaneously, the instruction packet reversible fingerprint calculation module encodes each random instruction packet to generate fingerprint information;

[0013] Step S3: Then, the instruction packet fingerprint encoding sequence and the corresponding coverage file of the instruction packet are sent to the instruction packet and its coverage self-training module to mine the influence of the instruction packet on the coverage and reorganize the instruction packet according to the results to obtain the coverage as feedback for reinforcement learning.

[0014] In one embodiment of the present invention, in order to prevent large-scale random testing from being interrupted by abnormal execution of random instructions, a variety of exception and interrupt entry point processing assemblers are set up; wherein, the instruction package generation and sampling module, according to the user-configurable instruction package length, pseudo-randomly samples different types of instructions and automatically adds assembler context and exception handling logic to form an instruction package.

[0015] In one embodiment of the present invention, the instruction packet reversible fingerprint calculation module first calculates the number of occurrences of each type of Opcode in the extracted instruction packet, adds the two with the smallest occurrences and uses them as the left and right subtrees, and repeats this process until the probability value is 1; then, the left side of each binary tree is designated as 0 and the right side as 1; along the path from the top of the binary tree to each character, the encoding of each Opcode is obtained; finally, the Opcode encodings are merged together to form the Opcode distribution feature code of the instruction packet.

[0016] In one embodiment of the present invention, the coverage test collection module is responsible for compiling and simulating an instruction package in a simulation verification environment, and for the final coverage collection; the UVM phase mechanism is used to implement the simulation of the instruction package; "gen_top.sv" is the top-level file, which calls the UVM "run_test" function to start the simulation; "test_base.sv" inherits from UVM_TEST and is used to define the basic execution order; "Random_all_test.sv" inherits from "test_base.sv" and defines the execution order of specific instances. Different sequences are instantiated according to requirements to implement different functional scenarios. Different "random_all_test.sv" correspond to different "random_all_seq.sv" in the "sequence" module.

[0017] The environment is implemented using Makefiles and Python scripts, including simulation execution scripts, scripts for building functional scenarios, and text processing scripts. After simulation, the output results are processed into a format suitable for the DUT. The script files include "Makefile", "vcs.mk", "xrun.mk", "dsim.mk", "umsim.mk", "make.sh", and "bins.sh". It can support different simulation tools, select different simulation scripts according to different simulators, and collect statistical files of the coverage results of the corresponding tool running the random instruction package.

[0018] In one embodiment of the present invention, the instruction packet and its coverage training module utilize a BP network to reflect the potential relationship between the coverage magnitude brought by the simulated instruction packet sequence and the next instruction packet sequence; a dual BP self-training process is used, which predicts the Q1 value [coverage] after the execution of the sequence by combining the historical execution instruction sequence with multiple sets of randomly selected instruction sequences; then the instruction sequence with the maximum expected coverage value plus the historical execution instruction sequence (minus the predicted instruction sequence length) is used as the historical instruction sequence in the new sequence, and a new set of random instruction sequences is added as the input of the second BP network to predict Q2, which represents the predicted coverage after executing the best predicted sequence and then performing the next step of execution space exploration;

[0019] Then, add the reward value (the difference between the coverage obtained by speculating the optimal instruction sequence in actual operation and Q1 / Q2) to Q2 and Q1, and use it as training feedback to achieve reinforcement learning self-training and improve the trajectory of finding the optimal instruction sequence.

[0020] The technical solution of the present invention has the following advantages compared with the prior art:

[0021] Timeliness: Automatically generates legal instruction sequences that conform to design specifications, and can complete the generation of a large number of random test vectors in a relatively short time;

[0022] Efficiency: Compared to large-scale pseudo-random verification, it can reduce the number of inputs required for verification to achieve rare combinations of internal states;

[0023] Intelligent and convenient: Python is used to automate the management process without human intervention in incentive selection, and reinforcement learning enables self-iteration;

[0024] High reusability: By making minor adjustments to the reinforcement learning input, it can be applied to different validation tasks. Attached Figure Description

[0025] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0026] Figure 1 This is a schematic diagram of the CPU instruction random generator environment structure provided by the present invention, which is based on reversible encoding and combines recurrent neural network algorithm and reinforcement learning algorithm.

[0027] Figure 2 This is a schematic diagram illustrating the specific application of the CPU instruction random generator environment described in this invention, which is based on reversible encoding and combines recurrent neural network algorithms and reinforcement learning algorithms. Detailed Implementation

[0028] This embodiment provides a CPU instruction random generator, wherein the generator is designed based on reversible encoding, combined with recurrent neural network algorithms and reinforcement learning algorithms, including:

[0029] The instruction package generation and sampling module extracts a fixed number of instructions from a large number of existing random instructions, adds an instruction execution environment header and instruction execution exception handling logic tail, executes the cross-compilation toolchain, and generates a large number of hexadecimal HEX files.

[0030] The instruction packet reversible fingerprint calculation module is used to encode a fixed-length random instruction packet to generate fingerprint information for subsequent backpropagation (BP) training. The instruction packet reversible fingerprint calculation module includes a function function, which contains two parts of functional logic. The first part extracts the opcode field from each instruction in the instruction packet, calculates the probability distribution of various opcode fields, and inputs them into the Huffman encoder to obtain a fingerprint encoding that can characterize the instruction packet, which serves as a sample for supervised training.

[0031] The coverage test collection module calls the CPU simulation verification environment, automatically packages random instructions into the package one by one, runs the simulation one by one, and outputs the coverage file for each instruction package.

[0032] The instruction packet and its coverage self-training module uses a BP network to explore the impact of test instruction packet sequences on coverage. Instruction packet fingerprint encoding sequences are used as samples, and coverage is used as a label for model training. The predicted instruction sequences are then reverse-engineered from Huffman coding to obtain the instruction type operation distribution. The instruction packets are reorganized and fed into a simulated verification environment to obtain the true coverage value, which serves as feedback for reinforcement learning.

[0033] The verification framework's software architecture consists of three interacting layers, such as... Figure 1 .

[0034] The instruction packet generation and sampling module, and the coverage test collection module constitute the hardware layer. This layer includes hardware designs written in HDLs like Verilog and a UVM environment (a test platform written in SystemVerilog). These two parts interact with each other, leading to the verification logic. Traditional hardware verification environments written in SystemVerilog previously only contained these two parts.

[0035] Building upon this foundation, an RL layer and a cocoon layer are added. The cocoon layer contains a module for reversible fingerprint calculation of instruction packets, an instruction packet and its coverage self-training module, and the RL layer completes the RL feedback loop by interacting with the instruction packet fingerprint encoding and the coverage results generated by the hardware layer. As mentioned earlier, one of the goals of this framework is to achieve modularity between components.

[0036] The signal-switching RL layer and UVM layer run in separate processes, aided by Python's native pipe objects from its multiprocessing module. Through this pipe, the RL agent sends actions for the next step, according to the policies it follows. The cocoon layer receives this action through the pipe and process, reassembles the instruction packets to generate an accurate sequence of input signals, which must be driven to the DUT as part of this step. The DUT is then simulated using this input sequence, and elements of interest within the DUT, such as registers and buffers, are monitored using UVM co-routines. Once the DUT simulation for this step is complete, the information needed for Markov state and reward calculations is piped back to the RL layer. The RL agent receives this information and uses it for training.

[0037] This invention also provides user-adjustable parameters whose values ​​will be input to the DUT excitation. Each value is a real number located within a user-specified interval and can be used collectively to specify the input signal sequence for that time step to be sent to the DUT. The user can select these operating parameters for the required operating space and specify how to generate the DUT's input sequence from these parameter values. Finite discrete set values ​​can also be included in the operating space. By default, this invention assumes a single-step, single Markov state environment as recommended in [the original text].

[0038] During the verification process, a crucial design choice is the selection of a reward signal. We define the reward as a function that is part of the DUT reporting, tracking functional events of interest. Let (+1) represent the first functional event tracked. For example, it could be a register being reset. The reward at each time step is calculated based on a user-specified multiplier, tracking functional events as part of the coverage.

[0039]

[0040] Here, represents the number of events monitored in the step, represents the multiplier specified by the user, and represents the contribution to the reward. Therefore, if 3 events are tracked as part of the coverage, and 0 = 1 = 0, 2 = 1, the agent prefers the operation that results in higher coverage based on the reward function in the equation, specifically the coverage of event 2. This will increase the credibility of the verification logic associated with event 2, potentially leading to a greater chance of hitting corner cases that only occur a small fraction of the time.

[0041] In a single-step RL setup, the agent's goal is to increase the functional coverage of certain events to generate interest. These events can be chosen as those that occur less frequently in the DUT under normal randomized validation runs. However, if 2 corresponds to a coverage black hole that rarely occurs during exploration, it means that the learning reward is too sparse to occur within a reasonable timeframe. Addressing this challenge requires multi-step MDP and RL algorithms for sparse reward environments.

[0042] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A CPU instruction random generator, wherein the generator is designed based on reversible encoding, combined with recurrent neural network algorithms and reinforcement learning algorithms, characterized in that, include: The instruction package generation and sampling module is used to extract a fixed number of instructions from a massive amount of existing random instructions, add an instruction execution environment header and instruction execution exception handling logic tail, execute the cross-compilation toolchain, and generate a massive amount of hexadecimal HEX files. The instruction package generation and sampling module includes a random instruction generation and sampling module. The random instruction generation and sampling module constructs instructions in segments according to the proprietary CPU instruction architecture template. The instruction template includes opcode, Ra register, Rb register, Rc register, function, disp, and imm fields, and adds basic constraints. Different combinations of fields are concatenated to form different instructions. The instruction packet reversible fingerprint calculation module is used to encode a fixed-length random instruction packet to generate fingerprint information for subsequent backpropagation (BP) training. The instruction packet reversible fingerprint calculation module includes a function function, which contains two parts of functional logic. The first part extracts the opcode field from each instruction in the instruction packet, calculates the probability distribution of various opcode fields, and inputs them into the Huffman encoder to obtain the fingerprint encoding that represents the instruction packet, which serves as a sample for supervised training. The coverage test collection module calls the CPU simulation verification environment, automatically packages random instructions into the package one by one, runs the simulation one by one, and outputs the coverage file for each instruction package. The instruction packet and its coverage self-training module uses a BP network to explore the impact of test instruction packet sequences on coverage. The fingerprint encoding sequence of the instruction packet is used as a sample, and the coverage is used as a label to carry out model training. The predicted instruction sequence is reverse-engineered from Huffman encoding to obtain the instruction type operation distribution. The instruction packet is reorganized and fed into a simulation verification environment to obtain the real coverage value as reinforcement learning feedback. The random generator also includes the following steps: Step S1: First, the instruction package generation and sampling module generates multiple random instruction packages. The random instruction packages are then compiled into hexadecimal files using the cross-compilation toolchain. The coverage test collection module calls the CPU simulation verification environment, loads the hexadecimal files, runs the simulation, and outputs the coverage file of the instruction package corresponding to each hexadecimal file. Step S2: Simultaneously, the instruction packet reversible fingerprint calculation module encodes each random instruction packet to generate fingerprint information; Step S3: Then, the instruction packet fingerprint encoding sequence and the corresponding coverage file of the instruction packet are sent to the instruction packet and its coverage self-training module to mine the influence of the instruction packet on the coverage and reorganize the instruction packet according to the results to obtain the coverage as feedback for reinforcement learning.

2. The CPU instruction random generator according to claim 1, characterized in that, Meanwhile, to prevent large-scale random testing from being interrupted by abnormal execution of random instructions, multiple exception and interrupt entry point assemblers are set up. Among them, the instruction package generation and sampling module, based on the user-configurable instruction package length, pseudo-randomly selects different types of instructions and automatically adds the assembler context and exception handling logic to form the instruction package.

3. The CPU instruction random generator according to claim 1, characterized in that, The instruction packet reversible fingerprint calculation module first calculates the number of times each type of Opcode appears in the extracted instruction packet, adds the two with the smallest number of occurrences and uses them as the left and right subtrees, and repeats this process until the probability value is 1; then, the left side of each binary tree is assigned 0 and the right side is assigned 1. By traversing the path from the top of the binary tree to each character, we obtain the encoding of each Opcode; Finally, the Opcode encodings are combined to form the Opcode distribution feature code of the instruction packet.

4. The CPU instruction random generator according to claim 1, characterized in that, The coverage test collection module is responsible for compiling and simulating an instruction package in a simulation verification environment, and finally collecting the coverage. The UVMphase mechanism is used to simulate and run instruction packages; "gen_top.sv" is the top-level file, which calls the UVM "run_test" function to start the simulation; "test_base.sv" inherits from UVM_TEST and is used to define the basic execution order; "Random_all_test.sv" inherits from "test_base.sv" and defines the execution order of specific instances. Different sequences are instantiated according to requirements to implement different functional scenarios. Different "random_all_test.sv" correspond to different "random_all_seq.sv" in the "sequence" module.

5. The CPU instruction random generator according to claim 1, characterized in that, The instruction package and its coverage training module utilize a BP network to reflect the potential relationship between the coverage magnitude brought by the simulated instruction package sequence and the next instruction package sequence. A dual BP self-training process is used, which predicts the Q1 value [coverage] after the execution of the sequence by combining the historical executed instruction sequence with multiple randomly selected instruction sequences. Then, the instruction sequence with the maximum expected coverage value plus the historical executed instruction sequence is used as the historical instruction sequence in the new sequence. A new set of random instruction sequences is added as the input of the second BP network to predict Q2, which represents the predicted coverage after executing the best predicted sequence and then performing the next step of execution space exploration. Then, Q2 and Q1 are combined with the reward value and used as feedback for training, thereby achieving reinforcement learning self-training and improving the trajectory of finding the best instruction sequence.

Images