Method and device for designing and testing electronic apparatus using artificial intelligence

Abstract

To provide a method, a device, a system, and an article for utilizing an artificial intelligence (AI)-based model to update a product design on the basis of an execution result of a generated verification code.SOLUTION: A device including a programmable circuit instantiates: generating, using a first trained artificial intelligence (AI)-based model, on the basis of an input design, a verification code; performing the verification code to generate a verifiability score for the input design; and adjusting, using a second trained AI-based model, on the basis of the verifiability score, the input design.SELECTED DRAWING: Figure 7

Description

[Background technology]

[0001] When an electronic device (e.g., a system, chip, integrated circuit, and / or software) is designed for one or more specific applications, the design is tested to verify that the electronic device operates as intended. To verify the design, software (e.g., verification code) is developed to simulate, predict, and / or detect errors in the design. When executed, the verification code can identify whether the design works as intended and / or whether the design exhibits errors. If the output of the verification code identifies the design as valid and / or verified, the design is verified. If the output of the verification code identifies the design as invalid and / or unverifiable, the design may be adjusted or discarded, thereby attempting to improve the design (e.g., before the corresponding product is sold and / or released for sale). [Brief explanation of the drawings]

[0002] [Figure 1] FIG. 1 is a block diagram of an example computing device for training and / or utilizing artificial intelligence-based models that generate verification codes and / or increase the verifiability of electronic devices.

[0003] [Figure 2] FIG. 2 is a block diagram of an example implementation of the model training circuit of FIG. 1.

[0004] [Figure 3] FIG. 2 is a block diagram of an exemplary implementation of the design and verification circuitry of FIG. 1.

[0005] [Figure 4] FIG. 4 is a block diagram of an example implementation of the multi-mode model of FIG.

[0006] [Figure 5] 3 is a flowchart representative of example machine-readable instructions and / or operations that may be embodied, instantiated, and / or executed by a programmable circuit to implement the model training circuit of FIG. 2.

[0007] [Figure 6] 3 is a flowchart representative of example machine-readable instructions and / or operations that may be embodied, instantiated, and / or executed by a programmable circuit to implement the model training circuit of FIG. 2.

[0008] [Figure 7] 4 is a flowchart representative of example machine-readable instructions and / or operations that may be embodied, instantiated, and / or executed by a programmable circuit to implement the design generation and verification circuit of FIG. 3.

[0009] [Figure 8] 5 is a flowchart representative of example machine-readable instructions and / or operations that may be embodied, instantiated, and / or executed by a programmable circuit to implement the multi-mode model circuit of FIG. 4.

[0010] [Figure 9] FIG. 10 is a block diagram of an exemplary processor platform including programmable circuitry configured to implement, instantiate, and / or execute the computer-readable instructions of FIGS. 5-8 and / or perform exemplary operations to implement at least one of the model training circuitry, multi-mode model circuitry, and / or design generation and verification circuitry of FIGS. 1-4.

[0011] [Figure 10] FIG. 10 is a block diagram of an example implementation of the programmable circuitry of FIG.

[0012] [Figure 11]FIG. 10 is a block diagram of another exemplary implementation of the programmable circuitry of FIG. 9.

[0013] [Figure 12] 10 is a block diagram of an example software / firmware / instruction distribution platform (e.g., one or more servers) for distributing software, instructions, and / or firmware (e.g., corresponding to the example machine-readable instructions of FIG. 9 ) to client devices associated with end users and / or consumers (e.g., for licensing, sale, and / or use), retailers (e.g., for sale, resale, license, and / or sublicense), and / or original equipment manufacturers (OEMs) (e.g., for inclusion in products distributed to retailers and / or other end users, such as direct purchase customers).

[0014] Generally, the same reference numbers are used throughout the drawings and the accompanying specification to refer to the same or like parts. The drawings are not necessarily to scale. DETAILED DESCRIPTION OF THE INVENTION

[0015] When creating and / or developing new systems (e.g., hardware systems, firmware systems, integrated circuits, field programmable gate arrays (FPGAs), application-specific integrated circuits, chips, software, etc.), proper operation of the electronic device and / or electronic system must be verified before manufacturing. The verification phase of development involves generating verification code (e.g., also referred to as verification scripts) to check the functionality of and / or verify the electronic device and / or electronic system design (e.g., checking whether the electronic device and / or electronic system operates as intended, identifying potential errors or bugs, etc.). Execution of the verification code may identify errors and / or verify functionality in the electronic device and / or electronic system design. In many instances, the time required to generate the verification code is significantly longer than the time required to develop the design. The success and yield of products corresponding to the design are highly dependent on the success of the design verification phase.

[0016] Traditionally, generating verification code during the verification phase has been largely a manual process. Not only is the manual process of generating verification code time-consuming, but if a problem is identified during the verifiable phase (e.g., the design has an error or does not function as intended), the process returns to the design phase to adjust or redesign the product to eliminate or reduce the identified error. After the design is adjusted, the verification phase is restarted based on the updated design. Accordingly, the time required to verify the design is significant due to the back-and-forth process between the design and verification phases. Furthermore, the success of the verification process depends on the skill and creativity of the engineers who designed the verification code. For example, if the engineers or personnel who designed the verification code do not consider certain worst-case scenarios, the verification code may not be able to properly identify whether the design will function properly when implemented.

[0017] Examples disclosed herein utilize generative artificial intelligence (AI) to generate verification code for validating a product design. As used herein, a product design refers to the design of any electronic system, product, device, hardware, firmware, integrated circuit, field programmable gate array (FPGA), application-specific integrated circuit, chip, software, etc. Generative AI-based models may include code generation modes, code auto-completion models, code copilot models, etc. that generate verification code for validating a product design. Because product designs can be generated in different formats (e.g., a graphical representation of the system, a textual description of the system, a flowchart representation of the system, a schematic diagram of the system, etc.), examples disclosed herein train a generative AI model to generate verification code and / or scripts based on inputs corresponding to one or more different formats. Furthermore, examples disclosed herein may train the generative AI model using prompts obtained from a user to generate more robust and accurate verification code. The prompt is an example that can help the model develop verification code. For example, if the product design corresponds to a 32-bit adder circuit, the prompt can be an example of a 1-bit adder circuit with example inputs and example outputs.

[0018] Additionally, examples disclosed herein utilize an AI-based model to update a product design based on the results of executing the generated verification code. For example, after a first AI-based model generates verification code for a product design, examples disclosed herein execute the verification code to generate a verification score. A better (e.g., higher) score indicates fewer errors have been identified in the product design. If the verification score meets (e.g., exceeds) a threshold, the product design passes the verification phase. However, if the verification score does not meet (e.g., falls below) the threshold, a second AI-based model in some examples adjusts the product design to increase the verifiability (e.g., reduce errors) of the product design. The second AI-based model generates an adjusted product design based on the product design, the verification code, and / or the verifiability score. The verification code and / or updated verification code are then executed to generate an updated verifiability score for the adjusted product design, and the process continues until the design's verifiability score meets the threshold. Examples disclosed herein result in more efficient, faster, and more robust verification and design phases when generating a product design.

[0019] FIG. 1 is a block diagram of an exemplary computing device 100 in which an AI-based model is trained and implemented to generate validation code and update a product design. The exemplary computing device 100 of FIG. 1 includes an exemplary model training circuit 102, an exemplary model storage 104, an exemplary product design generation and validation circuit 106, and an exemplary user interface 108. The computing device 100 may be a server, a computer, a mobile device, a tablet, and / or any other device capable of training and / or implementing an AI-based model. While the computing device 100 of FIG. 1 trains and implements the AI-based model, there may be two or more computing devices. For example, a first computing device may include the model training circuit 102, the model storage 104, and the user interface 108 and train the AI-based model, and a second computing device may include the product design generation and validation circuit 106 and the user interface 108 and implement the deployed AI-based model trained by the first computing device.

[0020] The model training circuit 102 of FIG. 2 trains and / or fine-tunes an AI-based model (e.g., a generative AI-based model such as a large language model (LLM)) using any combination of unsupervised learning, supervised learning, or semi-supervised learning. The model training circuit 102 includes one or more databases having training data used to train the pre-trained AI-based model. As described further below in connection with FIG. 2, the model training circuit 102 can train and / or fine-tune a verification code generation model that generates verification code for the product design based on an input product design and / or input verification prompts. The model training circuit 102 fine-tunes the pre-trained verification code generation model to generate verification code based on various input modalities (e.g., a graphical representation of the system, a textual description of the system, a flowchart representation of the system, a schematic diagram of the system, etc.) or combinations of input modalities. When the verification code is implemented, the product design is inspected for errors, and a verifiability score for the design is generated based on the inspection.

[0021] 2, the model training circuit 102 of FIG. 1 can train a circuit design generation model that generates and / or adjusts product designs to increase the verifiability score of designs whose verifiability score does not meet (e.g., is below) a threshold. The model training circuit 102 stores the trained model (e.g., the verification code generation model and / or the circuit design generation model) in the model storage 104. For example, if the trained model corresponds to multiple weight values ​​and / or thresholds for neurons of the trained AI-based model, the model training circuit 102 can store the multiple weight values ​​and / or thresholds in the model storage 104.

[0022] After model training is complete, the product design generation and verification circuitry 106 accesses the trained model data from the model storage 104 and implements the trained model into a product design for the verifiability stage. For example, the product design generation and verification circuitry 106 retrieves the trained verifiability code generation model data from the model storage 104 and applies the product design and / or verification prompts to the trained verifiability code generation model to generate verification code for the product design. The product design generation and verification circuitry 106 executes the verification code to verify the product design and outputs a verifiability score corresponding to whether the design operates as intended. If the verification code meets (e.g., exceeds) a threshold, the product design generation and verification circuitry 106 outputs the product design as a final design and proceeds to the next manufacturing stage. If the verification code does not meet (e.g., falls below) the threshold, the product design generation and verification circuitry 106 applies the product design to the trained circuit design generation circuitry to adjust and / or redesign the product design to increase the verifiability score. The product design generation and verification circuitry 106 is further described below in relation to FIG.

[0023] 8 interfaces with a user of computing device 100. For example, user interface 108 may obtain a user-generated product design from the user. Additionally, user interface 108 may provide information to the user regarding the training and / or implementation of one or more models to obtain feedback. User interface 108 may provide user feedback regarding the training and / or implementation of the models.

[0024] Figure 2 is a block diagram of an example implementation of the model training circuit 102 of Figure 1. The example model training circuit 102 of Figure 2 includes an example pre-trained model database 200, an example multiple language code database 202, an example language-specific code database 204, an example product design database 206, an example prompt database 208, and an example weight adjustment circuit 210. Although the example implementation of the model training circuit 102 of Figure 2 includes multiple separate databases, the model training circuit 102 may be implemented with fewer databases by combining data stored in multiple databases into one or more databases.

[0025] The pre-trained model database 200 of FIG. 2 stores one or more pre-trained models (e.g., data that can be used to implement one or more pre-trained models). For example, the pre-trained model database 200 may store a first pre-trained verification code model to generate verification code. The pre-trained model may be a large language model (LLM). Additionally, the pre-trained model database 200 may store a second pre-trained circuit design model to generate and / or adjust a product design. The pre-trained model serves as a starting point model. However, to make the pre-trained model more accurate, robust, and effective, the weight adjustment circuit 210 uses information from the other databases 202-208 to fine-tune the pre-trained model, as further described below.

[0026] The multiple language code database 202 of Figure 2 includes alignment data corresponding to multiple different code languages. In some examples, the alignment data is verification code generalized for multiple different languages ​​(e.g., Verilog, C+, circuit descriptions, schematics, flowcharts, product designs, multimodal descriptions, etc.). The verification code may be linked or labeled to a specific product design. In some examples, the alignment data includes product designs generalized for multiple different languages ​​or outputs (e.g., Verilog, C+, circuit descriptions, schematics, flowcharts, product designs, multimodal descriptions, etc.). The product designs may be linked or labeled to verification code, verifiability scores, and / or original product designs.

[0027] The language-specific code database 204 in Figure 2 includes training data corresponding to a particular language. When used for fine-tuning, the training data can convert a generic model fine-tuned in multiple languages ​​into a model corresponding to a specific language or one or more specific modalities. The training data can link a specific language to a generalized validation code or product design. As further described below, in addition to fine-tuning a model pre-trained in a specific language, overfitting of input data can be reduced by first fine-tuning the model to a generic model and then fine-tuning the generalized modality to a language-specific one.

[0028] 2 stores product designs and / or product design adjustments to improve verifiability. The product design database can be used as input for fine-tuning a verification code generation model or as input and / or output for fine-tuning a circuit design generation model. Product designs can be linked or labeled with corresponding verification code, corresponding verifiability scores, and / or corresponding updated product designs.

[0029] The example prompt database 208 of FIG. 2 stores prompts used to fine-tune the verification code generation model. The prompts can be examples that help the model develop the verification code. For example, if the product design corresponds to a 32-bit adder circuit, the prompt can be an example of a 1-bit adder circuit with example inputs and example outputs. The prompts can be linked or labeled to the corresponding product design and / or the corresponding verification code.

[0030] The weight adjustment circuit 210 in FIG. 2 adjusts the model weights to fine-tune (e.g., also referred to as tuning) the model into a final trained model. For example, the weight adjustment circuit 210 can fine-tune a pre-trained model using training data from the multiple language code databases 202 and / or corresponding product design data from the product design database 206 to generate a generalized validation code model or a generalized product design generation model. In some examples, the weight adjustment circuit 210 can further fine-tune the pre-trained validation code model based on prompts from the prompt database 208. After the generic model is generated, the weight adjustment circuit 210 further fine-tunes the generic model using language-specific (e.g., one of Verilog, C++, etc.) tuning data in the language-specific code database 204. The specific language can be based on user and / or manufacturer preferences. In contrast to directly training a pre-trained model using language-specific code, training a pre-trained model into a generic model and then turning the generic model into a language-specific model can reduce overfitting to specific input types.

[0031] When tuning the generalized verification code generation model to a language-specific code generation model, the weight adjustment circuit 210 of FIG. 2 can fine-tune the language-specific verification code model to an instruction-based verification code model, a non-instruction-based model, or a task-specific verification code model based on user and / or manufacturer preferences. An instruction-based design corresponds to the trained model's input being instructions (e.g., code, circuit descriptions, etc.). A non-instruction-based design corresponds to the trained model's input being non-instruction (e.g., circuit diagrams, flowcharts, etc.). When training to an instruction-based verification model, the weight adjustment circuit 210 uses language-specific tuning data that is instruction-based. When training to a non-instruction-based verification model, the weight adjustment circuit 210 uses language-specific tuning data that is non-instruction-based. In some examples, the weight adjustment circuit 210 can further fine-tune the non-instruction-based verification model to a long text-based verification model by fine-tuning the task-based verification model for long text implementations (e.g., when product designs perform various tasks or functions).

[0032] When tuning the pre-trained product design generative model into a generalized product design generative model, the weight adjustment circuit 210 of FIG. 2 can obtain user feedback at length and use the user feedback to fine-tune the product design generative model. For example, the user interface 108 may provide the user with a portion of the tuning data used to inspect the product design based on the generalized product design generative model. The user can provide feedback regarding the output of the generalized system-generated model via the user interface 108. The weight adjustment circuit 210 can use the user feedback to further fine-tune the generalized product design generative model.

[0033] 2 performs fine-tuning by adjusting the weights, thresholds, and / or any other data of the model. The weight adjustment circuit 210 can perform fine-tuning using any fine-tuning technique. For example, the weight adjustment circuit 210 can perform full fine-tuning, parameter-efficient fine-tuning, transfer learning, task-specific fine-tuning, multi-task learning, sequential fine-tuning, etc.

[0034] Figure 3 is a block diagram of an example implementation of the product design generation and validation circuit 106 of Figure 1. The product design generation and validation circuit 106 of Figure 3 includes an example product design 302, an example verification code generation circuit 304, an example multimodal encoder circuit 305, an example proposal verification prompt 306, an example verification code 308, an example design validation circuit 310, an example final product design 312, an example circuit design generation circuit 314, and an example updated product design 316.

[0035] The product design 302 of FIG. 2 is a design for a new system (e.g., an integrated circuit, ASIC, etc.) for performing one or more operations. The product design 302 may be created by a user (e.g., obtained via the user interface 108 of FIG. 1) and / or may be created by another system (e.g., an AI-based model). The product design 302 may be described in one or more different manners. For example, the product design 302 may be one or more of a graph-based description, a text-based description, a circuit diagram, etc. The product design 302 describes how the design is intended to operate after it is manufactured. The product design 302 is an input to the verification code generation circuit 304.

[0036] The verification code generation circuit 304 of FIG. 3 implements the verification code generation model trained by the model training circuit 102 of FIGS. 1-2. For example, the verification code generation circuit 304 can retrieve AI-based model data (e.g., weights, thresholds, structure, etc., of the trained verification code generation circuit) from the model storage 104 to implement the trained verification code generation model. The verification code generation circuit 304 retrieves the product design 302 and the verification prompt 306 and generates an output verification code 308 based on the product design 302 and / or the verification prompt 306. If the product design 302 is described in two or more different modalities (e.g., a textual description and a circuit description), the multimodal encoder circuit 405 combines information from the two or more different modalities into a single representation of the input product design, as described further below in connection with FIG. 4. As described above, the verification prompt 306 can be an example that the model uses to develop the verification code. For example, the verification prompt 306 can be example input and output values. The output values ​​may be expected output values ​​of the product design when the example input values ​​are applied to the product design. Additionally, the verification code 308 is code or script that, when implemented, verifies the functionality of the product design 302. For example, the verification code may examine the product design with various values ​​and / or configurations to ensure that the product design 302 operates as intended. The verification code 308 of FIG. 3 outputs a verifiability score for the product design 302. A higher verifiability score corresponds to finding a smaller number of errors in the product design 302. A lower verifiability score corresponds to finding a larger number of errors in the product design 302. The verification code generation circuit 304 outputs the verification code 308 to the design verification circuit 310.

[0037] The design verification circuitry 310 of FIG. 3 executes the verification code 308 for the product design 302 to generate a verifiability score for the product design 302. As described above, the fewer errors that occur during execution of the verification code 308, the higher the verifiability score for the product design 302. Furthermore, the design verification circuitry 310 compares the output verifiability score to a threshold verifiability score. If the design verification circuitry 310 determines that the output verifiability score meets (e.g., exceeds) the threshold verifiability score, the design verification circuitry 310 outputs the product design as a final product design 312. The final product design 312 can move on to the next manufacturing stage. If the design verification circuitry 310 determines that the output verifiability score does not meet (e.g., is below) the threshold verifiability score, the design verification circuitry 310 outputs the verification score to the circuit design generation circuitry 314 to trigger the circuit design generation circuitry 314 to update or rewrite the product design 302.

[0038] The circuit design generation circuit 314 of FIG. 3 implements the circuit design generation model trained by the model training circuit 102 of FIGS. 1-2. For example, the circuit design generation circuit 314 can retrieve AI-based model data (e.g., weights, thresholds, structure, etc. of the trained verification code generation circuit) from the model storage 104 to implement the trained verification code generation model. The circuit design generation circuit 314 receives as input the product design 302, the verification code 308, and / or the results of the design verification circuit 310. The circuit design generation circuit 314 uses the inputs in the trained circuit design generation model to attempt to improve the verifiability of the product design and generate an output of an updated product design 316 that maintains the initial functionality of the original product design 302. The circuit design generation circuit 314 outputs the updated product design 316 to the verification code generation circuit 304 to generate new verifiability code for testing the updated product design.

[0039] Figure 4 is a block diagram of an example implementation of the multimodal encoder circuit 305 of Figure 3. The example multimodal encoder circuit 305 of Figure 4 includes an example text encoder circuit 400, an example visual encoder circuit 402, and an example cross-modal encoder circuit 404.

[0040] The text encoder circuit 400 of Figure 4 performs text encoding on a text-based product design. For example, the text encoder circuit 300 can extract features from input text data and convert the input data into a common representation or format understood by a model. In some examples, the text encoder circuit 300 can transform the text into word embeddings, in which similar words are represented as numerical vectors.

[0041] The visual encoder circuit 402 of Figure 4 performs visual encoding on a vision-based product design. For example, the visual encoder circuit 402 can extract features from an input visual design and convert the input data into a common representation. In some examples, the visual encoder circuit 402 can use a convolutional neural network to encode the image.

[0042] 4 performs cross-modal encoding (also referred to as multimodal fusion) to generate a representation between text and a corresponding visual representation. The cross-modal encoder circuit 404 combines information from different modalities into a single representation. For example, the cross-modal encoder circuit 404 may perform a weighted sum of the modal feature outputs of the text encoder circuit 300 and the visual encoder circuit 402.

[0043] 1 are illustrated in Figures 2-3, one or more of the elements, processes, and / or devices illustrated in Figure 2 may be combined, divided, rearranged, omitted, eliminated, and / or implemented in any other manner. Furthermore, the weight adjustment circuit 210, the verification code generation circuit 304, the multimodal encoder circuit 305, the design verification circuit 310, the circuit design generation circuit 314, the text encoder circuit 400, the vision encoder circuit 402, the cross-modal encoder circuit 404, and / or, more generally, the model training circuit 102 and the product design generation and validation circuit 106 of Figures 2-3 may be implemented by hardware alone or by hardware in combination with software and / or firmware. Thus, for example, any of the weight adjustment circuit 210, verification code generation circuit 304, multimodal encoder circuit 305, design verification circuit 310, circuit design generation circuit 314, text encoder circuit 400, visual encoder circuit 402, cross-modal encoder circuit 404, and / or more generally, the model training circuit 102, and the product design generation and verification circuit 106 of FIGS. 2-3 may be implemented by machine-readable instructions (e.g., firmware or software), processor circuitry, analog circuitry, digital circuitry, logic circuitry, programmable processors, programmable microcontrollers, graphics processing units (GPUs), digital signal processors (DSPs), ASICs, programmable logic devices (PLDs), and / or programmable circuitry in combination with field programmable logic devices (FPLDs) such as FPGAs. Furthermore, the model training circuitry 102 and product design generation and validation circuitry 106 of FIGS. 2-3 may include one or more elements, processes, and / or devices in addition to or instead of those illustrated in FIGS. 1-4, and / or may include more than one of any one or all of the illustrated elements, processes, and devices.

[0044] Flowcharts representing example machine-readable instructions that may be executed by a programmable circuit for implementing and / or instantiating the model training circuit 102 and the product design generation and validation circuit 106 of FIGS. 2-3 and / or representing example operations that may be performed by a programmable circuit for implementing and / or instantiating the model training circuit 102 and the product design generation and validation circuit 106 of FIGS. 2-3 are shown in FIGS. 5-8. The machine-readable instructions may be one or more executable programs, or portions of one or more executable programs, for execution by a programmable circuit, such as the programmable circuit 912 shown in the example processor platform 900 described below in connection with FIG. 9, and / or may be one or more functions or portions of functions performed by the example programmable circuit (e.g., FPGA) described below in connection with FIGS. 10 and / or 11. In some examples, the machine-readable instructions cause operations, tasks, etc. to be performed and / or executed in an automated manner in the real world. As used herein, "automated" means without human involvement.

[0045] The program may be embodied in instructions (e.g., software and / or firmware) stored on one or more non-transitory computer-readable and / or machine-readable storage media, such as cache memory, magnetic storage devices or disks (e.g., floppy disks, hard disk drives (HDDs), etc.), optical storage devices or disks (e.g., Blu-ray disks, compact disks (CDs), digital versatile disks (DVDs), etc.), redundant arrays of independent disks (RAIDs), registers, ROM, solid-state drives (SSDs), SSD memory, non-volatile memory (e.g., electrically erasable programmable read-only memory (EEPROM), flash memory, etc.), volatile memory (e.g., any type of random access memory (RAM), etc.), and / or any other storage device or disk. The instructions of the non-transitory computer-readable and / or machine-readable storage medium can program and / or be executed by programmable circuitry located in one or more hardware devices, although all and / or portions of the program can alternatively be performed and / or instantiated by one or more hardware devices other than programmable circuitry and / or embodied in dedicated hardware. The machine-readable instructions may be distributed across multiple hardware devices and / or executed by two or more hardware devices (e.g., a server and a client hardware device). For example, a client hardware device may be implemented by an endpoint client hardware device (e.g., a hardware device associated with a human and / or machine user) or an intermediate client hardware device gateway (e.g., a radio access network (RAN) that can facilitate communication between a server and an endpoint client hardware device). Similarly, a non-transitory computer-readable storage medium may include one or more media.Furthermore, although the exemplary programs are described with reference to the flowcharts illustrated in FIGS. 5-8 , many other ways of implementing the model training circuit 102 and product design generation and validation circuit 106 of FIGS. 2-3 may alternatively be used. For example, the order of execution of the flowchart blocks may be changed, and / or some of the described blocks may be modified, eliminated, or combined. Additionally or alternatively, any or all of the flowchart blocks may be implemented by one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, FPGAs, ASICs, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured to perform corresponding operations without executing software or firmware. The programmable circuitry may be distributed across different network locations and / or local to one or more hardware devices (e.g., single-core processors (e.g., single-core CPUs), multi-core processors (e.g., multi-core CPUs, XPUs, etc.)). For example, the programmable circuitry may be a CPU and / or FPGA located in the same package (e.g., the same integrated circuit (IC) package or two or more separate housings), one or more processors in a single machine, multiple processors distributed across multiple servers in a server rack, multiple processors distributed across one or more server racks, and / or any combination thereof.

[0046] The machine-readable instructions described herein may be stored in one or more of a compressed format, an encrypted format, a fragmented format, a compiled format, an executable format, a packaged format, etc. The machine-readable instructions described herein may be stored as data (e.g., computer-readable data, machine-readable data, one or more bits (e.g., one or more computer-readable bits, one or more machine-readable bits, etc.), a bitstream (e.g., a computer-readable bitstream, a machine-readable bitstream, etc.)) or a data structure (e.g., a portion of instructions, code, a representation of code, etc.) that can be utilized to create, manufacture, and / or generate machine-executable instructions. For example, the machine-readable instructions may be fragmented and stored in one or more storage devices, disks, and / or computing devices (e.g., servers) located in the same location or in different locations of a network or collection of networks (e.g., cloud, edge devices, etc.). The machine-readable instructions may require one or more of installing, modifying, adapting, updating, combining, supplementing, configuring, decrypting, decompressing, unpacking, distributing, reassigning, compiling, etc. to make them directly readable, interpretable, and / or executable by computing devices and / or other machines. For example, the machine-readable instructions may be individually compressed, encrypted, and / or stored in multiple portions that are stored on separate computing devices and / or stored on another computing device, where the portions, when decrypted, decompressed, and / or combined, form a set of computer-executable and / or machine-executable instructions that implement one or more functions and / or operations that may together form a program as described herein.

[0047] In another example, machine-readable instructions may be stored in a state that may be read by a programmable circuit, but require the addition of a library (e.g., a dynamic link library (DLL)), a software development kit (SDK), an application programming interface (API), etc., to execute the machine-readable instructions on a particular computing device or other device. In another example, configuration of the machine-readable instructions (e.g., stored settings, data input, recorded network addresses, etc.) may be required before the machine-readable instructions and / or corresponding program can be executed in whole or in part. Thus, as used herein, machine-readable, computer-readable, and / or machine-readable medium may include instructions and / or programs regardless of the particular format or state of the machine-readable instructions and / or programs.

[0048] The machine-readable instructions described herein may be expressed in any past, present, or future command language, scripting language, programming language, etc. For example, the machine-readable instructions may be expressed using any of the following languages: C, C++, Java®, C#, Perl, Python®, JavaScript®, Hypertext Markup Language (HTML), Structured Query Language (SQL), Swift®, etc.

[0049] 3 can be implemented using executable instructions (e.g., computer-readable and / or machine-readable instructions) stored on one or more non-transitory computer-readable and / or machine-readable storage media. As used herein, the terms non-transitory computer-readable medium and non-transitory computer-readable storage medium, non-transitory machine-readable medium, and / or non-transitory machine-readable storage medium are expressly defined to include any type of computer-readable storage device and / or storage disk, exclude propagating signals, and exclude transmission media. Examples of such non-transitory computer-readable medium, non-transitory computer-readable storage medium, non-transitory machine-readable medium, and / or non-transitory machine-readable storage medium on which information is stored for any period of time (e.g., long-term, persistent, short-term instance, temporary buffering, and / or caching of information) include optical storage devices, magnetic storage devices, HDDs, flash memory, read-only memory (ROM), CDs, DVDs, caches, any type of RAM, registers, and / or any other storage device or storage disk. As used herein, the terms "non-transitory computer-readable storage" and "non-transitory machine-readable storage" are defined to include any physical (mechanical, magnetic, and / or electrical) hardware for retaining information for a period of time, but excluding propagating signals and excluding transmission media. Examples of non-transitory computer-readable storage and / or non-transitory machine-readable storage include any type of random access memory, any type of memory only, solid-state memory, flash memory, optical disks, magnetic disks, disk drives, and / or redundant array of independent disks (RAID) systems. As used herein, the term "device" refers to a physical structure, such as mechanical and / or electrical equipment, hardware, and / or circuitry, that may or may not be configured with and / or manufactured to carry out computer-readable instructions, machine-readable instructions, etc.

[0050] The terms "including" and "comprising" (and all their forms and tenses) are used herein as open-ended terms. Thus, whenever a claim uses any form of "include" or "comprise" (e.g., "comprises," "includes," "comprising," "including," "having," etc.) as a preamble or within any type of claim recitation, it is to be understood that additional elements, terms, etc. may be present without departing from the scope of the corresponding claim or recitation. As used herein, the phrase "at least" is open-ended in the same way that the terms "including" and "comprising" are open-ended when used, for example, as a transitional phrase in a claim preamble. When used, the term "and / or" in the form of, for example, A, B, and / or C, refers to any combination or subset of A, B, C, such as (1) A only, (2) B only, (3) C only, (4) A and B, (5) A and C, (6) b and C, or (7) A, B, and C. As used herein, in the text of a structure, component, item, object, and / or Thing Description, the phrase "at least one of A and B" is intended to refer to an implementation that includes any of (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Similarly, as used herein, in the text of a structure, component, item, object, and / or Thing Description, the phrase "at least one of A or B" is intended to refer to an implementation that includes any of (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.As used herein, in the text of a description of the performance or execution of a process, instruction, action, action, and / or operation, the phrase "at least one of A and B" is intended to refer to an implementation that includes any of (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Similarly, as used herein, in the text of a description of the performance or execution of a process, instruction, action, action, and / or operation, the phrase "at least one of A or B" is intended to refer to an implementation that includes any of (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.

[0051] As used herein, singular references (e.g., "a," "an," "first," "second," etc.) do not exclude a plurality. The term "a" or "an" object, as used herein, refers to one or more of that object. The terms "a" (or "an"), "one or more," and "at least one" are used interchangeably herein. Furthermore, although individually listed, multiple means, elements, or actions may be implemented, for example, by the same entity or object. In addition, although individual features may be included in different examples or claims, they may in some cases be combined, and their inclusion in different examples or claims does not imply that a combination of features is not feasible and / or advantageous.

[0052] Descriptors such as "first," "second," and "third" are used herein to identify multiple elements or components that may be referenced separately. Unless otherwise specified or understood based on the context of their usage, such descriptors are not intended to imply any sense of priority or chronological order, but are merely intended as labels for individually referencing multiple elements or components to facilitate understanding of the disclosed embodiments. In some instances, the detailed description may refer to an element using the descriptor "first," while the claims may refer to the same element using a different descriptor, such as "second" or "third." In such cases, it should be understood that such descriptors are used merely to facilitate referring to multiple elements or components.

[0053] As used herein, the phrase "communicate" (including variations thereof) encompasses direct communication and / or indirect communication through one or more intermediary components, and does not require direct physical (e.g., wired) and / or constant communication, and additionally includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and / or one-time events.

[0054] As used herein, "programmable circuitry" is defined to include (i) one or more special-purpose electrical circuits (e.g., application-specific circuits (ASICs)) having a structure to perform a particular operation and including one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors), and / or (ii) one or more general-purpose semiconductor-based electrical circuits that are programmable with instructions to perform a particular function and / or operation and include one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors). Examples of programmable circuitry include programmable microprocessors such as a central processor unit (CPU) that can execute first instructions to perform one or more operations and / or functions, a field programmable gate array (FPGA) that can be programmed with second instructions to cause the configuration and / or structure of the FPGA to instantiate one or more operations and / or functions corresponding to the first instructions, a graphics processor unit (GPU) that can execute first instructions to perform one or more operations and / or functions, a digital signal processor (DSP), XPU, network processing unit (NPU) that can execute first instructions to perform one or more operations and / or functions, one or more microcontrollers that can execute first instructions to perform one or more operations and / or functions, and / or an integrated circuit such as an application specific integrated circuit (ASIC). For example, an XPU may be implemented by a heterogeneous computing system that includes multiple types of programmable circuitry (e.g., one or more FPGAs, one or more CPUs, one or more GPUs, one or more NPUs, one or more DSPs, etc., and / or any combination thereof) and orchestration technology (e.g., an application programming interface (API)) that can assign computational tasks to any of the multiple types of programmable circuitry that are suitable and available to perform the computational task.

[0055] As used herein, an integrated circuit / circuit configuration is defined as one or more semiconductor packages that include one or more circuit elements, such as transistors, capacitors, inductors, resistors, current paths, diodes, etc. For example, an integrated circuit may be implemented as one or more of an ASIC, an FPGA, a chip, a microchip, a programmable circuit, a semiconductor substrate with multiple circuit elements coupled together, a system on a chip (SoC), etc.

[0056] FIG. 5 is a flowchart representative of example machine-readable instructions and / or example operations 500 that may be implemented, instantiated, and / or executed by a programmable circuit to train a verification code generation model. For example, the example operations 500 may be implemented, instantiated, and / or executed by the model training circuit 102 of FIG. 2. The example machine-readable instructions and / or example operations 500 of FIG. 5 begin at block 502, where the weight adjustment circuit 210 accesses pre-trained base models for generating verification codes from the pre-trained model database 200. The pre-trained model database 200 stores pre-trained models that serve as a starting point for generating a fully trained verification code generation model. In some examples, the weight adjustment circuit 210 can train a model from scratch using training data stored in one or more of the databases 200-208 of FIG. 2.

[0057] At block 504, the weight adjustment circuit 210 generates a generalized validation model by fine-tuning the pre-trained base model with the validation code-based training data. For example, the weight adjustment circuit 210 may access multiple language code data from the multiple language code databases 202 and / or prompts corresponding to product designs from the prompt database 208. The weight adjustment circuit 210 uses the accessed data to adjust the weights of the pre-trained model to generate a generalized validation code model.

[0058] In block 506 of FIG. 5 , the weight adjustment circuit 210 determines whether the generalized verification code model is to be tuned for an instruction-based design or a non-instruction-based design. An instruction-based design corresponds to inputs for the trained model that are instructions (e.g., code, circuit descriptions, etc.). A non-instruction-based design corresponds to inputs for the trained model that are non-instructions (e.g., circuit diagrams, flowcharts, etc.). The decision to select an instruction-based model or a non-instruction-based model may be based on user and / or manufacturer preference. If the weight adjustment circuit 210 determines to tune the model for instructions (block 506: Instructions), the weight adjustment circuit 210 generates an instruction-based verification model by fine-tuning the generalized verification model for the instruction-based design using language-specific product design data corresponding to the instruction-based model from the language-specific code database 204 (block 510). The trained instruction-based verification model outputs verification code based on the language-specific instruction-based input data.

[0059] If the weight adjustment circuit 210 determines to tune the model for instructions (block 506: non-instruction), the weight adjustment circuit 210 generates a non-instruction-based verification model by fine-tuning the generalized verification model for the non-instruction-based design using language-specific product design data corresponding to the non-instruction-based model from the language-specific code database 204 (block 512). The trained instruction-based verification model outputs verification code based on the language-specific non-instruction-based input data. At block 514, the weight adjustment circuit 210 generates a long text-based verification model by fine-tuning the task-based verification model for long text. The weight adjustment circuit 210 fine-tunes any model using any fine-tuning technique (e.g., feature extraction, full fine-tuning, etc.). At block 516, the weight adjustment circuit 210 stores the trained model in the exemplary model storage 104. For example, the weight adjustment circuit 210 can store data corresponding to values ​​of the structure of the trained model (e.g., number of neurons, number of layers, etc.) and weights and / or thresholds required to implement the trained model.

[0060] FIG. 6 is a flowchart representative of example machine-readable instructions and / or example operations 600 that may be implemented, instantiated, and / or executed by a programmable circuit to train a circuit design generative model. For example, the example operations 600 may be implemented, instantiated, and / or executed by the model training circuit 102 of FIG. 2. The example machine-readable instructions and / or example operations 600 of FIG. 6 begin at block 602, where the weight adjustment circuit 210 accesses pre-trained base models for generating product designs from the pre-trained model database 200. The pre-trained model database 200 stores pre-trained models that serve as a starting point for generating a fully trained product design generative model. In some examples, the weight adjustment circuit 210 can train a model from scratch using tuning data stored in one or more of the databases 200-208 of FIG. 2.

[0061] At block 604, the weight adjustment circuit 210 generates a generalized circuit design model by fine-tuning the pre-trained foundation model with the system design-based adjustment data. For example, the weight adjustment circuit 210 can access data from the multiple language code database 202 and / or corresponding data from the product design database 206 to fine-tune the pre-trained foundation model. At block 606, the weight adjustment circuit 210 displays a sample output of the generalized circuit design model. For example, the weight adjustment circuit 210 can apply a sample product design as an input to the generalized circuit design model and generate the sample output. The weight adjustment circuit 210 outputs the sample result to the user interface 108 to provide the sample result to the user for feedback and / or input the sample product design. The output can include a prompt to ask the user a question about the sample output.

[0062] At block 608, the weight adjustment circuit 210 obtains feedback from the user via the user interface 108. At block 610, the example weight adjustment circuit 210 further adjusts the generalized circuit design model based on the user feedback. At block 612, the example weight adjustment circuit 210 determines whether to continue adjusting the generalized circuit design model. In some examples, the weight adjustment circuit 210 determines whether to continue adjusting based on the user feedback and / or the accuracy of the generalized circuit design model. For example, the weight adjustment circuit 210 may use a portion of data from the product design database 206 to check the accuracy of the generalized circuit design model. If the weight adjustment circuit 210 determines that adjusting the generalized circuit design model will continue (block 612: YES), control returns to block 606. If the weight adjustment circuit 210 determines that adjusting the generalized circuit design model will not continue (block 612: NO), control continues to block 614.

[0063] At block 614, the weight adjustment circuit 210 generates a language-specific circuit design model by adjusting the generalized circuit design based on the language-specific adjustment data. For example, the weight adjustment circuit 210 further adjusts the generalized circuit design model using data from the language-specific code database 204. At block 616, the example weight adjustment circuit 210 stores the trained model in the example model storage 104. For example, the weight adjustment circuit 210 may store data corresponding to the structure of the trained model (e.g., number of neurons, number of layers, etc.) and the weight and / or threshold values ​​required to implement the trained model.

[0064] 7 is a flowchart representative of example machine-readable instructions and / or example operations 700 that may be implemented, instantiated, and / or performed by a programmable circuit to verify and / or adjust a product design. For example, the example operations 700 may be implemented, instantiated, and / or performed by the product design generation and verification circuit 106 of FIG. 3. The example machine-readable instructions and / or example operations 700 of FIG. 7 begin at block 702, where the verification code generation circuit 304 accesses the product design 302. As described above, the product design 302 may be one or more of a graph-based description, a text-based description, a circuit diagram, etc. In some examples, the verification code generation circuit 304 accesses the product design from the user interface 108, a network interface, and / or storage.

[0065] At block 702, the multimodal encoder circuit 405 determines whether the product design description is multimodal. For example, if the product design 302 includes some graph-based descriptions and some text-based descriptions, the multimodal encoder circuit 405 determines that the product design 302 is multimodal. If the multimodal encoder circuit 405 determines that the product design 302 is not multimodal (block 704: NO), control continues to block 708. If the multimodal encoder circuit 405 determines that the product design 302 is multimodal (block 706: YES), the multimodal encoder circuit 405 performs multimodal encoding on the product design 302 (block 706), as further described below in connection with the flowchart of FIG. 8. As discussed above, cross-modal encoding combines multiple modalities into a single modality.

[0066] At block 708, the verification code generation circuit 304 inputs the product design into the trained verification code model. The verification code generation circuit 304 may access information about the trained model from the example model storage 104 of FIG. 1 and implement the trained model based on the accessed information. At block 710, the example verification code generation circuit 304 determines whether a verification prompt 306 has been obtained. The verification prompt 306 is an example that a model can aid in developing a verification code. The verification code generation circuit 304 may obtain the verification prompt 306 via the user interface 108, a network interface, and / or from storage. If the verification code generation circuit 304 determines that a verification prompt 306 has not been obtained (block 710: NO), control continues to block 714. If the verification code generation circuit 304 determines that the verification prompt 306 has been obtained (block 710: YES), the verification code generation circuit 304 inputs the verification prompt 306 into the trained verification code model (block 712).

[0067] At block 714, the example verification code generation circuitry 304 uses the trained verification code generation model to generate verification code 308 based on the input product design 302 and / or verification prompt 306. As described above, the verification code 308 is code and / or script that, when executed, generates a verifiability score based on the product design 302. The verifiability score corresponds to the number of errors or bugs associated with the product design 302. At block 716, the example design verification circuitry 310 executes the verification code 308 to generate a verifiability score for the product design 302. At block 718, the example design verification circuitry 310 determines whether the verifiability score meets (e.g., exceeds) a threshold value. The threshold value may be based on user and / or manufacturer preferences. If the design verification circuitry 310 determines that the verifiability score does not meet the threshold value (block 718: NO), control continues to block 720, as further described below. If the design verification circuitry 310 determines that the verifiability score meets the threshold (block 718: YES), the design verification circuitry 310 outputs the verified product design as the final product design 312 (block 282) and the instructions end.

[0068] At block 720, the example circuit design generation circuit 314 inputs the verification code, the product design, and / or the verifiability analysis into the trained circuit design model. At block 722, the example circuit design generation circuit 314 generates an updated product design 316 based on the input. At block 724, the design verification circuit 310 executes the verification code for the updated product design 316 to generate an updated verifiability score for the updated product design 316. At block 726, the example design verification circuit 310 determines whether the updated verifiability score meets (e.g., exceeds) a threshold value. If the design verification circuit 310 determines that the updated verifiability score does not meet the threshold value (block 726: NO), control returns to block 704 to perform additional iterations to attempt to increase the verifiability of the product design. If the design verification circuit 310 determines that the updated verifiability score meets the threshold value (block 726: YES), the design verification circuit 310 outputs the final product design.

[0069] Figure 8 is representative of a flowchart of example machine-readable instructions and / or example operations 706 that may be implemented, instantiated, and / or executed by a programmable circuit to perform multi-modal encoding on a product design, as described above in connection with block 706 of Figure 7. For example, the example operations 706 may be implemented, instantiated, and / or executed by the multi-modal encoder circuit 305 of Figure 4. The example machine-readable instructions and / or example operations 706 of Figure 8 begin at block 800, where the text encoder circuit 300 performs text encoding on a text-based product design. For example, the text encoder circuit 300 extracts features from input text data and converts the input data into a common representation (e.g., by transforming the input data into a feature vector).

[0070] At block 802, the visual encoder circuit 402 performs visual encoding on the vision-based product design. For example, the visual encoder circuit 402 extracts features from the input visual design and converts the input data into a common representation. At block 804, the exemplary cross-modal encoder circuit 404 performs cross-modal encoding to generate a representation between the text and the corresponding visual representation. For example, the cross-modal encoder circuit 404 combines information from different modalities into a single representation (e.g., using a weighted sum of modal features).

[0071] 9 is an example block diagram of a programmable circuitry configuration platform 900 configured to execute and / or instantiate the example machine-readable instructions and / or example operations of Figures 5-8 to implement the model training circuit 102 and / or product design generation and validation circuit 106 of Figures 1-4. The programmable circuitry configuration platform 900 may be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., neural network), or any other type of computing and / or electronic device.

[0072] The programmable circuitry platform 900 of the illustrated example includes programmable circuitry 912. The programmable circuitry 912 of the illustrated example is hardware. For example, the programmable circuitry 912 may be implemented by one or more integrated circuits, logic circuits, FPGAs, microprocessors, CPUs, GPUs, DSPs, and / or microcontrollers from any desired family or manufacturer. The programmable circuitry 912 may be implemented by one or more semiconductor-based (e.g., silicon-based) devices. In this example, the weight adjustment circuit 210, verification code generation circuit 304, multimodal encoder circuit 305, design verification circuit 310, circuit design generation circuit 314, text encoder circuit 400, vision encoder circuit 402, and cross-modal encoder circuit 404 of FIGS. 2-4 are included.

[0073] The programmable circuitry 912 of the illustrated example includes local memory 913 (e.g., cache, registers, etc.). The programmable circuitry 912 of the illustrated example communicates with main memory 914, 916, including volatile memory 914 and nonvolatile memory 916, via a bus 918. The volatile memory 914 can be implemented with synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), RAMBUS® dynamic random access memory (RDRAM®), and / or any other type of RAM device. The nonvolatile memory 916 may be implemented with flash memory and / or any other desired type of memory device. Access to the main memory 914, 916 of the illustrated example is controlled by a memory controller 917. In some examples, the memory controller 917 can be implemented with one or more integrated circuits, logic circuits, microcontrollers, or any other type of circuitry from any desired family or manufacturer to manage the flow of data to and from the main memory 914, 916. Any one or more of the main memory 914, 916 or the local memory 913 may implement the model storage 104 and / or the databases 200, 202, 204, 206, 208 of FIG. 1 and / or FIG.

[0074] The programmable circuit configuration platform 900 of the illustrated example also includes interface circuitry 920. The interface circuitry 920 may be implemented by hardware according to any type of interface standard, such as an Ethernet interface, a Universal Serial Bus (USB) interface, a Bluetooth interface, a Near Field Communication (NFC) interface, a Peripheral Component Interconnect (PCI) interface, and / or a Peripheral Component Interconnect Express (PCIe) interface.

[0075] In the depicted example, one or more input devices 922 are connected to the interface circuit 920. The input devices 922 allow a user (e.g., a human user, a machine user, etc.) to input data and / or commands into the programmable circuitry 912. The input devices 922 may be implemented by, for example, a keyboard, buttons, a mouse, and / or a touch screen.

[0076] One or more output devices 924 are also connected to the interface circuitry 920 of the illustrated example. The output device(s) 924 may be implemented, for example, by a display device (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube (CRT) display, an in-place switching (IPS) display, a touch screen, etc.), a tactile output device, a printer, and / or a speaker. The interface circuitry 920 of the illustrated example therefore typically includes a graphics driver card, a graphics driver chip, and / or graphics processor circuitry such as a GPU.

[0077] The interface circuitry 920 of the depicted example also includes communication devices such as transmitters, receivers, transceivers, modems, residential gateways, wireless access points, and / or network interfaces to facilitate data exchange with external equipment (e.g., any type of computing device) over the network 926. Communication may be via, for example, an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, an optical fiber connection, a satellite system, a non-line-of-sight wireless system, a line-of-sight wireless system, a cellular phone system, an optical connection, etc.

[0078] The programmable circuit configuration platform 900 of the illustrated example also includes one or more mass storage disks or devices 928 for storing firmware, software, and / or data. Examples of such mass storage disks or devices 928 include magnetic storage devices (e.g., floppy disks, drives, HDDs, etc.), optical storage devices (e.g., Blu-ray disks, CDs, DVDs, etc.), RAID systems, and / or solid-state storage disks or devices such as flash memory devices and / or SSDs.

[0079] The machine-readable instructions 932, which may be implemented by the machine-readable instructions of Figures 5-8, may be stored in at least one non-transitory computer-readable storage medium, such as mass storage device 928, volatile memory 914, non-volatile memory 916, and / or a removable CD or DVD.

[0080] FIG. 10 is a block diagram of an exemplary implementation of the programmable circuitry 912 of FIG. 9. In this example, the programmable circuitry 912 of FIG. 9 is implemented by a microprocessor 1000. For example, the microprocessor 1000 may be a general-purpose microprocessor (e.g., general-purpose microprocessor circuitry). The microprocessor 1000 executes some or all of the machine-readable instructions of the flowchart of FIG. 3 to effectively instantiate the circuits of FIGS. 1 and / or 2 as logic circuits to perform operations corresponding to those machine-readable instructions. In some such examples, the circuits of FIGS. 10 and / or 11 are instantiated by the hardware circuitry of the microprocessor 1000 in combination with the machine-readable instructions. For example, the microprocessor 1000 may be implemented by multi-core hardware circuitry such as a CPU, DSP, GPU, XPU, etc. While any number of the exemplary cores 1002 (e.g., one core) may be included, the microprocessor 1000 in this example is a multi-core semiconductor device including N cores. The cores 1002 of the microprocessor 1000 may operate independently or may cooperate to execute machine-readable instructions. For example, a firmware program, embedded software program, or machine code corresponding to a software program may be executed by one of the cores 1002 or by multiple cores 1002 at the same or different times. In some examples, the machine code corresponding to the firmware program, embedded software program, or software program is divided into threads and executed in parallel by two or more cores 1002. The software program may correspond to some or all of the machine-readable instructions and / or operations represented by the flowchart of FIG. 3.

[0081] The cores 1002 may communicate via a first exemplary bus 1004. In some examples, the first bus 1004 may be implemented by a communication bus to achieve communications associated with one of the cores 1002. For example, the first bus 1004 may be implemented by at least one of an Inter-Integrated Circuit (I2C) bus, a Serial Peripheral Interface (SPI) bus, a PCI bus, or a PCIe bus. Additionally or alternatively, the first bus 1004 may be implemented by any other type of computing or electrical bus. The cores 1002 may obtain data, instructions, and / or signals from one or more external devices via the exemplary interface circuitry 1006. The cores 1002 may output data, instructions, and / or signals to one or more external devices via the interface circuitry 1006. The cores 1002 in this example include exemplary local memory 1020 (e.g., a level 1 (L1) cache that may be divided into an L1 data cache and an L1 instruction cache), but the microprocessor 1000 also includes exemplary shared memory 1010 (e.g., a level 2 (L2) cache) that may be shared by the cores for fast access to data and / or instructions. However, in some examples, for fast access to data and / or instructions, an L2 cache is connected to each core 1002 and the shared memory 1010 is implemented by a level 3 (L3) cache. Data and / or instructions may be transferred (e.g., shared) by writing to and / or reading from the shared memory 1010. The local memory 1020 and the shared memory 1010 of each core 1002 may be part of a hierarchy of storage devices that includes multiple levels of cache memory and main memory (e.g., main memories 914, 916 of FIG. 9 ). Typically, higher-level memories in the hierarchy exhibit slower access times and smaller storage capacities than lower-level memories. Changes at various levels of the cache hierarchy are governed (eg, coordinated) by a cache coherency policy.

[0082] Each core 1002 may be referred to as a CPU, DSP, GPU, etc., or any other type of hardware circuit. Each core 1002 includes control unit circuitry 1014, arithmetic and logic (AL) circuitry (sometimes referred to as an ALU) 1016, multiple registers 1018, local memory 1020, and a second exemplary bus 1022. Other structures may exist. For example, each core 1002 may include vector unit circuitry, single instruction multiple data (SIMD) unit circuitry, load / store unit (LSU) circuitry, branch / jump unit circuitry, floating point unit (FPU) circuitry, etc. The control unit circuitry 1014 includes semiconductor-based circuitry configured to control (e.g., coordinate) data movement within the corresponding core 1002. The AL circuitry 1016 includes semiconductor-based circuitry configured to perform one or more mathematical and / or logical operations on data within the corresponding core 1002. The AL circuitry 1016 in some examples performs integer-based operations. In another example, the AL circuitry 1016 also performs floating-point operations. In yet another example, the AL circuitry 1016 may include first AL circuitry that performs integer-based operations and second AL circuitry that performs floating-point operations. In some examples, the AL circuitry 1016 may be referred to as an arithmetic logic unit (ALU).

[0083] The registers 1018 are semiconductor-based structures for storing data and / or instructions, such as results of one or more operations performed by the AL circuitry 1016 of the corresponding core 1002. For example, the registers 1018 may include vector registers, SIMD registers, general-purpose registers, flag registers, segment registers, machine-specific registers, instruction pointer registers, control registers, debug registers, memory management registers, machine check registers, etc. The registers 1018 may be arranged in banks as shown in FIG. 10. Alternatively, the registers 1018 may be organized in any other arrangement, format, or structure, such as by being distributed throughout the core 1002 to reduce access time. The second bus 1022 may be implemented by at least one of an I2C bus, an SPI bus, a PCI bus, or a PCIe bus.

[0084] Each core 1002 and / or, more generally, microprocessor 1000 may include additional and / or alternative structures to those shown and described above. For example, one or more clock circuits, one or more power supplies, one or more power gates, one or more cache home agents (CHAs), one or more convergence / common mesh stops (CMSs), one or more shifters (e.g., barrel shifters), and / or other circuits may be present. Microprocessor 1000 is a semiconductor device fabricated to include many transistor interconnects to implement the structures described above in one or more integrated circuits (ICs) contained in one or more packages.

[0085] Microprocessor 1000 may include and / or interface with one or more accelerators (e.g., acceleration circuitry, hardware accelerators, etc.). In some examples, accelerators are implemented by logic circuitry to perform specific tasks more quickly and / or efficiently than can be performed by a general-purpose processor. Examples of accelerators include ASICs and FPGAs, such as those discussed herein. GPUs, DSPs, and / or other programmable devices may also be accelerators. Accelerators may be on-board microprocessor 1000 in the same chip package as microprocessor 1000 and / or in one or more packages separate from microprocessor 1000.

[0086] 11 is a block diagram of another exemplary implementation of programmable circuitry 912 of FIG. 9. In this example, programmable circuitry 912 is implemented by FPGA circuitry 1100. For example, FPGA circuitry 1100 may be implemented by an FPGA. FPGA circuitry 1100 may be used, for example, to perform operations that could otherwise be performed by exemplary microprocessor 1000 of FIG. 10 executing corresponding machine-readable instructions. However, once configured, FPGA circuitry 1100 instantiates the operations and / or functions corresponding to the machine-readable instructions in hardware, thereby often being able to perform the operations / functions faster than can be achieved when executed by a general-purpose microprocessor executing corresponding software.

[0087] More specifically, in contrast to the microprocessor 1000 of FIG. 10 described above (which can be programmed to implement some or all of the machine-readable instructions represented by the flowchart of FIG. 3, but which, once manufactured, is a general-purpose device whose interconnects and logic circuitry are fixed), the example FPGA circuit 1100 of FIG. 11 includes interconnects and logic circuitry that can be configured, structured, programmed, and / or interconnected in different ways after fabrication, e.g., to instantiate some or all of the operations / functions corresponding to the machine-readable instructions represented by the flowchart of FIG. 3. In particular, FPGA circuit 1100 can be thought of as an array of logic gates, interconnects, and switches. The switches can be programmed to change the manner in which the logic gates are interconnected by the interconnects, effectively forming one or more dedicated logic circuits (unless FPGA circuit 1100 is reprogrammed). The configured logic circuits allow the logic gates to cooperate in different ways to perform different operations on data received by input circuits. Those operations may correspond to some or all of the instructions (e.g., software and / or firmware) represented by the flowchart of FIG. 3. In this manner, FPGA circuitry 1100 may be configured and / or constructed to effectively instantiate some or all of the operations / functions corresponding to the machine-readable instructions of the flowchart of Figure 3 as dedicated logic circuitry that performs the operations / functions corresponding to those software instructions in a dedicated manner similar to an ASIC. Thus, FPGA circuitry 1100 may perform the operations / functions corresponding to some or all of the machine-readable instructions of Figure 3 faster than a general-purpose microprocessor could.

[0088] In the example of FIG. 11 , FPGA circuit 1100 is configured and / or constructed in response to being programmed (and / or reprogrammed one or more times) based on a binary file. In some examples, the binary file may be compiled and / or generated based on instructions in a hardware description language (HDL), such as Lucid, Very High-Speed ​​Integrated Circuits (VHSIC) Hardware Description Language (VHDL), or Verilog. For example, a user (e.g., a human user, a machine user, etc.) may write code or a program corresponding to one or more operations / functions in HDL. The code / program may be translated into a low-level language as needed. The code / program (e.g., code / program in a low-level language) may be converted into a binary file (e.g., by a compiler, a software application, etc.). In some examples, FPGA circuit 1100 of FIG. 11 may access and / or load the binary file, such that FPGA circuit 1100 of FIG. 11 is configured and / or constructed to perform one or more operations / functions. For example, the binary file may be implemented by a bitstream (e.g., one or more computer-readable bits, one or more machine-readable bits, etc.), data (e.g., computer-readable data, machine-readable data, etc.), and / or machine-readable instructions that are accessible to the FPGA circuit 1100 of FIG. 11 to produce the configuration and / or structure of the FPGA circuit 1100 of FIG. 11 or portions thereof.

[0089] In some examples, a binary file is compiled, generated, translated, and / or otherwise output from a uniform software platform utilized to program the FPGA. For example, the uniform software platform may translate first instructions (e.g., code or program) corresponding to one or more operations / functions in a high-level language (e.g., C, C++, Python®, etc.) into second instructions corresponding to one or more operations / functions in an HDL. In some such examples, a binary file is compiled, generated, and / or otherwise output from the uniform software platform based on the second instructions. In some examples, the FPGA circuit 1100 of FIG. 11 may access and / or load the binary file, such that the FPGA circuit 1100 of FIG. 11 is configured and / or constructed to perform one or more operations / functions. For example, the binary file may be implemented by a bitstream (e.g., one or more computer-readable bits, one or more machine-readable bits, etc.), data (e.g., computer-readable data, machine-readable data, etc.), and / or machine-readable instructions that are accessible to the FPGA circuit 1100 of FIG. 11 to produce the configuration and / or structure of the FPGA circuit 1100 of FIG. 11 or portions thereof.

[0090] The FPGA circuit 1100 of FIG. 11 includes exemplary input / output (I / O) circuitry 1102 for obtaining and / or outputting data from exemplary configuration circuitry 1104 and / or external hardware 1106. For example, the configuration circuitry 1104 may be implemented by interface circuitry that may obtain a binary file that may be implemented with bitstreams, data, and / or machine-readable instructions to configure the FPGA circuit 1100 or a portion thereof. In some such examples, the configuration circuitry 1104 may obtain the binary file from a user, a machine (e.g., hardware circuitry (e.g., programmable or dedicated circuitry) that may implement an artificial intelligence / machine learning (AI / ML) model to generate the binary file, etc., and / or any combination thereof). In some examples, the external hardware 1106 may be implemented by external hardware circuitry. For example, the external hardware 1106 may be implemented by the microprocessor 1000 of FIG. 10.

[0091] The FPGA circuit 1100 also includes an exemplary logic gate circuit 1108, a plurality of exemplary configurable interconnects 1110, and an array of exemplary storage circuits 1112. The logic gate circuit 1108 and the configurable interconnects 1110 are configurable to instantiate one or more operations / functions that may correspond to at least a portion of the machine-readable instructions of FIG. 3 and / or other desired operations. The logic gate circuitry 1108 shown in FIG. 11 is assembled in blocks or groups. Each block includes semiconductor-based electrical structures that may be configured into a logic circuit. In some examples, the electrical structures include logic gates (e.g., And gates, Or gates, Nor gates, etc.) that provide the basic building blocks of the logic circuit. Electrically controllable switches (e.g., transistors) are present in each of the logic gate circuitry 1108, enabling the configuration of the electrical structures and / or logic gates to form a circuit to perform a desired operation / function. The logic gate circuitry 1108 may also include other electrical structures, such as look-up tables (LUTs), registers (e.g., flip-flops or latches), multiplexers, etc.

[0092] The configurable interconnect 1110 in the illustrated example is a conductive path, trace, via, or the like that may include electrically controllable switches (e.g., transistors) whose state can be changed by programming (e.g., using an HDL instruction language) to activate or deactivate one or more connections between one or more of the logic gate circuitry 1108 to program a desired logic circuit.

[0093] The storage circuits 1112 in the illustrated example are configured to store one or more results of the operations performed by the corresponding logic gates. The storage circuits 1112 may be implemented by registers, etc. In the illustrated example, the storage circuits 1112 are distributed among the logic gate circuits 1108 for ease of access and increased execution speed.

[0094] The example FPGA circuit 1100 of FIG. 11 also includes example dedicated operational circuitry 1114. In this example, the dedicated operational circuitry 1114 includes dedicated circuitry 1116 that may be called upon to implement commonly used functions to avoid the need to program those functions in the field. Examples of such dedicated circuitry 1116 include memory (e.g., DRAM) controller circuitry, PCIe controller circuitry, clock circuitry, transceiver circuitry, memory, and multiplier-accumulator circuitry. Other types of dedicated circuitry may also be present. In some examples, the FPGA circuit 1100 may also include example general-purpose programmable circuitry 1118, such as an example CPU 1120 and / or an example DSP 1122. There may additionally or alternatively be other general-purpose programmable circuitry 1118 that can be programmed to perform other operations, such as a GPU, XPU, etc.

[0095] 10 and 11 illustrate two exemplary implementations of the programmable circuitry 912 of FIG. 9, many other approaches are contemplated. For example, the FPGA circuitry may include an on-board CPU, such as one or more of the exemplary CPUs 1120 of FIG. 10. Thus, the programmable circuitry 912 of FIG. 9 may additionally be implemented by combining at least the exemplary microprocessor 1000 of FIG. 10 and the exemplary FPGA circuitry 1100 of FIG. 11. In some such hybrid examples, the one or more cores 1002 of FIG. 10 may execute a first portion of the machine-readable instructions represented by the flowchart of FIG. 3 to perform a first operation / function, the FPGA circuitry 1100 of FIG. 11 may be configured and / or constructed to perform a second operation / function corresponding to a second portion of the machine-readable instructions represented by the flowchart of FIG. 3, and / or the ASIC may be configured and / or constructed to perform a third operation / function corresponding to a third portion of the machine-readable instructions represented by the flowchart of FIG. 3.

[0096] 10 and / or 11 may be instantiated at the same time or at different times. For example, the same and / or different portions of microprocessor 1000 of FIG. 10 may be programmed to implement portions of the machine-readable instructions at the same time and / or at different times. In some examples, the same and / or different portions of FPGA circuit 1100 of FIG. 11 may be configured and / or constructed to perform operations / functions corresponding to portions of the machine-readable instructions at the same time and / or at different times.

[0097] In some examples, some or all of the circuits of Figures 10 and / or 11 may be instantiated with one or more threads, e.g., executing simultaneously and / or sequentially. For example, microprocessor 1000 of Figure 10 may execute machine-readable instructions with one or more threads executing simultaneously and / or sequentially. In some examples, FPGA circuit 1100 of Figure 11 may be configured and / or constructed to perform operations / functions simultaneously and / or serially. Furthermore, in some examples, some or all of processor circuit 912 of Figures 10 and / or 11 may be implemented within one or more virtual machines and / or virtual execution environments that execute for microprocessor 1000 of Figure 10.

[0098] In some examples, the programmable circuitry 912 of Figure 9 may be in one or more packages. For example, the microprocessor 1000 of Figure 10 and / or the FPGA circuit 1100 of Figure 11 may be in one or more packages. In some examples, an XPU may be implemented by the programmable circuitry 912 of Figure 9, which may be in one or more packages. For example, an XPU may include a CPU (e.g., the microprocessor 1000 of Figure 10, the CPU 1120 of Figure 11, etc.) in one package, a DSP (e.g., the DSP 1122 of Figure 11) in another package, a GPU in yet another package, and an FPGA (e.g., the FPGA circuit 1100 of Figure 11) in yet another package.

[0099] 12, a block diagram illustrating an exemplary software distribution platform 1205 for distributing software, such as the exemplary machine-readable instructions 932 of FIG. 9, to other hardware devices (e.g., hardware devices owned and / or operated by a third party from the owner and / or operator of the software distribution platform) is shown. The exemplary software distribution platform 1205 may be implemented by any computer server, data facility, cloud service, etc. capable of storing and transmitting software to other computing devices. The third party may be a customer of the entity that owns and / or operates the software distribution platform 1205. For example, the entity that owns and / or operates the software distribution platform 1205 may be a developer, seller, and / or licensor of software, such as the exemplary machine-readable instructions 932 of FIG. 9. The third party may be a consumer, user, retailer, OEM, etc. that purchases and / or licenses the software for use and / or resale and / or sublicense. In the illustrated example, the software distribution platform 1205 includes one or more servers and one or more storage devices. The storage device stores machine-readable instructions 932, which may correspond to the example machine-readable instructions of Figure 3, as described above. One or more servers of the example software distribution platform 1205 are in communication with the example network 1210, which may correspond to network 114 of Figure 1. In some examples, the one or more servers respond to requests to transmit software to requesters as part of a commercial transaction. Payment for distribution, sale, and / or licensing of the software may be processed by one or more servers of the software distribution platform and / or a third-party payment entity. The server enables purchasers and / or licensors to download the machine-readable instructions 932 from the software distribution platform 1205.For example, software that may correspond to the example machine-readable instructions of Figure 3 may be downloaded to the example programmable circuit configuration platform 900 that executes the machine-readable instructions 932 to implement the processor circuit 912. In some examples, one or more servers of the software distribution platform 1205 periodically provide, transmit, and / or force updates to the software (e.g., the example machine-readable instructions 932 of Figure 9) to ensure that improvements, patches, updates, etc. are distributed and applied to the software on end-user devices. Although referred to above as software, the distributed "software" may also be firmware.

[0100] Disclosed herein are exemplary methods, apparatus, systems, and articles of manufacture for designing and testing electronic devices using artificial intelligence, further examples and combinations of which include the following: Example 1 includes a non-transitory computer-readable medium comprising instructions that cause at least one programmable circuit to perform the following steps: generating a verification code based on an input design using a first trained artificial intelligence (AI)-based model; executing the verification code to generate a verifiability score for the input design; and adjusting the input design using a second trained AI-based model based on the verifiability score.

[0101] Example 2 includes the non-transitory computer-readable medium of Example 1, wherein the input design is expressed in at least one manner.

[0102] Example 3 includes the non-transitory computer-readable medium of Example 1, wherein the instructions cause one or more of the at least one programmable circuit to perform multi-mode encoding on the input design based on the input design expressed in two or more ways.

[0103] Example 4 includes the non-transitory computer-readable medium of Example 1, wherein the instructions cause one or more of the at least one programmable circuit to input a prompt to the first trained AI-based model to generate the verification code.

[0104] Example 5 includes the non-transitory computer-readable medium of Example 1, wherein the verification code verifies functionality of the input design.

[0105] Example 6 includes the non-transitory computer-readable medium of Example 1, wherein the verifiability score is a first verifiability score and the instructions cause one or more of the at least one programmable circuit to execute the verification code to generate a second verifiability score for the adjusted input design.

[0106] Example 7 includes the non-transitory computer-readable medium of Example 1, wherein the verifiability score is a first verifiability score, and the instructions cause one or more of the at least one programmable circuit to output the adjusted input design as a final design based on a second verifiability score of the adjusted input design satisfying a threshold.

[0107] Example 8 includes the non-transitory computer-readable medium of Example 1, wherein the instructions cause one or more of the at least one programmable circuit to access a pre-trained base model; generate a generalized verification code model by fine-tuning the pre-trained base model, where training data corresponds to a plurality of code languages; and train the first trained AI-based model by fine-tuning the generalized verification code model with language-specific training data.

[0108] Example 9 includes the non-transitory computer-readable medium of Example 1, wherein the instructions cause one or more of the at least one programmable circuit to access a pre-trained base model; generate a generalized verification code model by fine-tuning the pre-trained base model, where training data corresponds to a plurality of code languages, the fine-tuning of the pre-trained base model being based on user feedback; and train the second trained AI-based model by fine-tuning the generalized verification code model with language-specific training data.

[0109] Example 10 includes an apparatus comprising: an interface circuit for acquiring a design; computer-readable instructions; and at least one programmable circuit that is programmable with the computer-readable instructions to cause a first trained artificial intelligence (AI)-based model to generate verification code based on the design; generate a verifiability score for the design based on the verification code; and cause a second trained AI-based model to update the design based on the verifiability score.

[0110] Example 11 includes the device of Example 10, wherein the design is expressed by at least one modality.

[0111] Example 12 includes the apparatus of example 10, wherein one or more of the at least one programmable circuit performs multi-mode encoding on the design when the design is expressed in at least two ways.

[0112] Example 13 includes the apparatus of Example 10, wherein one or more of the at least one programmable circuit inputs a prompt to the first trained AI-based model to generate the verification code.

[0113] Example 14 includes the apparatus of example 10, wherein the verification code finds errors in the design.

[0114] Example 15 includes the apparatus of Example 10, wherein the verifiability score is a first verifiability score and one or more of the at least one programmable circuit executes the verification code to generate a second verifiability score corresponding to the updated design.

[0115] Example 16 includes the apparatus of Example 10, wherein the verifiability score is a first verifiability score, and one or more of the at least one programmable circuit outputs the updated design as a final design if a second verifiability score of the updated design satisfies a threshold.

[0116] Example 17 includes the apparatus of Example 10, wherein one or more of the at least one programmable circuit access a pre-trained base model; generate a generalized verification code model by tuning the pre-trained base model with training data corresponding to a plurality of code languages; and train the first trained AI-based model by tuning the generalized verification code model with language-specific training data to generate the first trained AI-based model.

[0117] Example 18 includes the apparatus of Example 10, wherein one or more of the at least one programmable circuit access a pre-trained base model; generate a generalized verification code model by adjusting the pre-trained base model with training data corresponding to a plurality of code languages, wherein adjusting the pre-trained base model is based on user feedback; and train the second trained AI-based model by generating the second trained AI-based model by adjusting the generalized verification code model with language-specific training data.

[0118] Example 19 includes an apparatus comprising: an interface circuit that obtains a design; machine-readable instructions; and a programmable circuit that at least one of instantiates or implements the machine-readable instructions to cause a first artificial intelligence (AI)-based model to generate verification code based on the design; execute the verification code to generate a verifiability score corresponding to the design; and cause a second AI-based model to iteratively adjust the design until the verifiability score meets a threshold.

[0119] Example 20 includes the device of example 19, wherein the design is expressed in at least one manner.

[0120] From the foregoing, it should be appreciated that exemplary systems, apparatus, articles, and methods of manufacturing are disclosed for designing and testing electronic devices using artificial intelligence. The examples disclosed herein generate a verifiability score to test the design of an electronic device. In response, the examples disclosed herein improve the functionality and / or design of the electronic device by generating code that detects errors in the design. Furthermore, the examples disclosed herein adjust the design if testing of the design identifies errors. Accordingly, the disclosed systems, apparatus, articles, and methods of manufacturing are accordingly directed to one or more improvements in the operation of machines, such as computers or other electronic devices and / or mechanical devices.

[0121] Although specific exemplary methods, apparatus, and articles of manufacture have been disclosed herein, the scope of this patent is not so limited. On the contrary, this patent covers all systems, methods, apparatus, and articles of manufacture fairly falling within the scope of the claims of this patent. [Other possible items] [Item 1] generating verification code based on the input design using the first trained artificial intelligence (AI)-based model; executing the verification code to generate a verifiability score for the input design; and adjusting the input design using a second trained AI-based model based on the verifiability score. a non-transitory computer-readable medium comprising instructions for causing at least one programmable circuit to execute the [Item 2] Item 10. The non-transitory computer-readable medium of item 1, wherein the input design is expressed in at least one manner. [Item 3] 3. The non-transitory computer-readable medium of claim 1, wherein the instructions cause one or more of the at least one programmable circuit to perform multi-mode encoding on the input design based on the input design expressed in two or more ways. [Item 4] 4. The non-transitory computer-readable medium of any one of items 1 to 3, wherein the instructions cause one or more of the at least one programmable circuit to input a prompt to the first trained AI-based model to generate the verification code. [Item 5] 5. The non-transitory computer-readable medium of any one of items 1 to 4, wherein the verification code verifies functionality of the input design. [Item 6] 6. The non-transitory computer-readable medium of any one of items 1 to 5, wherein the verifiability score is a first verifiability score, and the instructions cause one or more of the at least one programmable circuit to execute the verification code to generate a second verifiability score for the adjusted input design. [Item 7] 7. The non-transitory computer-readable medium of any one of items 1 to 6, wherein the verifiability score is a first verifiability score, and the instructions cause one or more of the at least one programmable circuit to output the adjusted input design as a final design based on a second verifiability score of the adjusted input design satisfying a threshold. [Item 8] The instructions may be to cause one or more of the at least one programmable circuit to: Access to pre-trained underlying models; generating a generalized validation code model by fine-tuning the pre-trained base model, the training data of which corresponds to a plurality of code languages; and Fine-tuning the generalized verification code model with language-specific training data. 8. The non-transitory computer-readable medium of any one of items 1 to 7, wherein the first trained AI-based model is trained by: [Item 9] The instructions may be to cause one or more of the at least one programmable circuit to: Access to pre-trained underlying models; generating a generalized validation code model by fine-tuning the pre-trained base model, the training data of which corresponds to a plurality of code languages, wherein the fine-tuning of the pre-trained base model is based on user feedback; and Fine-tuning the generalized verification code model with language-specific training data. 9. The non-transitory computer-readable medium of any one of items 1 to 8, wherein the second trained AI-based model is trained by: [Item 10] Interface circuit for obtaining design; computer-readable instructions; and The computer readable instructions causing a first trained artificial intelligence (AI) based model to generate verification code based on the design; generating a verifiability score for the design based on the verification code; and having a second trained AI-based model update the design based on the verifiability score. At least one programmable circuit that is programmable to An apparatus comprising: [Item 11] Item 11. The apparatus of item 10, wherein the design is expressed in at least one manner. [Item 12] Item 12. The apparatus of item 10 or 11, wherein one or more of the at least one programmable circuit performs multi-mode encoding on the design when the design is expressed in at least two ways. [Item 13] 13. The apparatus of claim 10, wherein one or more of the at least one programmable circuitry inputs prompts to the first trained AI-based model to generate the verification code. [Item 14] 14. The apparatus of any one of items 10 to 13, wherein the verification code finds errors in the design. [Item 15] 15. The apparatus of claim 10, wherein the verifiability score is a first verifiability score, and one or more of the at least one programmable circuit executes the verification code to generate a second verifiability score corresponding to the updated design. [Item 16] 16. The apparatus of claim 10, wherein the verifiability score is a first verifiability score, and the one or more of the at least one programmable circuit outputs the updated design as a final design if a second verifiability score of the updated design satisfies a threshold. [Item 17] One or more of the at least one programmable circuit Access to pre-trained underlying models; generating a generalized validation code model by training the pre-trained base model where training data corresponds to a plurality of code languages; and generating the first trained AI-based model by training the generalized verification code model with language-specific training data; 17. The apparatus of any one of items 10 to 16, wherein the first trained AI-based model is trained by: [Item 18] One or more of the at least one programmable circuit Access to pre-trained underlying models; generating a generalized validation code model by adjusting the pre-trained base model, the training data corresponding to a plurality of code languages, wherein adjusting the pre-trained base model is based on user feedback; and generating the second trained AI-based model by training the generalized verification code model with language-specific training data; 18. The apparatus of any one of items 10 to 17, wherein the second trained AI-based model is trained by: [Item 19] Design and obtain the interface circuit; machine-readable instructions; and causing a first artificial intelligence (AI) based model to generate verification code based on the design; Executing the verification code to generate a verifiability score corresponding to the design; and Having a second AI-based model iteratively adjust the design until the verifiability score meets a threshold. a programmable circuit for at least one of instantiating or implementing said machine-readable instructions for An apparatus comprising: [Item 20] 20. The apparatus of claim 19, wherein the design is expressed in at least one manner. (Item 21) generating verification code based on the input design using the first trained artificial intelligence (AI) based model; executing the verification code to generate a verifiability score for the input design; and adjusting the input design using a second trained AI-based model based on the verifiability score. A method for providing (Item 22) Item 22. The method of item 21, wherein the input design is expressed in at least one format. (Item 23) 23. The method of claim 21 or 22, wherein the instructions cause one or more of the at least one programmable circuit to perform multi-mode encoding on the input design based on the input design expressed in two or more ways. (Item 24) means for generating verification code based on the input design; means for executing the verification code to generate a verifiability score for the input design; and means for adjusting the input design based on the verifiability score. (Item 25) Item 25. The apparatus of item 24, wherein the input design is expressed in at least one manner. (Item 26) 10. A computer-readable medium storing a computer program according to any one of items 1 to 9.

Claims

1. generating verification code based on the input design using the first trained artificial intelligence (AI) based model; executing the verification code to generate a verifiability score for the input design; and adjusting the input design using a second trained AI-based model based on the verifiability score.

1. A computer program comprising instructions for causing at least one programmable circuit to execute:

2. The computer program of claim 1 , wherein the input design is expressed in at least one manner.

3. 2. The computer program product of claim 1, wherein the instructions cause one or more of the at least one programmable circuit to perform multi-mode encoding on the input design based on the input design expressed in two or more ways.

4. 2. The computer program product of claim 1, wherein the instructions cause one or more of the at least one programmable circuit to input a prompt to the first trained AI-based model to generate the verification code.

5. The computer program product of claim 1 , wherein the verification code verifies functionality of the input design.

6. 2. The computer program product of claim 1, wherein the verifiability score is a first verifiability score, and the instructions cause one or more of the at least one programmable circuit to execute the verification code to generate a second verifiability score for the adjusted input design.

7. 2. The computer program product of claim 1, wherein the verifiability score is a first verifiability score, and the instructions cause one or more of the at least one programmable circuit to output the adjusted input design as a final design based on a second verifiability score of the adjusted input design satisfying a threshold.

8. The instructions may be to cause one or more of the at least one programmable circuit to: Access to pre-trained underlying models; generating a generalized validation code model by fine-tuning the pre-trained base model whose training data corresponds to multiple code languages; and Fine-tuning the generalized verification code model with language-specific training data.

2. The computer program product of claim 1, wherein the first trained AI-based model is trained by:

9. The instructions may be to cause one or more of the at least one programmable circuit to: Access to pre-trained underlying models; generating a generalized validation code model by fine-tuning the pre-trained base model, the training data of which corresponds to a plurality of code languages, wherein the fine-tuning of the pre-trained base model is based on user feedback; and Fine-tuning the generalized verification code model with language-specific training data.

2. The computer program product of claim 1, wherein the second trained AI-based model is trained by:

10. interface circuit for obtaining the design; computer-readable instructions; and The computer readable instructions generating verification code based on the design using a first trained artificial intelligence (AI)-based model; generating a verifiability score for the design based on the verification code; and having a second trained AI-based model update the design based on the verifiability score. At least one programmable circuit that is programmable to An apparatus comprising:

11. The apparatus of claim 10 , wherein the design is expressed in at least one manner.

12. 11. The apparatus of claim 10, wherein one or more of the at least one programmable circuit performs multi-mode encoding on the design based on the design expressed in at least two ways.

13. 11. The apparatus of claim 10, wherein one or more of the at least one programmable circuit inputs prompts to the first trained AI-based model to generate the verification code.

14. The apparatus of claim 10 , wherein the verification code finds errors in the design.

15. 11. The apparatus of claim 10, wherein the verifiability score is a first verifiability score, and one or more of the at least one programmable circuit executes the verification code to generate a second verifiability score corresponding to the updated design.

16. 11. The apparatus of claim 10, wherein the verifiability score is a first verifiability score, and wherein one or more of the at least one programmable circuit output the updated design as a final design based on a second verifiability score of the updated design satisfying a threshold.

17. One or more of the at least one programmable circuit Access to pre-trained underlying models; generating a generalized validation code model by training the pre-trained base model, the training data corresponding to multiple code languages; and generating the first trained AI-based model by training the generalized verification code model with language-specific training data; 11. The apparatus of claim 10, wherein the first trained AI-based model is trained by:

18. One or more of the at least one programmable circuit Access to pre-trained underlying models; generating a generalized validation code model by adjusting the pre-trained base model, the training data corresponding to a plurality of code languages, wherein adjusting the pre-trained base model is based on user feedback; and generating the second trained AI-based model by training the generalized verification code model with language-specific training data; The apparatus of any one of claims 10 to 17, wherein the second trained AI-based model is trained by:

19. interface circuit to obtain the design; machine-readable instructions; and generating verification code based on the design using a first artificial intelligence (AI)-based model; Executing the verification code to generate a verifiability score corresponding to the design; and Having a second AI-based model iteratively adjust the design until the verifiability score meets a threshold. a programmable circuit for at least one of instantiating or implementing said machine-readable instructions for An apparatus comprising:

20. 20. The apparatus of claim 19, wherein the design is expressed in at least one manner.

21. generating verification code based on the input design using the first trained artificial intelligence (AI) based model; executing the verification code to generate a verifiability score for the input design; and adjusting the input design using a second trained AI-based model based on the verifiability score. A method for providing

22. The method of claim 21 , wherein the input design is expressed in at least one manner.

23. 23. The method of claim 21 or 22, wherein instructions cause one or more of the at least one programmable circuit to perform multi-mode encoding on the input design based on the input design being expressed in two or more ways.

24. means for generating a verification code based on an input design; means for executing the verification code to generate a verifiability score for the input design; and means for adjusting the input design based on the verifiability score. An apparatus comprising:

25. 25. The apparatus of claim 24, wherein the input design is expressed in at least one manner.

26. A computer readable medium storing a computer program according to any one of claims 1 to 9.

Images