Method, device, computer equipment, readable storage medium and program product for non-graph optimization based on reinforcement learning

Abstract

The present application relates to a method, apparatus, computer equipment, readable storage medium and program product for optimizing a non-graph based on reinforcement learning. Through a reinforcement learning model, according to the embedded features of the non-graph to be optimized and a preset logic optimization tool, multiple states corresponding to the non-graph and each target preset non-graph optimization algorithm corresponding to each state are determined, and an optimization sequence corresponding to each target preset non-graph optimization algorithm is output, and the non-graph is optimized based on each optimization algorithm in the optimization sequence. Compared with the traditional optimization based on artificial experience, this scheme trains the reinforcement learning model by using the reinforcement learning framework and the logic optimization tool, uses the trained reinforcement learning model to predict and learn the embedded features of the non-graph, identifies multiple states corresponding to the features and the optimization algorithms corresponding to each state, and optimizes the non-graph based on each optimization algorithm output by the model, thereby improving the efficiency of optimizing the non-graph.

Description

Technical Field

[0001] The present application relates to the field of electronic automation technology, and in particular to a reinforcement learning-based non-AND graph optimization method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] In electronic design automation, the process of logic synthesis includes logic conversion, logic optimization and process mapping. Logic optimization is to convert the logic expression into the "best implementation" and comprehensively consider the best parameters such as complexity, area, delay, etc., that is, to simplify the existing logic expression. Among them, the optimization of the NAND graph is a necessary step in the logic optimization process. At present, the optimization of the NAND graph is usually done manually. However, optimization based on manual experience is time-consuming and labor-intensive, which leads to low efficiency.

[0003] Therefore, the current methods for optimizing non-AND graphs have the defect of low optimization efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a reinforcement learning-based and non-graph optimization method, device, computer equipment, computer-readable storage medium and computer program product that can improve the optimization efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a non-graph optimization method based on reinforcement learning, the method comprising:

[0006] Obtaining a NAND graph to be optimized, and extracting embedding features corresponding to the NAND graph;

[0007] The embedded features are input into a trained reinforcement learning model, and the reinforcement learning model determines the multiple states corresponding to the non-graph and the respective target presets and non-graph optimization algorithms corresponding to the respective states according to the embedded features and a preset logic optimization tool, and outputs an optimization sequence according to the respective target presets and non-graph optimization algorithms; the reinforcement learning model is trained in a preset reinforcement learning framework based on non-graph samples and the preset logic optimization tool;

[0008] According to each target preset AND non-graph optimization algorithm in the optimization sequence, the AND non-graph is optimized.

[0009] In one embodiment, the method further comprises:

[0010] According to the preset logic optimization tool, a preset reinforcement learning framework is constructed; the preset logic optimization tool includes a plurality of preset and non-graph optimization algorithms;

[0011] Obtaining a reinforcement learning model to be trained, a non-graph sample, and an optimized sequence sample corresponding to the non-graph sample, and extracting an embedded feature sample corresponding to the non-graph sample;

[0012] In the preset reinforcement learning framework, the embedded feature sample is input into the reinforcement learning model, and the reinforcement learning model determines the current state corresponding to the non-graph sample based on the embedded feature sample;

[0013] According to the current state, determining a corresponding preset and non-graph optimization algorithm sample from a plurality of preset and non-graph optimization algorithms in the preset logic optimization tool;

[0014] Execute the preset and non-graph optimization algorithm sample to obtain a new current state corresponding to the non-graph sample;

[0015] Returning to the step of determining a corresponding preset and non-graph optimization algorithm sample from a plurality of preset and non-graph optimization algorithms in the preset logic optimization tool according to the current state, until a preset prediction end condition is met, outputting optimization sequence prediction information according to each of the preset and non-graph optimization algorithm samples;

[0016] The optimization sequence prediction information and the optimization sequence samples are input into a preset loss function, and the model parameters of the reinforcement learning model are adjusted according to the output value of the preset loss function until the preset training end condition is met, thereby obtaining the trained reinforcement learning model.

[0017] In one embodiment, determining a corresponding preset and non-graph optimization algorithm sample from a plurality of preset and non-graph optimization algorithms in the preset logic optimization tool according to the current state includes:

[0018] For each preset and non-graph optimization algorithm in the preset logic optimization tool, obtaining a simulation execution result corresponding to the preset and non-graph optimization algorithm in the current state;

[0019] Determining the action probability corresponding to the preset non-graph optimization algorithm according to the current state and the simulation execution result;

[0020] According to the preset and non-graph optimization algorithm with the largest action probability among the preset and non-graph optimization algorithms, a preset and non-graph optimization algorithm sample corresponding to the current state is obtained.

[0021] In one embodiment, the embedded features are input into a trained reinforcement learning model, and the reinforcement learning model determines the multiple states corresponding to the non-graph and the respective target presets and non-graph optimization algorithms corresponding to the respective states according to the embedded features and a preset logic optimization tool, and outputs an optimization sequence according to the respective target presets and non-graph optimization algorithms, including:

[0022] Determining the current state corresponding to the non-graph based on the embedded feature;

[0023] According to the current state, determining a corresponding target preset and a non-graph optimization algorithm from the preset logic optimization tool;

[0024] Execute the corresponding target preset and non-graph optimization algorithm to obtain the new current state corresponding to the non-graph;

[0025] Return to the step of determining the corresponding target preset and non-graph optimization algorithm from the preset logic optimization tool according to the current state, until the preset prediction end condition is met, and output the optimization sequence according to each of the target presets and non-graph optimization algorithms.

[0026] In one embodiment, the optimizing algorithm of the AND graph is preset according to each target in the optimization sequence, and the optimizing algorithm of the AND graph is optimized, including:

[0027] The first target preset and non-graph optimization algorithm in the optimization sequence are used as the current algorithm to be executed;

[0028] Optimizing the NAND graph to be optimized according to the currently executed algorithm to obtain a new NAND graph to be optimized;

[0029] Obtaining the next target preset and non-graph optimization algorithm corresponding to the current algorithm to be executed in the optimization sequence as a new current algorithm to be executed;

[0030] Return to the step of optimizing the NAND graph to be optimized according to the currently executed algorithm until all the target preset NAND graph optimization algorithms in the optimization sequence are executed to obtain the optimized NAND graph.

[0031] In one embodiment, extracting the embedding feature corresponding to the non-graph includes:

[0032] Obtaining the graph embedding feature corresponding to the non-graph;

[0033] Extracting node embedding features according to each node in the AND-NOT graph;

[0034] The embedding feature corresponding to the non-graph is obtained according to the graph embedding feature and the node embedding feature.

[0035] In a second aspect, the present application provides a non-graph optimization device based on reinforcement learning, the device comprising:

[0036] An acquisition module, used for acquiring a NAND graph to be optimized and extracting embedding features corresponding to the NAND graph;

[0037] A prediction module, used for inputting the embedded features into a trained reinforcement learning model, wherein the reinforcement learning model determines the multiple states corresponding to the non-graph and the respective target presets and non-graph optimization algorithms corresponding to the respective states according to the embedded features and a preset logic optimization tool, and outputs an optimization sequence according to the respective target presets and non-graph optimization algorithms; the reinforcement learning model is trained in a preset reinforcement learning framework based on non-graph samples and the preset logic optimization tool;

[0038] The optimization module is used to optimize the AND-NORMAL graph according to the preset AND-NORMAL graph optimization algorithm of each target in the optimization sequence.

[0039] In a third aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0041] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0042] The above-mentioned reinforcement learning-based AND-NON graph optimization method, device, computer equipment, computer-readable storage medium and computer program product determine the multiple states corresponding to the AND-NON graph and the respective target preset AND-NON graph optimization algorithms corresponding to the respective states through the reinforcement learning model according to the embedded features of the AND-NON graph to be optimized and the preset logic optimization tool, output the optimization sequence corresponding to the respective target preset AND-NON graph optimization algorithm, and optimize the AND-NON graph based on the respective optimization algorithms in the optimization sequence. Compared with the traditional optimization based on artificial experience, this scheme trains the reinforcement learning model by using the reinforcement learning framework and the logic optimization tool, uses the trained reinforcement learning model to predict and learn the embedded features of the AND-NON graph, identifies the multiple states corresponding to the features and the optimization algorithms corresponding to the respective states, and optimizes the AND-NON graph based on the respective optimization algorithms output by the model, thereby improving the efficiency of the AND-NON graph optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0044] FIG1 is a schematic diagram of a process flow of a non-graph optimization method based on reinforcement learning in one embodiment;

[0045] FIG2 is a schematic diagram of the structure of an AND-NOT graph in one embodiment;

[0046] FIG3 is a schematic diagram of the structure of a reinforcement learning framework in one embodiment;

[0047] FIG4 is a schematic diagram of a structure of a reinforcement learning model prediction in one embodiment;

[0048] FIG5 is a structural block diagram of a non-AND graph optimization device based on reinforcement learning in one embodiment;

[0049] FIG. 6 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] In one embodiment, as shown in FIG1 , a method for optimizing a non-AND graph based on reinforcement learning is provided. This embodiment uses the method applied to a server as an example for illustration. It is understandable that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server, including the following steps S202 to S206. Among them:

[0052] Step S202, obtaining the NAND graph to be optimized, and extracting the embedding features corresponding to the NAND graph.

[0053] Among them, in electronic design automation, it is necessary to perform logic synthesis steps on the circuit, including logic conversion, logic optimization and process mapping. Among them, logic conversion is to convert other forms of logic expressions into logic expressions that are easy to process, and logic optimization is to convert logic expressions into "optimal implementations", taking into account the best parameters such as complexity, area, and delay. In order to obtain these indicators and parameters, process mapping can realize the conversion of gate circuits into standard cell netlists of specified processes, and physical design analysis can obtain parameters such as delay and area based on the cell library. Among them, area refers to the area occupied by the logic circuit in the chip after the process mapping stage. Delay refers to the unit time length required for the logic circuit from input to output in the chip after the process mapping stage.

[0054] Logic optimization of NAND graphs is a necessary measure to ensure the circuit effect. At present, it is impossible to complete logic optimization of complex circuit design in one go. Therefore, for NAND graphs, a series of optimization algorithms need to be used to generate optimization sequences to gradually optimize the NAND graphs. Among them, NAND graphs refer to logic networks that only contain two-input AND gates and NOT gates.

[0055] The server may be a computer device that optimizes the NAND graph to be optimized of the circuit to be optimized. The program in the server may optimize the NAND graph to be optimized by means of reinforcement learning. For example, the program in the server is constructed based on a deep reinforcement learning framework and a reinforcement learning model, and an adaptive learning mechanism is introduced, which can adjust the optimization inverse direction according to real-time feedback. This enables the reinforcement learning framework to continuously optimize its own model during training and subsequent testing, thereby improving the adaptability and effect under different NAND graphs and constraints.

[0056] In order to optimize the above-mentioned AND non-graph, it is necessary to extract relevant features corresponding to the AND non-graph. In the preprocessing stage, the program in the server can extract the embedded features corresponding to the above-mentioned AND non-graph after obtaining the AND non-graph to be optimized. Among them, the above-mentioned embedded features include multiple types of embedded features. The program in the server can combine multiple types of embedded features to obtain the embedded features corresponding to the above-mentioned AND non-graph.

[0057] In one embodiment, extracting embedding features corresponding to non-graphs includes: obtaining graph embedding features corresponding to the non-graphs; extracting node embedding features according to each node in the non-graphs; and obtaining embedding features corresponding to the non-graphs according to the graph embedding features and the node embedding features.

[0058] In this embodiment, the above-mentioned embedding features corresponding to non-graphs to be optimized may include multiple types, such as graph embedding features and node embedding features.

[0059] Wherein, as shown in Figure 2, Figure 2 is a schematic diagram of the structure of an AND-NON graph in one embodiment. It can be known that the AND-NON graph to be optimized includes multiple nodes and multiple AND gates and NOT gates. The program in the server can obtain the graph embedding features corresponding to the above-mentioned AND-NON graph. The graph embedding features mainly include converting the above-mentioned AND-NON graph data into a low-dimensional vector representation, while retaining the structure of the original graph and the relationship information between the nodes as much as possible. The above-mentioned AND-NON graph includes multiple nodes, and the program in the server can extract the corresponding node embedding features according to each node in the AND-NON graph. Among them, the node embedding feature can be to map the nodes in the AND-NON graph to a low-dimensional vector space to retain the structural information of the nodes in the network. Thereby, the program in the server can obtain the embedding features corresponding to the above-mentioned AND-NON graph according to the above-mentioned graph embedding features and node embedding features.

[0060] Step S204, input the embedded features into the trained reinforcement learning model, and the reinforcement learning model determines multiple states corresponding to the non-graph and each target preset and non-graph optimization algorithm corresponding to each state according to the embedded features and the preset logic optimization tool, and outputs an optimization sequence according to each target preset and non-graph optimization algorithm; the reinforcement learning model is trained in the preset reinforcement learning framework based on the non-graph samples and the preset logic optimization tool.

[0061] Among them, reinforcement learning refers to the intelligent agent learning in a "trial and error" manner, and the behavior is guided by the rewards obtained by interacting with the environment, with the goal of enabling the intelligent agent to obtain the maximum reward. The backend of the program training in the server can pre-train the reinforcement learning model to be trained in a preset reinforcement learning framework based on non-graph samples and preset logic optimization tools to obtain the above-mentioned trained reinforcement learning model. Among them, the above-mentioned reinforcement learning model can also be a deep learning model. The deep learning model is a method of machine learning that uses a deep neural network for data analysis and feature extraction. The above-mentioned preset logic optimization tool may include a variety of preset non-graph optimization algorithms. Among them, the preset non-graph optimization algorithm can be used to optimize the non-graph, and the preset non-graph optimization algorithm is provided by the above-mentioned preset logic optimization tool. The above-mentioned preset logic optimization tool can be an integrated circuit design tool.

[0062] After the server completes training of the above reinforcement learning model, the above acquired embedded features can be output to the trained reinforcement learning model. The reinforcement learning model can determine the above multiple states corresponding to the non-graph and the various target presets and non-graph optimization algorithms corresponding to each state based on the above embedded features and preset logic optimization tools. For example, the reinforcement learning model can use the above input embedded features as a starting point to gradually determine the target presets and non-graph optimization algorithms corresponding to each non-graph state and the new state obtained after optimizing the current state based on each target preset and non-graph optimization algorithm, and then the reinforcement learning model can obtain an optimization sequence including multiple target presets and non-graph optimization algorithms. Among them, the various target presets and non-graph optimization algorithms in the optimization sequence can be sorted according to the order in which the reinforcement learning model determines the algorithm.

[0063] The program front end in the server can obtain the above optimization sequence output by the reinforcement learning model, and then use the optimization sequence to optimize the non-graph accordingly. Among them, each preset non-graph optimization algorithm in the above preset logic optimization tool can be operated through visualization. Through the visualization of the optimization process and decision basis, engineers can understand the reasons for each step more clearly, thereby effectively improving reliability and decision quality.

[0064] Step S206, optimizing the AND-NORMAL graph according to the preset AND-NORMAL graph optimization algorithm for each target in the optimization sequence.

[0065] Wherein, the above-mentioned optimization sequence can be a sequence output by a reinforcement learning model. Wherein, the optimization sequence includes multiple target presets and non-graph optimization algorithms. The program in the server can automatically optimize the above-mentioned AND non-graph to be optimized according to the order of multiple target presets and non-graph optimization algorithms in the optimization sequence. For example, the program in the server executes the corresponding target presets and non-graph optimization algorithms on the above-mentioned AND non-graph in sequence according to the order of the target presets and non-graph optimization algorithms, and then after the above-mentioned presets and non-graph optimization algorithms are executed on the above-mentioned AND non-graph, the optimized AND non-graph is obtained. Wherein, any AND non-graph in the circuit design can obtain the corresponding optimization sequence through the above-mentioned reinforcement learning model, and then the program in the server can realize the generation and optimization of the optimization sequence for any AND non-graph.

[0066] In the above-mentioned optimization method of the AND-NON graph based on reinforcement learning, the reinforcement learning model determines the multiple states corresponding to the AND-NON graph and the preset AND-NON graph optimization algorithms corresponding to each state according to the embedded features of the AND-NON graph to be optimized and the preset logic optimization tool, outputs the optimization sequence corresponding to each target preset AND-NON graph optimization algorithm, and optimizes the AND-NON graph based on each optimization algorithm in the optimization sequence. Compared with the traditional optimization based on artificial experience, this scheme trains the reinforcement learning model by using the reinforcement learning framework and the logic optimization tool, uses the trained reinforcement learning model to predict and learn the embedded features of the AND-NON graph, identifies the multiple states corresponding to the features and the optimization algorithms corresponding to each state, optimizes the AND-NON graph based on each optimization algorithm output by the model, improves the efficiency of the optimization of the AND-NON graph, and uses the reinforcement learning algorithm to automatically generate the optimization sequence, which significantly improves the optimization efficiency. By analyzing the structural features of the AND-NON graph and the existing optimization algorithms, the system can quickly generate high-quality optimization sequences, thereby reducing the dependence on artificial experience and avoiding the time-consuming and labor-intensive manual design process.

[0067] In one embodiment, it also includes: constructing a preset reinforcement learning framework according to a preset logic optimization tool; the preset logic optimization tool includes multiple preset and non-graph optimization algorithms; obtaining a reinforcement learning model to be trained, a non-graph sample, and an optimization sequence sample corresponding to the non-graph sample, and extracting an embedded feature sample corresponding to the non-graph sample; in the preset reinforcement learning framework, inputting the embedded feature sample into the reinforcement learning model, and the reinforcement learning model determines the current state corresponding to the non-graph sample based on the embedded feature sample; according to the current state, determining the corresponding preset and non-graph optimization algorithm sample from multiple preset and non-graph optimization algorithms in the preset logic optimization tool; executing the preset and non-graph optimization algorithm sample to obtain a new current state corresponding to the non-graph sample; returning to the step of determining the corresponding preset and non-graph optimization algorithm sample from multiple preset and non-graph optimization algorithms in the preset logic optimization tool according to the current state, until the preset prediction end condition is met, outputting the optimization sequence prediction information according to each preset and non-graph optimization algorithm sample; inputting the optimization sequence prediction information and the optimization sequence sample into the preset loss function, and adjusting the model parameters of the reinforcement learning model according to the output value of the preset loss function until the preset training end condition is met, and a trained reinforcement learning model is obtained.

[0068] In this embodiment, the server may pre-train the reinforcement learning model to be trained to obtain the above-mentioned trained reinforcement learning model. Among them, the server may pre-build a preset reinforcement learning framework for training and recognition. For example, the above-mentioned preset logic optimization tool may include multiple preset and non-graph optimization algorithms, then a preset reinforcement learning framework may be constructed according to the above-mentioned preset logic optimization tool, an environment that satisfies the use of the preset logic optimization tool may be constructed, and the preset and non-graph optimization algorithms that can be used may be determined based on the preset logic optimization tool.

[0069] After the server constructs the above-mentioned preset reinforcement learning framework, in order to realize the training of the reinforcement learning model, the server can obtain the reinforcement learning model to be trained, the non-graph sample and the optimization sequence sample corresponding to the non-graph sample. Among them, there can be multiple non-graph samples, and the optimization sequence sample can be an optimization sequence corresponding to the non-graph sample that meets the optimization requirements. The server can extract the embedded feature sample corresponding to the non-graph sample. When training the reinforcement learning model, the server can input the above-mentioned embedded feature sample into the reinforcement learning model to be trained within the preset reinforcement learning framework, and the reinforcement learning model determines the current state corresponding to the non-graph sample based on the embedded feature sample. Among them, the current state represents the state of the non-graph sample in the reinforcement learning model. The reinforcement learning model can determine the corresponding preset non-graph optimization algorithm sample from multiple preset non-graph optimization algorithms in the preset logic optimization tool according to the current state. Among them, the preset non-graph optimization algorithm sample determined above can be an algorithm determined based on the reward mechanism, that is, the reinforcement learning model believes that the use of the preset non-graph optimization algorithm sample can improve the state of the non-graph sample.

[0070] The reinforcement learning model can execute preset and non-graph optimization algorithm samples to obtain a new current state corresponding to the non-graph sample. Based on the above new current state, the reinforcement learning model returns the step of determining the corresponding preset and non-graph optimization algorithm samples from multiple preset and non-graph optimization algorithms in the preset logic optimization tool according to the current state until the preset prediction end condition is met, such as reaching the total learning step length, the number of preset and non-graph optimization algorithm samples reaches a preset number threshold, or the number of training iterations reaches a preset number threshold. The reinforcement learning model outputs optimization sequence prediction information based on each preset and non-graph optimization algorithm sample. Among them, the optimization sequence prediction information includes multiple determined preset and non-graph optimization algorithm samples.

[0071] Thus, the server can input the above-mentioned optimization sequence prediction information and optimization sequence samples into the preset loss function, and obtain the output value of the preset loss function based on the above-mentioned optimization sequence prediction information and the corresponding optimization sequence samples. The server can adjust the model parameters of the reinforcement learning model according to the output value of the preset loss function, and iteratively train the reinforcement learning model until the preset training end condition is met, thereby obtaining a trained reinforcement learning model. Among them, the preset training end condition may include that the output value of the above-mentioned loss function is the minimum within the preset training iteration number; or the iterative training number of the above-mentioned model reaches the preset training iteration number threshold.

[0072] Specifically, when the program in the server trains the reinforcement learning model, it can pre-install the preset logic optimization tool and process mapping tool, and prepare a large amount of non-graph data as samples. The server can pre-build a reinforcement learning framework, which includes the definition of reinforcement learning environment, agent, state, action and reward. As shown in Figure 3, Figure 3 is a schematic diagram of the structure of the reinforcement learning framework in one embodiment. Among them, the above-mentioned reinforcement learning model can be a deep Q network model, or it can be other neural network models or prediction models. The deep Q network model can be a deep neural network, and the server uses the deep neural network to approximate the value function Q (s, a). Among them, s represents the state, a represents the action, r represents the reward of the action, and s' represents the next state after executing action a. The above-mentioned environment is the use environment of the logic optimization tool and the process mapping tool; the state is the statistical information of the current non-graph network and other useful information from historical experience; the action is the preset non-graph optimization algorithm provided in the current optimization tool, and the action space refers to a set of valid actions that can be performed by the current non-graph. The reward is the distance between the original state and the target state after the environment reaches a new state after the action is executed. Improvement, used to guide the agent to select actions. Among them, the intelligent agent is a structure in the reinforcement learning framework used to interact with the prediction results and the environment, and can determine whether to take an action based on the action score returned by the server.

[0073] For example, the server uses the deep Q network in the reinforcement learning model to obtain the optimization sequence prediction information, which can be sampled by the intelligent agent and then executed by each action, that is, each preset and non-graph optimization algorithm sample, to obtain the corresponding new state and reward. The intelligent agent combines the optimization sequence prediction information and the corresponding optimization sequence samples, and inputs them into the preset loss function. Then, based on the output value of the preset loss function, the reinforcement learning model can be replayed to learn the experience, thereby realizing the training of the reinforcement learning model.

[0074] When the server trains the reinforcement learning model, PyTorch can be used to build a deep Q network model as a reinforcement learning model. PyTorch is a deep learning framework. In the reinforcement learning model, randomly initialized orthogonal vectors are used as initial values, and the Adam optimizer with a learning rate of 0.001 is used for training. The number of training iterations is a preset training iteration threshold, for example, 1000 times. During the training process, the AND-NON graph is used as input, and the optimized sequence prediction information is finally output. The Adam optimizer is an algorithm for deep learning to optimize the target network.

[0075] Specifically, when training the reinforcement learning model, the reinforcement learning framework in the server can initialize the reinforcement learning environment and network parameters, determine the total learning step length N, optimize the maximum sequence length L and the number of training iterations n k Within the total learning step length N, the reinforcement learning model can perform the following steps multiple times: reset the environment, initialize the optimization sequence, perform the extraction state s within the maximum length of the optimization sequence t , and sample action a from the current network model t , that is, the above preset and non-graph optimization algorithm samples, perform action a in the environment t , update the parameters (state) and set action a t Add the step of optimization sequence. Wherein, t represents the number of times the above steps are executed. If the optimization sequence is completed, that is, there are multiple preset non-graph optimization algorithm samples in the optimization sequence that can make the non-graph state meet the requirements, or the number of preset non-graph optimization algorithm samples in the optimization sequence is greater than the above L, jump out of the above loop and execute the following steps multiple times within the number of iterations: Based on the prediction information of the above optimization sequence and the optimization sequence samples, determine the output value of the loss function, and then determine the current reward. According to the reward and the back propagation algorithm, update the model parameters of the reinforcement learning model. After adjusting the model parameters multiple times, the server can obtain a trained reinforcement learning model.

[0076] Through this embodiment, the server can combine the reinforcement learning framework and multiple non-graph samples to iteratively train the reinforcement learning model, and then use the trained reinforcement learning model to predict and identify the optimized sequence of the non-graph, thereby improving the efficiency of optimizing the non-graph.

[0077] In one embodiment, according to the current state, corresponding preset and non-graph optimization algorithm samples are determined from multiple preset and non-graph optimization algorithms in the preset logic optimization tool, including: for each preset and non-graph optimization algorithm in the preset logic optimization tool, obtaining the simulation execution result corresponding to the preset and non-graph optimization algorithm in the current state; determining the action probability corresponding to the preset and non-graph optimization algorithm according to the current state and the simulation execution result; and obtaining the preset and non-graph optimization algorithm samples corresponding to the current state according to the preset and non-graph optimization algorithm with the largest action probability among the preset and non-graph optimization algorithms.

[0078] In this embodiment, during the above training process, the reinforcement learning model can determine the preset and non-graph optimization algorithm samples that can improve the non-graph samples based on the current state, so that the reinforcement learning model can obtain multiple preset and non-graph optimization algorithm samples. Among them, each preset and non-graph optimization algorithm sample can be determined from multiple preset and non-graph optimization algorithms based on the action score. For example, for each preset and non-graph optimization algorithm in the preset logic optimization tool, the reinforcement learning model can obtain the simulation execution result corresponding to the preset and non-graph optimization algorithm in the current state, so that the reinforcement learning model can determine the action probability corresponding to the preset and non-graph optimization algorithm according to the current state and the simulation execution result. Among them, the above-mentioned action probability can be determined by the reinforcement learning model by scoring the action, and the action score and the action probability are positively correlated. Thus, the reinforcement learning model can obtain the preset and non-graph optimization algorithm sample corresponding to the current state according to the preset and non-graph optimization algorithm with the largest action probability among the preset and non-graph optimization algorithms.

[0079] Among them, the above-mentioned action score represents the degree of improvement of the non-graph after the preset and non-graph optimization algorithm is executed. The greater the degree of improvement, the higher the action score, and thus the higher the action probability. Specifically, as shown in Figure 4, Figure 4 is a structural diagram of the reinforcement learning model prediction in one embodiment. Taking the reinforcement learning model as a deep Q network model as an example, after the server inputs the node embedding features and the graph embedding features into the deep Q network model, the action scores of each preset and non-graph optimization algorithm can be output through deep Q network learning, which can be specifically expressed as {Q(st, "balance"; w t ); Q(st, "rewrite"; w t );…;Q(st,“rw-zl”;w t )}. Among them, w trepresents the weight when the algorithm predicts the current state for the tth time, balance (balance optimization algorithm), rewrite (rewrite optimization algorithm), rw-zl, etc. represent different preset and non-graph optimization algorithms. It should be noted that the preset and non-graph optimization algorithms are not limited to the algorithms recorded above. Therefore, the reinforcement learning model can determine the corresponding action probability according to each action score, and based on the size of the action probability, determine the optimal action corresponding to the current state, that is, the current optimal preset and non-graph optimization algorithm, as the preset and non-graph optimization algorithm sample corresponding to the current state. Among them, the Q function of the above-mentioned deep Q network model can be represented by a simple deep perceptron model.

[0080] Through this embodiment, the reinforcement learning framework in the server can use the method of action probability and action score to determine the optimal preset non-graph optimization algorithm corresponding to each current state, and then based on the optimization sequence containing multiple determined non-graph optimization algorithms, realize the optimization of non-graph, and improve the efficiency of optimizing non-graph. And the server can also perform adaptive optimization, and ensure the consistency of optimization direction through deep learning mechanism. In this way, the most preferred optimization sequence for various types of non-graphs can be generated while meeting the area and delay constraints, so that the optimization results are more consistent and predictable, and the result differences between different engineers are reduced. In addition, the server can also adopt a modular design, and different optimization strategies and model algorithms can be transplanted to different systems by the transfer learning method, and customized for different non-graphs. The optimization knowledge base and model experience formed by the system can be accumulated and applied to new non-graph designs, thereby improving its generalizability and reusability.

[0081] In one embodiment, the embedded features are input into a trained reinforcement learning model, and the reinforcement learning model determines multiple states corresponding to the non-graph and each target preset and non-graph optimization algorithm corresponding to each state according to the embedded features and a preset logic optimization tool, and outputs an optimization sequence according to each target preset and non-graph optimization algorithm, including: determining a current state corresponding to the non-graph based on the embedded features; determining the corresponding target preset and non-graph optimization algorithm from the preset logic optimization tool according to the current state; executing the corresponding target preset and non-graph optimization algorithm to obtain a new current state corresponding to the non-graph; returning to the step of determining the corresponding target preset and non-graph optimization algorithm from the preset logic optimization tool according to the current state, until the preset prediction end condition is met, and then outputting an optimization sequence according to each target preset and non-graph optimization algorithm.

[0082] In this embodiment, the server can use a trained reinforcement learning model to optimize the output of the sequence. Among them, the above-mentioned input reinforcement model can be the embedded features of the non-graph to be optimized, including graph embedding features and node embedding features. After the server inputs the embedded features into the trained reinforcement learning model, it can determine the current state corresponding to the non-graph based on the embedded features. The reinforcement learning model can determine the corresponding target preset and non-graph optimization algorithm from the preset logic optimization tool according to the current state. For example, for each preset and non-graph optimization algorithm in the preset logic optimization tool, the reinforcement learning model can obtain the simulation execution results corresponding to the preset and non-graph optimization algorithm in the current state, so that the reinforcement learning model can determine the action probability corresponding to the preset and non-graph optimization algorithm according to the current state and the simulation execution results. Thus, the reinforcement learning model can obtain the target preset and non-graph optimization algorithm corresponding to the current state according to the preset and non-graph optimization algorithm with the largest action probability among the preset and non-graph optimization algorithms.

[0083] The reinforcement learning model can execute the target preset and non-graph optimization algorithm corresponding to the above current state, obtain a new current state corresponding to the non-graph, and return based on the new current state. According to the current state, the corresponding target preset and non-graph optimization algorithm steps are determined from the preset logic optimization tool, and multiple target preset and non-graph optimization algorithm predictions are performed until the preset prediction end conditions are met, and the optimization sequence is output according to each target preset and non-graph optimization algorithm. Specifically, the above reinforcement learning model can be a deep Q network model. The server uses the deep Q network model to realize the output of the optimization sequence. It is similar to the output process of the optimization sequence prediction information during the training of the reinforcement learning model, which will not be repeated here.

[0084] Through this embodiment, the server can use the trained reinforcement learning model to predict multiple states of the features of the non-graph in sequence, and determine the target preset and non-graph optimization algorithm corresponding to each state, so as to finally output the optimization sequence, and use the optimization sequence to optimize the non-graph, thereby improving the efficiency of optimizing the non-graph. In addition, through the adaptive learning mechanism and deep model learning method, higher reliability is also achieved.

[0085] In one embodiment, the AND-NON graph is optimized according to each target preset AND-NON graph optimization algorithm in the optimization sequence, including: taking the first target preset AND-NON graph optimization algorithm in the optimization sequence as the current algorithm to be executed; optimizing the AND-NON graph to be optimized according to the current algorithm to be executed to obtain a new AND-NON graph to be optimized; obtaining the next target preset AND-NON graph optimization algorithm corresponding to the current algorithm to be executed in the optimization sequence as the new current algorithm to be executed; returning to the step of optimizing the AND-NON graph to be optimized according to the current algorithm to be executed, until each target preset AND-NON graph optimization algorithm in the optimization sequence is executed, and an optimized AND-NON graph is obtained.

[0086] In this embodiment, the optimization sequence output by the above reinforcement learning model may include multiple target presets and non-graph optimization algorithms. Each target preset and non-graph optimization algorithm is sorted in the order predicted by the reinforcement learning model. The server can use the above optimization sequence to optimize the non-graph to be optimized. For example, the server obtains the first target preset and non-graph optimization algorithm from the optimization sequence, that is, the first target preset and non-graph optimization algorithm determined by the reinforcement learning model after determining the current state based on the embedded features of the input.

[0087] The server uses the above-mentioned first target preset AND non-graph optimization algorithm as the current algorithm to be executed, and optimizes the AND non-graph to be optimized according to the current algorithm to be executed to obtain a new AND non-graph to be optimized. Among them, the new AND non-graph to be optimized can be an AND non-graph with certain improvements, but it still needs to be optimized. Then the server can obtain the next target preset AND non-graph optimization algorithm corresponding to the current algorithm to be executed in the optimization sequence as the new current algorithm to be executed, and based on the new current algorithm to be executed and the new AND non-graph to be optimized, return the steps of optimizing the AND non-graph to be optimized according to the current algorithm to be executed, and perform multiple optimizations until all the target preset AND non-graph optimization algorithms in the optimization sequence have completed the execution of the AND non-graph, and based on the AND non-graph output after the last execution of the target preset AND non-graph optimization algorithm, obtain the optimized AND non-graph.

[0088] Through this embodiment, the server can use the reinforcement learning model to perform multiple state recognitions on the features of the AND-NO graph, and output an optimization sequence, and use the optimization sequence to gradually optimize the AND-NO graph, thereby improving the efficiency of optimizing the AND-NO graph.

[0089] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0090] Based on the same inventive concept, the embodiment of the present application also provides a reinforcement learning-based AND-NON graph optimization device for implementing the reinforcement learning-based AND-NON graph optimization method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the reinforcement learning-based AND-NON graph optimization device provided below can refer to the limitations of the reinforcement learning-based AND-NON graph optimization method above, and will not be repeated here.

[0091] In an exemplary embodiment, as shown in FIG5 , a reinforcement learning-based AND-NON graph optimization device is provided, comprising: an acquisition module 500, a prediction module 502, and an optimization module 504, wherein:

[0092] The acquisition module 500 is used to acquire the NAND graph to be optimized and extract the embedding features corresponding to the NAND graph.

[0093] The prediction module 502 is used to input the embedded features into the trained reinforcement learning model, and the reinforcement learning model determines multiple states corresponding to the non-graph and each target preset and non-graph optimization algorithm corresponding to each state according to the embedded features and the preset logic optimization tool, and outputs an optimization sequence according to each target preset and non-graph optimization algorithm; the reinforcement learning model is trained in the preset reinforcement learning framework based on the non-graph samples and the preset logic optimization tool.

[0094] The optimization module 504 is used to optimize the AND-NORM graph according to the preset AND-NORM graph optimization algorithm for each target in the optimization sequence.

[0095] In one embodiment, the above-mentioned device also includes: a training module, which is used to construct a preset reinforcement learning framework according to a preset logic optimization tool; the preset logic optimization tool includes multiple preset and non-graph optimization algorithms; obtain the reinforcement learning model to be trained, the non-graph sample and the optimization sequence sample corresponding to the non-graph sample, and extract the embedded feature sample corresponding to the non-graph sample; in the preset reinforcement learning framework, the embedded feature sample is input into the reinforcement learning model, and the reinforcement learning model determines the current state corresponding to the non-graph sample based on the embedded feature sample; according to the current state, the corresponding preset and non-graph optimization algorithm sample is determined from the multiple preset and non-graph optimization algorithms in the preset logic optimization tool; the preset and non-graph optimization algorithm sample is executed to obtain a new current state corresponding to the non-graph sample; return to the step of determining the corresponding preset and non-graph optimization algorithm sample from the multiple preset and non-graph optimization algorithms in the preset logic optimization tool according to the current state, until the preset prediction end condition is met, output the optimization sequence prediction information according to each preset and non-graph optimization algorithm sample; input the optimization sequence prediction information and the optimization sequence sample into the preset loss function, and adjust the model parameters of the reinforcement learning model according to the output value of the preset loss function until the preset training end condition is met, and a trained reinforcement learning model is obtained.

[0096] In one embodiment, the above-mentioned training module is specifically used to obtain the simulation execution results corresponding to the preset and non-graph optimization algorithms in the preset logic optimization tool under the current state; determine the action probability corresponding to the preset and non-graph optimization algorithms according to the current state and the simulation execution results; and obtain the preset and non-graph optimization algorithm samples corresponding to the current state according to the preset and non-graph optimization algorithms with the largest action probability among the preset and non-graph optimization algorithms.

[0097] In one embodiment, the above-mentioned prediction module 502 is specifically used to determine the current state corresponding to the non-graph based on the embedded features; according to the current state, determine the corresponding target preset and non-graph optimization algorithm from the preset logic optimization tool; execute the corresponding target preset and non-graph optimization algorithm to obtain a new current state corresponding to the non-graph; return to the step of determining the corresponding target preset and non-graph optimization algorithm from the preset logic optimization tool according to the current state, until the preset prediction end condition is met, and output the optimization sequence according to each target preset and non-graph optimization algorithm.

[0098] In one embodiment, the above-mentioned optimization module 504 is specifically used to use the first target preset and non-graph optimization algorithm in the optimization sequence as the current algorithm to be executed; optimize the non-graph to be optimized according to the current algorithm to be executed to obtain a new non-graph to be optimized; obtain the next target preset and non-graph optimization algorithm corresponding to the current algorithm to be executed in the optimization sequence as the new current algorithm to be executed; return to the step of optimizing the non-graph to be optimized according to the current algorithm to be executed, until all the target preset and non-graph optimization algorithms in the optimization sequence are executed, and the optimized non-graph is obtained.

[0099] In one embodiment, the acquisition module 500 is specifically used to acquire graph embedding features corresponding to the non-graph; extract node embedding features according to each node in the non-graph; and obtain embedding features corresponding to the non-graph according to the graph embedding features and the node embedding features.

[0100] Each module in the above-mentioned reinforcement learning-based AND-non-graph optimization device can be implemented in whole or in part by software, hardware, and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0101] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG6. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store non-graphic data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a text evaluation method is implemented.

[0102] Those skilled in the art will understand that the structure shown in FIG. 6 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0103] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the above-mentioned reinforcement learning and non-graph optimization method is implemented.

[0104] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned reinforcement learning and non-graph optimization method is implemented.

[0105] In one embodiment, a computer program product is provided, including a computer program, which implements the above-mentioned reinforcement learning and non-graph optimization method when executed by a processor.

[0106] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0107] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0108] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0109] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A circuit and non-graph optimization method based on reinforcement learning, characterized in that: The method comprises: Acquire a non-AND graph to be optimized, and extract embedding features corresponding to the non-AND graph; the embedding features include graph embedding features and node embedding features corresponding to the non-AND graph; the graph embedding features include features that convert the non-AND graph data into a low-dimensional vector representation and retain the structure of the original graph and the relationship information between nodes; the node embedding features represent the feature of mapping the nodes in the non-AND graph to a low-dimensional vector space to retain the structural information of the nodes in the network; The embedded features are input into a trained reinforcement learning model, and the reinforcement learning model determines the multiple states corresponding to the non-graph and the target preset and non-graph optimization algorithms corresponding to the states according to the embedded features and the preset logic optimization tool, and outputs an optimization sequence according to the target preset and non-graph optimization algorithms, including: after the embedded features are input into the trained reinforcement learning model, the current state corresponding to the non-graph is determined based on the embedded features, and for each preset and non-graph optimization algorithm in the preset logic optimization tool, the simulation execution result corresponding to the preset and non-graph optimization algorithm in the current state is obtained, and the action probability corresponding to the preset and non-graph optimization algorithm is determined according to the current state and the simulation execution result, and the action probability corresponding to the preset and non-graph optimization algorithm is determined according to the preset and non-graph optimization algorithm with the largest action probability. The preset and non-graph optimization algorithm is used to obtain the target preset and non-graph optimization algorithm corresponding to the current state, and the corresponding target preset and non-graph optimization algorithm is executed to obtain the new current state corresponding to the non-graph; returning to the step of determining the corresponding target preset and non-graph optimization algorithm from the preset logic optimization tool according to the current state until the preset prediction end condition is met, and outputting the optimization sequence according to each of the target preset and non-graph optimization algorithms; the reinforcement learning model is trained in the preset reinforcement learning framework based on the non-graph sample and the preset logic optimization tool; the preset logic optimization tool includes multiple preset and non-graph optimization algorithms; the multiple preset and non-graph optimization algorithms include a balance optimization algorithm and a rewrite optimization algorithm; the reinforcement learning model is a deep Q network model; According to the order of each target preset and non-graph optimization algorithm in the optimization sequence, the non-graph is optimized, including: taking the first target preset and non-graph optimization algorithm in the optimization sequence as the current algorithm to be executed; optimizing the non-graph to be optimized according to the current algorithm to be executed to obtain a new non-graph to be optimized; obtaining the next target preset and non-graph optimization algorithm corresponding to the current algorithm to be executed in the optimization sequence as the new current algorithm to be executed; returning to the step of optimizing the non-graph to be optimized according to the current algorithm to be executed until each target preset and non-graph optimization algorithm in the optimization sequence is executed to obtain an optimized non-graph; the optimized non-graph is optimal in terms of complexity, area, and delay parameters; wherein the area refers to the area occupied by the logic circuit in the chip after the process mapping stage, and the delay refers to the unit time length required for the logic circuit from input to output in the chip after the process mapping stage.

2. The method according to claim 1, characterized in that The method further comprises: According to the preset logic optimization tool, a preset reinforcement learning framework is constructed; Obtaining a reinforcement learning model to be trained, a non-graph sample, and an optimized sequence sample corresponding to the non-graph sample, and extracting an embedded feature sample corresponding to the non-graph sample; In the preset reinforcement learning framework, the embedded feature sample is input into the reinforcement learning model, and the reinforcement learning model determines the current state corresponding to the non-graph sample based on the embedded feature sample; According to the current state, determining a corresponding preset and non-graph optimization algorithm sample from a plurality of preset and non-graph optimization algorithms in the preset logic optimization tool; Execute the preset and non-graph optimization algorithm sample to obtain a new current state corresponding to the non-graph sample; Returning to the step of determining a corresponding preset and non-graph optimization algorithm sample from a plurality of preset and non-graph optimization algorithms in the preset logic optimization tool according to the current state, until a preset prediction end condition is met, outputting optimization sequence prediction information according to each of the preset and non-graph optimization algorithm samples; The optimization sequence prediction information and the optimization sequence samples are input into a preset loss function, and the model parameters of the reinforcement learning model are adjusted according to the output value of the preset loss function until the preset training end condition is met, thereby obtaining the trained reinforcement learning model.

3. The method according to claim 2, characterized in that Determining a corresponding preset and non-graph optimization algorithm sample from a plurality of preset and non-graph optimization algorithms in the preset logic optimization tool according to the current state includes: For each preset and non-graph optimization algorithm in the preset logic optimization tool, obtaining a simulation execution result corresponding to the preset and non-graph optimization algorithm in the current state; Determining the action probability corresponding to the preset non-graph optimization algorithm according to the current state and the simulation execution result; According to the preset and non-graph optimization algorithm with the largest action probability among the preset and non-graph optimization algorithms, a preset and non-graph optimization algorithm sample corresponding to the current state is obtained.

4. A circuit and non-graph optimization device based on reinforcement learning, characterized in that: The device comprises: An acquisition module is used to acquire the NAND graph to be optimized and extract the embedding features corresponding to the NAND graph; the embedding features represent the graph embedding features and node embedding features corresponding to the NAND graph; the graph embedding features represent the features of converting the NAND graph data into a low-dimensional vector representation and retaining the structure of the original graph and the relationship information between nodes; the node embedding features represent the features of mapping the nodes in the NAND graph to a low-dimensional vector space to retain the structural information of the nodes in the network; A prediction module is used to input the embedded features into a trained reinforcement learning model, and the reinforcement learning model determines the multiple states corresponding to the non-graph and the respective target preset and non-graph optimization algorithms corresponding to the respective states according to the embedded features and the preset logic optimization tool, and outputs an optimization sequence according to the respective target preset and non-graph optimization algorithms, including: after inputting the embedded features into the trained reinforcement learning model, determining the current state corresponding to the non-graph based on the embedded features, obtaining the simulation execution result corresponding to the preset and non-graph optimization algorithm in the preset logic optimization tool under the current state, determining the action probability corresponding to the preset and non-graph optimization algorithm according to the current state and the simulation execution result, and outputting an optimization sequence according to the respective action in the preset and non-graph optimization algorithm. The preset and non-graph optimization algorithm with the highest probability is used to obtain the target preset and non-graph optimization algorithm corresponding to the current state, and the corresponding target preset and non-graph optimization algorithm is executed to obtain the new current state corresponding to the non-graph; returning to the step of determining the corresponding target preset and non-graph optimization algorithm from the preset logic optimization tool according to the current state until the preset prediction end condition is met, and outputting the optimization sequence according to each of the target preset and non-graph optimization algorithms; the reinforcement learning model is trained in the preset reinforcement learning framework based on the non-graph sample and the preset logic optimization tool; the preset logic optimization tool includes multiple preset and non-graph optimization algorithms; the multiple preset and non-graph optimization algorithms include a balance optimization algorithm and a rewrite optimization algorithm; the reinforcement learning model is a deep Q network model; An optimization module is used to optimize the NAND graph according to the order of each target preset and NAND graph optimization algorithm in the optimization sequence, specifically used to: use the first target preset and NAND graph optimization algorithm in the optimization sequence as the current algorithm to be executed; optimize the NAND graph to be optimized according to the current algorithm to be executed to obtain a new NAND graph to be optimized; obtain the next target preset and NAND graph optimization algorithm corresponding to the current algorithm to be executed in the optimization sequence as the new current algorithm to be executed; return to the step of optimizing the NAND graph to be optimized according to the current algorithm to be executed until each target preset and NAND graph optimization algorithm in the optimization sequence is executed to obtain an optimized NAND graph; the optimized NAND graph is optimal in terms of complexity, area and delay parameters; wherein the area refers to the area occupied by the logic circuit in the chip after the process mapping stage, and the delay refers to the unit time length required for the logic circuit from input to output in the chip after the process mapping stage.

5. The device according to claim 4, characterized in that The device also includes a training module, which is used to: According to the preset logic optimization tool, a preset reinforcement learning framework is constructed; the preset logic optimization tool includes a plurality of preset and non-graph optimization algorithms; Obtaining a reinforcement learning model to be trained, a non-graph sample, and an optimized sequence sample corresponding to the non-graph sample, and extracting an embedded feature sample corresponding to the non-graph sample; In the preset reinforcement learning framework, the embedded feature sample is input into the reinforcement learning model, and the reinforcement learning model determines the current state corresponding to the non-graph sample based on the embedded feature sample; According to the current state, determining a corresponding preset and non-graph optimization algorithm sample from a plurality of preset and non-graph optimization algorithms in the preset logic optimization tool; Execute the preset and non-graph optimization algorithm sample to obtain a new current state corresponding to the non-graph sample; Returning to the step of determining a corresponding preset and non-graph optimization algorithm sample from a plurality of preset and non-graph optimization algorithms in the preset logic optimization tool according to the current state, until a preset prediction end condition is met, outputting optimization sequence prediction information according to each of the preset and non-graph optimization algorithm samples; The optimization sequence prediction information and the optimization sequence samples are input into a preset loss function, and the model parameters of the reinforcement learning model are adjusted according to the output value of the preset loss function until the preset training end condition is met, thereby obtaining the trained reinforcement learning model.

6. The device according to claim 5, characterized in that The training module is specifically used for: For each preset and non-graph optimization algorithm in the preset logic optimization tool, obtaining a simulation execution result corresponding to the preset and non-graph optimization algorithm in the current state; Determining the action probability corresponding to the preset non-graph optimization algorithm according to the current state and the simulation execution result; According to the preset and non-graph optimization algorithm with the largest action probability among the preset and non-graph optimization algorithms, a preset and non-graph optimization algorithm sample corresponding to the current state is obtained.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 3 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images