Method, system and device for routing optimization of network-on-chip in vlsi design

Abstract

The application discloses a routing optimization method, system and device of a network on chip in a VLSI design, relates to the field of reinforcement learning and operation scheduling, utilizes the advantages of a deep reinforcement learning model and an evolutionary algorithm, simultaneously optimizes multiple targets of a system, and trains data collected or generated by the deep reinforcement learning model and the evolutionary algorithm, to generate a preliminary solution of a multi-target problem, and then to generate a group of approximate optimal Pareto frontiers through decomposition and synchronous optimization of the evolutionary algorithm, to provide feasible solutions for on-chip communication of different targets. The model and the algorithm of the deep reinforcement learning are used to let model parameters autonomously fit and learn a path selection strategy by a black box training method, so that an optimal communication scheme can be calculated without a large time cost, and efficient communication is realized.

Description

Technical Field

[0001] This invention relates to the fields of reinforcement learning and operations research and scheduling, and in particular to a routing optimization method, system and device for on-chip networks in VLSI design. Background Technology

[0002] Currently, with the development of computer intelligence, natural language processing (NLP) and operations research technologies have seen significant advancements and applications. NLP technologies have been validated and used in machine translation, information extraction and summarization, text classification and sentiment analysis, and speech recognition. Operations research technologies mainly include dynamic programming, heuristic algorithms, exact algorithms, and the emerging machine learning approach for solving operations research problems. However, when dealing with large-scale multi-objective optimization problems, traditional methods have gradually revealed shortcomings such as slow execution speed, low efficiency in handling high-dimensional problems, and the need for expert strategies. Therefore, a new algorithm capable of overcoming these problems is needed for solving multi-objective optimization problems.

[0003] In recent decades, the rapid development of Very Large Scale Integration (VLSI) technology has enabled the integration of an increasing number of IP components onto a single chip. Multi-core architectures have become a key explanation for the ever-growing demands for system performance. To leverage the advantages of this multi-core architecture and the ever-shrinking feature size of electronic circuits, sophisticated communication between cores is essential. Networks on-chip (NOC) have proven to solve architectural scalability challenges and concurrent interconnection problems. While NOC architectures possess the scalability and modularity of general-purpose networks, their high design complexity and intricate configuration make implementing general-purpose network communication schemes as on-chip communication methods extremely difficult.

[0004] Although on-chip networks currently have communication solutions, the complexity of the NOC architecture means that calculating the optimal solution always requires a significant time cost, such as:

[0005] 1. Routing Complexity: A NOC architecture may contain a large number of nodes and routers, each of which needs to be able to independently find the optimal path to forward data packets. This requires complex path calculation and selection algorithms to ensure good performance under high load.

[0006] 2. Physical Layout: NOC architecture typically involves the layout of numerous on-chip resources, including nodes, routers, and interconnects. To implement on-chip communication, precise planning of the physical layout is required, including location and connection methods, to ensure reliable signal transmission and minimal interference.

[0007] In summary, how to calculate the optimal solution without incurring significant time costs, thereby enabling efficient communication, has become a pressing issue for those skilled in the art. Summary of the Invention

[0008] The purpose of this invention is to provide a routing optimization method, system, and device for on-chip networks in VLSI design, which can calculate the optimal solution without incurring a large time cost, thereby enabling efficient communication.

[0009] To achieve the above objectives, the present invention provides the following solution:

[0010] A routing optimization method for on-chip networks in VLSI design, the method comprising:

[0011] Step S1: Obtain the target of the entire communication process of the on-chip network; the target includes f1 and f2; f1 represents the sum of the path distances that the signal travels from the source router to the target router when it visits all routers and reaches the target router under the current router distribution; f2 represents the total power cost required to travel f1.

[0012] Step S2: Based on the goal of the entire communication process of the on-chip network, generate a set of uniformly distributed weight vectors to divide the original multi-objective routing optimization problem into a set of multi-objective sub-problems, thus obtaining a set of multi-objective sub-problems; the original multi-objective routing optimization problem is to minimize f1 and f2.

[0013] Step S3: Select one multi-objective sub-problem from the set of multi-objective sub-problems as the problem that needs to be trained and solved;

[0014] Step S4: Generate a set of random data for the problem that needs to be trained and solved, and process the generated random data to generate the initial edge embedding graph data;

[0015] Step S5: Use an encoder to process the initial edge embedding graph data, extract the information features from the initial edge embedding graph data, and finally integrate them into graph embedding data;

[0016] Step S6: Generate the context embeddings required by the decoder;

[0017] Step S7: Use the decoder to combine the context embedding and the graph embedding data to solve the selection probability of each candidate edge at the current time step, and select the next edge according to the selection strategy;

[0018] Step S8: Determine whether a communication scheme has been established.

[0019] If the output of step S8 is negative, then return to step S6;

[0020] If the output of step S8 is yes, then proceed to step S9: obtain the objective function value of the multi-objective sub-problem based on the weight coefficients of the multi-objective sub-problem, calculate the loss function value, and train the model corresponding to the multi-objective sub-problem using the policy gradient algorithm.

[0021] Step S10: Determine whether the current loss function value has reached the optimal state;

[0022] If the output of step S10 is negative, then return to step S4;

[0023] If the output of step S10 is yes, then proceed to step S11: determine whether the multi-objective sub-problems have been trained.

[0024] If the output of step S11 is negative, then return to step S3;

[0025] If the output of step S11 is yes, then proceed to step S12: input the test data into the model set to generate an initial population; the model set includes the model corresponding to each multi-objective sub-problem;

[0026] Step S13: Take the current population as the parent population and select an individual i in the population as the object of the current operation;

[0027] Step S14: Perform gene crossover operation based on the model parameters of individuals i-1 and i+1 adjacent to individual i to obtain a new individual i_new;

[0028] Step S15: Determine whether the new individual i_new satisfies the mutation condition;

[0029] If the output of step S15 is yes, then proceed to step S16: perform gene mutation operation on the new individual i_new based on the gradient of the attention compatibility score obtained from the test data during model operation;

[0030] If the output of step S15 is negative, then proceed to step S17: compare the new individual i_new with individual i-1, individual i, and individual i+1, and perform corresponding population updates and Pareto optimal set updates.

[0031] Step S18: Determine whether the maximum number of iterations for population evolution has been reached;

[0032] If the output of step S18 is negative, then return to step S13;

[0033] If the output of step S18 is yes, then the final Pareto optimal solution set is output as the optimal communication scheme; the optimal communication scheme minimizes f1 and f2.

[0034] Optionally, the context embedding required by the generator decoder specifically includes:

[0035] If the current node is the source router of the network, the initial context embedding is generated through the LSTM network; otherwise, the current edge embedding information is input into the LSTM network and the corresponding context embedding is updated using the multi-head attention layer.

[0036] Optionally, the selected strategy is a random sampling strategy or a greedy strategy.

[0037] Optionally, determining whether a communication scheme has been established specifically includes:

[0038] The judgment criteria determine whether a communication scheme has been formed. The judgment criteria are whether a feasible path has been formed between the source node and the target node. If a feasible path has been formed between the source node and the target node, it is determined that a communication scheme has been formed. If no feasible path has been formed between the source node and the target node, it is determined that a communication scheme has not been formed.

[0039] Optionally, determining whether the new individual i_new satisfies the mutation condition specifically includes:

[0040] The mutation condition of a new individual i_new is determined by whether the random number is greater than the mutation probability. When the random number is greater than the mutation probability, the new individual i_new meets the mutation condition and needs to undergo gene mutation. When the random number is less than the mutation probability, the new individual i_new does not meet the mutation condition and does not need to undergo gene mutation. The range of the random number is 0 to 1, and the mutation probability is 0.9.

[0041] The present invention also provides the following solutions:

[0042] A routing optimization system for on-chip networks in VLSI design, the system comprising:

[0043] The target acquisition module is used to acquire the target of the entire communication process of the on-chip network; the target includes f1 and f2; f1 represents the sum of the path distances that the signal travels from the source router to the target router when it visits all routers and reaches the target router under the current router distribution; f2 represents the total power cost required to travel f1.

[0044] The sub-problem partitioning module is used to generate a set of uniformly distributed weight vectors based on the goal of the entire communication process of the on-chip network to divide the original multi-objective routing optimization problem into a set of multi-objective sub-problems, thus obtaining a set of multi-objective sub-problems; the original multi-objective routing optimization problem is to minimize f1 and f2.

[0045] The sub-problem selection module is used to select one multi-objective sub-problem from the set of multi-objective sub-problems as the problem that needs to be trained and solved at the present time;

[0046] The edge embedding graph data generation module is used to generate a set of random data for the current problem that needs to be trained and solved, and to process the generated random data to generate the initial edge embedding graph data.

[0047] The graph embedding data generation module is used to process and manipulate the initial edge embedding graph data using an encoder, extract information features from the initial edge embedding graph data, and finally integrate them into graph embedding data.

[0048] The context embedding generation module is used to generate the context embeddings required by the decoder;

[0049] The next edge selection module is used to use the decoder to combine the context embedding and the graph embedding data to solve the selection probability of each candidate edge at the current time step, and select the next edge according to the selection strategy.

[0050] The first judgment module is used to determine whether a communication scheme has been formed.

[0051] The first return module is used to return to the context embedding generation module when the output result of the first judgment module is negative;

[0052] The model training module is used to obtain the objective function value of the multi-objective sub-problem based on the weight coefficients of the multi-objective sub-problem, calculate the loss function value, and train the model corresponding to the multi-objective sub-problem through the policy gradient algorithm when the output result of the first judgment module is yes.

[0053] The second judgment module is used to determine whether the current loss function value has reached the optimal state.

[0054] The second return module is used to return to the edge-embedded chart data generation module when the output result of the second judgment module is negative;

[0055] The third judgment module is used to determine whether the multi-objective sub-problems have been trained when the output of the second judgment module is yes.

[0056] The third return module is used to return to the sub-problem selection module when the output result of the third judgment module is negative;

[0057] The population generation module is used to input test data into the model set to generate an initial population when the output result of the third judgment module is yes; the model set includes the model corresponding to each multi-objective sub-problem.

[0058] The individual selection module is used to take the current population as the parent population and select an individual i in the population as the object of the current operation.

[0059] A new individual generation module is used to perform gene crossover operations on the model parameters of individuals i-1 and i+1 that are adjacent to individual i to obtain a new individual i_new;

[0060] The fourth judgment module is used to determine whether the new individual i_new meets the mutation conditions;

[0061] The gene mutation operation module is used to perform gene mutation operation on the new individual i_new based on the gradient of the attention compatibility score obtained by the test data in the model operation when the output result of the fourth judgment module is yes.

[0062] The update module is used to compare the new individual i_new with individuals i-1, i, and i+1 when the output of the fourth judgment module is negative, and to perform corresponding population updates and Pareto optimal set updates.

[0063] The fifth judgment module is used to determine whether the maximum number of iterations for population evolution has been reached;

[0064] The fourth return module is used to return to the individual selection module when the output result of the fifth judgment module is negative;

[0065] The optimal communication scheme output module is used to output the final Pareto optimal solution set as the optimal communication scheme when the output result of the fifth judgment module is yes; the optimal communication scheme minimizes f1 and f2.

[0066] In another aspect, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the routing optimization method for on-chip network in the VLSI design.

[0067] Optionally, the memory is a non-transitory computer-readable storage medium.

[0068] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0069] This invention discloses a routing optimization method, system, and device for on-chip networks in VLSI design. Leveraging the advantages of deep reinforcement learning models and evolutionary algorithms, it simultaneously optimizes multiple system objectives. Deep reinforcement learning and evolutionary algorithms train the model with collected or self-generated data to first generate preliminary solutions to the multi-objective problem. Then, through decomposition and simultaneous optimization using evolutionary algorithms, a set of near-optimal Pareto fronts is generated, providing feasible solutions for on-chip communication with different objectives. This invention designs a multi-head attention mechanism model combining graph attention networks and edge features based on the problem to be solved, and uses reinforcement learning... The training model quickly obtains the initial solution to the problem in an end-to-end manner, significantly improving the speed of problem solving and providing new possibilities for solving large-scale integrated circuit design. This invention combines a multi-task evolutionary algorithm, utilizing the similarity between individuals in the population for knowledge learning. Secondly, it uses the attention compatibility score gradient generated when the model solves test cases to update individuals, thereby further improving the quality of the final solution. This invention uses a deep reinforcement learning model and algorithm to train the model parameters autonomously to learn the path selection strategy through black-box training, calculating the optimal communication scheme without a large time cost, thus achieving efficient communication. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 This is a flowchart of a routing optimization method for on-chip networks in VLSI design according to the present invention;

[0072] Figure 2 This is a structural diagram of the model of the present invention;

[0073] Figure 3 This is a flowchart of the algorithm of the present invention. Detailed Implementation

[0074] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0075] The purpose of this invention is to provide a routing optimization method, system, and device for on-chip networks in VLSI design, which can calculate the optimal solution without incurring a large time cost, thereby enabling efficient communication.

[0076] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0077] Figure 1 This is a flowchart illustrating a routing optimization method for on-chip networks in VLSI design according to the present invention. Figure 1 As shown, this invention provides a routing optimization method for on-chip networks in VLSI design, comprising the following steps:

[0078] Step S1: Obtain the target of the entire communication process of the on-chip network; the target includes f1 and f2; f1 represents the sum of the path distances the signal travels from the source router to the target router when it visits all routers under the current router distribution, and f2 represents the total power cost required to travel f1.

[0079] Step S2: Based on the goal of the entire communication process of the on-chip network, generate a set of uniformly distributed weight vectors to divide the original multi-objective routing optimization problem into a set of multi-objective sub-problems, thus obtaining a set of multi-objective sub-problems; the original multi-objective routing optimization problem is to minimize f1 and f2.

[0080] Step S3: Select one multi-objective sub-problem from the set of multi-objective sub-problems as the problem that needs to be trained and solved.

[0081] Step S4: Generate a set of random data for the problem that needs to be trained and solved, and process the generated random data to generate the initial edge embedding graph data.

[0082] Step S5: Use the encoder to process and refine the initial edge embedding graph data, extract the information features from the initial edge embedding graph data, and finally integrate them into graph embedding data.

[0083] Step S6: Generate the context embeddings required by the decoder.

[0084] Step S6 specifically includes:

[0085] If the current node is the source router of the network, the initial context embedding is generated through the LSTM network; otherwise, the current edge embedding information is input into the LSTM network and the corresponding context embedding is updated using the multi-head attention layer.

[0086] Step S7: Use the decoder to combine context embedding and graph embedding data to solve the selection probability of each candidate edge at the current time step, and select the next edge according to the selection strategy.

[0087] In step S7, the strategy selected is either a random sampling strategy or a greedy strategy.

[0088] Step S8: Determine whether a communication scheme has been established.

[0089] Step S8 specifically includes:

[0090] The judgment criteria determine whether a communication scheme has been formed. The judgment criteria are whether a feasible path has been formed between the source node and the target node. If a feasible path has been formed between the source node and the target node, a communication scheme has been formed. If no feasible path has been formed between the source node and the target node, a communication scheme has not been formed.

[0091] If the output of step S8 is negative, then return to step S6.

[0092] If the output of step S8 is yes, then proceed to step S9: obtain the objective function value of the multi-objective sub-problem based on the weight coefficients of the multi-objective sub-problem, calculate the loss function value, and train the model corresponding to the multi-objective sub-problem using the policy gradient algorithm.

[0093] Step S10: Determine whether the current loss function value has reached the optimal state.

[0094] If the output of step S10 is negative, then return to step S4.

[0095] If the output of step S10 is yes, then proceed to step S11: determine whether the multi-objective sub-problems have been trained.

[0096] If the output of step S11 is negative, then return to step S3.

[0097] If the output of step S11 is yes, then proceed to step S12: input the test data into the model set to generate the initial population; the model set includes the model corresponding to each multi-objective sub-problem.

[0098] Step S13: Use the current population as the parent population and select an individual i in the population as the object of the current operation.

[0099] Step S14: Perform gene crossover operation based on the model parameters of individuals i-1 and i+1 adjacent to individual i to obtain a new individual i_new.

[0100] Step S15: Determine whether the new individual i_new satisfies the mutation condition.

[0101] Step S15 specifically includes:

[0102] The mutation condition of a new individual i_new is determined by whether the random number is greater than the mutation probability. When the random number is greater than the mutation probability, the new individual i_new meets the mutation condition and needs to undergo gene mutation. When the random number is less than the mutation probability, the new individual i_new does not meet the mutation condition and does not need to undergo gene mutation. The range of the random number is 0 to 1, and the mutation probability is 0.9.

[0103] If the output of step S15 is yes, then proceed to step S16: perform gene mutation operation on the new individual i_new based on the gradient of the attention compatibility score obtained from the test data during model operation.

[0104] If the output of step S15 is negative, then proceed to step S17: compare the new individual i_new with individuals i-1, i, and i+1, and perform corresponding population updates and Pareto optimal set updates.

[0105] Step S18: Determine whether the maximum number of iterations for population evolution has been reached.

[0106] If the output of step S18 is negative, then return to step S13.

[0107] If the output of step S18 is yes, then proceed to step S19: output the final Pareto optimal solution set as the optimal communication scheme; the optimal communication scheme minimizes f1 and f2.

[0108] The technical solution of the present invention is illustrated below with a specific embodiment:

[0109] This invention proposes a routing optimization algorithm (method) for Network on-Chip (NOC) in VLSI design. This algorithm leverages the advantages of both deep reinforcement learning models and evolutionary algorithms to simultaneously optimize multiple system objectives, such as minimum path length for packet transmission, minimum coupling capacitance, and minimum power. The combination of these two approaches has high practical value in VLSI design, as it can automatically learn and optimize routing strategies, significantly improving the efficiency and performance of VLSI design. Furthermore, this method can adapt to different network routing layouts and requirements, exhibiting strong robustness and scalability. Specifically, the deep reinforcement learning and evolutionary algorithm components of this invention involve: training a deep reinforcement learning model with collected or self-generated data to first generate preliminary solutions to the multi-objective problem; then, through evolutionary algorithm decomposition and simultaneous optimization, generating a set of near-optimal Pareto fronts, providing feasible solutions for on-chip communication with different objectives.

[0110] This invention provides an on-chip network routing algorithm design based on edge feature graph attention networks. Primarily targeting the routing design of on-chip networks in VLSI, it proposes a multi-objective routing optimization algorithm combining deep reinforcement learning and evolutionary algorithms. This invention plans paths and power in multi-objective routing problems through two main stages: "offline learning and real-time application." Furthermore, this invention provides a method that uses a deep reinforcement learning model and algorithm with black-box training to allow the model parameters to autonomously fit and learn path selection strategies, achieving efficient communication.

[0111] To achieve the above objectives, this invention combines the advantages of deep reinforcement learning and evolutionary algorithms, designing a multi-objective combinatorial optimization algorithm that integrates a graph attention network structure with an edge attention mechanism and an evolutionary algorithm. The invention mainly includes the following technical points: 1. Dividing the original multi-objective routing problem into multiple sub-problems according to requirements, and mathematically modeling each sub-problem based on its actual situation (using Visual Studio Code and Python as the programming language), and determining the optimization objective; 2. Introducing deep reinforcement learning to train the problem model (i.e., the model obtained by mathematically modeling each sub-problem based on its actual situation), and through continuous learning and optimization strategies, ultimately obtaining a set of models capable of solving preliminary solutions; 3. Using the trained model parameters and combining them with a multi-task optimization algorithm to optimize and obtain the Pareto optimal solution set for the multi-objective routing problem.

[0112] The phrase "dividing the original multi-objective routing problem into multiple sub-problems according to requirements" refers to the allocation of different objective weight vectors. Assume there are two objectives, f1 and f2, but these objectives conflict (i.e., optimizing f2 cannot be achieved while optimizing f1). Furthermore, different people have different preferences; for example, some people prioritize objective f1 while others prefer objective f2. Therefore, preference weights λ are introduced. For the two objectives, there are two weights, λ1 and λ2, where λ1 + λ2 = 1. The larger weight indicates greater preference. Since different combinations of preference weights represent different needs, a Pareto optimal solution set is provided to satisfy the different needs of different people for the final objective. This solution set contains many final solutions with different preference weights, representing the optimal solution for each weight.

[0113] The original multi-objective routing problem refers to the communication process of an on-chip network. Specifically, the original multi-objective routing problem refers to the fact that the entire communication process of an on-chip network has two objectives, f1 and f2. The ideal optimization situation is to achieve the optimal state for both objectives, which is in the form of minF = {f1, f2}.

[0114] There are many possible optimization objectives for a problem, such as: First, the total distance the signal travels along the path in the entire communication scheme. Second, the total power consumption required for the signal to travel along the planned path in the communication scheme. Third, the total time consumed by the signal to travel along the planned path. In this invention, the first and second objectives are used as the optimization objectives.

[0115] The steps for solving the multi-objective routing optimization algorithm proposed in this invention (i.e., the steps for the routing optimization method of on-chip networks in VLSI design) are as follows:

[0116] (1) The algorithm generates a set of uniformly distributed weight vectors based on the two objectives of the problem (total path distance and total power consumption) to divide the original multi-objective routing optimization problem into a set of multi-objective sub-problems.

[0117] Among them, optimizing the total path distance and total power consumption are two objectives that enable the communication scheme obtained by the algorithm to achieve the minimum path distance (thus achieving the effect of minimum latency); the minimum total power consumption means that since the signal will pass through the routers on the planned route after route planning, each router will incur power consumption when receiving and forwarding signals, and optimizing this objective will achieve the minimum total consumption.

[0118] (2) Select one subproblem from the set of multi-objective subproblems as the problem that needs to be trained and solved.

[0119] (3) Generate a set of random data for the current subproblem, and process the data to generate the initial edge embedding graph data h. 0 Proceed to step (4).

[0120] (4) The encoder receives the edge embedding data h generated in step (3). 0 The data is processed and analyzed to extract information features and ultimately integrate them into graph embedding data h. Graph Then proceed to step (5).

[0121] The formula for embedding graph data is as follows: Since the dimensions of the embedded data generated after encoder processing are (batch_size, node_num, node_num, embedding_dim), where batch_size refers to the batch size of the generated data (the number of data instances); node_num is the number of nodes; and embedding_dim is the dimension of the embedded data, after dimension transformation, the dimensions become (batch_size, node_num * node_num, embedding_dim), where h mThis is one dimension of the transformed data, with dimensions (batch_size, 1, embedding_dim), where M equals node_num * node_num.

[0122] Encoders and decoders are techniques in the field of machine learning. The combination of the two can take the initial feature data of the input (such as the coordinate features of each router node, the power consumption required for each router to work, and the path distance between each node in this invention) and pass it through a fully connected layer (i.e., the matrix formula is y = wx + b, where x is the initial feature data, w is the matrix, and y is the initialized embedding) to obtain the initial embedding data. Then, after the encoder encodes (extracts and processes information), the final graph embedding data is obtained. The graph embedding data is then handed over to the decoder for decoding (gradually obtaining edges to build a complete communication scheme).

[0123] (5) Generate the context embedding required by the decoder. If the current node is the source router of the network (i.e., there is no edge in the current solution sequence), the initial context embedding is generated by LSTM; otherwise, the current edge embedding information (i.e., the edge embedding information corresponding to the previous selected edge) is input into LSTM, and the corresponding context embedding is updated using the multi-head attention layer (MHA). Proceed to step (6).

[0124] The encoder is connected to the LSTM, the LSTM is connected to the multi-head attention layer (the multi-head attention layer is connected after the LSTM network), and the multi-head attention layer is connected to the decoder.

[0125] The encoder consists of the following modules: a fully connected layer (for initializing feature data), a multi-head attention layer, a feedforward layer (for further processing of the data after MHA processing), and a data normalization operation (for data normalization and unification operations).

[0126] The decoder consists of the following parts: a multi-head attention layer (used to calculate the compatibility score between the currently selected edge and its neighbors), and a layer that uses the calculated compatibility score to derive the selection probability of its neighbors (specifically implemented using the softmax function).

[0127] An LSTM network exists between the encoder and decoder, which is used to derive the context embedding. The context embedding is used to let the decoder know which edge is currently selected and what the neighboring edges of the current edge are. This information is used as context until the decoder performs decoding (to obtain the probability).

[0128] The overall operation flow is as follows: 1. Feed the encoder data to process it and finally produce graph embedding data (the encoder is used once in a complete process). 2. Use an LSTM network to obtain the context embedding (initially, this context embedding is also obtained by LSTM but is random) and feed the context embedding to the decoder. 3. Use the context embedding and the graph embedding data given by the encoder to perform step-by-step decoding to obtain the probability of selecting the next edge for the current edge. 4. Select the next edge according to the selection strategy (there are two strategies: selecting based on the maximum probability and random selection). 5. Determine whether the current communication scheme is formed (i.e., whether the signal can travel from the source node to the target node). If the construction is completed, calculate the loss function value of this scheme (based on the two objectives f1 and f2 and the weights assigned to them: F = λ1f1 + λ2f2) and update the gradient accordingly; if a complete communication scheme is not formed, return to step 2.

[0129] (6) The decoder combines context embedding and graph embedding information to solve the selection probability p of each candidate edge at the current time step t, selects the next edge according to the selected strategy (random sampling strategy or greedy strategy), and proceeds to step (7).

[0130] (7) Determine whether a communication scheme has been formed (the criterion for judgment is whether a feasible path has been formed between the source node and the target node). If yes, proceed to step (8); otherwise, return to step (5).

[0131] (8) Obtain the objective function value of the subproblem based on the weight coefficients of the subproblem, calculate the loss function value, and train the model using the Policy Gradient algorithm (each subproblem divided by different weight coefficients will correspond to a model). Proceed to step (9).

[0132] Each subproblem represents a problem under different preference weights. That is, before the problem was divided, the formula was minF = {f1, f2}, and after dividing into subproblems, it becomes minF... i =λ1f1 + λ2f2. Therefore, different weight preferences will result in different subproblems F. i The weighting coefficients are randomly generated at the beginning of the program to ensure that both weighting coefficients are greater than 0 and less than 1, and that they add up to 1.

[0133] (9) Determine whether the current loss function value has reached the optimal situation (when solving the gradient of the objective function of each subproblem, the case where λ1f1+λ2f2 is minimized is the optimal situation). If the optimal situation has been reached, proceed to step (10); otherwise, return to step (3).

[0134] (10) Determine whether the multi-objective sub-problems have been trained. If all have been trained, proceed to step (11); otherwise, return to step (2).

[0135] (11) Input the test data into the model set obtained from the model training part to generate the initial population. Proceed to step (12).

[0136] The test data can be real data, such as the coordinates and power consumption between routers in a real-world network-on-a-chip. However, the cost of obtaining real-world data may be very high. Therefore, randomly generated data (i.e., randomly generated coordinates and power consumption of each node) can be used here. Reinforcement learning does not have labeled data.

[0137] (12) Take the current population as the parent population and select an individual i in the population as the object of the current operation. Proceed to step (13).

[0138] (13) Perform gene crossover operation on the model parameters of the two adjacent individuals i-1 and i+1 of individual i (since there is similarity between adjacent multi-objective sub-problems, transfer learning can be performed) to obtain a new individual i_new. Proceed to step (14).

[0139] (14) Determine whether individual i_new meets the mutation condition (in this invention, it is determined by whether the random number is greater than the mutation probability, and the mutation probability is set to 0.9). If the mutation condition is met, proceed to step (15); otherwise, proceed directly to step (16).

[0140] The random number is user-defined (ranging from 0 to 1), and the mutation probability is also user-defined, for example, 0.9. In this code step, a random number is set, such as a = random(). When this random number a is greater than 0.9, it means that the individual i_new needs to undergo mutation; if a < 0.9, it means that mutation is not needed.

[0141] (15) Perform gene mutation operation on individual i_new based on the gradient of the attention compatibility score obtained during the model run of the test sample (test data) (the final output of the decoder). Proceed to step (16).

[0142] The decoder will generate a compatibility score, and then perform gradient calculation on this compatibility score to obtain the gradient magnitude of each compatibility score. The larger the gradient, the greater the probability of its change. Then, this characteristic is used in conjunction with the subsequent formula to perform gene mutation operations.

[0143] (16) Compare the new individual i_new with individuals i-1, i, i+1, and perform corresponding population updates and Pareto optimal set updates. Proceed to step (17).

[0144] (17) Determine whether the maximum number of iterations for population evolution has been reached. If the maximum number of iterations has been reached, output the final Pareto optimal solution set; otherwise, return to step (12).

[0145] In simple terms, this invention first divides the problem into sub-problems, then trains each sub-model, and inputs test data into each sub-model to obtain the corresponding initial optimal solutions. Next, an evolutionary algorithm is used to mutate the parameters of each model and compare them to retain the better individuals (this step requires multiple iterations). After adjustment, the algorithm framework collects the solutions from all models and combines them into a Pareto optimal front for output. The individuals here are essentially the same models from before, but with some parameters changed.

[0146] To more clearly illustrate the content of this invention, the relevant methods of this invention are described in detail below:

[0147] The ultimate goal of this invention is to construct an algorithmic structure that combines deep reinforcement learning and multi-task evolutionary algorithms, wherein the network model is as follows: Figure 2 As shown, the algorithm flow is as follows: Figure 3 As shown, the algorithm utilizes the "offline learning, online allocation" characteristic of deep reinforcement learning to quickly derive network routing plans for test samples using the trained model parameters, and then uses a multi-task evolutionary algorithm to perform population evolution to obtain a higher-quality planning solution.

[0148] (i) In this invention, f1 and f2 represent two objectives. The two objective functions of the multi-objective routing optimization problem to be solved (optimizing the objective function composed of the total distance path and total power consumption under different preferences, ultimately giving the Pareto front) can be expressed as:

[0149]

[0150]

[0151] In the two objective function formulas above, f1 represents the path distance dis from the source router to the target router, after visiting all routers and reaching the target router, given the current router distribution. i The sum, assuming a complete path that visits all routers is: E = {e1, e2, ..., e...} N}, where e i It is an edge, and |e i |=dis iAnd it is required that each edge in E is connected to a router that is visited exactly once, i.e., ∑ i∈V y ij =1,∑ k∈V y jk =1, where y ij It is a binary quantity, i.e., a Boolean value, indicating whether to choose to travel from router (node) i to router (node) j. If the choice is made to travel from node i to node j, then y ij =1, otherwise y ij =0, y jk Similarly, k represents any other node in the current router node set excluding nodes i and j, V represents the current router node set, and N represents the total number of edges in E. The sum of all edges in E then yields the final f1, as shown in the formula. medium power i This represents the total power cost required to complete a certain path. Therefore, the model for the multi-objective path optimization problem in this invention is defined as min F = {f1, f2}. It is decomposed into several scalar subproblems using a set of uniformly distributed weight vectors, each subproblem having its corresponding objective function. (in These are the weight coefficients of the two objective functions corresponding to the i-th scalar quantum problem. Different weight values ​​indicate the focus of each subproblem; some emphasize the distance the signal travels, some prioritize total power consumption, and some combine both. The subproblem objective function is optimized using the following formula:

[0152]

[0153] stπ∈Ω

[0154] Among them, g te Here, is the representation of the objective function, g is the function name, te indicates that g is the objective function obtained using the Chef-Shabi decomposition strategy, and λ represents the weight preferences used in this function. It is a reference point. That is, z * It is an m-dimensional reference vector consisting of the minimum values ​​of all objective functions, where m represents the number of objective functions, T represents the transpose, and f i (π) is the i-th objective function value obtained by solving π (here referring to the sequence of accessing the router), where x and π both represent solutions, and Ω represents the solution space, which can contain many x or π and is the set space of solutions; This represents the weight value of the i-th objective function in the j-th subproblem. Proceed to step (II).

[0155] (ii) Select one unselected multi-objective sub-problem from the set of multi-objective sub-problems divided in step (i) as the current training problem and proceed to step (iii).

[0156] (III) Generate a set of random data for the current subproblem. This random data is a set of data nodes in a Cartesian coordinate system that satisfy the condition of being within the identity matrix. Python tools can directly generate a batch of data between (0 and 1). The tool generates (x, y) coordinates for both the x and y coordinates of the nodes. These coordinates exist within the matrix range of (0-1) x and y coordinates in the Cartesian coordinate system, hence the term "identity matrix." The Euclidean distance between nodes in the dataset is then calculated to obtain the graph data. Finally, the graph data undergoes a preliminary linear projection to generate the initial edge-embedded graph data h. 0 The linear projection formula is as follows:

[0157]

[0158] Where, x in_i This represents the input edge feature information. `in_i` simply indicates the edge feature information corresponding to the i-th batch of data, just like the index `i`. It is the initial edge embedding of all data in the i-th sample (each sample contains a complete graph data, which includes the path distance and power consumption from each router to other routers), w 0 and b 0 These are the trainable weight parameters. Proceed to step (four).

[0159] (iv) The encoder receives the edge embedding data h generated in the previous step. 0 The edge embeddings are fed into multiple processing layers for feature extraction. Each processing layer is a neural structure represented by the following formula:

[0160]

[0161]

[0162] Among them, BN i () is a data normalization function. The superscript i indicates the i-th data operation, where i = {1, 2, ..., n}, and n is the number of processing layers. It is the initial edge embedding after linear projection. This refers to the edge embedding data after the i-th processing layer. ReLU is an activation function that sets values ​​less than zero to zero and leaves values ​​greater than zero unchanged. LN is a batch normalization function, and MHA is a function for multi-head attention operations. This represents the trainable matrix used to combine the embedding vectors obtained from MHA(). Indicates used for combination The trainable matrix of the obtained embedding vectors, Indicates used for combination The trainable matrix of the obtained embedding vectors, This represents the edge embedding data obtained after the (i-1)th processing step. After multiple processing steps, the resulting edge embeddings are averaged to obtain the graph embedding of the entire graph data, and then proceed to step (five). The formula for calculating the graph embedding is as follows:

[0163]

[0164] (v) Generate the context embedding required by the decoder at the current time step t. If the current node is the source router of the network (i.e., there are no edges in the current solution sequence), then the initial context embedding is generated using LSTM; otherwise, the current edge embedding information is input into LSTM and the corresponding context embedding is updated using a multi-head attention layer (MHA), as shown in the following formula:

[0165]

[0166]

[0167]

[0168]

[0169] Among them It is a partial solution π 1:t-1 The edge embedding sequence. At t=1, the partial solution is an empty set, therefore, g1 is a d h dimensional vector (d in this invention) h =256), which are considered as trainable parameters. All of these are trainable parameters. ij e represents the edge embedding from router i to router j. ij H represents an edge ij. t This represents the set of candidate edge embeddings for the selected edge at the current time step t, i.e., the set of neighboring edges of node i at the current time t. Since this is also a multi-head attention operation used in the decoder, a final information aggregation operation is also required during the final update of the context embedding information. This is the matrix used for information aggregation; This represents the value obtained after performing a softmax operation on z. Softmax is a probabilistic operation that yields values ​​that are all less than 1, but whose sum is equal to 1. For example, performing a softmax operation on [1, 2, 3, 4] will result in the values ​​[1 / 10, 2 / 10, 3 / 10, 4 / 10]. 'a' represents the current query matrix g. t After querying the key-value matrix h ij The attention probabilities (probabilistic representation of attention coefficients) obtained afterwards are... The "attention coefficient" represents the current context embedding g. t This is obtained by matrix multiplication with the "embedding of the neighboring edge" of the current edge; where i,j represent the edges from node i to node j; and m represents a number in M. Proceed to step (VI).

[0170] (vi) The decoder combines context embedding and graph embedding information to solve for the next edge. During the decoding process, this invention will use the set context embedding and the edge e selected in the previous time step. t-1 Neighbor set We calculate the selection probability of an edge in the neighbor set, and then select the edge e at the current time step according to different strategies (specifically two strategies: random sampling strategy and greedy strategy). t For context embedding, this invention utilizes edge information from the graph data (edge ​​information mainly includes distance and time features). The probability calculation process is as follows:

[0171]

[0172] Where, p θ This indicates that, given the selection of the corresponding solution from time T = 1 to time T = t-1, edge e is chosen at time T = t. t The probability of the selected edge e is a holistic concept; it doesn't simply represent the probability of a single edge, but rather the probability of the selected edge e. t-1 The set of probabilities of choosing all edges under the given condition (in a matrix or vector representation), u (c)j π represents the compatibility score between the current edge and the candidate edges in its neighboring edge set. 1:t-1 Let u represent the current solution sequence obtained from time step 1 to t-1, and u (c)j The formula is shown below:

[0173]

[0174] In the formula, C t That is, the context embedding of the update at time step t, h j It is an edge embedding in the adjacent edge set. Proceed to step (seven).

[0175] (vii) Determine whether a communication scheme has been formed. If so, proceed to step (viii); otherwise, return to step (v).

[0176] (viii) Calculate the objective function and loss function values ​​for the current subproblem, and train the model. This is based on the weight coefficients assigned to the current subproblem. and To calculate the objective function value Then, a greedy strategy is used to input the test data G into the model to obtain the baseline B(G). The loss function of the algorithm is then defined as: Since the test data is not from a single batch but from multiple batches, the algorithm of this invention operates simultaneously on all batches of data. This represents all test sets. The same operation is performed on each batch of data, and the final loss function value is obtained by summing the results. Afterwards, gradient optimization and updates are performed. Then, the sub-problem model is trained using the policy gradient. Proceed to step (nine).

[0177] (ix) Determine whether the current loss function value has reached the optimal situation or whether the maximum number of training iterations has been reached. If the above conditions are met, proceed to step (x); otherwise, return to step (iii).

[0178] (x) Determine whether the multi-objective sub-problems have been trained. If all multi-objective sub-problems have been trained, proceed to step (xi). Otherwise, return to step (ii).

[0179] (xi) Generate an initial population using test data and the already trained set of multi-objective sub-problem models. Because the different weight coefficients lead to differences in the objective functions of each multi-objective sub-problem, the resulting trained model parameters also differ. Therefore, each model parameter in the multi-objective sub-problem set can be used as an individual (hereafter referred to as θ). i Therefore, the initial population is represented as P = {θ0, θ1, ..., θ...} N}, where N is the total number of individuals. Proceed to step (twelve).

[0180] (xii) Take the current population P as the parent population, and select an individual θ from the population. i As the object to be operated on, proceed to step (thirteen).

[0181] (XIII) Utilize the model similarity of adjacent multi-objective sub-problems to perform gene crossover operations. Since a set of uniformly distributed weight vectors was given in step (I), the similarity between adjacent weight vectors is relatively high. Therefore, the corresponding multi-objective sub-problem model parameters will also have a certain degree of similarity. Select individuals θ obtained in step (XII). i The two adjacent individuals θ i-1 and θ i+1 Perform random operations on each of the two individuals to obtain the number of random parameter swaps, num. j j∈{i-1,i+1} and random swapping of parameter positions and θ i-1 and θ i+1 The parameters at the corresponding positions are respectively related to θ i The parameters at corresponding positions are swapped to obtain a new individual θ. i_new Proceed to step fourteen.

[0182] (XIV) Determining individual θ i_new Does the mutation condition meet? In this invention, the mutation probability of an individual is set to p. m =0.9, when a random number is generated using a random function random>p m At that time, it conforms to the individual θ i_new If the conditions for self-mutation are met, proceed to step (xv); otherwise, individual θ i_new If the conditions for self-mutation are not met, proceed directly to step (sixteen).

[0183] (xv) For individual θ i_new Perform self-mutation. Input the test data G into the individual θ obtained after gene crossover. i_new In the decoding process, an attention compatibility score table is generated for each time step. The gradients of the data in these tables are used to assist the individual θ. i_new The parameter update formula is shown below:

[0184]

[0185]

[0186] Where K is the number of edges in the adjacent edge set of each edge. Since the graph data obtained in step (iii) is fully symmetric, the number of adjacent edge sets for each edge is consistent. B is the batch size of the test data G, t is the decoding time step, which is selected from {1,2,...,T}, and π t-1,j It is instance G jIn the partial solution at time step t-1, ∈ represents the obtained coefficients. The content corresponding to ∈ is a matrix. ∈ represents the probabilities generated by storing and using the decoder during the testing process using test data (probabilities p are generated once for each batch of test data, so they need to be accumulated). k represents the selectable neighboring edges of the current edge (since there is more than one selectable neighboring edge, we assume the total number of neighboring edges is K). In summary, the coefficients here are "the values ​​of all possible neighboring edges obtained by using the gradient of the current solution probability p for all batches of test data". This represents the gradient sign with respect to the model parameters. σ indicates that the result is a matrix with the same dimensions as ∈, where the values ​​are all normally distributed data. ij Indicates the currently selected edge. This is to obtain normally distributed data with the same dimensions as u. After performing the mutation operation, proceed to step (sixteen).

[0187] (xvi) The new individual θ i_new Compared with the original individual θ i-1 ,θ i+1 Compare the advantages and disadvantages. After obtaining θ... i_new Then, the reference point is updated first, that is, for j = 1, 2, ..., m, if but The solution This refers to inputting test data x into individual θ i_new The function value of the j-th objective generated in the process is represented by the solution as follows: m represents the number of targets, and in this invention, m = 2, z j This represents the optimal value for the j-th objective. In this problem, the desired optimal value is the minimum value (obtained by inputting all test data into the objective function of the corresponding objective). Then, the neighboring solutions are updated, for each neighboring individual θ... i-1 (Here, individual θ) i-1 For example, if there is Then the individual θ i_new The parameter coverage parameter θ i-1 The parameters for individual θ i+1 This is also the case. Finally, the solution is removed from the Pareto optimal set. The dominated solution, and if there is no dominant solution in the Pareto optimal set. The solution, then... Add it to the Pareto optimal set; otherwise, ignore the solution. Proceed to step (seventeen).

[0188] (XVII) Determine whether the maximum number of iterations for population evolution has been reached. If the maximum number of iterations max_num has been reached, output the final Pareto optimal solution set; otherwise, return to step (XII).

[0189] In simple terms, the design steps of the on-chip network routing algorithm based on edge feature graph attention network provided by this invention are as follows:

[0190] [1] Mathematical modeling is performed based on the actual situation of on-chip network routing distribution, and its optimization objective is determined.

[0191] The specific steps in this step [1] are as follows:

[0192] Based on the routing distribution of the on-chip network, a model for routing line planning is constructed. The design and rationality of routing planning aim to minimize the length of the routes traversed by data packets and their power consumption. This reduces power consumption while minimizing the transmission path. The two factors most influential on routing line planning are: the total path a data packet takes to reach the target router after traversing the routing network, and the total power consumed by the data packet after passing through a series of routers to reach the target router. The specific modeling process is shown in step (I): "In this invention, f1 and f2 represent two objectives... Therefore, the model of the multi-objective path optimization problem in this invention is defined as min F = {f1, f2}". This invention mainly aims to derive a communication scheme on a network with many distributed routers and routing lines between them. The initial objective function is defined as: min F = {f1, f2}. However, due to different preference weights (many people may need many different schemes), many preference weights λ1 and λ2 are randomly generated in advance, dividing the multi-objective problem into many sub-problems. The model of the sub-problems is defined as follows: Where i∈{1,2,3,....,n}, and n is the total number of subproblems. The constraints only need to be reasonable for the training and test data.

[0193] [2] Using data packets as carriers, the target value is used as the reward value (loss function). The Markov policy idea is applied to construct a graph attention model based on edge features. Reinforcement learning is used to train it, and the optimal model is obtained by optimizing the loss function.

[0194] The specific steps in this step [2] are as follows:

[0195] [2.1] The problem class, problem state and other environment are defined according to the actual routing distribution in the on-chip network. The edge data that is randomly generated and processed is used as training data, and the constructed edge feature graph attention network model is trained by the policy gradient algorithm.

[0196] [2.2] Collect the actual data distribution of the on-chip network router and use it as test data to input into the pre-trained model set to finally obtain the initial solution for routing path planning.

[0197] [3] Optimize using the already trained model and combined with a multi-task optimization algorithm.

[0198] The specific steps in this step [3] are as follows:

[0199] [3.1] The initial solution obtained in step [2.2] is used as the initial population. Genes are subjected to random crossover and the compatibility score gradient obtained from solving the test dataset is used to perform gene self-mutation to obtain new individuals. The new individuals are compared with the individuals in the parent population to update the population and the Pareto optimal set.

[0200] [3.2] Set a threshold for the number of generations. When the number of generations reaches the threshold, output the final Pareto optimal set obtained by iterative update as the final solution.

[0201] Compared with the prior art, the present invention has the following advantages:

[0202] This invention designs a multi-head attention mechanism model that combines graph attention networks and edge features based on the problem to be solved, and uses reinforcement learning to train the model to quickly obtain the initial solution to the problem in an end-to-end manner, which greatly improves the speed of problem solving and provides new possibilities for solving large-scale integrated circuit design.

[0203] This invention combines a multi-task evolutionary algorithm, which uses the similarity between individuals in the population to learn knowledge, and then uses the attention compatibility score gradient generated when the model solves the test cases to update the individuals, thereby further improving the quality of the final solution.

[0204] Based on the routing optimization method for on-chip networks in VLSI design provided by this invention, this invention also provides a routing optimization system for on-chip networks in VLSI design, which includes the following modules:

[0205] The target acquisition module is used to acquire the target of the entire communication process of the on-chip network. The target includes f1 and f2. f1 represents the sum of the path distances that the signal travels from the source router to the target router when it visits all routers and reaches the target router under the current router distribution. f2 represents the total power cost required to travel f1.

[0206] The subproblem partitioning module is used to generate a set of uniformly distributed weight vectors based on the goal of the entire communication process of the on-chip network to divide the original multi-objective routing optimization problem into a set of multi-objective subproblems, thus obtaining a set of multi-objective subproblems; the original multi-objective routing optimization problem is to minimize f1 and f2.

[0207] The subproblem selection module is used to select one multi-objective subproblem from the set of multi-objective subproblems as the current problem to be trained and solved.

[0208] The edge embedding chart data generation module is used to generate a set of random data for the problem that needs to be trained and solved, and to process the generated random data to generate the initial edge embedding chart data.

[0209] The graph embedding data generation module is used to process and manipulate the initial edge embedding graph data using an encoder, extract information features from the initial edge embedding graph data, and finally integrate them into graph embedding data.

[0210] The context embedding generation module is used to generate the context embeddings required by the decoder.

[0211] The next edge selection module is used to solve the selection probability of each candidate edge at the current time step by combining the decoder with context embedding and graph embedding data, and select the next edge according to the selection strategy.

[0212] The first judgment module is used to determine whether a communication scheme has been formed.

[0213] The first return module is used to return to the context embedding generation module when the output of the first judgment module is negative.

[0214] The model training module is used to obtain the objective function value of the multi-objective sub-problem based on the weight coefficients of the multi-objective sub-problem when the output of the first judgment module is yes, calculate the loss function value, and train the model corresponding to the multi-objective sub-problem through the policy gradient algorithm.

[0215] The second judgment module is used to determine whether the current loss function value has reached the optimal state.

[0216] The second return module is used to return to the edge-embedded chart data generation module when the output result of the second judgment module is negative.

[0217] The third judgment module is used to determine whether the multi-objective sub-problem has been trained when the output of the second judgment module is yes.

[0218] The third return module is used to return to the sub-problem selection module when the output of the third judgment module is negative.

[0219] The population generation module is used to input test data into the model set to generate an initial population when the output of the third judgment module is yes; the model set includes the model corresponding to each multi-objective sub-problem.

[0220] The individual selection module is used to select an individual i from the current population as the parent population and to select an individual i from the current population as the object of the current operation.

[0221] A new individual generation module is used to perform gene crossover operations on the model parameters of individuals i-1 and i+1 adjacent to individual i to obtain a new individual i_new.

[0222] The fourth judgment module is used to determine whether the new individual i_new meets the mutation conditions.

[0223] The gene mutation operation module is used to perform gene mutation operations on the new individual i_new based on the gradient of the attention compatibility score obtained during model operation, when the output of the fourth judgment module is yes.

[0224] The update module is used to compare the new individual i_new with individuals i-1, i, and i+1 when the output of the fourth judgment module is negative, and to perform corresponding population updates and Pareto optimal set updates.

[0225] The fifth judgment module is used to determine whether the maximum number of iterations for population evolution has been reached.

[0226] The fourth return module is used to return to the individual selection module when the output of the fifth judgment module is negative.

[0227] The optimal communication scheme output module is used to output the final Pareto optimal solution set as the optimal communication scheme when the output result of the fifth judgment module is yes; the optimal communication scheme minimizes f1 and f2.

[0228] Furthermore, the present invention also provides an electronic device, which may include: a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The processor can call a computer program stored in the memory to execute the routing optimization method for the on-chip network in the VLSI design.

[0229] Furthermore, when the computer program in the aforementioned memory is implemented as a software functional unit and sold or used as an independent product, it can be stored in a non-transitory computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0230] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0231] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A routing optimization method for on-chip networks in VLSI design, characterized in that, The method includes: Step S1: Obtain the target of the entire communication process of the on-chip network; the target includes f1 and f2; f1 represents the sum of the path distances that the signal travels from the source router to the target router when it visits all routers and reaches the target router under the current router distribution; f2 represents the total power cost required to travel f1. Step S2: Based on the goal of the entire communication process of the on-chip network, generate a set of uniformly distributed weight vectors to divide the original multi-objective routing optimization problem into a set of multi-objective sub-problems, thus obtaining a set of multi-objective sub-problems; the original multi-objective routing optimization problem is to minimize f1 and f2. Step S3: Select one multi-objective sub-problem from the set of multi-objective sub-problems as the problem that needs to be trained and solved; Step S4: Generate a set of random data for the problem that needs to be trained and solved, and process the generated random data to generate the initial edge embedding graph data; Step S5: Use an encoder to process and refine the initial edge embedding graph data, extract the information features from the initial edge embedding graph data, and finally integrate them into graph embedding data. Step S6: Generate the context embeddings required by the decoder; Step S7: Use the decoder to combine the context embedding and the graph embedding data to solve the selection probability of each candidate edge at the current time step, and select the next edge according to the selection strategy; Step S8: Determine whether a communication scheme has been established. If the output of step S8 is negative, then return to step S6; If the output of step S8 is yes, then proceed to step S9: obtain the objective function value of the multi-objective sub-problem based on the weight coefficients of the multi-objective sub-problem, calculate the loss function value, and train the model corresponding to the multi-objective sub-problem using the policy gradient algorithm. Step S10: Determine whether the current loss function value has reached the optimal state; If the output of step S10 is negative, then return to step S4; If the output of step S10 is yes, then proceed to step S11: determine whether the multi-objective sub-problems have been trained. If the output of step S11 is negative, then return to step S3; If the output of step S11 is yes, then proceed to step S12: input the test data into the model set to generate an initial population; the model set includes the model corresponding to each multi-objective sub-problem; Step S13: Take the current population as the parent population and select an individual i in the population as the object of the current operation; Step S14: Perform gene crossover operation based on the model parameters of individuals i-1 and i+1 adjacent to individual i to obtain a new individual i_new; Step S15: Determine whether the new individual i_new satisfies the mutation condition; If the output of step S15 is yes, then proceed to step S16: perform gene mutation operation on the new individual i_new based on the gradient of the attention compatibility score obtained from the test data during model operation; If the output of step S15 is negative, then proceed to step S17: compare the new individual i_new with individual i-1, individual i, and individual i+1, and perform corresponding population updates and Pareto optimal set updates. Step S18: Determine whether the maximum number of iterations for population evolution has been reached; If the output of step S18 is negative, then return to step S13; If the output of step S18 is yes, then the final Pareto optimal solution set is output as the optimal communication scheme; the optimal communication scheme minimizes f1 and f2.

2. The routing optimization method for on-chip networks in VLSI design according to claim 1, characterized in that, The context embedding required for generating the decoder specifically includes: If the current node is the source router of the network, the initial context embedding is generated through the LSTM network; otherwise, the current edge embedding information is input into the LSTM network and the corresponding context embedding is updated using the multi-head attention layer.

3. The routing optimization method for on-chip networks in VLSI design according to claim 1, characterized in that, The selected strategy is either a random sampling strategy or a greedy strategy.

4. The routing optimization method for on-chip networks in VLSI design according to claim 1, characterized in that, The determination of whether a communication scheme has been established specifically includes: The judgment criteria determine whether a communication scheme has been formed. The judgment criteria are whether a feasible path has been formed between the source node and the target node. If a feasible path has been formed between the source node and the target node, it is determined that a communication scheme has been formed. If no feasible path has been formed between the source node and the target node, it is determined that a communication scheme has not been formed.

5. The routing optimization method for on-chip networks in VLSI design according to claim 1, characterized in that, The determination of whether the new individual i_new satisfies the mutation condition specifically includes: The mutation condition of a new individual i_new is determined by whether the random number is greater than the mutation probability. When the random number is greater than the mutation probability, the new individual i_new meets the mutation condition and needs to undergo gene mutation. When the random number is less than the mutation probability, the new individual i_new does not meet the mutation condition and does not need to undergo gene mutation. The range of the random number is 0 to 1, and the mutation probability is 0.

9.

6. A routing optimization system for on-chip networks in VLSI design, characterized in that, The system includes: The target acquisition module is used to acquire the target of the entire communication process of the on-chip network; the target includes f1 and f2; f1 represents the sum of the path distances that the signal travels from the source router to the target router when it visits all routers and reaches the target router under the current router distribution; f2 represents the total power cost required to travel f1. The sub-problem partitioning module is used to generate a set of uniformly distributed weight vectors based on the goal of the entire communication process of the on-chip network to divide the original multi-objective routing optimization problem into a set of multi-objective sub-problems, thus obtaining a set of multi-objective sub-problems; the original multi-objective routing optimization problem is to minimize f1 and f2. The sub-problem selection module is used to select one multi-objective sub-problem from the set of multi-objective sub-problems as the current problem to be trained and solved; The edge embedding graph data generation module is used to generate a set of random data for the current problem that needs to be trained and solved, and to process the generated random data to generate the initial edge embedding graph data. The graph embedding data generation module is used to process and manipulate the initial edge embedding graph data using an encoder, extract information features from the initial edge embedding graph data, and finally integrate them into graph embedding data. The context embedding generation module is used to generate the context embeddings required by the decoder; The next edge selection module is used to use the decoder to combine the context embedding and the graph embedding data to solve the selection probability of each candidate edge at the current time step, and select the next edge according to the selection strategy. The first judgment module is used to determine whether a communication scheme has been formed. The first return module is used to return to the context embedding generation module when the output result of the first judgment module is negative; The model training module is used to obtain the objective function value of the multi-objective sub-problem based on the weight coefficients of the multi-objective sub-problem, calculate the loss function value, and train the model corresponding to the multi-objective sub-problem through the policy gradient algorithm when the output result of the first judgment module is yes. The second judgment module is used to determine whether the current loss function value has reached the optimal state. The second return module is used to return to the edge-embedded chart data generation module when the output result of the second judgment module is negative; The third judgment module is used to determine whether the multi-objective sub-problems have been trained when the output of the second judgment module is yes. The third return module is used to return to the sub-problem selection module when the output result of the third judgment module is negative; The population generation module is used to input test data into the model set to generate an initial population when the output result of the third judgment module is yes; the model set includes the model corresponding to each multi-objective sub-problem. The individual selection module is used to take the current population as the parent population and select an individual i in the population as the object of the current operation. A new individual generation module is used to perform gene crossover operations on the model parameters of individuals i-1 and i+1 adjacent to individual i to obtain a new individual i_new; The fourth judgment module is used to determine whether the new individual i_new meets the mutation conditions; The gene mutation operation module is used to perform gene mutation operation on the new individual i_new based on the gradient of the attention compatibility score obtained by the test data in the model operation when the output result of the fourth judgment module is yes. The update module is used to compare the new individual i_new with individuals i-1, i, and i+1 when the output of the fourth judgment module is negative, and to perform corresponding population updates and Pareto optimal set updates. The fifth judgment module is used to determine whether the maximum number of iterations for population evolution has been reached; The fourth return module is used to return to the individual selection module when the output result of the fifth judgment module is negative; The optimal communication scheme output module is used to output the final Pareto optimal solution set as the optimal communication scheme when the output result of the fifth judgment module is yes; the optimal communication scheme minimizes f1 and f2.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the routing optimization method for on-chip networks in a VLSI design as described in any one of claims 1-5.

8. The electronic device according to claim 7, characterized in that, The memory is a non-transitory computer-readable storage medium.

Images