Printed circuit board element placement method and device based on traveling salesman problem and medium

Abstract

The invention discloses a printed circuit board element placement method and device based on a traveling salesman problem and a medium, belongs to the technical field of traveling salesman problems and printed circuit boards, and aims to solve the technical problem that diversified constraint conditions and efficiency are difficult to consider when the placement sequence of traditional printed circuit board elements is optimized. According to the technical scheme, the method comprises the following steps: constructing a training data set; performing feature coding on the placement requirement of the printed circuit board element by using an encoder to obtain a coding result; decoding a coding result through a decoder based on a double-pointer network, and generating an element placement sequence meeting the priority constraint; and training the encoder-decoder architecture by adopting a near-end strategy optimization algorithm based on reinforcement learning, and finally obtaining a deep reinforcement learning model for solving the priority constraint traveling salesman problem for placing the printed circuit board element.

Description

Technical Field

[0001] The invention relates to the technical field of traveling salesman problem and printed circuit board, in particular to a method, device and medium for placing components on a printed circuit board based on the traveling salesman problem. Background Art

[0002] With the increasing complexity and miniaturization of electronic equipment functions, the design requirements for printed circuit boards have increased significantly. The reasonable placement of components is a key link in the design of printed circuit board component placement. This process not only directly affects electrical performance, signal transmission efficiency and heat dissipation, but is also closely related to production efficiency, manufacturing costs and the convenience of later maintenance. The sharp increase in the number of components in modern electronic devices has caused the complexity of printed circuit board component placement to grow exponentially. How to achieve efficient automated component placement has become a core technical challenge that needs to be solved in the field of electronic manufacturing.

[0003] From an optimization perspective, the printed circuit board component placement problem is essentially a complex combinatorial optimization problem with constraints, especially when it comes to the priority order and mutual constraints between components, the complexity of its solution is further increased. In this modeling, each component on the printed circuit board is regarded as a "city", and the "path" represents the order and connection relationship of component placement. The priority constraints reflect the physical, functional or electrical order requirements between components. For example, high-speed signal components need to be as close to the processor as possible (such as the crystal oscillator needs to be placed near the clock input pin of the microcontroller and keep the wiring short), heat-generating components need to be away from sensitive devices (such as power MOS tubes need to be close to the heat sink or the area with a large copper foil area, and away from temperature-sensitive MEMS sensors) or the location of some components is limited by mechanical structure (such as the HDMI interface is placed at the edge of the printed circuit board to ensure that it matches the opening of the shell). The solution goal is to maximize the overall efficiency and minimize resource consumption while optimizing the path so that the component placement order meets the priority constraints.

[0004] Traditional methods for solving the traveling salesman problem with priority constraints usually rely on heuristic or meta-heuristic algorithms, such as genetic algorithms, simulated annealing algorithms, and particle swarm optimization. These methods can improve the solution efficiency to a certain extent, but they still have obvious limitations. For example, they usually rely on empirical rules and it is difficult to fully model complex priority constraint relationships. When dealing with large-scale problems, they are prone to fall into local optimal solutions, and the solution efficiency is difficult to meet practical needs. Therefore, achieving effective processing of complex constraints and optimizing the solution performance of large-scale problems have become the difficulties in the study of printed circuit board component placement problems. In theory, the printed circuit board component placement problem can be modeled as a traveling salesman problem with priority constraints. The model aims to determine the optimal order of component placement by path optimization to achieve comprehensive optimization of distance and placement efficiency. However, traditional methods perform poorly in modeling complex constraint relationships and achieving global optimization, resulting in the accuracy and efficiency of the solution being difficult to meet the requirements of practical applications, that is, it is difficult for traditional printed circuit board components to take into account diversified constraints and efficiency when optimizing the placement order, and the solution efficiency of long sequence problems is low.

[0005] In recent years, the application of deep reinforcement learning in complex optimization problems has gradually attracted widespread attention, especially in solving high-dimensional and complex constraint problems. In the printed circuit board component placement problem, deep reinforcement learning can automatically discover the optimal placement strategy by training intelligent agents, thereby overcoming the limitations of manual design rules and heuristic strategies in traditional methods. However, existing deep reinforcement learning methods face challenges such as insufficient feature extraction and limited ability to model complex constraint relationships when solving printed circuit board component placement problems, resulting in low learning efficiency and difficulty in ensuring the quality and stability of the solution.

[0006] Therefore, how to use deep reinforcement learning methods to take into account diverse constraints and efficiency in the process of optimizing the placement order, so as to comprehensively improve the accuracy and efficiency of printed circuit board component placement is a technical problem that needs to be solved urgently. Summary of the invention

[0007] The technical task of the present invention is to provide a printed circuit board component placement method, device and medium based on the traveling salesman problem to solve the problem of how to use deep reinforcement learning methods to balance diversified constraints and efficiency in the process of optimizing the placement order, thereby comprehensively improving the accuracy and efficiency of printed circuit board component placement.

[0008] The technical task of the present invention is achieved in the following manner: a printed circuit board component placement method based on the traveling salesman problem, the method is specifically as follows:

[0009] The training data set is constructed by randomly generating 10,000 data with n nodes in a unit square. The number of priority constraint nodes in each data set is 0.33n, where n represents the number of component placement points on the printed circuit board. The priority constraints are generated by randomly selecting node pairs to form an ordered pair set C to ensure the diversity and reality of the constraint relationship in the test samples.

[0010] The encoder is used to encode the placement requirements of printed circuit board components to obtain encoding results; wherein the encoder includes a convolutional neural network embedding layer and several encoder layers based on heterogeneous attention mechanisms; the convolutional neural network embedding layer is used to extract the features of input nodes and capture the local spatial information of nodes based on the translation invariance and domain relationship of the component positions in the printed circuit board;

[0011] The encoding result is decoded by a decoder based on a double-pointer network to generate a component placement sequence that satisfies the priority constraints;

[0012] The encoder-decoder architecture is trained using a reinforcement learning-based proximal policy optimization algorithm, and the final result is a deep reinforcement learning model for solving the priority constrained traveling salesman problem for printed circuit board component placement, which optimizes the quality of the solution and the convergence speed.

[0013] Preferably, the convolutional neural network embedding layer generates an embedding vector by extracting local spatial information from the input node, and passes the embedding vector to the encoder layer based on the heterogeneous attention mechanism; specifically, as follows:

[0014] The concatenation of the feature vector of the i-th node and the features of the k nearest neighbor nodes is expressed as The embedding vector of the i-th node The formula is as follows:

[0015]

[0016] Where d represents the dimension of the feature space; represents a d-dimensional real vector space; Represents the learnable parameters of the node embedding; represents a d×d real matrix space; x i Represents the feature vector of the i-th input node; Represents the local feature matrix of node i and its k nearest neighbor nodes (forming k+1) Perform convolution operation to extract local features;

[0017] Through local neighborhood operations, the embedding vector It not only contains the information of the node itself, but also captures the positional relationship of the node in the local topology, providing a strong feature representation for the subsequent heterogeneous attention mechanism modeling;

[0018] Embedding vector As the input of the encoder layer based on the heterogeneous attention mechanism, the initial feature representation of each node is provided.

[0019] Preferably, each encoder layer based on the heterogeneous attention mechanism includes a multi-head heterogeneous attention sub-layer, a feed-forward network layer, and a residual connection and normalization layer;

[0020] Among them, the multi-head heterogeneous attention sub-layer is used to capture the different layers of relationships and priority constraints between nodes; for example, the crystal oscillator components need to be placed near the constant input pins of the microcontroller and keep the wiring as short as possible;

[0021] Feed-forward network layers are used to perform nonlinear transformations;

[0022] Residual connections and normalization layers are used to ensure gradient stability and speed up training.

[0023] Preferably, the heterogeneous attention mechanism of the encoder layer is represented as u∈{ps,sp,nn,mm,gr}; where ps represents the predecessor node to the successor node, which is used to limit the attention of the predecessor node to the successor node; sp represents the successor node to the predecessor node, which is used to capture the dependence of the successor node on the predecessor node; nn represents the neighbor-to-neighbor attention, which is used to focus on the node relationship within the local neighborhood; mm represents the attention between the same constraint groups, which is used to model the relationship between nodes in the same priority chain; gr represents the global relationship attention, which is used to capture the cross-chain or global node association;

[0024] ps, sp, nn, mm, and gr are all equipped with independent trainable weight matrices: query vector, key vector, and value vector are The feature representation of each layer node is The initial feature representation of the l=1th layer is represented by the embedding vector Provide, that is In subsequent layers, Through the heterogeneous attention mechanism and the feedforward network, the multi-level node relations and priority constraints are captured. For each node i∈{1,...,n} and each layer l, it is defined as follows:

[0025]

[0026]

[0027]

[0028] For each heterogeneous attention mechanism u, the attention scores of nodes i and j The calculation method is as follows, and the constraint is implemented by masking the attention scores that do not meet the constraint by -∞:

[0029]

[0030]

[0031]

[0032]

[0033]

[0034] Among them, V i represents the neighborhood of node i, is a normalization factor used to balance the numerical range of the dot product operation in the heterogeneous attention mechanism; globalrelation(i,j) indicates whether nodes i and j have a global association relationship; M represents the set of all priority constraint chains, where each node can belong to at most one chain; function chain(i) is used to return the chain to which node i belongs;

[0035] Attention weights are obtained by applying softmax normalization The formula is as follows:

[0036]

[0037] Attention Weight With value vector Multiply and accumulate to get the embedding of each head, as follows:

[0038]

[0039] A multi-head heterogeneous attention function MHA with B = 8 heads is applied to the embedding h as follows:

[0040]

[0041] Among them, W O is a trainable weight matrix;

[0042] The embedding calculation of each layer l includes an MHA function, a feedforward layer, a residual connection, and two batch normalizations to calculate the embedding of node i The formula is as follows:

[0043]

[0044]

[0045] Among them, BN is batch normalization, and FF is feedforward neural network;

[0046] The mean of the final node embedding is used as the context of the decoder, and the formula is as follows:

[0047]

[0048] Preferably, the decoder based on the dual-pointer network gradually generates access nodes that meet the constraints by specifying the query order, thereby building a feasible path for each instance without relying on additional search components; at the same time, the decoder uses context embedding to dynamically represent the current state, and uses the current state as a query to interact with the embedding of the candidate node; wherein, the context embedding is dynamically updated in each decoding step until the complete path is generated.

[0049] Preferably, the decoder adopts a dual-pointer network structure to fully exploit the potential of embedding from the perspective of context and node relations. The dual-pointer network structure uses two sets of linear projections (i.e., a dual-head mechanism) to capture more complex and multi-level relationships between nodes. Each set of linear projections independently calculates the interaction score and improves robustness through averaging. In the printed circuit board component placement problem, nodes need to meet a strict priority access order. The dual-pointer network can dynamically adjust the node selection strategy when generating a path, so that the generated path always meets the constraints.

[0050] More preferably, the context embedding in the decoder is a dynamic state representation of the decoder, which is used to reflect the historical information generated by the current path. The dynamic state identifier interacts with the encoder node embedding to capture the priority constraints that change dynamically during the decoding process. The node embedding generated by the encoder represents the static features of the unvisited nodes, including the physical location of the unvisited nodes and the priority constraint relationship. The static features are an important basis for the decoder to make the next decision. The encoder's node embedding vector is simultaneously input into the context embedding and the dual-pointer network structure to ensure that the decoder can pay attention to the dynamic state and use global information, providing strong support for the decoding of complex constraint problems.

[0051] At each decoding step, the context embedding interacts with the unvisited node embeddings to generate a probability distribution of unvisited nodes; specifically:

[0052] The query vector q t and key vector k j (j-th node embedding generated by the encoder) linearly projected to the same dimension d k , and design H different linear projections for each interaction (H = 2);

[0053] The interaction between query and j is calculated by dot product and divided by After normalization, the final interaction score is calculated by taking the average of multiple interactions: Based on the consideration of priority constraints and distance costs, NAdjust the interaction score by subtracting the access cost from the last node in the current partial path to all other nodes: score ij =P N -cost(i,j). In this way, the model tends to choose the nearest node as the next visit target, so as to start training from a good strategy initialization. Compared with random initialization, this method significantly improves the training efficiency. The probability distribution of the visited nodes is as follows:

[0054]

[0055] The output probability vector p is generated by the softmax function to guide the decoder's node selection and iteratively generate the node access order until all nodes are visited and the complete path is output.

[0056] As a preferred approach, the proximal policy optimization algorithm based on reinforcement learning adopts an actor-critic network architecture, in which the core of the actor network includes an encoder-decoder architecture; specifically, the encoder combines convolutional neural networks and heterogeneous attention mechanisms to achieve efficient extraction of complex features; the decoder adopts a dual-pointer network architecture to achieve more accurate output prediction;

[0057] The critic network consists of two layers of feedforward neural network, the first layer is a fully connected layer with ReLU activation function, and the second layer is a linear layer; the critic network is used to estimate the state value function V φ (s), relying on the efficient computing power and powerful function approximation characteristics of the feedforward network, it provides more accurate feedback signals for training;

[0058] Advantage function A t It represents the improvement of the current action relative to the benchmark strategy, and the formula is as follows:

[0059] S t =-ΔL(σ / s)-V φ (s);

[0060] Where -ΔL(σ / s) represents the path length reduction value; V φ (s) represents the expected reward value in state s, estimated by the critic network;

[0061] The strategy ratio value r(θ) is used to quantify the difference between the new and old strategies. The formula is as follows

[0062]

[0063] Among them, a t represents the action selection at time step t; p θ(a / s) represents the policy function, which refers to the probability of selecting action a in state s. During the training process, the unbiased Monte Carlo estimate of the policy gradient is obtained by reducing the run length by -L(σ / s), and the goal is to optimize the parameters θ of the actor network to generate a better policy.

[0064] In order to stabilize the optimization process and increase the exploration of the strategy, a truncated proxy objective function L is used CLIP (θ) and policy entropy loss L E (θ) Optimize the actor network:

[0065]

[0066] L E (θ)=Entropy(V φ (σ / s));

[0067] Among them, clip(r(θ),1-∈,1+∈ is used to limit the amplitude of policy updates to avoid instability caused by excessive policy updates; ∈ represents a hyperparameter, which is usually small; Entropy represents the entropy of policy distribution. The higher the entropy value, the more dispersed the policy distribution is and the stronger the exploratory nature is.

[0068] For the critic network, the mean square error loss function L MSE (φ) Update parameters to estimate the discounted reward in state s, as follows:

[0069] L MSE (φ)=MSE(-ΔL(σ / s),V φ (s));

[0070] Among them, MSE is the mean square error function, which represents the error between the predicted value and the target value;

[0071] The overall loss function of the encoder-decoder architecture combines strategy optimization, entropy regularization, and state value estimation, and is given by:

[0072] Loss(φ,θ)=c p L CLIP (θ)+c b L MSE (φ)-c e L E (θ);

[0073] Among them, c p 、c v and c e L CLIP , L MSE and L E The weight coefficient is used to balance the impact of various losses in the optimization objective.

[0074] An electronic device comprising: a memory and at least one processor;

[0075] Wherein, the memory stores a computer program;

[0076] The at least one processor executes the computer program stored in the memory, so that the at least one processor performs the printed circuit board component placement method based on the traveling salesman problem as described above.

[0077] A computer-readable storage medium stores a computer program, which can be executed by a processor to implement the printed circuit board component placement method based on the traveling salesman problem as described above.

[0078] The printed circuit board component placement method, device and medium based on the traveling salesman problem of the present invention have the following advantages:

[0079] (i) The present invention can effectively capture the complex dependencies and constraints in the placement process by introducing a convolutional neural network embedding layer and a heterogeneous attention mechanism, and improve the accuracy and efficiency of the placement results by using proximal strategy optimization algorithm training;

[0080] (ii) The present invention uses a convolutional neural network embedding layer combined with an encoder of a heterogeneous attention mechanism to effectively capture the complex constraint relationship between components. The decoder based on the dual-pointer network takes into account both the priority constraints and the global optimization goals when generating the placement path, and the proximal strategy optimization algorithm is used to train the encoder-decoder architecture. By truncation strategy updates and strategy entropy regularization, the learning efficiency and solution stability are improved. It has important application value in the field of automated placement of printed circuit board components, and can significantly improve placement accuracy and save time costs. Compared with traditional methods, the present invention achieves efficient modeling of node features and constraints through deep learning, and significantly improves the solution efficiency in combination with reinforcement learning.

[0081] (III) The present invention is used to solve the traveling salesman problem with priority constraints in printed circuit board component placement. Specifically, by combining a convolutional neural network with an encoder of a heterogeneous attention mechanism and an autoregressive decoder based on a dual-pointer network, it is possible to efficiently generate an optimal path that satisfies the priority constraints, thereby improving the efficiency of printed circuit board component placement and the accuracy of path planning. Furthermore, a proximal policy optimization algorithm is used to train the model, and stable convergence is achieved through the effective design of policy gradients and advantage functions. Compared with traditional optimization methods, the present invention shows significant advantages in path planning accuracy and computational efficiency, and provides a practical deep learning solution for efficient path optimization in complex industrial scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] The present invention is further described below in conjunction with the accompanying drawings.

[0083] Attached Figure 1 A schematic diagram of a printed circuit board component placement method based on the traveling salesman problem;

[0084] Attached Figure 2 Schematic diagram of the heterogeneous attention mechanism between two consecutive multi-head attention layers in the encoder. DETAILED DESCRIPTION

[0085] The printed circuit board component placement method, device and medium based on the traveling salesman problem of the present invention are described in detail below with reference to the accompanying drawings and specific embodiments of the specification.

[0086] Embodiment 1:

[0087] The problem definition is as follows:

[0088] Traveling Salesman Problem with Priority Constraints: The core goal of the traveling salesman problem is to determine a path given a set of nodes V, so that each node is visited only once and eventually returns to the starting node, while minimizing the total length of the path. In the actual application of printed circuit board component placement optimization, it is necessary to further consider priority constraints - that is, the order of visiting certain nodes (components) must meet specific rules, so as to optimize the placement process and meet actual design requirements.

[0089] The printed circuit board component placement is modeled as a traveling salesman problem with precedence constraints, which is defined as follows: The set of all component nodes that need to be placed V = {v1, v2, ..., v n}, the starting point of the printed circuit board component placement is v0, and the distance matrix D = [d i,j ], where d i,j Represents the slave node v i To node v j The priority constraint set C represents the constraints on the order of component placement. For example, if component v i Must be placed in v j Before, it is recorded as an ordered pair (v i ,v j )∈C. Our goal is to find an optimal node order σ=[v0,v σ(1) ,v σ(2) ,…,v σ(n) ,v0], so that the total travel length L is minimized, and its mathematical expression is as follows:

[0090]

[0091] Among them, σ(0)=0,σ(n+1)=0 to ensure that the path starts from the starting node and returns to the starting node; To ensure that each node on the path is visited only once; for all (v i ,v j )∈C must have σ(i)<σ(j) to satisfy the node access order and the priority constraint.

[0092] Reinforcement learning modeling: To efficiently solve the priority-constrained traveling salesman problem in printed circuit board component placement, we model it as a Markov decision process and solve it based on deep reinforcement learning. The Markov construction and its elements are defined as follows:

[0093] State space (S): represents the current layout of component placement, including the position and arrangement order of placed components, the sequence of unplaced components and their priority constraints.

[0094] Action space (A): Indicates the component placement actions that can be selected at the current moment.

[0095] State transfer function (P(s ′ |s,a): After taking action a∈A, transition from the current state s to the next state s ′ In this problem, the state transition is deterministic and is determined by the current layout and the selected components.

[0096] Reward function (R(s,a)): used to evaluate the quality of the current decision. The reward function is the negative increment of the distance length -L.

[0097] Discount factor (γ): The weight used to weigh immediate rewards against future rewards.

[0098] Node v corresponding to action a i In a precedence relation, a node can be selected only when all its predecessor nodes have been placed. For any state S, define a valid action set A valid (s), such that

[0099]

[0100] As attached Figure 1 As shown, this embodiment provides a printed circuit board component placement method based on the traveling salesman problem, and the method is specifically as follows:

[0101] S1. A training data set is constructed by randomly generating 10,000 data with n nodes in a unit square. The number of priority constraint nodes in each data set is 0.33n; where n represents the number of placement points of printed circuit board components; priority constraints are generated by randomly selecting node pairs to form an ordered pair set C to ensure the diversity and reality of constraint relationships in the test samples;

[0102] S2. Using an encoder to perform feature encoding on the placement requirements of printed circuit board components to obtain encoding results; wherein the encoder includes a convolutional neural network embedding layer and a plurality of encoder layers based on a heterogeneous attention mechanism; the convolutional neural network embedding layer is used to extract features of input nodes and capture local spatial information of nodes based on translation invariance and domain relationship of component positions in the printed circuit board;

[0103] S3, decoding the encoding result through a decoder based on a double-pointer network to generate a component placement sequence that satisfies the priority constraint;

[0104] S4. A proximal policy optimization algorithm based on reinforcement learning is used to train the encoder-decoder architecture, and finally a deep reinforcement learning model for solving the priority constrained traveling salesman problem for printed circuit board component placement is obtained, which optimizes the quality of the solution and the convergence speed.

[0105] The convolutional neural network embedding layer in step S2 of this embodiment generates an embedding vector by extracting local spatial information from the input node, and passes the embedding vector to the encoder layer based on the heterogeneous attention mechanism; specifically, as follows:

[0106] S2-101, the concatenation of the feature vector of the i-th node and the features of the k nearest neighbor nodes is expressed as The embedding vector of the i-th node The formula is as follows:

[0107]

[0108] Where d represents the dimension of the feature space; represents a d-dimensional real vector space; Represents the learnable parameters of the node embedding; represents a d×d real matrix space; x i Represents the feature vector of the i-th input node; Represents the local feature matrix of node i and its k nearest neighbor nodes (forming k+1) Perform convolution operation to extract local features;

[0109] S2-102, embedding vectors through local neighborhood operations It not only contains the information of the node itself, but also captures the positional relationship of the node in the local topology, providing a strong feature representation for the subsequent heterogeneous attention mechanism modeling;

[0110] S2-103, embedding vector As the input of the encoder layer based on the heterogeneous attention mechanism, the initial feature representation of each node is provided.

[0111] In this embodiment, each encoder layer based on the heterogeneous attention mechanism includes a multi-head heterogeneous attention sub-layer, a feed-forward network layer, and a residual connection and normalization layer;

[0112] Among them, the multi-head heterogeneous attention sub-layer is used to capture the different layers of relationships and priority constraints between nodes; for example, the crystal oscillator components need to be placed near the constant input pins of the microcontroller and keep the wiring as short as possible;

[0113] Feed-forward network layers are used to perform nonlinear transformations;

[0114] Residual connections and normalization layers are used to ensure gradient stability and speed up training.

[0115] To meet the priority constraint requirements in printed circuit board component placement, the encoder layer designs a heterogeneous attention mechanism to capture different levels of relationships and order constraints between nodes. For example, crystal oscillator components need to be placed near the microcontroller's input pins and keep the wiring short.

[0116] The heterogeneous attention mechanism of the encoder layer in step S2 of this embodiment is as follows:

[0117] S2-201. The heterogeneous attention mechanism of the encoder layer is represented as u∈{ps,sp,nn,mm,gr}, as shown in the attached Figure 2 As shown in the figure, ps represents the predecessor node to the successor node, which is used to limit the attention of the predecessor node to the successor node; sp represents the successor node to the predecessor node, which is used to capture the dependence of the successor node on the predecessor node; nn represents the neighborhood to neighborhood attention, which is used to focus on the node relationship within the local neighborhood; mm represents the attention between the same constraint groups, which is used to model the relationship between nodes in the same priority chain; gr represents the global relationship attention, which is used to capture the cross-chain or global node association;

[0118] S2-202, ps, sp, nn, mm, gr are all equipped with independent trainable weight matrices: query vector, key vector and value vector are The feature representation of each layer node is The initial feature representation of the l=1th layer is represented by the embedding vector Provide, that is In subsequent layers, Through the heterogeneous attention mechanism and the feedforward network, the multi-level node relations and priority constraints are captured. For each node i∈{1,...,n} and each layer l, it is defined as follows:

[0119]

[0120]

[0121]

[0122] S2-203. For each heterogeneous attention mechanism u, the attention scores of nodes i and j The calculation method of is as follows, and the constraint is implemented by -∞ masking the attention scores that do not meet the constraint:

[0123]

[0124]

[0125]

[0126]

[0127]

[0128] Among them, V i represents the neighborhood of node i, is a normalization factor used to balance the numerical range of the dot product operation in the heterogeneous attention mechanism; globalrelation(i,j) indicates whether nodes i and j have a global association relationship; M represents the set of all priority constraint chains, where each node can belong to at most one chain; function chain(i) is used to return the chain to which node i belongs;

[0129] S2-204, get the attention weight by applying softmax normalization The formula is as follows:

[0130]

[0131] S2-205, Attention Weight With value vector Multiply and accumulate to get the embedding of each head, as follows:

[0132]

[0133] S2-206. Apply a multi-head heterogeneous attention function MHA with B=8 heads to the embedding h, as follows:

[0134]

[0135] Among them, W O is a trainable weight matrix;

[0136] S2-207, the embedding calculation of each layer l includes an MHA function, a feedforward layer, a residual connection, and two batch normalizations to calculate the embedding of node i The formula is as follows:

[0137]

[0138]

[0139] Among them, BN is batch normalization, and FF is feedforward neural network;

[0140] S2-208, use the mean of the final node embedding as the context of the decoder, the formula is as follows:

[0141]

[0142] The double-pointer network-based decoder in step S3 of this embodiment gradually generates access nodes that meet the constraints by specifying the query order, thereby building a feasible path for each instance without relying on additional search components; at the same time, the decoder uses context embedding to dynamically represent the current state, and uses the current state as a query to interact with the embedding of the candidate node; wherein the context embedding is dynamically updated in each decoding step until the complete path is generated.

[0143] The decoder in step S3 of this embodiment adopts a dual-pointer network structure to fully exploit the potential of embedding from the perspective of context and node relationship; wherein, the dual-pointer network structure adopts two sets of linear projections (i.e., a dual-head mechanism) to capture more complex and multi-level relationships between nodes, each set of linear projections independently calculates the interaction score, and improves robustness through averaging processing; in the printed circuit board component placement problem, nodes must meet a strict priority access order, and the dual-pointer network can dynamically adjust the node selection strategy when generating a path, so that the generated path always meets the constraints.

[0144] The context embedding in the decoder in step S3 of this embodiment is a dynamic state representation of the decoder, which is used to reflect the historical information of the current path generation. The dynamic state identifier interacts with the encoder node embedding to capture the priority constraints that change dynamically during the decoding process; the node embedding generated by the encoder represents the static features of the unvisited nodes, including the physical position of the unvisited nodes and the priority constraint relationship. The static features are an important basis for the decoder to make the next decision; the node embedding vector of the encoder is simultaneously input into the context embedding and the dual-pointer network structure to ensure that the decoder can pay attention to the dynamic state and use global information, providing strong support for the decoding of complex constraint problems;

[0145] At each decoding step, the context embedding interacts with the unvisited node embeddings to generate a probability distribution of unvisited nodes; specifically:

[0146] S301: query vector q t and key vector k j (j-th node embedding generated by the encoder) linearly projected to the same dimension dk , and design H different linear projections for each interaction (H = 2);

[0147] S302, calculate the interaction result between query and j by dot product, and divide by After normalization, the final interaction score is calculated by taking the average of multiple interactions: Considering the priority constraint and distance cost, from p N Adjust the interaction score by subtracting the access cost from the last node in the current partial path to all other nodes: score ij =p N -cost(i,j). In this way, the model tends to choose the nearest node as the next visit target, so as to start training from a good strategy initialization. Compared with random initialization, this method significantly improves the training efficiency. The probability distribution of the visited nodes is as follows:

[0148]

[0149] S303, generating an output probability vector p through the softmax function to guide the decoder's node selection, and iteratively generating a node access sequence until all nodes are visited, and outputting a complete path.

[0150] The proximal policy optimization algorithm based on reinforcement learning in step S4 of this embodiment adopts an actor-critic network architecture, wherein the core of the actor network includes an encoder-decoder architecture; specifically, the encoder combines a convolutional neural network and a heterogeneous attention mechanism to achieve efficient extraction of complex features; the decoder adopts a dual-pointer network architecture to achieve more accurate output prediction;

[0151] The critic network consists of two layers of feedforward neural network, the first layer is a fully connected layer with ReLU activation function, and the second layer is a linear layer; the critic network is used to estimate the state value function V φ (s), relying on the efficient computing power and powerful function approximation characteristics of the feedforward network, it provides more accurate feedback signals for training;

[0152] S401, advantage function A t It represents the improvement of the current action relative to the benchmark strategy, and the formula is as follows:

[0153] A t =-ΔL(σ / s)-V φ (s);

[0154] Where -ΔL(σ / s) represents the path length reduction value; V φ (s) represents the expected reward value in state s, estimated by the critic network;

[0155] S402, the strategy ratio value r(θ) is used to quantify the difference between the new and old strategies, and the formula is as follows

[0156]

[0157] Among them, a t represents the action selection at time step t; p θ (a / s) represents the policy function, which refers to the probability of selecting action a in state s. During the training process, the unbiased Monte Carlo estimate of the policy gradient is obtained by reducing the run length by -L(σ / s), and the goal is to optimize the parameters θ of the actor network to generate a better policy.

[0158] S403: To stabilize the optimization process and increase the exploratory nature of the strategy, a truncated proxy objective function L is used. CLIP (θ) and policy entropy loss L E (θ) Optimize the actor network:

[0159]

[0160] L E (θ)=Entropy(V φ (σ / s));

[0161] Among them, clip(r(θ),1-∈,1+∈ is used to limit the amplitude of policy updates to avoid instability caused by excessive policy updates; ∈ represents a hyperparameter, which is usually small; Entropy represents the entropy of policy distribution. The higher the entropy value, the more dispersed the policy distribution is and the stronger the exploratory nature is.

[0162] S404. For the critic network, the mean square error loss function L MSE (φ) Update parameters to estimate the discounted reward in state s, as follows:

[0163] L MSE (φ)=MSE(-ΔL(σ / s),V φ (s));

[0164] Among them, MSE is the mean square error function, which represents the error between the predicted value and the target value;

[0165] S405, the total loss function of the encoder-decoder architecture integrates strategy optimization, entropy regularization and state value estimation, and the formula is as follows:

[0166] Loss(θ,θ)=c p L CLIP (θ)+c v L MSE (φ)-c e L E(θ);

[0167] Among them, c p 、c v and c e L CLIP , L MSE and L E The weight coefficient is used to balance the impact of various losses in the optimization objective.

[0168] Embodiment 2:

[0169] This embodiment also provides an electronic device, including: a memory and a processor;

[0170] Wherein, the memory stores computer-executable instructions;

[0171] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the printed circuit board component placement method based on the traveling salesman problem in any embodiment of the present invention.

[0172] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or any conventional processor, etc.

[0173] The memory can be used to store computer programs and / or modules. The processor realizes various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory can also include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, at least one disk storage period, a flash memory device, or other volatile solid-state storage devices.

[0174] Embodiment 3:

[0175] This embodiment also provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions are loaded by a processor, so that the processor executes the printed circuit board component placement method based on the traveling salesman problem in any embodiment of the present invention. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code that implements the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0176] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0177] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0178] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0179] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0180] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A printed circuit board component placement method based on the traveling salesman problem, characterized in that: The method is as follows: The training data set is constructed by randomly generating 10,000 data with n nodes in a unit square. The number of priority constraint nodes in each data set is 0.33n, where n represents the number of component placement points on the printed circuit board. The priority constraints are generated by randomly selecting node pairs to form an ordered pair set C to ensure the diversity and reality of the constraint relationship in the test samples. The encoder is used to encode the placement requirements of printed circuit board components to obtain encoding results; wherein the encoder includes a convolutional neural network embedding layer and several encoder layers based on heterogeneous attention mechanisms; the convolutional neural network embedding layer is used to extract the features of input nodes and capture the local spatial information of nodes based on the translation invariance and domain relationship of the component positions in the printed circuit board; The encoding result is decoded by a decoder based on a double-pointer network to generate a component placement sequence that satisfies the priority constraints; The encoder-decoder architecture is trained using a reinforcement learning-based proximal policy optimization algorithm, and the final result is a deep reinforcement learning model for solving the priority constrained traveling salesman problem for printed circuit board component placement, which optimizes the quality of the solution and the convergence speed.

2. The printed circuit board component placement method based on the traveling salesman problem according to claim 1 is characterized in that: The convolutional neural network embedding layer generates an embedding vector by extracting local spatial information from the input node and passes the embedding vector to the encoder layer based on the heterogeneous attention mechanism; specifically: The concatenation of the feature vector of the i-th node and the features of the k nearest neighbor nodes is expressed as The embedding vector of the i-th node The formula is as follows: Where d represents the dimension of the feature space; represents a d-dimensional real vector space; Represents the learnable parameters of the node embedding; represents a d×d real matrix space; x i Represents the feature vector of the i-th input node; Represents the local feature matrix of node i and its k nearest neighbor nodes Perform convolution operation to extract local features; Through local neighborhood operations, the embedding vector It not only contains the information of the node itself, but also captures the position relationship of the node in the local topology; Embedding vector As the input of the encoder layer based on the heterogeneous attention mechanism, the initial feature representation of each node is provided.

3. The printed circuit board component placement method based on the traveling salesman problem according to claim 1 is characterized in that: Each encoder layer based on the heterogeneous attention mechanism includes a multi-head heterogeneous attention sub-layer, a feed-forward network layer, and a residual connection and normalization layer; Among them, the multi-head heterogeneous attention sub-layer is used to capture the different layers of relationships and priority constraints between nodes; Feed-forward network layers are used to perform nonlinear transformations; Residual connections and normalization layers are used to ensure gradient stability and speed up training.

4. The printed circuit board component placement method based on the traveling salesman problem according to claim 3 is characterized in that: The heterogeneous attention mechanism of the encoder layer is represented as u∈{ps,sp,nn,mm,gr}; where ps represents the predecessor node to the successor node, which is used to limit the attention of the predecessor node to the successor node; sp represents the successor node to the predecessor node, which is used to capture the dependence of the successor node on the predecessor node; nn represents the neighbor-to-neighbor attention, which is used to focus on the node relationship within the local neighborhood; mm represents the attention between the same constraint groups, which is used to model the relationship between nodes in the same priority chain; gr represents the global relationship attention, which is used to capture the cross-chain or global node association; ps, sp, nn, mm, and gr are all equipped with independent trainable weight matrices: query vector, key vector, and value vector are The feature representation of each layer node is The initial feature representation of the l=1th layer is represented by the embedding vector Provide, that is In subsequent layers, Through the heterogeneous attention mechanism and the feedforward network, the multi-level node relations and priority constraints are captured. For each node i∈{1,...,n} and each layer l, it is defined as follows: For each heterogeneous attention mechanism u, the attention scores of nodes i and j The calculation method is as follows, and the constraint is implemented by masking the attention scores that do not meet the constraint by -∞: Among them, V i represents the neighborhood of node i, is a normalization factor used to balance the numerical range of the dot product operation in the heterogeneous attention mechanism; globalrelation(i,j) indicates whether nodes i and j have a global association relationship; M represents the set of all priority constraint chains, where each node can belong to at most one chain; function chain(i) is used to return the chain to which node i belongs; Attention weights are obtained by applying softmax normalization The formula is as follows: Attention Weight With value vector Multiply and accumulate to get the embedding of each head, as follows: A multi-head heterogeneous attention function MHA with B = 8 heads is applied to the embedding h as follows: Among them, W O is a trainable weight matrix; The embedding calculation of each layer l includes an MHA function, a feedforward layer, a residual connection, and two batch normalizations to calculate the embedding of node i The formula is as follows: Among them, BN is batch normalization, and FF is feedforward neural network; The mean of the final node embedding is used as the context of the decoder, and the formula is as follows:

5. The printed circuit board component placement method based on the traveling salesman problem according to claim 1 is characterized in that: The decoder based on the dual-pointer network gradually generates access nodes that meet the constraints by specifying the query order, thereby building a feasible path for each instance without relying on additional search components; at the same time, the decoder uses context embedding to dynamically represent the current state, and uses the current state as a query to interact with the embedding of the candidate node; among them, the context embedding will be dynamically updated in each decoding step until the complete path is generated.

6. The printed circuit board component placement method based on the traveling salesman problem according to claim 5 is characterized in that: The decoder adopts a dual-pointer network structure to fully tap the potential of embedding from the perspective of context and node relations. The dual-pointer network structure uses two sets of linear projections to capture more complex and multi-level relationships between nodes. Each set of linear projections independently calculates the interaction score and improves robustness through averaging. In the printed circuit board component placement problem, nodes must meet a strict priority access order. The dual-pointer network can dynamically adjust the node selection strategy when generating a path so that the generated path always meets the constraints.

7. The printed circuit board component placement method based on the traveling salesman problem according to claim 5 is characterized in that: The context embedding in the decoder is a dynamic state representation of the decoder, which is used to reflect the historical information generated by the current path. The dynamic state identification interacts with the encoder node embedding to capture the priority constraints that change dynamically during the decoding process; At each decoding step, the context embedding interacts with the unvisited node embeddings to generate a probability distribution of unvisited nodes; specifically: The query vector q t and key vector k j Linear projection to the same dimension d k , and design H different linear projections for each interaction; The interaction between query and j is calculated by dot product and divided by After normalization, the final interaction score is calculated by taking the average of multiple interactions: From P N Adjust the interaction score by subtracting the access cost from the last node in the current partial path to all other nodes: score ij =P N -cost(i,j); the probability distribution of visiting a node is as follows: The output probability vector p is generated by the softmax function to guide the decoder's node selection and iteratively generate the node access order until all nodes are visited and the complete path is output.

8. The printed circuit board component placement method based on the traveling salesman problem according to claim 1 is characterized in that: The proximal policy optimization algorithm based on reinforcement learning adopts the actor-critic network architecture, in which the core of the actor network includes an encoder-decoder architecture; specifically, the encoder combines convolutional neural networks and heterogeneous attention mechanisms to achieve efficient extraction of complex features; the decoder adopts a dual-pointer network architecture to achieve more accurate output prediction; The critic network consists of two layers of feedforward neural network, the first layer is a fully connected layer with ReLU activation function, and the second layer is a linear layer; the critic network is used to estimate the state value function V φ (s), relying on the efficient computing power and powerful function approximation characteristics of the feedforward network, it provides more accurate feedback signals for training; Advantage function A t It represents the improvement of the current action relative to the benchmark strategy, and the formula is as follows: TO t =-ΔL(σ / s)-V φ (s); Where -ΔL(σ / s) represents the path length reduction value; V φ (s) represents the expected reward value in state s, estimated by the critic network; The strategy ratio value r(θ) is used to quantify the difference between the new and old strategies. The formula is as follows Among them, a t represents the action selection at time step t; p θ (a / s) represents the policy function, which refers to the probability of selecting action a in state s. During the training process, the unbiased Monte Carlo estimate of the policy gradient is obtained by reducing the run length by -L(σ / s), and the goal is to optimize the parameters θ of the actor network to generate a better policy. Use a truncated proxy objective function L CLIP (θ) and policy entropy loss L E (θ) Optimize the actor network: L E (θ)=Entropy(V φ (s / s)); Among them, clip(r(θ),1-∈,1+∈ is used to limit the amplitude of policy updates to avoid instability caused by excessive policy updates; ∈ represents a hyperparameter; Entropy represents the entropy of policy distribution. The higher the entropy value, the more dispersed the policy distribution is and the stronger the exploratory nature is. For the critic network, the mean square error loss function L MSE (φ) Update parameters to estimate the discounted reward in state s, as follows: L MSE (φ)=MSE(-ΔL(σ / s),V φ (s)); Among them, MSE is the mean square error function, which represents the error between the predicted value and the target value; The overall loss function of the encoder-decoder architecture combines strategy optimization, entropy regularization, and state value estimation, and is given by: Loss(φ,θ)=c p L CLIP (θ)+c v L MSE (φ)-c e L E (i); Among them, c p 、c v and c e L CLIP , L MSE and L E The weight coefficient is used to balance the impact of various losses in the optimization objective.

9. An electronic device, characterized in that: include: memory and at least one processor; Wherein, the memory stores a computer program; The at least one processor executes the computer program stored in the memory, so that the at least one processor performs the printed circuit board component placement method based on the traveling salesman problem according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which can be executed by a processor to implement the printed circuit board component placement method based on the traveling salesman problem as claimed in any one of claims 1 to 8.

Images