Printed circuit board layout method, model training method and device

Abstract

The invention discloses a printed circuit board layout method and device and a model training method and device. The method comprises the steps of obtaining layout task data of a printed circuit board; inputting the layout task data into a printed circuit board layout model, determining a layout environment and an action mask according to the layout task data, determining a current state according to the layout environment and the action mask, determining an action strategy according to the current state, and updating the current state according to the action strategy to determine a next action strategy, and outputting a layout result until iteration is completed. And updating the printed circuit board layout model according to the layout result and the reward function. The printed circuit board layout model is constructed based on a deep reinforcement learning algorithm, and the action mask is used for representing an available action space. According to the invention, the automation and optimization level of PCB layout can be effectively improved, manual intervention is reduced, and the layout efficiency and quality are improved at the same time.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a printed circuit board layout method, a model training method, and a device. Background Art

[0002] In the traditional printed circuit board (PCB) design process, layout and routing are crucial steps that directly impact the performance and production cost of the board. Prior to this, engineers primarily relied on manual component layout and routing, placing components according to the logical relationships within the circuit schematic and minimizing routing paths. However, this approach faced multiple challenges: low efficiency, single layout results, a lack of consideration for routing constraints, and unsatisfactory routing results. As circuit designs increase in complexity, the limitations of manual layout and routing become increasingly apparent. Furthermore, complex PCB designs require more flexible and efficient layout and routing strategies, which traditional methods struggle to meet. Summary of the Invention

[0003] The embodiments of the present application provide a printed circuit board layout method, model training method, and device that can effectively improve the automation and optimization level of PCB layout, reduce manual intervention, and improve layout efficiency and quality. The above technical solutions are as follows:

[0004] In a first aspect, an embodiment of the present application provides a method for training a printed circuit board layout model, the method comprising:

[0005] Acquire layout task data of a printed circuit board;

[0006] Inputting the layout task data into a printed circuit board layout model, determining a layout environment and an action mask based on the layout task data, determining a current state based on the layout environment and the action mask, determining an action strategy based on the current state, updating the current state based on the action strategy to determine a next action strategy, and outputting a layout result after iteration is completed;

[0007] updating the printed circuit board layout model according to the layout result and the reward function;

[0008] Among them, the above-mentioned printed circuit board layout model is constructed based on a deep reinforcement learning algorithm.

[0009] In the above technical solution, a PCB layout model based on a deep reinforcement learning algorithm utilizes a Markov decision process (MDP) to achieve intelligent processing of PCB layout tasks. The current state is determined by analyzing the layout task data. The current state includes an action mask that represents the available action space, thereby guiding the PCB layout model to generate the optimal action strategy. This process is iterative, continuously updating the current state and determining the next action strategy based on the new action strategy until the iteration is completed and a complete layout result is generated. Finally, the PCB layout model is updated based on the layout results and a preset reward function to continuously improve the model's performance in future layout tasks. The application of this technical solution has significantly improved the performance of the PCB layout model, improved layout efficiency and quality, reduced the workload of engineers, and helped meet the modern electronics manufacturing industry's demand for accurate and efficient PCB design.

[0010] In a possible implementation, determining the layout environment and the action mask according to the layout task data includes:

[0011] Determine the layout environment according to the above layout task data;

[0012] Determine layout constraints and routing constraints based on the above layout environment;

[0013] Determine the position mask according to the above layout constraints;

[0014] Determine the line length mask according to the above wiring constraints;

[0015] The above position mask and the above line length mask are superimposed to determine the action mask.

[0016] In this technical solution, by combining placement and routing constraints to determine action masks, the accuracy and compliance of the PCB layout model when handling layout tasks are effectively enhanced. Combining the position mask that reflects layout requirements with the line length mask that reflects routing requirements can more comprehensively guide the model's layout decisions, improve model performance, and ensure that the actions it takes are optimal.

[0017] In a possible implementation, in the action mask, the initial probability of the action within the overlapping area of ​​the position mask and the line length mask is greater than the initial probability of the action outside the overlapping area.

[0018] In the above technical solution, positions within the overlapping area of ​​the available action space are assigned a higher action probability, which ensures that the model's layout decisions are more concentrated on those areas that meet both layout and routing constraints, which can significantly improve the quality and efficiency of layout.

[0019] In a possible implementation, the initial probability of the action at each position in the action mask is determined based on the congestion degree.

[0020] In the above technical solution, the congestion of each position is taken into account to determine the action probability, which further optimizes the layout effect and avoids the problem of excessive concentration or uneven distribution of components. It can significantly improve the quality and efficiency of the layout, reduce the need for subsequent adjustments and redoing, and make the PCB layout more compact, reasonable and in compliance with electrical engineering standards.

[0021] In a possible implementation, determining the current state according to the layout environment and the action mask includes:

[0022] Extracting real-valued vector features from the above layout environment through a multi-layer perceptron;

[0023] Extract graph features from the above action mask through graph convolutional neural network;

[0024] The current state is determined based on the above real-valued vector features and the above graph features.

[0025] In the above technical solution, the process of determining the current state utilizes multi-layer perceptron (MLP) and graph convolutional neural network (GCN) technology to extract real-valued vector features describing the entire layout, as well as graph features about the interactions between components and layout constraints. By combining these two types of features, the model can accurately determine the current layout state, enabling the deep reinforcement learning model to more effectively understand and process complex PCB layout tasks.

[0026] In one possible implementation, updating the current state according to the action strategy to determine the next action strategy includes:

[0027] Update the layout environment and the action mask according to the action strategy;

[0028] Update the current state according to the updated layout environment and the action mask;

[0029] The next action strategy is determined based on the updated current state.

[0030] In this technical solution, the process of updating the current state based on the action strategy allows the current state to reflect changes caused by layout decisions in real time. This iterative update method ensures that the model can adapt to dynamic changes during the layout process, ensuring that each layout decision is based on the latest layout situation, improving layout accuracy and efficiency.

[0031] In a second aspect, an embodiment of the present application provides a printed circuit board layout method, the method comprising:

[0032] Acquire target layout task data of a printed circuit board;

[0033] Inputting target layout task data of a printed circuit board into a printed circuit board layout model, determining a layout environment and an action mask according to the target layout task data, determining a current state according to the layout environment and the action mask, determining an action strategy according to the current state, updating the current state according to the action strategy to determine a next action strategy, and outputting a first target layout result after iteration is completed;

[0034] The printed circuit board layout model is obtained by training based on any one of the above training methods.

[0035] In the above technical solution, by inputting the target layout task data into a trained PCB layout model, the model can determine the optimal layout strategy based on the latest layout status in each iteration, until the target layout result that meets all layout requirements is output after the iteration is completed. This can effectively realize the intelligent and automated PCB layout, optimize PCB layout efficiency, and improve design quality and performance. Especially when dealing with complex or high-density layout tasks, it makes the PCB design process faster, more accurate, and more efficient.

[0036] In a possible implementation, the method further includes:

[0037] Adjusting the initial motion probability distribution of target components in the target layout task data based on the target layout result set, inputting the adjusted target layout task data into the printed circuit board layout model, and outputting a second target layout result; the target layout result set includes one or more target layout results;

[0038] The target layout result set is updated according to the second target layout result.

[0039] In the above technical solution, by adjusting the initial action probability distribution through the target layout result set, the PCB layout model can provide multiple different layout results, effectively expanding the engineer's design ideas and ensuring that the layout results can meet the actual layout requirements.

[0040] In a possible implementation, updating the target layout result set according to the second target layout result includes:

[0041] When the similarity between the second target layout result and any target layout result in the target layout result set is less than a first preset threshold, adding the second target layout result to the target layout result set;

[0042] When there is a third target layout result in the target layout result set whose reward difference with the second target layout result is less than the second preset threshold, and the reward value of the second target layout result is greater than the third target layout result, the third target layout result in the target layout result set is updated to the second target layout result.

[0043] In the above technical solution, the update process of the layout result set adopts a similarity and reward value comparison mechanism. Only layout solutions that are significantly different from those in the layout result set are added to the layout result set, thereby ensuring that each layout solution in the layout result set is unique and most valuable, effectively expanding engineers' design ideas and improving layout efficiency.

[0044] In a third aspect, an embodiment of the present application provides a training device for a printed circuit board layout model, characterized in that the device comprises:

[0045] A first acquisition module is used to acquire layout task data of a printed circuit board;

[0046] A first training module is configured to input the layout task data into a printed circuit board layout model, determine a layout environment and an action mask based on the layout task data, determine a current state based on the layout environment and the action mask, determine an action strategy based on the current state, update the current state based on the action strategy to determine a next action strategy, and output a layout result after iteration is completed;

[0047] A second training module is used to update the printed circuit board layout model according to the layout result and the reward function;

[0048] Among them, the above-mentioned printed circuit board layout model is constructed based on a deep reinforcement learning algorithm.

[0049] In a possible implementation, the first training module is specifically configured to:

[0050] Determine the layout environment according to the above layout task data;

[0051] Determine layout constraints and routing constraints based on the above layout environment;

[0052] Determine the position mask according to the above layout constraints;

[0053] Determine the line length mask according to the above wiring constraints;

[0054] The above position mask and the above line length mask are superimposed to determine the action mask.

[0055] In a possible implementation, in the action mask, the initial probability of the action within the overlapping area of ​​the position mask and the line length mask is greater than the initial probability of the action outside the overlapping area.

[0056] In a possible implementation, the initial probability of the action at each position in the action mask is determined based on the congestion degree.

[0057] In a possible implementation, the first training module is specifically configured to:

[0058] Extracting real-valued vector features from the above layout environment through a multi-layer perceptron;

[0059] Extract graph features from the above action mask through graph convolutional neural network;

[0060] The current state is determined based on the above real-valued vector features and the above graph features.

[0061] In a possible implementation, the first training module is specifically configured to:

[0062] Update the layout environment and the action mask according to the action strategy;

[0063] Update the current state according to the updated layout environment and the action mask;

[0064] The next action strategy is determined based on the updated current state.

[0065] In a fourth aspect, an embodiment of the present application provides a printed circuit board layout device, the device comprising:

[0066] A second acquisition module is used to acquire target layout task data of a printed circuit board;

[0067] a first prediction module, configured to input the target layout task data into a printed circuit board layout model, determine a layout environment and an action mask based on the target layout task data, determine a current state based on the layout environment and the action mask, determine an action strategy based on the current state, update the current state based on the action strategy to determine a next action strategy, and output a first target layout result after iteration is completed;

[0068] The printed circuit board layout model is obtained by training based on any one of the above training methods.

[0069] In one possible implementation, the device includes:

[0070] a second prediction module, configured to adjust the initial motion probability distribution of target components in the target layout task data based on a target layout result set, input the adjusted target layout task data into the printed circuit board layout model, and output a second target layout result; the target layout result set may include one or more target layout results;

[0071] An updating module is used to update the target layout result set according to the second target layout result.

[0072] In a possible implementation, the update module is specifically configured to:

[0073] When the similarity between the second target layout result and any target layout result in the target layout result set is less than a first preset threshold, adding the second target layout result to the target layout result set;

[0074] When there is a third target layout result in the target layout result set whose reward difference with the second target layout result is less than the second preset threshold, and the reward value of the second target layout result is greater than the third target layout result, the third target layout result in the target layout result set is updated to the second target layout result.

[0075] In a fifth aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores instructions, and the instructions are suitable for being loaded by a processor and executing the above method steps.

[0076] In a sixth aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above method steps. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0078] Figure 1 A schematic diagram of an exemplary printed circuit board layout system architecture provided in an embodiment of the present application;

[0079] Figure 2 A flow chart of a method for training a printed circuit board layout model provided in an embodiment of the present application;

[0080] Figure 3A schematic diagram of a sequential decision-making process of an exemplary PCB layout agent provided in an embodiment of the present application;

[0081] Figure 4 A schematic diagram of an exemplary action mask generation process provided in an embodiment of the present application;

[0082] Figure 5 A schematic diagram of an exemplary feature extraction process of the current state provided in an embodiment of the present application;

[0083] Figure 6 A schematic diagram of a flow chart of a printed circuit board layout method provided in an embodiment of the present application;

[0084] Figure 7 A schematic structural diagram of a printed circuit board layout model training device provided in an embodiment of the present application;

[0085] Figure 8 A schematic structural diagram of a printed circuit board layout device provided in an embodiment of the present application;

[0086] Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0087] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0088] In the description of this application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances. In addition, in the description of this application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0089] In electronics manufacturing, PCB design and layout is a complex and meticulous process. Traditional PCB layout methods rely heavily on engineers' experience and manual labor. These methods typically involve manually placing components according to the circuit schematic and meticulously adjusting each component's position to meet requirements for electrical performance, space utilization, and manufacturing processes. During the layout process, engineers must comprehensively consider factors such as inter-component interactions, trace routing, and thermal distribution. Limited by their experience and habits, complex PCB layout tasks often make it difficult for engineers to achieve satisfactory results. Manual layout often involves numerous repetitive adjustments and modifications, significantly reducing layout efficiency. Furthermore, as the complexity of electronic devices continues to increase, this process becomes increasingly time-consuming and error-prone.

[0090] Despite the emergence of some automated PCB layout tools in recent years, these tools still have limitations. They are typically based on a set of predefined rules or simple optimization algorithms. These methods often lack flexibility and intelligence, making them difficult to adapt to the increasingly complex and diverse design requirements. For example, these tools may not perform well when handling high-density layouts or complex routing constraints, and they also struggle to automatically optimize the overall layout to improve efficiency and performance. Furthermore, these traditional methods often require engineers to perform extensive manual adjustments and optimizations, which not only increases design time but also increases the risk of errors.

[0091] In response to these limitations and shortcomings of the existing technology, this application provides a new PCB layout method and PCB layout model training method, which uses a policy neural network based on deep reinforcement learning to optimize the layout process. This method uses deep reinforcement learning technology to intelligently analyze layout data and automatically generate efficient and constraint-compliant layout solutions. Through continuous learning and iterative optimization, this method can adapt to various complex layout requirements while reducing the workload of engineers. Compared with traditional methods, this technical solution improves the degree of automation and optimization efficiency of layout, significantly improves the quality and efficiency of PCB design, and better meets the high standards of modern electronics manufacturing.

[0092] The present application is described in detail below with reference to specific embodiments.

[0093] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the architecture of a printed circuit board layout system provided by an exemplary embodiment of the present application. Figure 1 As shown, the printed circuit board layout system may include a terminal 110 and a server 120 .

[0094] Terminal 110 is connected to server 120 via network 130. Optionally, terminal 110 is a smartphone, tablet computer, laptop computer, desktop computer, smartwatch, etc., but is not limited thereto. Terminal 110 has an application installed and running that supports printed circuit board layout. Network 130 can be a wireless network or a wired network.

[0095] Server 120 is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, a Content Delivery Network (CDN), and big data and artificial intelligence platforms. Server 120 provides background services for applications running on terminal 110. In the embodiment of the present application, server 120 provides background services for applications running on the terminal that support printed circuit board layout, such as running or training a printed circuit board layout model.

[0096] Those skilled in the art will appreciate that the number of the terminals 110 and servers 120 may be greater or lesser. For example, there may be only one terminal 110 and server 120, or there may be dozens, hundreds, or even more terminals 110 and servers 120. In this case, the system architecture may also include other terminals and servers. The embodiment of the present application does not limit the number of terminals and device types.

[0097] The printed circuit board layout method or printed circuit board layout model training method involved in one or more embodiments of the present application can be implemented by a computer program and can be run on a printed circuit board layout device or a printed circuit board layout model training device based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application. The execution subject corresponding to the printed circuit board layout method or the printed circuit board layout model training method can be the above-mentioned terminal 110, the server 120, or a system composed of the above-mentioned terminal 110 and the server 120, which is determined based on the actual application environment and is not limited in the embodiments of the present application.

[0098] Next, the training method of the printed circuit board layout model provided by the embodiment of this specification is introduced with the terminal device as the execution subject. Figure 2 As shown, the training method of a printed circuit board layout model includes the following steps:

[0099] S201, obtaining layout task data of a printed circuit board.

[0100] Specifically, the layout task data of the printed circuit board can be converted from the component data on the circuit diagram (for example, converted into JSON format), and the specific data content may include but is not limited to:

[0101] Layout area information: specifically, it may include the size of the area to be laid out, information on prohibited areas, etc.

[0102] Component information: This may include the component size, shape, center point, relative coordinates of the pins, initial angle, relative coordinates of the component pad center point, pad size, pad initial angle, pad network name, etc.

[0103] Constraint information: This can include layout constraints and routing constraints. Layout constraints can include component placement constraints, such as aesthetic requirements, component orientation requirements, safe distance requirements between components with different attributes, and maximum distance requirements between components in the same circuit. Routing constraints can include maximum length limits for circuit traces, routing order constraints for components within a circuit, and circuit line width constraints.

[0104] In some embodiments, after obtaining the layout task data, component grouping processing and component placement order processing are required to optimize the quality of the layout task data and improve the performance of the model.

[0105] Specifically, component grouping involves combining multiple components with the same circuit and consistent attributes into a unified component based on the original layout task data. This component is then treated as a new component and used to generate new layout task data, reducing the size of layout tasks and reducing duplication. For example, consider three components with the same attributes: resistors, connected in series in the same circuit. These three components can be combined into a single unified component, generating new component information with new dimensions, shapes, and other data for subsequent layout tasks.

[0106] The component placement order processing is to determine the placement priority of each component according to the constraints of the layout task data. The component with higher priority is placed first.

[0107] It should be noted that during the training phase of the printed circuit board layout model, the printed circuit board layout task data is used as the training data for the model. This data may include layout task data for multiple sets of different PCB layout tasks. Using a large amount of diverse training data can help improve model performance and robustness, making the model more adaptable to various possible environmental changes, including complex layout constraints, different types of device arrangements, etc. It can also enhance generalization capabilities and avoid overfitting.

[0108] S202, input the layout task data into the printed circuit board layout model, determine the layout environment and action mask according to the layout task data, determine the current state according to the layout environment and action mask, determine the action strategy according to the current state, update the current state according to the action strategy to determine the next action strategy, and output the layout result after the iteration is completed.

[0109] Specifically, the PCB layout model is a deep reinforcement learning model based on the Markov decision process. The Markov decision process is a mathematical framework used to model the process of making a series of decisions in an uncertain environment. The Markov decision process mainly consists of the following key components:

[0110] States: States are variables that describe the environment in which a decision is made. At any given moment, the state provides complete information about the environment. In an MDP, the next state depends only on the current state and the action taken, and is independent of previous states.

[0111] Actions: Actions are behaviors that an agent can take in a specific state. The set of actions can vary from state to state.

[0112] Transition Probabilities: Transition probabilities describe the probability of reaching each possible next state given the current state and the action taken. They define the dynamic characteristics of the environment.

[0113] Reward Function: A reward function assigns a value (reward) to each state (or combination of state and action) to evaluate the quality of that state (or combination of state and action). The reward function drives the agent to learn how to choose actions to maximize the long-term cumulative reward.

[0114] Policy: A policy is a mapping from state to action, defining which action should be chosen in a given state. A policy can be deterministic (e.g., given a state, the policy always produces the same action) or stochastic (e.g., the policy provides a probability distribution for each action, and the action is chosen randomly based on this probability distribution).

[0115] In the field of reinforcement learning, an agent typically refers to an entity that interacts with its environment and learns from these interactions to make decisions. In the context of PCB layout models, the agent can be understood as the entity that executes the PCB layout model, responsible for performing layout actions, evaluating the results, and adjusting its behavioral strategy based on feedback. The agent operates according to the principles of Markov decision processes, utilizing policy neural networks to make decisions and applying deep reinforcement learning algorithms to continuously optimize these decisions, effectively solving PCB layout tasks.

[0116] For example, the PCB layout task is constructed as a deep reinforcement learning model based on Markov decision process for PCB automatic layout. In this model, the policy π θ (a t \s t ) is defined as a neural network, which transforms the current state s t As the input of the policy neural network, action a t As the output of the policy neural network, the policy neural network is updated using a deep reinforcement learning algorithm. Figure 3 As shown in the figure, in each successive decision-making process of PCB automatic layout, the strategy neural network makes decisions based on the state of the current PCB layout area (S0-S T ), determines the action of the components to be laid out at each decision time t (a0-a T-1 ). Subsequently, the agent generates corresponding rewards (r0-r T ) and infer the state at the next moment. This process continues until the deep reinforcement learning agent completes the placement of all components in the PCB module. In this way, the model effectively learns the strategy for optimal component placement in a variety of different states.

[0117] Therefore, after inputting the layout task data into the PCB layout model, the current state of the PCB must be determined. Then, the action strategy to be executed based on the current state is determined. Finally, the current state is updated based on the action strategy. This iteration continues until completion. Iterations terminate when all components have been placed or when there is no space for them. Once all components are placed, the final layout result is obtained.

[0118] In this embodiment, before determining the current state of the PCB, the state space of the PCB layout model must first be defined. The state space comprises the set of all possible states, each of which lies within this space. In the application scenario of PCB layout tasks, the definition of the state space encompasses multiple key dimensions. For example, the state space includes information about the PCB layout area. In the initial state, the PCB is empty of components. As components are gradually placed, the PCB layout area gradually forms different available layout areas. The state space also includes component information, including information about placed and unplaced components. Component information includes component center coordinates, shape, rotation angle, type, information about all pins (including pin coordinates, shape, and connected nets), and the connectivity between components. Furthermore, the state space includes various constraint information, such as whether components overlap, the aesthetics of the layout, whether components are located within the permitted layout area, and whether routing constraints are met. This comprehensive state space definition ensures that the model fully understands the complexity of the PCB layout task and makes reasonable decisions based on this understanding.

[0119] Similarly, before determining the action strategy to be executed in the current state, it is necessary to first define the action space of the PCB layout model. The action space includes the set of all possible actions, and each action strategy is located in this action space. For example, the action space can include the coordinates of the center point of the component to be placed and the rotation angle. According to the characteristics of the PCB layout task, the action space in the PCB layout model is discrete, so the entire action space is composed of A = {(c i ,w j ,v k )|c i ∈C,w j ∈W,v k ∈V}, where C={c1,c2,...,c n} represents the rotation angle of the component, W={w1,w2,...,w l} represents the horizontal coordinate of the center point of the component, V={v1,v2,...,v m} represents the vertical coordinate of the center point of the component.

[0120] The action space can be divided into available action space and unavailable action space. In a specific layout state, there may be some empty areas on the PCB board suitable for placing new components, or there may be enough space to move or rotate existing components. These areas constitute the available action space. For example, if an area is large enough to accommodate a new chip, and placing it does not violate any layout constraints, then the action of placing the chip in this area belongs to the available action space. On the other hand, if certain areas are already occupied by other components, or if placing a new component would cause a layout rule violation (such as overcrowding of components or crossing of traces), then the actions in these areas belong to the unavailable action space. For example, if an area has insufficient space for additional components, or if placing a new component in this area would cause the traces to be too long or too complex, then all placement actions in this area are considered unavailable. The available and unavailable action spaces change with the current state, and the PCB layout model dynamically adjusts the available and unavailable action spaces based on the current state.

[0121] Different actions taken during the PCB layout model decision-making process will result in different relative positions of each component placed in the layout area, which will lead to different final layout results and layout results with different degrees of constraint satisfaction.

[0122] In this embodiment, the current state of the PCB is a state in the state space that can be determined based on the current layout environment and action mask of the PCB. It is a comprehensive description of the current layout environment of the PCB. The layout environment refers to the position and layout of each component on the PCB at any given moment. It can include information about layout-available areas, already-placed areas, placed components, unplaced components, and constraints. The action mask is a filter set based on the rules and constraints of the PCB layout, such as component overlap, a certain safety distance, and the prohibition of component placement in certain areas. It is used to limit the actions that can be taken within a given state. For example, if an area is already occupied by a component, the action of placing another component in that area will be marked as infeasible by the mask. The current state of the PCB is a comprehensive representation that includes layout environment information and action masks. It provides the deep reinforcement learning model with a comprehensive view of the current layout situation and guides the PCB layout model on the legal actions that can be taken within this environment. Among them, the action mask can be used to characterize the available action space of the current state. It can be understood that the action mask can not only help define the current state (as part of the input information of the policy neural network), but also can be used to guide the action selection in the current state after the current state is determined (defining the available action space).

[0123] In some embodiments, the above-mentioned determining the layout environment and action mask based on the layout task data includes: determining the layout environment based on the layout task data; determining the layout constraints and routing constraints based on the layout environment; determining the position mask based on the layout constraints; determining the line length mask based on the routing constraints; and superimposing the position mask and the line length mask to determine the action mask.

[0124] Specifically, such as Figure 4 As shown, first, an initial layout environment can be determined based on the layout task data of the input PCB layout model. The initial layout environment can be a blank environment without any components placed, or an environment with some components pre-placed. Figure 4 View (a) in Figure 1 shows an exemplary layout environment, including an area 401 where components have been placed and an area 402 where no components have been placed. Next, based on this layout environment, layout constraints (such as the minimum distance between components and layout restrictions for specific areas) and routing constraints (such as trace length limits and trace path requirements) are determined.

[0125] Then, the position mask 403 is determined according to the layout constraints, i.e. Figure 4 The gray part in the view (b) in FIG, the position mask indicates which areas in the layout space can be used to place components. At the same time, the line length mask 404 is determined according to the routing constraints, that is, Figure 4 In the gray part of view (c), the wire length mask involves routing-related constraints, such as the maximum length the routing can reach and the possible paths.

[0126] Finally, the position mask 403 and the line length mask 404 are superimposed to form a comprehensive view mask (i.e., action mask) 405, i.e. Figure 4 In the gray portion of view (d) in Figure 3, the darker areas 406 are overlapping regions that satisfy both the position mask and the wire length mask, while the lighter areas 407 are regions that satisfy only the position mask or the wire length mask. This overlaid view mask comprehensively considers all placement and routing constraints, ensuring that any action will comply with these predefined constraints.

[0127] This method combines the position mask that reflects layout requirements with the line length mask that reflects routing requirements, which can more comprehensively guide the model's layout decisions and improve the model's performance, thereby optimizing the PCB layout effect and helping to improve the accuracy and reliability of layout decisions.

[0128] In some embodiments, the above-mentioned determination of the current state based on the layout environment and the action mask includes: extracting real-valued vector features in the layout environment through a multi-layer perceptron; extracting graph features in the action mask through a graph convolutional neural network; and determining the current state based on the real-valued vector features and the graph features.

[0129] Specifically, deep learning technology can be used to extract relevant features from the layout environment and action mask to determine the current state of the PCB layout. Figure 5 As shown in the figure, a multi-layer perceptron (MLP) is used to process the layout environment data. MLP is a neural network structure that can convert the layout environment data into real-valued vector features. The real-valued vector features are a quantitative representation of the layout environment, capturing the key information of the environment and helping the PCB layout model to fully understand the distribution and utilization of the layout space.

[0130] At the same time, a graph convolutional neural network (GCN) is used to process action masks, which define which actions are feasible in the current layout environment, such as which areas can place components and which areas are prohibited. GCN is capable of processing this graph-structured data and extracting graph features from it, which reflect the structure and constraints of the available action space.

[0131] Finally, the real-valued vector features extracted from the MLP and the graph features extracted from the GCN are combined to form a comprehensive current state representation. This combined feature set contains key information about the layout environment and the available action space, providing the deep reinforcement learning model with the comprehensive data needed to make decisions in the current layout environment, enabling the deep reinforcement learning model to more effectively understand and handle complex PCB layout tasks.

[0132] In this embodiment, after determining the current state of the PCB, the action strategy to be executed can be determined based on the current state. Specifically, the current state can be input into the policy neural network of the PCB layout model. The policy neural network can evaluate the effect of each action in the current environment and output a probability distribution for each possible action. This probability distribution reflects the feasibility and priority of each action in the current state, thereby providing decision-making guidance for the intelligent agent. Based on the output probability distribution, the intelligent agent can adopt different strategies to select the action strategy output. The action strategy is an action in the available action space corresponding to the current state. For example, an action strategy can be randomly selected based on the probability distribution to explore new possibilities; the action strategy with the highest probability can be selected, that is, the action most likely to produce the best results in the current state. The specific strategy used to select the action strategy can be determined based on the training stage of the PCB layout model and actual needs, and is not limited here.

[0133] In this embodiment, the current state of the PCB includes features of the action mask, which characterizes the available action space corresponding to the current state and can affect the probability distribution of the policy neural network output. For example, the policy neural network considers the available action space defined by the action mask and assigns a higher non-zero probability to actions within the action mask (i.e., actions in the available action space), while the probability of actions outside the action mask (i.e., actions in the unavailable action space) is significantly reduced or directly set to 0.

[0134] When the policy neural network assigns probabilities to actions in the action mask, it also considers the initial probability of each action, that is, the initial probability of the action in the action mask will affect the probability distribution of the policy neural network output.

[0135] In some embodiments, in the action mask, the initial probability of the action within the overlapping area of ​​the position mask and the line length mask is greater than the initial probability of the action outside the overlapping area.

[0136] For example, Figure 4 As shown in view (d), the dark region 406 represents the overlap of the position mask and the wire length mask, meaning that the actions in this region satisfy both placement and routing constraints. The light region 407 represents the overlap of a single position mask or wire length mask, meaning that the actions in this region satisfy only placement or routing constraints. By setting a higher initial probability for actions within the overlap region and a lower initial probability for actions outside the overlap region, the model's layout decisions are focused on areas that satisfy both placement and routing constraints, significantly improving layout quality and efficiency.

[0137] In some embodiments, the initial probability of the action at each position in the action mask is determined based on the congestion level.

[0138] The congestion level of each available placement location within the PCB layout area is an important factor in action selection. A location's congestion level is calculated as the inverse of the semi-perimeter of the net's envelope area multiplied by the importance of the net. Importance weights can be set based on expert experience or historical data.

[0139] For example, the congestion degree may be calculated as follows:

[0140] Congestion(Mx,My,Mw,Mh)=∑ (x,y)∈net a net / (Mw+Mh)

[0141] Among them, Mx, My are used to represent the coordinates of the location point, Mw, Mh are used to represent the length and width of the envelope rectangle of the network loop involved in the location point, and a netThe weight used to characterize the importance of the network loop is used. Finally, the sum of the congestion corresponding to each network loop involved in the location point is taken as the total congestion of the location point.

[0142] For example, if a region has high congestion, placing additional components there may result in an overly dense layout or complicated routing, thus reducing the action probability at that location. Conversely, regions with lower congestion are more suitable for placing new components, so the action probability at these locations increases accordingly. In this way, the action mask not only helps the agent adhere to layout rules but also guides it towards more efficient and optimized layouts. This congestion-based action probability adjustment mechanism effectively combines the technical requirements of the layout task with the optimization goals of real-world applications, thereby improving the overall quality and performance of PCB designs.

[0143] By considering the congestion of each position to determine the action probability, the layout effect is further optimized, avoiding the problem of excessive concentration or uneven distribution of components. It can significantly improve the quality and efficiency of the layout, reduce the need for subsequent adjustments and redoing, and make the PCB layout more compact, reasonable and in compliance with electrical engineering standards.

[0144] In some embodiments, updating the current state according to the action strategy to determine the next action strategy includes: updating the layout environment and action mask according to the action strategy; updating the current state according to the updated layout environment and action mask; and determining the next action strategy according to the updated current state.

[0145] In this embodiment, the agent determines an action strategy based on its current state and executes that action. Executing this action changes the PCB layout environment, including placed components and unplaced space, causing the action mask to be updated to reflect the new layout constraints. This update ensures that the agent has the latest environment information at each decision moment, allowing it to continuously and dynamically make layout decisions based on the most accurate current state.

[0146] As the agent executes layout decisions step by step, the model continuously receives updated current states through the policy neural network and outputs the next action strategy based on this current state. This continuous decision-making process not only enables the agent to adapt to real-time changes in the layout task, but also allows it to continuously optimize the layout design until the final layout goal is reached and the layout result is output. This method ensures that the agent's decisions are always based on the latest and most complete layout environment through an iterative update mechanism, thereby improving the accuracy of layout decisions and the efficiency of the overall layout process.

[0147] S203: Update the printed circuit board layout model according to the layout result and the reward function.

[0148] Specifically, the agent updates the PCB layout model based on the layout results after executing actions and a preset reward function, allowing the PCB layout model to learn and improve from the results of its actions. The reward function can be defined based on factors such as layout quality, component placement density, routing efficiency, and adherence to design rules. The PCB layout model learns how to improve its layout strategy by maximizing the reward.

[0149] For example, the agent calculates a reward based on the layout results output by the current policy neural network. The effectiveness of this layout result can be evaluated using a reward function, which can provide a numerical reward based on predefined criteria (such as constraint satisfaction, aesthetics, layout efficiency, etc.) to indicate the quality of the layout result. The policy optimization algorithm used in deep reinforcement learning is then used to update the policy neural network. This update is based on the layout results and the corresponding rewards. The policy optimization algorithm uses this information as a learning signal to adjust the weights of the policy neural network to update the PCB layout model and optimize future layout decisions.

[0150] In some embodiments, the total reward can be defined as a neural network that obtains the total reward of the layout result by inputting the state of the layout result.

[0151] In some embodiments, the reward function is used to evaluate the quality of the layout results. For the layout results, improving the utilization of the layout space and reducing the total wiring length while satisfying the constraints are the criteria for judging the layout quality. Therefore, for this goal, the reward function can be set as follows:

[0152]

[0153]

[0154] st

[0155] dist(P(i,j)) <constraint(i,j) (1)

[0156] overlap(Mx,My,Mw,Mh)=0 (2)

[0157] Congestion(Mx,My,Mw,Mh) <Cth (3)

[0158] Among them, the objective function is to minimize the envelope rectangle formed by each network loop; α net is the importance weight of the network loop to which the current component belongs, which can be derived from expert experience; (i,j) ∈net represents p (i,j)A position point in the envelope rectangle of the network loop net, Px and Py represent the horizontal and vertical coordinate values ​​of the component center respectively, Mx and My are used to represent the coordinates of the position point, and Mw and Mh are used to represent the length and width of the envelope rectangle of the network loop involved in the position point.

[0159] Constraint (1) represents the placement constraints of components, specifically the distance between pins must comply with layout and routing constraints;

[0160] Constraint (2) states that devices are not allowed to overlap.

[0161] Constraint (3) indicates that the congestion degree of each location point within the envelope rectangle cannot exceed a certain threshold Cth.

[0162] The congestion degree is calculated as follows:

[0163] Congestion(Mx,My,Mw,Mh)=∑ (x,y)∈net a net / (Mw+Mh)

[0164] The congestion degree of each position point that can be placed in the layout area is calculated by multiplying the inverse of the semi-perimeter of the envelope area of ​​the network loop where each point is located by the importance weight of each network loop; where Mx and My are used to represent the coordinates of the position point, Mw and Mh are used to represent the length and width of the envelope rectangle of the network loop involved in the position point, and a net The weight used to characterize the importance of the network loop is used. Finally, the sum of the congestion corresponding to each network loop involved in the location point is taken as the total congestion of the location point.

[0165] Policy optimization algorithms in deep reinforcement learning algorithms can include PPO (Proximal Policy Optimization), DDPG (Deep Deterministic Policy Gradient), and SAC (Soft Actor-Critic). For example, in the PPO algorithm, the agent optimizes the policy network by comparing the rewards generated by the new and old policies to achieve a more stable learning process. The DDPG algorithm combines deep learning and policy gradient methods, using the actor-critic framework to simultaneously learn a policy function (actor) and a value function (critic). The SAC algorithm, based on the actor-critic framework, incorporates entropy as an additional objective in the policy optimization process, thereby encouraging the agent to explore more action combinations and enhancing the robustness of the policy.

[0166] By continuously updating the model based on actual layout results and corresponding rewards, the intelligent agent is not only able to better adapt to the specific requirements of PCB layout, but also can quickly adjust its strategy when encountering new layout challenges, improving the performance and quality of the overall design.

[0167] Next, the printed circuit board layout method provided by the embodiment of the present application is introduced with the terminal device as the execution subject. Figure 6 As shown, the printed circuit board layout method includes the following steps:

[0168] S601: Acquire target layout task data of a printed circuit board.

[0169] Specifically, the target layout task data is obtained by converting the component data on the circuit diagram that currently needs to perform the layout task (for example, converting it into json format).

[0170] The specific data acquisition and processing steps are the same as step S201 in the training phase and will not be repeated here.

[0171] S602: Input the target layout task data of the printed circuit board into the printed circuit board layout model, determine the layout environment and action mask according to the target layout task data, determine the current state according to the layout environment and action mask, determine the action strategy according to the current state, update the current state according to the action strategy to determine the next action strategy, and output the first target layout result after the iteration is completed.

[0172] The PCB layout model is trained based on the training method described in the above embodiment. The first target layout result is the layout result calculated by the trained PCB layout model based on the input target layout task data. The specific execution steps of the PCB layout model are the same as those in step S202 of the training phase and are not further described here.

[0173] In this embodiment, by inputting target layout task data into a trained PCB layout model, the model can determine the optimal layout strategy based on the latest layout status in each iteration, and output the target layout result that meets all layout requirements after the iteration is completed. This can effectively realize intelligent and automated PCB layout, optimize PCB layout efficiency, and improve design quality and performance. Especially when handling complex or high-density layout tasks, it makes the PCB design process faster, more accurate, and more efficient.

[0174] In some embodiments, the above method also includes: adjusting the initial action probability distribution of target components in the target layout task data based on the target layout result set, inputting the adjusted target layout task data into the printed circuit board layout model, and outputting a second target layout result; and updating the target layout result set according to the second target layout result.

[0175] Specifically, the probability of important component placement actions can be initialized, such as initializing the placement angles of components such as chips and inductors. Based on the important component information of the generated layout results, the placement actions of important components are recorded, and the probability distribution of the placement actions of each important component is adjusted to reduce the placement probability of important components in the existing layout results, making the new result layout generated by the printed circuit board layout model more diverse. Among them, the target layout result set is a collection of all integrated layout results. The target layout result set can include one or more target layout results. Each time a layout result is generated, the target layout result set is updated. The newly generated second target layout result can be added to the target layout result set, or the second target layout result can be not added to the target layout result set, or the target result in the target layout result set can be replaced with the second target layout result, so that the layout results in the target layout result set have diversity.

[0176] In this embodiment, by adjusting the initial action probability distribution through the target layout result set, the PCB layout model can provide multiple different layout results, effectively expanding the design ideas of engineers and ensuring that the layout results can meet actual layout requirements.

[0177] In some embodiments, updating the target layout result set according to the second target layout result includes:

[0178] When the similarity between the second target layout result and any target layout result in the target layout result set is less than a first preset threshold, adding the second target layout result to the target layout result set;

[0179] When there is a third target layout result in the target layout result set whose reward difference with the second target layout result is less than a second preset threshold, and the reward value of the second target layout result is greater than the third target layout result, the third target layout result in the target layout result set is updated to the second target layout result.

[0180] Specifically, based on the component information of the generated layout results, the placement positions of the components are compared for similarity. If the new layout result is highly similar to the already generated layout result, the layout result will not be output. The position similarity is calculated as follows:

[0181] Q=∑ i∈I ∑ j∈I ,d ij

[0182] Where i represents the i-th component, j represents the j-th component, and d ijIt represents the distance between the center point of component i and the center point of component j. The closer the Q values ​​of two layout results are, the more similar the device positions of the two layout results are.

[0183] In addition, the reward values ​​of the layout results need to be compared, and layout results with similar reward values ​​will be discarded. This is because when the reward values ​​are the same, the corresponding two layout results may be obtained through rotation or translation. Therefore, when the reward value of a layout result differs from the reward value of an already generated layout result by no more than 5%, and when the reward value of the newly generated layout is the largest, the layout result is replaced, so that the layout results in the target layout result set are diverse and all are optimal layout results.

[0184] In this embodiment, the update process of the layout result set adopts a similarity and reward value comparison mechanism. Only layout solutions that are significantly different from those in the layout result set are added to the layout result set, thereby ensuring that each layout solution in the layout result set is unique and most valuable, effectively expanding the design ideas of engineers and improving layout efficiency.

[0185] See Figure 7 , Figure 7 This is a schematic diagram of a printed circuit board layout model training device provided in accordance with an exemplary embodiment of the present application. The printed circuit board layout model training device can be implemented as all or part of a device using software, hardware, or a combination of both. The printed circuit board layout model training device 700 includes:

[0186] A first acquisition module 710 is configured to acquire layout task data of a printed circuit board;

[0187] A first training module 720 is configured to input the layout task data into a printed circuit board layout model, determine a layout environment and an action mask based on the layout task data, determine a current state based on the layout environment and the action mask, determine an action strategy based on the current state, update the current state based on the action strategy to determine a next action strategy, and output a layout result after iteration is completed;

[0188] A second training module 730 is configured to update the printed circuit board layout model according to the layout result and the reward function;

[0189] Among them, the above-mentioned printed circuit board layout model is constructed based on a deep reinforcement learning algorithm.

[0190] In some embodiments, the first training module 720 is specifically configured to:

[0191] Determine the layout environment according to the above layout task data;

[0192] Determine layout constraints and routing constraints based on the above layout environment;

[0193] Determine the position mask according to the above layout constraints;

[0194] Determine the line length mask according to the above wiring constraints;

[0195] The above position mask and the above line length mask are superimposed to determine the action mask.

[0196] In some embodiments, in the available action space, the initial probability of the action within the overlapping area of ​​the position mask and the line length mask is greater than the initial probability of the action outside the overlapping area.

[0197] In some embodiments, the initial probability of the action at each position in the available action space is determined based on the congestion level.

[0198] In some embodiments, the first training module 720 is specifically configured to:

[0199] Extracting real-valued vector features from the above layout environment through a multi-layer perceptron;

[0200] Extract graph features from the above action mask through graph convolutional neural network;

[0201] The current state is determined based on the above real-valued vector features and the above graph features.

[0202] In some embodiments, the first training module 720 is specifically configured to:

[0203] Update the layout environment and the action mask according to the action strategy;

[0204] Update the current state according to the updated layout environment and the action mask;

[0205] The next action strategy is determined based on the updated current state.

[0206] The division of the modules in the above-mentioned training device for the printed circuit board layout model is for illustration only. In other embodiments, the training device for the printed circuit board layout model can be divided into different modules as needed to complete all or part of the functions of the above-mentioned training device for the printed circuit board layout model. The implementation of each module in the training device for the printed circuit board layout model provided in the embodiments of the present application can be in the form of a computer program. The computer program can be run on a terminal or a server. The program modules constituted by the computer program can be stored in the memory of the terminal or the server. When the computer program is executed by the processor, all or part of the steps of the training method for the printed circuit board layout model described in the embodiments of the present application are implemented.

[0207] Please refer to the following Figure 8 , Figure 8 This is a schematic diagram of the structure of a printed circuit board layout device provided in an exemplary embodiment of the present application. The printed circuit board layout device can be implemented as all or part of the device through software, hardware, or a combination of both. The printed circuit board layout device 800 includes:

[0208] The second acquisition module 810 is used to acquire target layout task data of a printed circuit board;

[0209] A first prediction module 820 is configured to input the target layout task data into a printed circuit board layout model, determine a layout environment and an action mask based on the target layout task data, determine a current state based on the layout environment and the action mask, determine an action strategy based on the current state, update the current state based on the action strategy to determine a next action strategy, and output a first target layout result after iteration is completed;

[0210] The printed circuit board layout model is obtained by training based on any one of the above training methods.

[0211] In some embodiments, the apparatus 800 includes:

[0212] a second prediction module, configured to adjust the initial motion probability distribution of target components in the target layout task data based on a target layout result set, input the adjusted target layout task data into the printed circuit board layout model, and output a second target layout result; the target layout result set may include one or more target layout results;

[0213] An updating module is used to update the target layout result set according to the second target layout result.

[0214] In some embodiments, the update module is specifically used to:

[0215] When the similarity between the second target layout result and any target layout result in the target layout result set is less than a first preset threshold, adding the second target layout result to the target layout result set;

[0216] When there is a third target layout result in the target layout result set whose reward difference with the second target layout result is less than the second preset threshold, and the reward value of the second target layout result is greater than the third target layout result, the third target layout result in the target layout result set is updated to the second target layout result.

[0217] The division of the modules in the above-described printed circuit board layout device is for illustrative purposes only. In other embodiments, the printed circuit board layout device can be divided into different modules as needed to perform all or part of the functions of the above-described printed circuit board layout device. The various modules in the printed circuit board layout device provided in the embodiments of this application can be implemented in the form of a computer program. This computer program can be executed on a terminal or server. The program modules comprising this computer program can be stored in a memory of the terminal or server. When executed by a processor, this computer program implements all or part of the steps of the printed circuit board layout method described in the embodiments of this application.

[0218] See also Figure 9 , Figure 9 This is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of this specification. Figure 9 As shown, the electronic device 900 may include: at least one processor 910, at least one communication bus 920, a user interface 930, at least one network interface 940, and a memory 950. The communication bus 920 may be used to implement connection and communication between the above components.

[0219] The user interface 930 may include a display screen (Display) and a camera (Camera), and the optional user interface may also include a standard wired interface and a wireless interface.

[0220] The network interface 940 may optionally include a Bluetooth module, a Near Field Communication (NFC) module, a Wireless Fidelity (Wi-Fi) module, and the like.

[0221] The processor 910 may include one or more processing cores. The processor 910 utilizes various interfaces and circuits to connect the various components within the electronic device 900. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 950, and accessing data stored in the memory 950, the processor 910 performs various functions and processes data for the routing electronic device 900. Optionally, the processor 910 may be implemented using at least one hardware form factor selected from the group consisting of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 910 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content displayed on the display; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 910 and may be implemented as a separate chip.

[0222] Among them, the memory 950 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 950 includes a non-transitory computer-readable medium. The memory 950 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 950 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a prediction function, a training function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 950 may also be optionally at least one storage device located away from the aforementioned processor 910. As Figure 9 As shown, the memory 950 as a computer storage medium may include an operating system, a network communication module, a user interface module, and program instructions.

[0223] Specifically, the electronic device 900 may be a training device for the printed circuit board layout model. The processor 910 may be configured to call program instructions stored in the memory 950 and perform the following operations:

[0224] Acquire layout task data of a printed circuit board;

[0225] Inputting the layout task data into a printed circuit board layout model, determining a layout environment and an action mask based on the layout task data, determining a current state based on the layout environment and the action mask, determining an action strategy based on the current state, updating the current state based on the action strategy to determine a next action strategy, and outputting a layout result after iteration is completed;

[0226] updating the printed circuit board layout model according to the layout result and the reward function;

[0227] Among them, the above-mentioned printed circuit board layout model is constructed based on a deep reinforcement learning algorithm.

[0228] In some possible embodiments, when the processor 910 determines the layout environment and the action mask according to the layout task data, it specifically performs:

[0229] Determine the layout environment according to the above layout task data;

[0230] Determine layout constraints and routing constraints based on the above layout environment;

[0231] Determine the position mask according to the above layout constraints;

[0232] Determine the line length mask according to the above wiring constraints;

[0233] The above position mask and the above line length mask are superimposed to determine the action mask.

[0234] In some possible embodiments, in the available action space, the initial probability of the action within the overlapping area of ​​the position mask and the line length mask is greater than the initial probability of the action outside the overlapping area.

[0235] In some possible embodiments, the initial probability of the action at each position in the available action space is determined based on the congestion level.

[0236] In some possible embodiments, when the processor 910 determines the current state according to the layout environment and the action mask, it specifically performs:

[0237] Extracting real-valued vector features from the above layout environment through a multi-layer perceptron;

[0238] Extract graph features from the above action mask through graph convolutional neural network;

[0239] The current state is determined based on the above real-valued vector features and the above graph features.

[0240] In some possible embodiments, when the processor 910 updates the current state according to the action strategy to determine the next action strategy, the processor 910 specifically performs:

[0241] Update the layout environment and the action mask according to the action strategy;

[0242] Update the current state according to the updated layout environment and the action mask;

[0243] The next action strategy is determined based on the updated current state.

[0244] In some possible embodiments, the electronic device 900 may be the aforementioned printed circuit board layout device, and the processor 910 may further specifically execute:

[0245] Acquire target layout task data of a printed circuit board;

[0246] Inputting target layout task data of a printed circuit board into a printed circuit board layout model, determining a layout environment and an action mask according to the target layout task data, determining a current state according to the layout environment and the action mask, determining an action strategy according to the current state, updating the current state according to the action strategy to determine a next action strategy, and outputting a first target layout result after iteration is completed;

[0247] The printed circuit board layout model is obtained by training based on any one of the above training methods.

[0248] In some possible embodiments, the processor 910 further specifically performs:

[0249] Adjusting the initial motion probability distribution of target components in the target layout task data based on the target layout result set, inputting the adjusted target layout task data into the printed circuit board layout model, and outputting a second target layout result; the target layout result set includes one or more target layout results;

[0250] The target layout result set is updated according to the second target layout result.

[0251] In some possible embodiments, when the processor 910 updates the target layout result set according to the second target layout result, it specifically performs:

[0252] When the similarity between the second target layout result and any target layout result in the target layout result set is less than a first preset threshold, adding the second target layout result to the target layout result set;

[0253] When there is a third target layout result in the target layout result set whose reward difference with the second target layout result is less than the second preset threshold, and the reward value of the second target layout result is greater than the third target layout result, the third target layout result in the target layout result set is updated to the second target layout result.

[0254] Embodiments of the present application also provide a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps of the aforementioned embodiments. The aforementioned printed circuit board layout model training device or the various modules of the printed circuit board layout device, if implemented as software functional units and sold or used as independent products, may be stored in the aforementioned computer-readable storage medium.

[0255] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The above-mentioned computer program product includes one or more computer instructions. When the above-mentioned computer program instructions are loaded and executed on a computer, the above-mentioned process or function according to the embodiment of the present application is generated in whole or in part. The above-mentioned computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The above-mentioned computer instructions can be stored in a computer-readable storage medium or transmitted by the above-mentioned computer-readable storage medium. The above-mentioned computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The above-mentioned computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The above-mentioned available media can be magnetic media (for example, floppy disks, hard disks, tapes), optical media (for example, digital versatile discs (DVDs)), or semiconductor media (for example, solid state disks (SSDs)).

[0256] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When executed, the program can include the processes of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks. The technical features of this embodiment and the implementation scheme can be combined in any manner unless they conflict.

[0257] The embodiments described above are merely preferred embodiments of the present application and are not intended to limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements made to the technical solutions of the present application by ordinary technicians in this field should fall within the scope of protection determined by the claims.

[0258] The foregoing description describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims and the specification can be performed in a different order than that described in the embodiments described in the specification and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for training a printed circuit board layout model, characterized in that: The method comprises: Acquire layout task data of a printed circuit board; Inputting the layout task data into a printed circuit board layout model, determining a layout environment and an action mask according to the layout task data, determining a current state according to the layout environment and the action mask, determining an action strategy according to the current state, updating the current state according to the action strategy to determine a next action strategy, and outputting a layout result after iteration is completed; updating the printed circuit board layout model according to the layout result and the reward function; The printed circuit board layout model is constructed based on a deep reinforcement learning algorithm.

2. The method according to claim 1, characterized in that The determining of the layout environment and the action mask according to the layout task data includes: determining a layout environment according to the layout task data; Determining layout constraints and routing constraints according to the layout environment; determining a position mask according to the layout constraint; Determining a line length mask according to the wiring constraint condition; The position mask and the line length mask are superimposed to determine an action mask.

3. The method according to claim 2, characterized in that In the action mask, the initial probability of the action within the overlapping area of ​​the position mask and the line length mask is greater than the initial probability of the action outside the overlapping area.

4. The method according to claim 3, characterized in that The initial probability of the action at each position in the action mask is determined based on the congestion degree.

5. The method according to claim 1, wherein The determining the current state according to the layout environment and the action mask includes: extracting real-valued vector features from the layout environment using a multi-layer perceptron; extracting graph features from the action mask using a graph convolutional neural network; A current state is determined according to the real-valued vector feature and the graph feature.

6. The method according to claim 1, wherein The updating of the current state according to the action strategy to determine a next action strategy includes: updating the layout environment and the action mask according to the action strategy; updating the current state according to the updated layout environment and the action mask; The next action strategy is determined according to the updated current state.

7. A printed circuit board layout method, characterized in that: The method comprises: Acquire target layout task data of a printed circuit board; Inputting the target layout task data into a printed circuit board layout model, determining a layout environment and an action mask according to the target layout task data, determining a current state according to the layout environment and the action mask, determining an action strategy according to the current state, updating the current state according to the action strategy to determine a next action strategy, and outputting a first target layout result after iteration is completed; The printed circuit board layout model is obtained by training based on the training method according to any one of claims 1 to 6.

8. The method according to claim 7, characterized in that The method further comprises: Adjusting the initial motion probability distribution of target components in the target layout task data based on the target layout result set, inputting the adjusted target layout task data into the printed circuit board layout model, and outputting a second target layout result; the target layout result set includes one or more target layout results; The target layout result set is updated according to the second target layout result.

9. The method according to claim 8, characterized in that Updating the target layout result set according to the second target layout result includes: When the similarity between the second target layout result and any target layout result in the target layout result set is less than a first preset threshold, adding the second target layout result to the target layout result set; When there is a third target layout result in the target layout result set whose reward difference with the second target layout result is less than a second preset threshold, and the reward value of the second target layout result is greater than the third target layout result, the third target layout result in the target layout result set is updated to the second target layout result.

10. A training device for a printed circuit board layout model, characterized in that: The device comprises: A first acquisition module is used to acquire layout task data of a printed circuit board; A first training module is configured to input the layout task data into a printed circuit board layout model, determine a layout environment and an action mask based on the layout task data, determine a current state based on the layout environment and the action mask, determine an action strategy based on the current state, update the current state based on the action strategy to determine a next action strategy, and output a layout result after iteration is completed; A second training module, configured to update the printed circuit board layout model according to the layout result and the reward function; The printed circuit board layout model is constructed based on a deep reinforcement learning algorithm.

11. A printed circuit board layout device, characterized in that: The device comprises: A second acquisition module is used to acquire target layout task data of a printed circuit board; a first prediction module, configured to input target layout task data of a printed circuit board into a printed circuit board layout model, determine a layout environment and an action mask according to the target layout task data, determine a current state according to the layout environment and the action mask, determine an action strategy according to the current state, update the current state according to the action strategy to determine a next action strategy, and output a first target layout result after iteration is completed; The printed circuit board layout model is obtained by training based on the training method according to any one of claims 1 to 6.

12. A computer storage medium, characterized in that The computer storage medium stores instructions, which are suitable for being loaded by a processor and executing the method steps of any one of claims 1 to 6 or any one of claims 7 to 9.

13. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps of any one of claims 1 to 6 or the method steps of any one of claims 7 to 9.

Images