Interconnection network, chip, topology generation method, product, device and medium

Abstract

The invention discloses an interconnection network, a chip, a topology generation method, a product, equipment and a medium, and relates to the technical field of chip integration, and the interconnection network comprises a core particle layer which comprises at least one core particle group; an active interposer connected to the core particle layer; the active interposer comprises a first active interposer and a second active interposer, and the first active interposer is located between the core particle layer and the second active interposer; the first active intermediate layer comprises an interface router device, an inter-group router device and a configurable crossbar switch network, and the second active intermediate layer comprises an inter-group relay router device and an inter-group bandwidth network; the central node of the configurable crossbar switch network configures an inter-group router device, the core grain configures an interface router device, and the interface router device is also connected with a gating node of the configurable crossbar switch network; the inter-group router device is connected with the inter-group bandwidth network, and the inter-group relay router device is configured in the inter-group bandwidth network, so that the problem of communication bottleneck is solved, and the effect of improving the communication bottleneck is achieved.

Description

Technical Field

[0001] The present application relates to the field of chip integration technology, and in particular to an interconnection network, a chip, a topology generation method, a product, a device and a medium. Background Art

[0002] With the continuous growth of computing demand and the development of chip technology, multi-chip integrated systems have gradually become an important solution for high-performance computing and complex application scenarios. Multi-chip integrated systems integrate multiple independent chips or chiplets in one package to form a powerful computing platform.

[0003] In the related technologies, communication methods based on bus architecture and mesh structure, although they perform well in single-chip systems, often face problems of insufficient bandwidth and excessive latency in multi-chip integrated systems. Especially in large-scale data transmission tasks, the communication bottleneck problem caused by traditional communication architecture becomes more obvious. Summary of the invention

[0004] The present application provides an interconnection network, a chip, a topology generation method, a product, a device and a medium to at least solve the communication bottleneck problem in the related technology.

[0005] The present application provides a multi-chip interconnection network, comprising: A core particle layer, comprising at least one core particle group, wherein the core particle group comprises at least two core particles; Active interposer layer, connecting the core particle layer; The active intermediary layer includes a first active intermediary layer and a second active intermediary layer, and the first active intermediary layer is located between the core particle layer and the second active intermediary layer; The first active intermediary layer includes an interface router component, an inter-group router component and a configurable cross switch network, and the second active intermediary layer includes an inter-group relay router component and an inter-group bandwidth network; the central node of the configurable cross switch network configures the inter-group router component, the core particle configures the interface router component, and the interface router component is also connected to the selection node of the configurable cross switch network; the inter-group router component is connected to the inter-group bandwidth network, and the inter-group relay router component is configured in the inter-group bandwidth network.

[0006] The present application also provides a chip, comprising: any one of the above-mentioned multi-chip interconnection networks.

[0007] The present application also provides a network topology generation method, which is implemented based on any of the above multi-chip interconnection networks; the network topology generation method includes: Obtaining the current strategy network based on the multi-chip interconnection network; Based on the current policy network, sample and execute multiple network topology change actions, and obtain the performance indicators of the network topology after the actions; Based on the performance indicators, the parameters of the policy network are updated, and the sampling and updating steps are iterated until the preset stopping condition is reached.

[0008] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned network topology generation methods when executing the computer program.

[0009] The present application also provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned network topology generation methods are implemented.

[0010] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned network topology generation methods when executed by a processor.

[0011] Through the present application, since the multi-chip interconnection network is based on a multi-layer active intermediary layer including a router device and a configurable cross switch network, and the interface router device, the inter-group router device and the configurable cross switch network are arranged in the first active intermediary layer, the inter-group relay router device and the inter-group bandwidth network are arranged in the second active intermediary layer, the central node of the configurable cross switch network configures the inter-group router device, the chip configures the interface router device, and the interface router device is also connected to the selection node of the configurable cross switch network; the inter-group relay router device connects the inter-group router device and the inter-group bandwidth network, thus, by setting the configurable cross switch network, the multi-chip interconnection network can flexibly and dynamically adjust the data flow direction, thereby improving the overall adaptability and resource By setting the inter-group router device of the first active intermediary layer and the inter-group relay router device of the second active intermediary layer, the core particles in the core particle group are connected to the inter-group bandwidth network through the configurable cross switch network, so as to realize the large data transmission between the core particle groups and improve the communication efficiency between the core particles; at the same time, by setting the configurable cross switch network, the inter-group router device and the inter-group relay router device, the data path can be effectively isolated and optimized, the parallel processing capability between different computing tasks is improved, and the delay problem caused by data conflict is avoided; thereby, the communication bottleneck problem caused by the traditional architecture is solved, and the technical effect of increasing bandwidth, reducing delay, and improving adaptability and resource utilization is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 A schematic diagram of the structure of a multi-chip interconnection network provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of another multi-chip interconnection network provided in an embodiment of the present application; Figure 3 A schematic diagram of a flow chart of a network topology generation method provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of a network topology generation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0015] It should be noted that, in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0016] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0017] First, the relevant technical content is explained.

[0018] The single-chip system / architecture in the related technology has been unable to meet the growing computing needs and communication bandwidth requirements when facing increasingly complex computing tasks. Especially when dealing with large-scale parallel computing tasks, the single-chip system / architecture in the related technology often faces many bottlenecks such as high power consumption, large latency, and poor scalability. Therefore, multi-chip integrated systems came into being and became one of the key technologies to improve computing performance, optimize resource utilization, and reduce power consumption.

[0019] The multi-chip integrated system integrates multiple independent chips or chiplets in one package to form a powerful computing platform. The multi-chip integrated system can integrate chips with different functions, give full play to the advantages of chips with different functions, and realize heterogeneous computing and multi-task parallel processing.

[0020] Compared with single-chip systems, multi-chip integrated systems not only have significantly improved computing performance, but also have higher flexibility and scalability. However, in the design and implementation of multi-chip integrated systems, there are also a series of new challenges, especially in terms of inter-chip communication, network topology design, power consumption management and system performance optimization. There is still room for optimization.

[0021] First of all, one of the problems in multi-chip integrated systems is the communication efficiency between chips. Since multiple chips need to exchange a large amount of data under high bandwidth and low latency, how to design an efficient inter-chip communication network has become the key to the design of multi-chip integrated systems. The communication methods in related technologies, such as communication methods based on bus architecture and grid structure, although they perform well in single-chip systems, often face problems of insufficient bandwidth and high latency in multi-chip integrated systems; especially in large-scale data transmission tasks, the bottleneck problem of traditional communication architecture becomes more obvious, seriously affecting the overall performance of multi-chip integrated systems.

[0022] Secondly, the network topology design of the multi-chip integrated system is also an optimizable direction. In a multi-chip system, the network topology directly determines the data transmission path and communication efficiency between chips. Although the fixed topology structure in the related art is simple in design and easy to implement, it often exhibits problems of poor flexibility and insufficient adaptability when faced with complex and changeable computing tasks. In response to this, the embodiment of the present application proposes a network topology structure that can be dynamically adjusted, so that it can flexibly adapt to different application scenarios, thereby improving the flexibility of the network topology structure.

[0023] In addition, with the continuous improvement of chip integration, multi-chip systems are also facing severe challenges in power consumption management. Although the power consumption optimization methods in related technologies, such as reducing frequency or reducing voltage, can reduce power consumption to a certain extent, they often sacrifice system performance and it is difficult to achieve the best balance between power consumption and performance. In view of this, the embodiments of the present application propose an optimization method that can reduce power consumption while maintaining high system performance.

[0024] Specifically, in response to at least one of the above-mentioned technical problems, an embodiment of the present application provides a multi-chip interconnection network, which is essentially an innovative multi-chip integrated system, which introduces an innovative reconfigurable network architecture and provides a dynamically optimized network topology generation method for the multi-chip interconnection network to improve the computing and communication performance of the system, reduce power consumption, and achieve flexible adaptability of the multi-chip system in different application scenarios.

[0025] Specifically, an embodiment of the present application provides a multi-chip interconnection network based on a multi-layer active interposer, wherein the multi-layer active interposer includes a configurable cross-switch network, which can specifically be a star-shaped configurable cross-switch network. The active interposer also includes a multi-layer router device structure. For example, by setting inter-group router devices, such as inter-group high-speed routing and inter-group relay router devices, large data transmission between chipsets / chip groups is achieved, greatly improving communication efficiency.

[0026] At the same time, the network topology is optimized through a policy network based on a graph convolutional network and a reinforcement learning algorithm. During each optimization process, the network topology can be gradually adjusted according to the feedback of performance indicators such as system latency, power consumption, and area, thereby achieving optimal performance under different workloads and improving the flexibility of the network topology.

[0027] Therefore, the embodiments of the present application not only provide a multi-chip interconnection network and realize an innovative multi-chip integrated system architecture, but also provide a network topology optimization method based on the multi-chip integrated system, which significantly improves the overall performance of the system and provides important technical support for the widespread application of multi-chip integrated systems in high-performance computing and complex application scenarios.

[0028] The embodiments of the present application are described below in conjunction with the accompanying drawings.

[0029] For example, Figure 1 A schematic diagram of a multi-chip interconnection network provided in an embodiment of the present application. Figure 1 The multi-chip interconnection network 10 includes a chip layer 11 and an active interposer 12. The chip layer 11 is used to arrange the chip 111. Specifically, the chip in the chip layer 11 can be arranged in groups, as described below. The active interposer 12 connects the chip layer 11; the active interposer 12 is used to arrange structures such as router devices, configurable cross-switch networks, and inter-group bandwidth networks. The connection relationship between the structures in the active interposer 12 and the connection relationship between the structures in the active interposer 12 and the chip layer 11 can be described below.

[0030] The core particle layer 11 includes at least one core particle group 110, and the core particle group 110 includes at least two core particles 111. Exemplarily, the number of core particle groups 110 may be 1, 2, 3 or more groups, which is not limited here; the number of core particles 111 in each core particle group 110 may be 2, 3 or more, which is not limited here.

[0031] Among them, the active intermediary layer 12 includes a first active intermediary layer 121 and a second active intermediary layer 122, and the first active intermediary layer 121 is located between the core particle layer 11 and the second active intermediary layer 122; the first active intermediary layer 121 includes an interface router component R1, an inter-group router component R2 and a configurable cross switch network 13, and the second active intermediary layer 122 includes an inter-group relay router component R3 and an inter-group bandwidth network 14; the central node of the configurable cross switch network 13 is configured with the inter-group router component R2, the core particle 111 is configured with the interface router component R1, and the interface router component R1 is also connected to the selection node of the configurable cross switch network 13; the inter-group router component R2 is connected to the inter-group bandwidth network 14, the inter-group relay router component R3 is configured in the inter-group bandwidth network 14, and the bandwidth network 14 can be connected to the central controller.

[0032] Specifically, the core particle 111 is configured with an interface router component R1, the interface router component R1 is connected to the selection node of the configurable cross switch network 13, the intermediate node of the configurable cross switch network 13 is configured with an inter-group router component R2, the inter-group router component R2 is connected to the inter-group bandwidth network 14, and the inter-group relay router component R3 is configured in the inter-group bandwidth network 14, thereby forming a module corresponding to a core particle group 110, including the interface router component R1, the configurable cross switch network 13, the inter-group router component R2, the inter-group bandwidth network 14 and the inter-group relay router component R3, realizing a layered architecture and modular design, improving the scalability of the multi-core interconnection network 10, and facilitating the flexible adjustment of the system architecture according to the application scenario requirements to adapt to computing tasks of different scales without having to significantly modify the system architecture.

[0033] In the multi-chip interconnection network 10 provided in the embodiment of the present application, by setting a configurable cross switch network 13, the multi-chip interconnection network 10 can flexibly and dynamically adjust the data flow direction, thereby improving the overall adaptability and resource utilization; by setting the inter-group router device R2 of the first active intermediary layer 121 and the inter-group relay router device R3 of the second active intermediary layer 122, the chiplets 111 in the chiplet group 110 are connected to the inter-group bandwidth network 14 through the configurable cross switch network 13, thereby realizing the transmission of large data between the chiplet groups 110 and improving the communication efficiency between the chiplets 111; at the same time, by setting the configurable cross switch network 13, the inter-group router device R2 and the inter-group relay router device R3, it is possible to effectively isolate and optimize the data path, improve the parallel processing capability between different computing tasks, and avoid the delay problem caused by data conflicts.

[0034] In some embodiments, Figure 2 A schematic diagram of another multi-chip interconnection network provided in an embodiment of the present application. Figure 2, the configurable crossbar switch network 13 includes a star-shaped configurable crossbar switch network; a core particle 111 is configured with at least two interface router components R1, the interface router component R1 is configured with a cumulative transmitter 15, and the cumulative transmitter 15 is configured to count the data flow of the current interface router component R1, which can provide node characteristic data for the optimization of the network topology structure. Exemplarily, the number of interface router components R1 configured by a core particle 111 can be 2, 3 or more, which is not limited here.

[0035] The configurable crossbar switch network is a grid network, specifically a star-shaped configurable crossbar switch network, which can realize flexible adjustment of the connection state to dynamically optimize the network topology. In other implementations, the configurable crossbar switch network can also be set as a configurable crossbar switch network of other structures, which is not limited here.

[0036] In the multi-chip interconnection network provided by the embodiment of the present application, a reconfigurable network architecture based on a star-shaped configurable cross switch network is provided. Based on the connection state adjustment of the star-shaped configurable cross switch network, the connection relationship between the chips can be dynamically adjusted. The architecture can adapt to the computing and communication requirements in different application scenarios, and improve the flexibility and adaptability of the system. At the same time, by setting a multi-layer active intermediary layer 12 including a router component, specifically, an inter-group router component R2 of the first active intermediary layer 121 and an inter-group relay router component R3 of the second active intermediary layer 122, the chip 111 in the chip group 110 is connected to the inter-group bandwidth network 14 through the configurable cross switch network 13, so that large data transmission between the chip groups 110 is realized, and the communication efficiency between the chip 111 is improved; at the same time, by setting the configurable cross switch network 13, the inter-group router component R2 and the inter-group relay router component R3, the data path can be effectively isolated and optimized, the parallel processing capability between different computing tasks is improved, and the delay problem caused by data conflict is avoided.

[0037] In some embodiments, continue to refer to Figure 2 The chip group 110 further includes an intra-group router component 112, which connects at least two chiplets 111 in the chip group 110. The intra-group router component 112 can communicate with the intra-group router components 112 in other chip groups 110 through the interface router component R1 and the configurable crossbar switch network 13. The communication paths between different chip groups 110 can also include an inter-group router component R2, an inter-group relay router component R3, and an inter-group bandwidth network 14, which can be flexibly adjusted based on the application scenario and are not limited here.

[0038] In the embodiment of the present application, by setting the intra-group router device 112, communication between different corelets 111 in the same corelet group 110 can be realized. At the same time, the intra-group router device 112 can also be connected to the configurable crossbar switch network 13 through the interface router device R1. Based on the connection state adjustment of the configurable crossbar switch network, the connection relationship between the corelets can be dynamically adjusted to adapt to the computing and communication requirements in different application scenarios, thereby improving the flexibility and adaptability of the system. At the same time, by setting up a multi-layer active intermediary layer 12 including a router device, specifically, an inter-group router device R2 of the first active intermediary layer 121 and an inter-group relay router device R3 of the second active intermediary layer 122, the chiplets 111 in the chiplet group 110 are connected to the inter-group bandwidth network 14 through a configurable cross-switch network 13, thereby realizing large data transmission between the chiplet groups 110 and improving the communication efficiency between the chiplets 111; at the same time, by setting up a configurable cross-switch network 13, an inter-group router device R2 and an inter-group relay router device R3, effective isolation and optimization of data paths can be achieved, thereby improving the parallel processing capability between different computing tasks and avoiding delay problems caused by data conflicts.

[0039] In some embodiments, in the multi-chip interconnection network 10, the configurable crossbar switch network may include a configurable crossbar switch and a configurable switch controller (not shown in the figure), and the configurable switch controller is configured to control the connection state of the configurable crossbar switch in the configurable crossbar switch network 13 and select the target selection node, that is, the configurable switch controller selectively connects the target selection node and the central node of the configurable crossbar switch network 13 by controlling the connection state of the configurable crossbar switch in the configurable crossbar switch network 13, so as to connect the corresponding inter-group router components and interface router components.

[0040] Exemplarily, the configurable crossbar switch may include a multiple-to-one multiplexer, such as an eight-to-one multiplexer, and select one of the paths to be turned on based on the configurable switch controller to achieve gating of the corresponding path.

[0041] In an embodiment of the present application, the configurable switch controller can also be connected to the network reconstruction main control core, which controls the connection state of the configurable cross switch through the configurable switch controller to determine the pairing relationship between each input port and output port. In addition, the input end and output end of the configurable cross switch correspond to one input and multiple optional outputs respectively.

[0042] Each core particle corresponds to two or more interface router components, and the interface router components are configured in the first active intermediary layer. The embodiment of the present application constructs a star connection network through a configurable cross switch, and the center of the star connection network is configured with an inter-group router component, which can be an inter-group high-speed routing. Two or more core particles constitute a core particle group. The inter-group router component is connected to the second active intermediary layer through an inter-group relay router component. The setting of inter-group high-speed routing and inter-group relay router components realizes large data transmission between core particle groups, greatly improves the communication efficiency between core particles, overcomes the communication bottleneck problem in traditional systems, and ensures high-bandwidth and low-latency data transmission.

[0043] In the embodiment of the present application, a chiplet routing network is set in the chiplet group, and the communication network between the chiplet groups includes a star connection network based on a configurable cross switch, an inter-group router device, an inter-group relay router device, and an inter-group bandwidth network. When the star connection network is implemented in the first active intermediary layer, the interface router device and the configurable cross switch are located in the first active intermediary layer, such as Figure 2 As shown. The communication between the routing network in the core particle and the interface router device is carried out through the micro-bump and the physical layer transceiver. The communication between the interface router device and the configurable crossbar switch does not need to pass through the micro-bump and the physical layer transceiver, which is conducive to reducing communication delay. The core of the star connection network is the inter-group high-speed routing. The inter-group high-speed routing is connected to the second active interposer through the inter-group relay router device to form an efficient data transmission channel, thereby realizing large data transmission between core particle groups.

[0044] The multi-chip interconnection network provided by the embodiments of the present application provides a reconfigurable network architecture, thereby improving the computing and communication performance of the chip system in variable application scenarios.

[0045] In an embodiment of the present application, a multi-chip interconnection network includes a chip layer and an active intermediary layer, wherein the active intermediary layer includes two or more layers of active intermediary layers; the first active intermediary layer includes a star-shaped configurable cross-switch network, each chip is configured with two or more interface router components, and each interface router component is configured with an accumulation transmitter; the central node of the star-shaped configurable cross-switch network is configured with an inter-group router component; the second active intermediary layer includes an inter-group relay router component and an inter-group bandwidth network, such as an inter-group high-bandwidth network. The embodiment of the present application supports more than three interconnection network topology reconstructions, and the delay between the CPU (central processing unit) and the chip is less than 60ns, and the bandwidth is higher than 120GBps.

[0046] On the basis of the above-mentioned implementation manner, an embodiment of the present application further provides a chip, which may include any one of the multi-chip interconnection networks provided in the above-mentioned implementation manner, and has corresponding beneficial effects, which may be understood with reference to the above text.

[0047] Among them, chips (such as AI chips) are modules used to process a large number of computing tasks in artificial intelligence applications, and may also be called AI accelerators or computing cards.

[0048] In the embodiments of the present application, by configuring the chip to include any of the multi-chip interconnection networks provided in the above embodiments, corresponding beneficial effects can be achieved. The multi-chip interconnection network / integrated system provided in the embodiments of the present application has the following advantages: High-bandwidth communication: By introducing inter-group high-bandwidth networks, inter-group relay routing, and inter-group high-speed routing in the active intermediary layer, the communication efficiency between chiplets is greatly improved, providing higher bandwidth support and ensuring smoother data transmission between chiplets, which is especially important in large-scale data processing tasks.

[0049] Flexibility and configurability: A reconfigurable network architecture based on a multi-layer active interposer is proposed. The flexibility of the system is increased by setting up a configurable crossbar switch network and configuring the interface router of the accumulator transmitter and high-speed routing between groups. Specifically, through the configurable crossbar switch network, the connection relationship between the core particles can be dynamically adjusted according to application requirements, and the data flow direction can be adjusted, which significantly improves the flexibility and communication efficiency of the system, and improves the adaptability and resource utilization of the overall system.

[0050] Low latency: Because each chip contains multiple interface router devices and performs direct and efficient routing through the active interposer, the latency in data transmission is reduced, allowing the system to better guarantee performance when performing high-speed computing tasks.

[0051] Inter-group isolation and efficient routing: The inter-group relay router components and inter-group router components configured in the active intermediary layer improve the parallel processing capabilities between different computing tasks and avoid delays caused by data conflicts by effectively isolating and optimizing data paths.

[0052] Enhanced modularity and scalability: Through the combination of layered architecture and multi-layer active intermediary layers, as well as modular settings, the system has strong scalability. The number of cores and routers can be flexibly increased or decreased according to application requirements, and the network topology can be adjusted to adapt to computing tasks of different scales without significantly modifying the system architecture.

[0053] The embodiments of the present application also provide a network topology generation method, which is essentially a dynamically optimized network topology generation method for a multi-core interconnected network, wherein by introducing a reinforcement learning algorithm, especially a policy network based on a graph convolutional network, reinforcement learning is combined with a graph convolutional network (i.e., a graph neural network, Graph Neural Network, GNN), providing a method that can dynamically optimize the network topology structure according to system performance indicators, for example, it can dynamically adjust the network topology structure according to feedback of performance indicators such as system delay, power consumption and area. Through this dynamic optimization method, the embodiments of the present application can achieve a balance between delay, area and power consumption, ensuring that the system can achieve optimal performance under different workloads, thereby not only improving system performance, but also achieving a balance between delay, area and power consumption.

[0054] In the multi-chip integrated system in the related art, the configuration method usually relies on a fixed topology and a static routing mechanism, in which a predefined topology like a grid is often used, and the connection relationship between nodes is determined in the design stage. Although this fixed configuration method is relatively simple to design and implement, it has poor flexibility and adaptability and cannot effectively cope with the changing computing and communication requirements in practical applications. For example, in large-scale parallel computing tasks, if inter-group communication relies on a fixed bus or low-bandwidth connection, it is easy to cause bottlenecks when facing large amounts of data transmission, resulting in limited performance; and the means for optimizing power consumption and area are limited, usually only achieved by reducing the frequency, and it is difficult to achieve the best performance balance.

[0055] In comparison, the multi-chip integrated system provided by the embodiments of the present application adopts an innovative reconfigurable network architecture. By setting up a configurable cross-switch network and interface router components, the system can dynamically adjust the connection relationship between the chips according to the actual operation needs. This dynamic configuration capability enables the system to flexibly adapt to different application scenarios, optimize data flow and resource utilization, thereby significantly improving computing and communication performance. In addition, the embodiments of the present application also introduce mechanisms such as inter-group high-speed routing and inter-group relay routing. Through efficient inter-group communication design, the data transmission capacity between chip groups is greatly improved, avoiding the communication bottleneck problem in related technologies. At the same time, in the above-mentioned multi-chip interconnection network, due to the large number of networks, many nodes and complex configuration, the embodiments of the present application specifically propose a network topology generation method based on any of the above-mentioned multi-chip interconnection networks / integrated systems to optimize the network topology structure.

[0056] For example, Figure 3 A schematic diagram of a flow chart of a network topology generation method provided in an embodiment of the present application. Figure 3 , the network topology generation method may include the following steps: S31, obtaining a current strategy network based on a multi-chip interconnection network.

[0057] Specifically, the current strategy network is used to process the network topology of the multi-chip interconnection network. Exemplarily, the current strategy network may use a graph convolutional network or other neural networks capable of processing graph structure data, which are not limited here.

[0058] S32. Based on the current policy network, sample and execute multiple network topology change actions, and obtain performance indicators of the network topology after the actions.

[0059] Specifically, according to the current policy network, multiple schemes (actions) of network topology changes are sampled from the action space (the action space composed of possible actions), these actions are executed, and the performance indicators of the network topology after each action is executed are recorded. The performance indicators are used to guide the selection of optimization actions.

[0060] S33. Based on the performance indicators, update the parameters of the policy network, and iterate the sampling and updating steps until the preset stopping condition is reached.

[0061] Specifically, according to the performance indicators of the network topology after the action is executed, the parameters of the strategy network are updated, that is, network optimization is performed; and the sampling and updating steps are continuously iterated until the preset stop condition is reached to generate an optimized network topology structure.

[0062] In the embodiments of the present application, the network topology can be dynamically adjusted based on the reinforcement learning algorithm. Specifically, in combination with a policy network based on a graph convolutional network, the network topology can be gradually adjusted during each optimization process based on the feedback of performance indicators such as system latency, power consumption, and area. This not only allows the network topology to be adjusted dynamically, but also allows a balance to be achieved between latency, area, and power consumption, ensuring that the system can achieve optimal performance under different workloads. Compared to the static configuration method in the related art, the network topology generation method provided in the embodiments of the present application can give full play to the potential of the multi-core system and help improve the overall system performance.

[0063] In some embodiments, the network structure of the policy network adopts a graph convolutional network; the graph convolutional network outputs a probability distribution of possible actions based on the graph structure data corresponding to the network topology; the probability distribution is used to determine the target action performed to update the network topology.

[0064] In an embodiment of the present application, a graph convolutional network can be used to process the network topology. A graph convolutional network is a neural network that can process graph structure data, can effectively capture the information of nodes and edges in the network topology graph structure, and output the action probability distribution for decision-making.

[0065] In some embodiments, the graph structure data corresponding to the network topology includes node features and an adjacency matrix; the node features include the computing power of the core particles, the adjacency matrix includes edge features, and the edge features include bandwidth and latency.

[0066] In an embodiment of the present application, in a method for optimizing a multi-chip interconnected network topology structure, a graph convolutional network is used to extract high-order features in the graph structure, such as node features and edge features. The edge features may be included in an adjacency matrix, and finally a probability distribution π(S, A) of each possible action is output, and the action performed to optimize the network topology structure is determined based on the probability distribution.

[0067] Among them, node features X v Represents the characteristics of each core node, such as computing power, storage capacity, etc. Edge features X e Represents the edge features between connected nodes, such as bandwidth, delay, etc. Adjacency matrix X Represents the connection relationship between nodes in the network, and the adjacency matrix X Extractable edge features X e .

[0068] In the embodiment of the present application, the network structure of the policy network adopts a graph convolutional network. In which, the graph convolutional layer (GCNLayer) connects the adjacency matrix X and the node features X v Perform graph convolution operations to extract high-level features in the graph structure.

[0069] Specifically, the formula for the graph convolution operation can be expressed as:

[0070] in, is the node feature matrix of the lth layer, and its initial value is , is the normalized adjacency matrix, is the learnable weight matrix of layer l, is an activation function, such as the ReLU function.

[0071] The features extracted by the last layer of graph convolution are flattened and converted into the probability distribution of the output action through a series of fully connected layers. Among them, the output layer uses the Softmax function to ensure that the sum of the action probability distribution is 1, which can be specifically expressed as:

[0072] in, is the weight matrix of the output layer, It is the feature representation after the last layer of graph convolution.

[0073] The input of the graph convolutional network is the current graph structure, including node features and edge features (or adjacency matrix), and the output is the probability distribution π(S,A) of each possible action. π(S,A) represents the probability distribution of executing each action A under the current state S.

[0074] For example, assuming that in the current state S, there is the following possible action set A={A1,A2,A3,A4}, the probability distribution π(S,A) that the policy network may output is as follows: π(S,A1)=0.35(add nodes) π(S,A2)=0.15(delete node) π(S,A3)=0.30 (add a connection between nodes 1 and 2) π(S,A4)=0.20 (delete the connection between nodes 3 and 4) The sum of the probability distributions of all possible actions can be 1. This action probability distribution guides the strategy selection process. In each round of optimization, these probabilities can be used to decide which action to perform, thereby gradually adjusting the network topology and optimizing system performance.

[0075] In some embodiments, the action of changing the network topology includes: adding a new core grain node, that is, adding a node, which means adding a new core grain node in the current topology; correspondingly, the action probability distribution π(S,A) / pi(S,A)π(S,A) will assign a probability to this action, indicating the possibility of adding a node in the current state; or removing an existing core grain node, that is, deleting a node, which means removing an existing core grain node from the current topology; correspondingly, the action probability distribution will assign a probability to this action, indicating the possibility of deleting a node in the current state; or adding a new connection path between two existing core grain nodes, that is, adding a connection, which means adding a new connection path (edge) between two existing nodes; correspondingly, the action probability distribution will assign a probability to a specific connection adding action; or removing an existing connection path between two core grain nodes, that is, deleting a connection, which means removing an existing connection between two nodes; correspondingly, the action probability distribution will assign a probability to a specific connection deleting action.

[0076] Among them, the probability distribution includes the probability corresponding to at least one action, which may specifically include the probability assigned to each possible action; based on this, updating the network topology includes: executing the action associated with the probability value exceeding the preset probability threshold and with the largest probability value.

[0077] In the embodiments of the present application, for the network topology optimization process of the multi-chip interconnected network, the actions that may be performed are operations on nodes and links (i.e., connection paths, or edges), which directly affect the topological structure of the network. Actions include adding links, deleting links, adding nodes, and deleting nodes. When a chip node has no link connected to the central controller (or other chip nodes), it means that the chip has no data transmission requirements, so the chip enters a non-computing dormant state and waits for reconnection and startup.

[0078] Specifically, adding a link means adding a new link to the current topology. The operation can be specifically expressed as:

[0079] Among them, G represents the current network topology, G′ represents the new topology after adding links, u and v Represents two different nodes.

[0080] Similarly, deleting a link means removing an existing link from the current topology. The operation can be specifically expressed as:

[0081] In an embodiment of the present application, the output π(S, A) of the graph convolutional network is a list or vector containing multiple actions and their corresponding probabilities. Each action corresponds to a possible topological structure adjustment, and the probability value represents the priority of the action in the current state. This probability distribution is used to guide the system on how to adjust its topological structure, such as executing the highest priority action to optimize performance indicators, such as balancing delay, area, and power consumption.

[0082] In some embodiments, the performance indicators include delay, area, and power consumption; based on the performance indicators, updating the parameters of the policy network may include the following steps.

[0083] Step 1: Calculate the reward value based on the performance index; the reward value is calculated based on delay, area, power consumption and corresponding weight coefficients.

[0084] The reward value can also be called a reward function, which represents the performance improvement of the topology. For example, reducing latency, power consumption, and area will receive positive rewards. The reward value can be calculated based on latency, area, power consumption, and corresponding weight coefficients.

[0085] For example, the reward value can be obtained based on the following formula:

[0086] Among them, Daug represents the average delay, M represents the total area, P represents the total power consumption, and α, β, and γ are the corresponding weight coefficients.

[0087] In the embodiment of the present application, the weight coefficient may also be referred to as a weight factor, which indicates the relative magnitude of the impact of the performance indicator on the system performance and may be dynamically updated during the iteration process.

[0088] In some embodiments, based on the performance indicator, calculating the reward value includes: Obtaining a first current weight coefficient corresponding to the delay, a second current weight coefficient corresponding to the area, and a third current weight coefficient corresponding to the power consumption; wherein the first current weight coefficient, the second current weight coefficient, and the third current weight coefficient are updated based on at least one of a relative change rate and an importance of the performance indicator; A reward value is calculated based on the delay, the first current weight coefficient, the area, the second current weight coefficient, the power consumption, and the third current weight coefficient.

[0089] In an embodiment of the present application, the weight coefficient is dynamically updated during the iteration process, and the reward value is calculated based on the adjusted weight coefficient.

[0090] For example, when a certain performance indicator (such as delay or power consumption) has a greater impact on system performance, the corresponding weight can be dynamically increased.

[0091] Exemplarily, the step of dynamically updating the weight coefficient may include: initial setting, performance indicator monitoring, dynamic adjustment and weight normalization.

[0092] The initial setting may specifically include: setting an initial value for each weight coefficient α, β and γ in the first iteration.

[0093] Performance indicator monitoring may specifically include: after each round of iteration, evaluating the performance improvement of the current system; if the improvement of a certain indicator is small, its weight may be increased accordingly.

[0094] The dynamic adjustment may specifically include: if the improvement of the delay is less than expected after the tth round of iterations, the value of the corresponding weight coefficient α may be increased, for example, by using the following formula for adjustment: α t+1 =α t ×(1+λ×ΔDt / Expected Improvement) Among them, α t+1 Represents the weight coefficient used in the t+1th iteration, α t represents the weight coefficient used in the tth iteration, λ represents the adjustment rate, ΔD t represents the delayed change after the tth iteration, and Expected Improvement represents the expected improvement value.

[0095] After each round of iteration, the weight coefficients can be updated according to the feedback information, and the weight combination that optimizes the system performance can be gradually found.

[0096] Weight normalization may specifically include: normalizing the sum of all weight coefficients to 1. Exemplarily, the normalization operation may be implemented by dividing the initial weight coefficients by their sum, as follows.

[0097]

[0098] In an embodiment of the present application, during the optimization process, a dynamic adjustment method can be used to determine the weight coefficients corresponding to different performance indicators. For example, the weight coefficients can be adjusted in real time according to the relative change rate or importance of the objective function corresponding to each performance indicator to obtain a weight combination that satisfies the system performance balance and optimizes the system performance.

[0099] Among them, the main goal of the topology performance evaluation of the chiplet interconnection network is to calculate the key performance indicators of the current network topology, including delay, area and power consumption. These performance indicators are used as objective functions in the optimization process and can be calculated in the following way.

[0100] Delay calculation: For the path P i The data packets on the transmission delay D(P i ) It can be expressed as:

[0101] in, L(u,v) Representation Node u To Node v The data length between C(u,v) Indicates the bandwidth capacity of the path.

[0102] Area calculation: The area M of the network topology can be expressed as:

[0103] in, M e Indicates link e The area occupied, M n Representation Node n Area occupied; E Indicates the path P i The set of links on N Indicates the path P i The collection of nodes on .

[0104] Power consumption calculation: The total power consumption P of the network can be expressed as:

[0105] in, P u,v Representation Node u To Node v The power consumption of the link between P n Representation Node n power consumption.

[0106] Based on this, by traversing all the chip nodes and connection paths on the multi-chip interconnection network, the average delay, total area and total power consumption of the multi-chip interconnection network can be calculated.

[0107] Step 2: Based on the reward value, determine the advantage function; the advantage function is configured to evaluate the relative goodness of the corresponding action.

[0108] In some embodiments, based on the reward value, determining an advantage function includes: Get the benchmark value corresponding to the reward value; the benchmark value is the average value of the previous positive returns, and the reward value is the actual reward obtained after the current strategy network executes the action; Based on the reward value and the corresponding benchmark value, an advantage function is determined.

[0109] For example, using represents the advantage function, the advantage function It is used to measure the performance of the current action relative to the average performance, and is the actual reward obtained. The benchmark value of the status The difference between is in state Next action The actual reward received.

[0110] Specifically, the advantage function can be calculated using the following formula:

[0111] in, Yes Status The benchmark value represents the average of past positive returns.

[0112] Step 3: Based on the advantage function, determine the loss function; the loss function includes the parameters of the policy network to be updated.

[0113] In some embodiments, based on the advantage function, determining the loss function includes: Obtaining a strategy ratio; the strategy ratio is configured to characterize the effect of a strategy update; performing a clipping operation on the strategy ratio; the clipping operation being configured to constrain the strategy ratio within a preset variation range; Based on the policy ratio, the policy ratio after clipping, and the advantage function, a loss function is determined.

[0114] Among them, the Probability Ratio represents the ratio between the new and old strategies, which is used to measure the effect of strategy update.

[0115] In some embodiments, obtaining a strategy ratio includes: Get the probability distribution of the current strategy and the probability distribution of the reference strategy; Based on the comparison between the probability distribution of the current strategy and the probability distribution of the reference strategy, the obtained ratio is taken as the strategy ratio.

[0116] For example, the strategy ratio can be calculated using the following formula:

[0117] in, is the probability distribution of the current strategy, is the probability distribution of the old strategy (i.e., the reference strategy).

[0118] In some embodiments, performing a clipping operation on the policy ratio includes: Get the clipping coefficient; the clipping coefficient is configured to limit the update amplitude; Based on the clipping coefficient, determining a preset variation range associated with the strategy ratio; Performs a clipping operation to constrain the strategy ratio to a preset range of variation.

[0119] In the embodiment of the present application, a clipping mechanism is introduced to control the magnitude of the policy update, and the clipped objective is to limit the range of change of the policy ratio.

[0120] Exemplarily, in an embodiment of the present application, the loss function can be expressed as:

[0121] Among them, θ represents the parameters of the network strategy, represents the advantage function, Represents a clipping operation, which is used to change the policy ratio Restricted to [ ] within the preset range of changes to avoid strategy ratios Big changes have occurred.

[0122] in, Represents the clipping coefficient, which is used to control the size of the update amplitude.

[0123] In the embodiments of the present application, a strategy comparison method is proposed to improve system performance by gradually updating the strategy, while limiting the magnitude of each update to avoid drastic changes.

[0124] Step 4: Minimize the loss function and update the parameters of the policy network.

[0125] Specifically, by minimizing the loss function , update the policy network parameters θ, thereby optimizing the strategy, making the policy change range appropriate, making the strategy more inclined to select high-quality actions, and realizing the policy network update optimization.

[0126] In some embodiments, reaching a preset stop condition includes: Generate a network topology that meets target performance indicators; or The number of iterations reaches the set threshold.

[0127] Specifically, the process of iterative sampling, evaluation, and updating the policy network is continued until a preset stopping condition is reached, such as policy convergence, such as generating a network topology that meets the target performance indicator; or the number of iterations reaches the maximum number of iterations.

[0128] In some embodiments, after a preset stop condition is reached, the method further includes: Output the updated network topology.

[0129] Specifically, after the iteration stops, the optimized network topology is output, which can balance the system performance indicators such as delay, area and power consumption.

[0130] In the embodiments of the present application, an optimal network topology generation method is provided for a multi-chip interconnection network / integrated system, and the specific steps may include setting objective functions, system actions, reward functions, policy networks, and optimization steps combining policy ratios and clipping operations. Specifically, based on the current policy network associated with the multi-chip interconnection network, multiple network topology change actions can be sampled from the action space, these actions can be executed, and the state transitions and rewards obtained after each action is executed can be recorded; specifically, the performance indicators of the network topology after each action is executed, such as the objective functions corresponding to the delay, area, and power consumption, are calculated, and the reward value is calculated according to the objective function; the advantage function can be used to evaluate the relative goodness of the currently possible actions; by minimizing the loss function, the parameters of the policy network are updated, thereby optimizing the policy, making the policy change range appropriate, and making the policy more inclined to select high-quality actions; the process of sampling, evaluating, and updating the policy network is continuously iterated until a preset stop condition is reached, such as policy convergence or reaching the maximum number of iterations, and finally the optimized network topology structure is output, so that the system achieves the optimal balance in performance indicators such as delay, area, and power consumption.

[0131] The embodiments of the present application can effectively perform reinforcement learning optimization in the network topology structure of a multi-chip integrated system, which can not only dynamically adjust the network topology structure, but also achieve the optimal balance between latency, area and power consumption, and ultimately improve the overall performance of the multi-chip system.

[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0133] The embodiment of the present application also provides a network topology generation device, referring to Figure 4 The network topology generation device may include: a first acquisition module 41, configured to acquire a current strategy network based on a multi-chip interconnection network; a second acquisition module 42, configured to sample and execute multiple network topology change actions based on the current strategy network, and acquire performance indicators of the network topology after the actions; a parameter update module 43, configured to update the parameters of the strategy network based on the performance indicators, and iterate the sampling and updating steps until a preset stop condition is reached.

[0134] In some embodiments, the network structure of the policy network adopts a graph convolutional network; the graph convolutional network outputs a probability distribution of possible actions based on the graph structure data corresponding to the network topology; the probability distribution is used to determine the target action performed to update the network topology.

[0135] In some embodiments, the graph structure data corresponding to the network topology includes node features and an adjacency matrix; the node features include the computing power of the core particles, the adjacency matrix includes edge features, and the edge features include bandwidth and latency.

[0136] In some embodiments, the action of changing the network topology includes: adding a new core particle node; or removing an existing core particle node; or adding a new connection path between two existing core particle nodes; or removing an existing connection path between two core particle nodes; wherein the probability distribution includes the probability corresponding to at least one action; the parameter update module is configured to update the network topology, which may specifically include: executing an action associated with a probability value exceeding a preset probability threshold and having the largest probability value.

[0137] In some embodiments, the performance indicators include delay, area and power consumption; the parameter update module is configured to update the parameters of the policy network based on the performance indicators, which may specifically include: calculating a reward value based on the performance indicators; the reward value is calculated based on delay, area, power consumption and corresponding weight coefficients; determining an advantage function based on the reward value; the advantage function is configured to evaluate the relative goodness of the corresponding action; determining a loss function based on the advantage function; the loss function includes the parameters of the policy network to be updated; minimizing the loss function and updating the parameters of the policy network.

[0138] In some embodiments, the parameter update module is configured to calculate a reward value based on a performance indicator, including: obtaining a first current weight coefficient corresponding to the delay, a second current weight coefficient corresponding to the area, and a third current weight coefficient corresponding to the power consumption; wherein the first current weight coefficient, the second current weight coefficient, and the third current weight coefficient are updated based on at least one of the relative change rate and the importance of the performance indicator; and calculating the reward value based on the delay, the first current weight coefficient, the area, the second current weight coefficient, the power consumption, and the third current weight coefficient.

[0139] In some embodiments, the parameter update module is configured to determine the advantage function based on the reward value, including: obtaining a benchmark value corresponding to the reward value; the benchmark value is the average of the previous positive returns, and the reward value is the actual reward obtained after the current strategy network executes the action; based on the reward value and the corresponding benchmark value, determine the advantage function.

[0140] In some embodiments, the parameter update module is configured to determine a loss function based on an advantage function, including: obtaining a strategy ratio; configuring the strategy ratio to characterize the effect of a strategy update; performing a clipping operation on the strategy ratio; the clipping operation is configured to limit the strategy ratio to a preset range of variation; and determining a loss function based on the strategy ratio, the strategy ratio after performing the clipping operation, and the advantage function.

[0141] In some embodiments, the parameter update module is configured to obtain a strategy ratio, including: obtaining a probability distribution of a current strategy and a probability distribution of a reference strategy; comparing the probability distribution of the current strategy and the probability distribution of the reference strategy, and using the obtained ratio as the strategy ratio.

[0142] In some embodiments, the parameter update module is configured to perform a clipping operation on the strategy ratio, including: obtaining a clipping coefficient; configuring the clipping coefficient to limit the update amplitude; based on the clipping coefficient, determining a preset range of variation associated with the strategy ratio; performing a clipping operation to limit the strategy ratio within the preset range of variation.

[0143] In some embodiments, reaching a preset stop condition includes: generating a network topology structure that meets a target performance indicator; or the number of iterations reaches a set threshold.

[0144] In some embodiments, the network topology generation device further includes an output module configured to output an updated network topology structure after a preset stop condition is reached.

[0145] In the embodiments of the present application, the description of the features in the embodiments corresponding to the network topology generation device can refer to the relevant description of the embodiments corresponding to the network topology generation method, which will not be repeated here.

[0146] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any of the above network topology generation method embodiments.

[0147] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned network topology generation method embodiments when running.

[0148] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0149] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above-mentioned network topology generation method embodiments are implemented.

[0150] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned network topology generation method embodiments are implemented.

[0151] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0152] The server main card, server, network topology generation method, product, device and medium provided by the present application are introduced in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A multi-chip interconnection network, characterized in that: include: A core particle layer, comprising at least one core particle group, wherein the core particle group comprises at least two core particles; An active interposer layer, connecting the core particle layer; Wherein, the active intermediary layer comprises a first active intermediary layer and a second active intermediary layer, and the first active intermediary layer is located between the core particle layer and the second active intermediary layer; The first active intermediary layer includes an interface router device, an inter-group router device, and a configurable crossbar switch network, and the second active intermediary layer includes an inter-group relay router device and an inter-group bandwidth network; The central node of the configurable cross switch network configures the inter-group router device, the core particle configures the interface router device, and the interface router device is also connected to the selection node of the configurable cross switch network; the inter-group router device is connected to the inter-group bandwidth network, and the inter-group relay router device is configured in the inter-group bandwidth network.

2. The multi-chip interconnection network according to claim 1, characterized in that: The configurable crossbar switch network includes a star-shaped configurable crossbar switch network; one of the core particles is configured with at least two of the interface router devices, and the interface router device is configured with an accumulation transmitter; The chiplet group further includes an intra-group routing component, and the intra-group routing component connects at least two of the chiplets in the chiplet group.

3. The multi-chip interconnection network according to claim 1, characterized in that: The configurable crossbar switch network includes a configurable crossbar switch and a configurable switch controller; The configurable switch controller is configured to control the connection state of the configurable crossbar switches in the configurable crossbar switch network and to gate a target gate node.

4. A chip, characterized in that: The invention comprises a multi-chip interconnection network as claimed in any one of claims 1 to 3.

5. A network topology generation method, characterized in that: Based on the multi-chip interconnection network implementation according to any one of claims 1 to 3; the network topology generation method comprises: Acquire a current strategy network based on the multi-chip interconnection network; Based on the current policy network, sampling and executing multiple network topology change actions, and obtaining performance indicators of the network topology after the actions; Based on the performance indicator, the parameters of the policy network are updated, and the sampling and updating steps are iterated until a preset stopping condition is reached.

6. The network topology generation method according to claim 5, characterized in that: The network structure of the strategy network adopts a graph convolutional network; The graph convolutional network outputs a probability distribution of possible actions based on the graph structure data corresponding to the network topology; the probability distribution is used to determine the target action performed to update the network topology; The graph structure data corresponding to the network topology includes node features and an adjacency matrix; the node features include the computing power of the core particle, the adjacency matrix includes edge features, and the edge features include bandwidth and delay.

7. The network topology generation method according to claim 6, characterized in that: The network topology change action includes: Add a new core particle node; or Remove an existing core node; or Add a new connection path between two existing core nodes; or Remove an existing connection path between two core nodes; Wherein, the probability distribution includes a probability corresponding to at least one of the actions; Updating the network topology includes: executing the probability-associated action whose probability value exceeds a preset probability threshold and has the largest probability value.

8. The network topology generation method according to claim 5, characterized in that: The performance indicators include delay, area, and power consumption; and updating the parameters of the policy network based on the performance indicators includes: Based on the performance indicator, a reward value is calculated; the reward value is calculated based on the delay, the area, the power consumption and a corresponding weight coefficient; Based on the reward value, determining an advantage function; the advantage function is configured to evaluate the relative goodness of the corresponding action; Based on the advantage function, determining a loss function; the loss function includes parameters of the policy network to be updated; The loss function is minimized and the parameters of the policy network are updated.

9. The network topology generation method according to claim 8, characterized in that: The step of calculating the reward value based on the performance indicator comprises: Obtaining a first current weight coefficient corresponding to the delay, a second current weight coefficient corresponding to the area, and a third current weight coefficient corresponding to the power consumption; wherein the first current weight coefficient, the second current weight coefficient, and the third current weight coefficient are updated based on at least one of a relative change rate and an importance of the performance indicator; calculating the reward value based on the delay, the first current weight coefficient, the area, the second current weight coefficient, the power consumption, and the third current weight coefficient; The determining of the advantage function based on the reward value comprises: Obtaining a reference value corresponding to the reward value; the reference value is the average value of the previous positive returns, and the reward value is the actual reward obtained after the current strategy network executes the action; determining the advantage function based on the reward value and the corresponding reference value; The step of determining a loss function based on the advantage function comprises: Obtaining a strategy ratio; the strategy ratio is configured to characterize the effect of the strategy update; performing a clipping operation on the strategy ratio; the clipping operation is configured to limit the strategy ratio to a preset variation range; The loss function is determined based on the policy ratio, the policy ratio after performing the clipping operation, and the advantage function.

10. The network topology generation method according to claim 9, characterized in that: Obtaining the strategy ratio, including: Get the probability distribution of the current strategy and the probability distribution of the reference strategy; Based on the comparison between the probability distribution of the current strategy and the probability distribution of the reference strategy, the obtained ratio is used as the strategy ratio.

11. The network topology generation method according to claim 9, characterized in that: Performing a clipping operation on the strategy ratio includes: Obtaining a clipping coefficient; the clipping coefficient is configured to limit an update amplitude; Based on the clipping coefficient, determining a preset range of variation associated with the strategy ratio; The clipping operation is performed to limit the strategy ratio within the preset variation range.

12. The network topology generation method according to claim 5, characterized in that: The reaching of the preset stop condition includes: Generate a network topology that meets target performance indicators; or The number of iterations reaches the set threshold; After the preset stop condition is reached, the method further includes: Output the updated network topology.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the network topology generation method according to any one of claims 5 to 12 are implemented.

14. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the network topology generation method according to any one of claims 5 to 12 when executing the computer program.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the network topology generation method according to any one of claims 5 to 12.

Images