Intelligent calculation center and configuration method thereof

Abstract

The invention discloses an intelligent calculation center and a configuration method thereof, and the intelligent calculation center comprises an inter-packet cluster which is arranged in a three-row and three-column manner, and the inter-packet cluster comprises a bridge network which is communicated with each inter-packet; the bridge network comprises inter-compartment bridges, each inter-compartment is provided with one inter-compartment bridge, and the inter-compartment bridges are connected with devices in the compartments; the same row of bridges are connected with the inter-compartment bridges in the two adjacent compartments in the same row; the same-column bridges are used for connecting the inter-compartment bridges in the two adjacent compartments in the same column; the inter-pack cluster comprises a core inter-pack located in the middle and computing power inter-packs surrounding the core inter-pack; the inter-pack clusters are distributed in bilateral symmetry about the center line of the column where the core inter-packs are located. According to the invention, the physical shortest distance from the POD in each computing power compartment to the core compartment is realized by reasonably distributing the computing power resource and the electric power resource of each compartment and placing the core compartment for placing the core network equipment at the central position, so that the cluster construction can adopt a multi-mode cable to realize full interconnection, and the construction cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of data centers, and in particular to an intelligent computing center and a configuration method thereof. Background Art

[0002] The construction of ultra-large-scale GPU clusters in intelligent computing centers relies on the design of underlying infrastructure. However, the existing traditional data center infrastructure architecture is often built with traditional CPU architecture as the target group. The single cabinet design has low power consumption and cannot match the power consumption requirements of high-power CPU servers. It has no elastic architecture and cannot match the power consumption growth requirements of its high-speed iteration, and cannot be compatible with the heterogeneous or different generation gap networking requirements of different GPU manufacturers. The layout of the computer room is evenly distributed on a floor-by-floor basis. The building layout, electrical and heating routing are optimized within the floor, while the optimal interconnection distance of the cluster is ignored. In addition, the current construction of large-scale computing power clusters is to fill GPU computing power in each module based on the layout of the IT modules of the built data center, and try to control the interconnection distance between each GPU server in each computing power unit POD and between units, so as to ensure that the full interconnection of GPU servers in the cluster can be achieved within a certain physical distance. In the current market, there is a lack of a data center design oriented to a large cluster architecture (generally 10,000 cards or more) to adapt to the short-distance and low-cost computing power cluster interconnection and the arrangement of supporting resources.

[0003] The "data center computer room" disclosed in the Chinese patent literature has a publication number of CN118223716A and a publication date of 2024.06.21. The data center computer room includes two air conditioning modules, two IT modules and two public corridor modules. Each air conditioning module includes a plurality of air conditioning boxes, and the plurality of air conditioning boxes are arranged in sequence along the second direction. The second direction is perpendicular to the first direction, and each air conditioning box is arranged along the second direction; each IT module includes a plurality of IT boxes, and the plurality of IT boxes are arranged in sequence along the second direction, and each IT box is arranged along the first direction; each public corridor module includes a plurality of public corridor boxes, and each public corridor box is arranged in sequence along the second direction. Each public corridor box is arranged between an air conditioning box and at least one IT box, and an air circulation channel is formed between the air conditioning box, the public corridor box and the IT box. In the data center computer room provided by this application, each component can be prefabricated and debugged in the factory, and only simple connection is required on site. The on-site construction workload is small, the setting is flexible, the computer room construction period is shortened, and the cost is low. This technology only arranges the equipment in a single computer room and also ignores the need for optimal cluster interconnection distance. It lacks a technical solution that is adapted to short-distance and low-cost computing cluster interconnection arrangements based on a large cluster architecture. Summary of the invention

[0004] The present invention aims to overcome the problem that the prior art ignores the need for optimal cluster interconnection distance and lacks a data center configuration solution oriented towards a large cluster architecture to adapt to the short-distance and low-cost computing cluster interconnection layout, and provides an intelligent computing center and its configuration method.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: An intelligent computing center includes a cluster of private rooms arranged in three rows and three columns, wherein the cluster of private rooms includes a bridge network connecting the private rooms; the bridge network includes: Intra-room bridge: Each private room is equipped with an intra-room bridge to connect the equipment in the room; In-row bridge, connecting the bridges in two adjacent rooms in the same row; The bridge in the same row connects the bridges in two adjacent rooms in the same row; The room cluster includes a core room located in the middle and computing power rooms surrounding the core room; the distribution of the room cluster is symmetrical about the center line of the column where the core room is located.

[0006] The present invention proposes a configuration method of the intelligent computing center structure and specific equipment resources from the overall consideration of the large cluster structure in order to solve the problem that the existing traditional intelligent computing center construction lacks a data center design scheme guided by a large cluster architecture to adapt to the short-distance and low-cost computing cluster interconnection and supporting resource arrangement. First, an intelligent computing center architecture considering the overall large cluster structure is given. The structure of the intelligent computing center with a scale of 10,000 cards is designed by using a nine-square grid-style package distribution method. The core package is placed in the most central position to achieve the shortest physical distance from the computing unit POD in each computing package to the core package, so that the cluster construction can use multi-mode cables to achieve full interconnection and reduce construction costs; in the overall layout, mirror symmetry is also taken into account. With the core package as the center, the scale of each package up, down, left, and right is a mirror-symmetrical layout, so that the equipment networking plan in the computing package can be quickly copied, and the cable resources can be purchased and reserved in advance, shortening the construction and delivery time of each computing unit POD in the cluster.

[0007] Preferably, the intelligent computing center includes: Cross-layer architecture, the second row of core rooms are located in the middle layer, and the first and third rows are located in the upper and lower layers adjacent to the middle layer respectively; Flat architecture, all private room clusters including core private rooms are located on the same layer.

[0008] Preferably, the core package room is used to place the core network equipment and storage equipment in the computing power cluster; the computing power package room is used to place the computing power equipment in the computing power cluster; The intra-room bridge frames include a plurality of transverse bridge frames and a plurality of longitudinal bridge frames. The transverse bridge frames and the longitudinal bridge frames are perpendicular to each other, and the transverse bridge frames are parallel to the center line of the row where the core package room is located.

[0009] Preferably, in the flat structure, the first row of private rooms includes at least two groups of bridge frames in the same row, wherein one group of bridge frames in the same row is arranged close to the wall between the first row of private rooms and the second row of private rooms; The second row of private rooms includes at least two groups of bridge frames in the same row, one group of bridge frames in the same row is arranged close to the wall between the first row of private rooms and the second row of private rooms, and the other group of bridge frames in the same row is arranged close to the wall between the third row of private rooms and the second row of private rooms.

[0010] Preferably, in the cross-layer structure, the bridge frames in the same row of the private rooms in the first row are arranged close to the wall between the private rooms in the first row and the private rooms in the second row, connecting the bridge frames in the same row in the same horizontal projection position in two adjacent private rooms in the first row; The second row of private rooms includes at least two groups of same-row bridge frames, one group of which is arranged close to the wall between the first row of private rooms and the second row of private rooms, connecting the same-row bridge frames at the same horizontal projection position in two adjacent private rooms in the second row.

[0011] Preferably, each private room in the private room cluster is provided with a plurality of rows of cabinet groups for placing equipment; The extension direction of each row of cabinet groups is parallel to the transverse bridge, and each transverse bridge is connected to the equipment in one row of cabinet groups.

[0012] A configuration method for an intelligent computing center, comprising: Calculate the number of computing units PODs that need to be configured based on the computing power requirements of the computing power cluster in the intelligent computing center; The distribution of the room cluster is symmetrical about the center line of the column where the core room is located. The number of computing unit PODs in each computing room is configured based on the preset distribution conditions. Configure the power resources of each private room according to the type and quantity of equipment in each private room.

[0013] Preferably, the number of computing power units PODs that need to be configured for the calculation includes: Pre-set the equipment combination of each computing unit POD to determine the computing power of a single computing unit POD; The computing power requirement of the computing power cluster is divided by the computing power of each computing power unit POD, and the number of computing power unit PODs that need to be configured is determined by considering the surplus.

[0014] Preferably, configuring the power resources of each private room includes: For the core package room, the power resources to be configured are calculated by summing the power requirements of the core network equipment and the storage equipment. For the computing power package room, the power resources that need to be configured are calculated based on the required power of the computing power equipment and the required power of the associated devices between the computing power equipment and the core network equipment.

[0015] Preferably, the preset distribution condition includes: dividing the computing power packages into Class A computing power packages adjacent to the core packages and the remaining Class B computing power packages; The number of PODs in the same type of computing power rooms is the same; The total number of configured PODs for all types of computing power packages is the number of computing power unit PODs that need to be configured; The path required to connect each POD in the computing power room to the core room is the shortest.

[0016] The present invention has the following beneficial effects: by reasonably distributing the computing power resources and power resources of each package room and placing the core package room for placing the core network equipment in the most central position, the shortest physical distance from the computing power unit POD in each computing power package room to the core package room is achieved, so that cluster construction can adopt multi-mode cables to achieve full interconnection and reduce construction costs; the overall layout also takes into account mirror symmetry, and the scales of the packages above, below, left and right are all mirror-symmetrical with the core package room as the center, so that the equipment networking plan in the computing power package room can be quickly copied, and the cable resources can be purchased and reserved in advance, shortening the construction and delivery time of each computing power unit POD in the cluster; the cabinet power design with multiple differences is adopted, which is compatible with server performance and power generation differences, and realizes the heterogeneous deployment of PODs of servers with different performance to form a cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a top view of the flat architecture of the intelligent computing center in the present invention.

[0018] Figure 2 It is a schematic diagram of the cross-layer architecture of the intelligent computing center in the present invention.

[0019] Figure 3 It is a top view of the top layer of the cross-layer architecture of the intelligent computing center in the present invention.

[0020] Figure 4 It is a schematic diagram of the middle layer of the cross-layer architecture of the intelligent computing center in the present invention.

[0021] Figure 5 It is a schematic diagram of the relationship between the bridge and the cabinet in a single private room of the intelligent computing center in the present invention.

[0022] In the figure: 1. Core room; 2. Computing power room; 3. Bridge network; 31. Intra-room bridge; 32. Bridge in the same row; 33. Bridge in the same column; 311. Horizontal bridge; 312. Vertical bridge; 4. Cabinet. DETAILED DESCRIPTION

[0023] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0024] The current construction of large-scale computing power clusters is to fill GPU computing power into each module based on the layout of the IT modules of the built data center, and try to control the interconnection distance between the GPU servers in each computing power unit POD and between units, so as to ensure that the full interconnection of GPU servers in the cluster can be achieved within a certain physical distance. The data center IDC is not built on demand, the interconnection cost is high, and it is based on the IT output of the cabinet, not the end use. In the current market, there is a lack of a data center design solution oriented to a large cluster architecture (generally 10,000 cards or more) to adapt to the short-distance and low-cost computing power cluster interconnection and the arrangement of supporting resources.

[0025] In order to solve the above problems, the present invention provides Figure 1 The intelligent computing center shown includes a cluster of private rooms arranged in three rows and three columns. The private room cluster includes a bridge network 3 connecting the private rooms. The bridge network 3 includes: Intra-room bridge 31, each private room is provided with an intra-room bridge 31, connecting the equipment in the private room; The same row bridge 32 connects the inner bridges 31 in two adjacent private rooms in the same row; The bridge frame 33 in the same row connects the bridge frames 31 in two adjacent private rooms in the same row; The room cluster includes the core room 1 located in the middle and the computing power room 2 surrounding the core room. The distribution of the room cluster is symmetrical about the center line of the column where the core room 1 is located.

[0026] It should be noted that the present invention proposes a configuration method of the intelligent computing center structure and specific equipment resources from the overall consideration of the large cluster structure in order to solve the problem that the existing traditional intelligent computing center construction lacks a data center design scheme oriented to the large cluster architecture to adapt to the short-distance and low-cost computing cluster interconnection and supporting resource arrangement. First, an intelligent computing center architecture considering the overall large cluster structure is given. The structure of the intelligent computing center with a scale of 10,000 cards is designed by using a nine-square grid-style package distribution method. The core package is placed in the most central position to achieve the shortest physical distance from the computing unit POD in each computing package to the core package, so that the cluster construction can use multi-mode cables to achieve full interconnection and reduce construction costs; in the overall layout, mirror symmetry is also taken into account. With the core package as the center, the scale of each package above, below, left and right is a mirror symmetric layout, so that the equipment group network shelf plan in the computing package can be quickly copied, and the cable resources can be purchased and reserved in advance, shortening the construction and delivery time of each computing unit POD in the cluster.

[0027] It is worth noting that the present invention is an intelligent computing center architecture designed as a whole. The arrangement of the private rooms in the form of a nine-square grid remains unchanged, and the core private room, as the most central private room, always only places the core network equipment of the computing power cluster and the storage equipment of the computing power, so that the distance from the computing power equipment of the computing power unit POD in other computing power private rooms surrounding the core private room to the core network equipment in the core private room is as short as possible and the path passed is as neat and consistent as possible, so that the bridge network in the private room cluster is designed accordingly, so that the devices in different private rooms are connected through the cables placed on the bridge network. That is, within the power range allowed by the intelligent computing center, the total computing power is adjusted by adjusting the number of computing power equipment in the computing power private rooms surrounding the core private room.

[0028] Specifically, in the cluster of rooms arranged in three rows and three columns, each room can be marked separately to facilitate subsequent explanations, wherein the room in the second row and second column is located in the middle, namely the core room; the three rooms in the first row are the first computing power room, the second computing power room, and the third computing power room from left to right; the three rooms in the second row are the fourth computing power room, the core room, and the fifth computing power room from left to right; the three rooms in the third row are the sixth computing power room, the seventh computing power room, and the eighth computing power room from left to right. The intra-room bridges and the same-row bridges in the rooms in the first row belong to the first row of bridges; the intra-room bridges and the same-row bridges in the rooms in the second row belong to the second row of bridges; the intra-room bridges and the same-row bridges in the rooms in the third row belong to the third row of bridges; the first row bridge and the second row bridges, and the second row bridges and the third row bridges are connected by bridges in the same column, and all bridges together form a bridge network.

[0029] As a specific embodiment, core room 1 is used to place core network devices and storage devices in the computing power cluster; computing power room 2 is used to place computing power devices in the computing power cluster; The intra-room bridge frames 31 include a plurality of transverse bridge frames 311 and a plurality of longitudinal bridge frames 312 . The transverse bridge frames 311 and the longitudinal bridge frames 312 are perpendicular to each other, and the transverse bridge frames 311 are parallel to the center line of the row where the core room 1 is located.

[0030] It should be noted that the main function of the core room is to place the black virtual network devices and storage devices in the computing power cluster, and the main function of the computing power room is to place the computing power devices to provide computing power; in addition, each room is also equipped with supporting equipment that must be configured in the room. The supporting equipment belongs to the content of the prior art and does not affect the improved technology of the present invention, so it will not be described in detail.

[0031] It is worth mentioning that by reasonably distributing the computing power resources and power resources of each package room and placing the core package room where the core network equipment is placed in the most central location, the shortest physical distance from the computing unit POD in each computing power package room to the core package room is achieved, so that cluster construction can use multi-mode cables to achieve full interconnection and reduce construction costs.

[0032] The length and width of each private room in a normal intelligent computing center are limited in range (generally limited to 40 meters), and the height of each floor is also specifically limited (generally limited to 6 meters). Therefore, the setting of the first-level bridge network in the private room of the present invention can control the physical distance from the computing power unit POD in each computing power private room to the core private room within 50 meters. In the actual configuration selection, the transmission distance of the 400G multi-mode optical module is 50 meters. If it exceeds 50 meters, it must be replaced with a single-mode optical module and cable, and the actual cost doubles. According to the computing power of 1000P, single-mode is more than 1000W more expensive than multi-mode. Therefore, through the structural design of the intelligent computing center of the present invention, cluster construction can be interconnected using multi-mode cables, thereby saving costs compared to single-mode cables.

[0033] As an optional embodiment, the intelligent computing center includes: Figure 1 In the flat architecture shown, the private room clusters, including the core private rooms, are all located on the same floor. In other words, except for the electromechanical system and other supporting facilities that must be provided by the intelligent computing center, the entire computing power cluster is placed on the same floor of the building, and the distribution of the private room clusters and the distribution of the bridge network are symmetrical about the center of the core private room.

[0034] Furthermore, in the flat structure, the first row of rooms includes at least two groups of bridge frames in the same row, one group of bridge frames in the same row is arranged close to the wall between the first row of rooms and the second row of rooms; the other group of bridge frames of the same type can be arranged in the middle of the first row of rooms; The second row of rooms includes at least two groups of bridge frames in the same row, one group of bridge frames in the same row is arranged close to the wall between the first row of rooms and the second row of rooms, and the other group of bridge frames in the same row is arranged close to the wall between the third row of rooms and the second row of rooms; The third row of rooms includes at least two groups of bridge frames in the same row, one group of which is arranged close to the wall between the third row of rooms and the second row of rooms; the other group of the same type of bridge frames can be arranged in the middle of the first row of rooms.

[0035] Specifically, the intra-room bridge in the first computing power package room and the intra-room bridge in the second computing power package room are connected through the same-in-line bridge, the intra-room bridge in the second computing power package room and the intra-room bridge in the third computing power package room are connected through the same-in-line bridge, the intra-room bridge in the fourth computing power package room and the intra-room bridge in the core package room are connected through the same-in-line bridge, the intra-room bridge in the core package room and the intra-room bridge in the fifth computing power package room are connected through the same-in-line bridge, the intra-room bridge in the sixth computing power package room and the intra-room bridge in the seventh computing power package room are connected through the same-in-line bridge, and the intra-room bridge in the seventh computing power package room and the intra-room bridge in the eighth computing power package room are connected through the same-in-line bridge.

[0036] One set of bridge frames in the same column between the core room and the second computing power room, and one set of bridge frames in the same column between the core room and the seventh computing power room are set close to the wall of the rooms in the first column, connecting the intra-room bridge frames in the rooms in the second column; according to the distribution of the room clusters, symmetrically about the center line of the column where the core room is located, another set of bridge frames in the same column between the core room and the second computing power room, and another set of bridge frames in the same column between the core room and the seventh computing power room are set close to the wall of the rooms in the third column, connecting the intra-room bridge frames in the rooms in the second column.

[0037] One group of bridges in the same row between the first and fourth computing power rooms, and one group of bridges in the same row between the fourth and sixth computing power rooms are set close to the wall of the second row of rooms; another group of bridges in the same row are set in the middle of the rooms to connect the bridges in the first row of rooms. According to the distribution of the room cluster, one group of bridges in the same row between the third and fifth computing power rooms, and one group of bridges in the same row between the fifth and eighth computing power rooms are set close to the wall of the second row of rooms; another group of bridges in the same row are set in the middle of the rooms to connect the bridges in the third row of rooms.

[0038] Specifically, for a computing power unit POD, it can place all the computing power equipment in a computing power package room. However, when configuring specific computing power equipment according to the package room cluster, in order to ensure that the distribution of the package room cluster is symmetrical about the center line of the column where the core package room is located, as well as some other configuration preset conditions, there will be a situation where a computing power unit POD is divided into two parts and configured in two adjacent computing power packages, such as 0.5 POD in the first computing power package room and 0.5 POD in the second computing power package room, and the two form a complete POD. In this case, the connection cables of the complete computing power unit POD in the first computing power package room are connected to the core network device of the core package room in turn through the same column bridge between the first computing power package room and the fourth computing power package room, and the same row bridge between the fourth computing power package room and the core package room. The 0.5 PODs in the first computing power package room are first aggregated into a complete POD through the same-row bridge between the first computing power package room and the second computing power package room with another 0.5 PODs, and then connected to the core network device of the core package room through the same-row bridge between the second computing power package room and the core package room.

[0039] Further, such as Figure 5 As shown, in each private room in the private room cluster, 4 groups of cabinets for placing equipment are arranged in several rows; the extension direction of each row of 4 groups of cabinets is parallel to the horizontal bridge, and each horizontal bridge is connected to the equipment in a row of cabinet groups.

[0040] As an optional embodiment, the intelligent computing center includes: Figure 2 In the cross-layer architecture shown, the second row of core rooms is located in the middle of the middle layer, and the first and third rows are located on the upper and lower floors adjacent to the middle layer, respectively; that is, except for the electromechanical system and other supporting facilities that must be provided by the intelligent computing center, the entire computing cluster is placed in three consecutive adjacent floors of the building. The distribution of equipment in the entire room cluster is symmetrical about the center line of the column where the core rooms are located, and is also symmetrical about the plane of the middle layer; the distribution of the entire bridge network is symmetrical about the center line of the column where the core rooms are located, and is also symmetrical about the second row of bridges in the middle layer.

[0041] Furthermore, in the cross-layer structure, the bridge frames in the same row of the private rooms in the first row are arranged close to the wall between the private rooms in the first row and the private rooms in the second row, connecting the bridge frames in the same row in the same horizontal projection position in two adjacent private rooms in the first row; The second row of private rooms includes at least two groups of bridge frames in the same row, one of which is arranged close to the wall between the first row of private rooms and the second row of private rooms, connecting the bridge frames in the same row in the same horizontal projection position in two adjacent private rooms in the second row; the other group of bridge frames in the same row is arranged close to the wall between the third row of private rooms and the second row of private rooms, connecting the bridge frames in the same row in the same horizontal projection position in two adjacent private rooms in the second row; The bridge frames in the same row of the private rooms in the third row are arranged close to the wall between the private rooms in the third row and the private rooms in the second row, connecting the bridge frames in the same row at the same horizontal projection position in two adjacent private rooms in the third row.

[0042] Specifically, Figure 3 The top view of the bridge network in the uppermost layer of the cross-layer structure is shown. The first row of bridges in the top view is structurally the same as the second row of bridges in the middle layer and the third row of bridges in the bottom layer. Figure 3 The first row of bridges in the top layer is used as an example. From left to right in the top layer, there are the first computing power package room, the second computing power package room, and the third computing power package room. The bridges in the first computing power package room and the bridges in the second computing power package room are connected by at least two bridges in the same row, and the bridges in the second computing power package room and the bridges in the third computing power package room are connected by at least two bridges in the same row. They constitute the first row of bridges.

[0043] One set of bridge frames in the same column between the core room and the second computing power room, and one set of bridge frames in the same column between the core room and the seventh computing power room are set close to the wall of the rooms in the first column, connecting the bridge frames in the same column located at the same horizontal projection position in the rooms in the second column (the same horizontal projection position refers to the bridge frames in the same column that are in an overlapping state between different layers from the overall bird's-eye view); according to the distribution of the room cluster, the other set of bridge frames in the same column between the core room and the second computing power room, and the other set of bridge frames in the same column between the core room and the seventh computing power room are set close to the wall of the rooms in the third column, connecting the bridge frames in the same column located at the same horizontal projection position in the rooms in the second column.

[0044] The bridges in the same column between the first and fourth computing power rooms, and the bridges in the same column between the fourth and sixth computing power rooms are set close to the wall of the second column of rooms; they connect the bridges in the same column at the same horizontal projection position in the first column of rooms. According to the distribution of the room cluster, the bridges in the same column between the third and fifth computing power rooms, and the bridges in the same column between the fifth and eighth computing power rooms are set close to the wall of the second column of rooms; they connect the bridges in the same column at the same horizontal projection position in the third column of rooms.

[0045] like Figure 4 The figure shows a schematic diagram of the bridge network of the middle layer in the cross-layer architecture. The structure of the second row of bridges in the middle layer is the same as that of the first row of bridges. The first row of bridges and the second row of bridges, as well as the second row of bridges and the third row of bridges are all connected through bridges in the same column.

[0046] One set of bridge frames in the same column between the core room and the second computing power room, and one set of bridge frames in the same column between the core room and the seventh computing power room are set close to the wall of the rooms in the first column, connecting the bridge frames in the same column located at the same horizontal projection position in the rooms in the second column (the same horizontal projection position refers to the bridge frames in the same column that are in an overlapping state between different layers from the overall bird's-eye view); according to the distribution of the room cluster, the other set of bridge frames in the same column between the core room and the second computing power room, and the other set of bridge frames in the same column between the core room and the seventh computing power room are set close to the wall of the rooms in the third column, connecting the bridge frames in the same column located at the same horizontal projection position in the rooms in the second column.

[0047] The bridges in the same column between the first and fourth computing power rooms, and the bridges in the same column between the fourth and sixth computing power rooms are set close to the wall of the second column of rooms; they connect the bridges in the same column at the same horizontal projection position in the first column of rooms. According to the distribution of the room cluster, the bridges in the same column between the third and fifth computing power rooms, and the bridges in the same column between the fifth and eighth computing power rooms are set close to the wall of the second column of rooms; they connect the bridges in the same column at the same horizontal projection position in the third column of rooms.

[0048] Specifically, similar to the flat-layer architecture, for a computing unit POD in a cross-layer architecture, all computing devices can be placed in one computing package room. However, when configuring specific computing devices according to the package room cluster, in order to ensure that the distribution of the package room cluster is symmetrical about the center line of the column where the core package room is located, as well as some other configuration preset conditions, there will be a situation where a computing unit POD is divided into two parts and configured in two adjacent computing packages, such as 0.5 POD in the first computing package room and 0.5 POD in the second computing package room, and the two form a complete POD. In this case, the connection cables of the complete computing unit POD in the first computing package room are connected to the core network device of the core package room in turn through the same bridge in the first computing package room, the same column bridge between the first computing package room and the fourth computing package room, and the same bridge between the fourth computing package room and the core package room. The 0.5 PODs in the first computing power package room are first aggregated into a complete POD through the same-row bridge between the first computing power package room and the second computing power package room with another 0.5 PODs, and then connected to the core network device of the core package room through the same-row bridge between the second computing power package room and the core package room.

[0049] Further, such as Figure 5 As shown, in each private room in the private room cluster, 4 groups of cabinets for placing equipment are arranged in several rows; the extension direction of each row of 4 groups of cabinets is parallel to the horizontal bridge, and each horizontal bridge is connected to the equipment in a row of cabinet groups.

[0050] In addition to an intelligent computing center, the present invention also provides a configuration method for the intelligent computing center, including: Calculate the number of computing units PODs that need to be configured based on the computing power requirements of the computing power cluster in the intelligent computing center; The distribution of the room cluster is symmetrical about the center line of the column where the core room is located. The number of computing unit PODs in each computing room is configured based on the preset distribution conditions. Configure the power resources of each private room according to the type and quantity of equipment in each private room.

[0051] It should be noted that the present invention first derives a reasonable cabinet layout based on the required cluster scale, then modularizes the layout of each private room, and then designs the power distribution, cabinet layout and intra-building interconnection of the data center, and configures each private room with supporting power, refrigeration, security, dynamic environment and other resources to achieve short-distance and low-cost training network construction and centralized storage network construction.

[0052] It is worth mentioning that the overall layout architecture of the intelligent computing center is compatible with both cross-layer and large-floor deployment, and is suitable for the construction of intelligent computing centers in different regions. In terms of the structure of the private room cluster and the equipment configuration method, it follows a mirror-symmetrical design, which facilitates the standardized and rapid construction of each private room regardless of the construction of the intelligent computing center or the cluster. It adopts a double-difference cabinet power design, which is compatible with server performance and power generation differences, and realizes heterogeneous deployment of the entire POD of servers with different performance to form a cluster, ensuring the uniformity of the number of cabinets in each private room, and can be deployed in a standardized manner based on the private room. The equipment in each private room adopts a star-shaped interconnection structure to achieve the shortest wiring distance, greatly saving costs.

[0053] The number of computing units PODs that need to be configured includes: Pre-set the equipment combination of each computing unit POD to determine the computing power of a single computing unit POD; The computing power requirement of the computing power cluster is divided by the computing power of each computing power unit POD, and the number of computing power unit PODs that need to be configured is determined by considering the surplus.

[0054] Configuring the power resources for each private room includes: For the core package room, the power resources to be configured are calculated by summing the power requirements of the core network equipment and the storage equipment. For the computing power package room, the power resources that need to be configured are calculated based on the required power of the computing power equipment and the required power of the associated devices between the computing power equipment and the core network equipment.

[0055] The preset distribution conditions include: dividing the computing power rooms into Class A computing power rooms adjacent to the core rooms and the remaining Class B computing power rooms; The number of PODs in the same type of computing power rooms is the same; The total number of configured PODs for all types of computing power packages is the number of computing power unit PODs that need to be configured; The path required to connect each POD in the computing power room to the core room is the shortest.

[0056] It should be noted that, in addition to the above-mentioned distribution conditions, in order to make the physical path from the computing unit POD in each computing power package room to the core package room as short as possible, even if the number of computing unit PODs in each package room is not an integer, that is, there are some computing unit PODs which are divided into two parts and placed in two adjacent computing power packages, the network requirements of the same computing unit POD must be adjacent to the same layer, and there cannot be cross-layer adjacent situations.

[0057] Specifically, when configuring computing power equipment in a computing power package, firstly, each computing power package has the same number n of complete computing power unit PODs, that is, first complete the division of 8n computing power unit PODs, and the number of computing power unit PODs that need to be configured is m, then there are m-8n computing power unit PODs left (the remaining number is less than 8). The remaining computing power units are then distributed to each computing power package.

[0058] If only one POD is left, this POD is configured to the second computing power package room or the seventh computing power package room based on the preset distribution conditions. It is also possible to combine a complete POD in the second computing power package room with this POD to form two PODs, with one POD placed half in the first computing power package room and half in the second computing power package room, and the other POD placed half in the second computing power package room and half in the third computing power package room. The same configuration method can also be used for the sixth computing power package room, the seventh computing power package room, and the eighth computing power package room.

[0059] If there are only 2 PODs left, place one POD in the second computing power package and the seventh computing power package, or place one POD in the fourth computing power package and the fifth computing power package. Alternatively, add 0.5 POD to the first computing power package, the third computing power package, the sixth computing power package, and the eighth computing power package, so that the 0.5 POD in the first computing power package and the 0.5 POD in the second computing power package are a whole, the other 0.5 POD in the second computing power package and the 0.5 POD in the third computing power package are a whole, the 0.5 POD in the sixth computing power package and the 0.5 POD in the seventh computing power package are a whole, and the other 0.5 POD in the third computing power package and the 0.5 POD in the eighth computing power package are a whole.

[0060] In the same way, when there are 3 PODs left, place one POD in the fourth and fifth computing power packages respectively, and the third POD in the second or seventh computing power package. When there are 4 PODs left, place one POD in the second, fourth, fifth, and seventh computing power packages respectively; or place one POD in the first, third, sixth, and eighth computing power packages respectively; or add 0.5 POD in the first, third, sixth, and eighth computing power packages respectively, and add 1 POD in the fourth and fifth computing power packages respectively. And so on to complete the configuration of the remaining number of PODs.

[0061] In an embodiment of the present invention, based on the construction of a 10,000-card GPU cluster, in order to ensure that there is no obstruction in the operation of the computing power cluster, the network convergence ratio within the cluster is configured according to 1:1, that is, half of the ports of each non-top-level switch are connected to the upper switch, and half of the ports are connected to the GPU server (the number of switch ports is generally a power of 2, and the current mainstream is 64 ports). Therefore, the number of GPU servers in a minimum network full configuration (all ports are fully used) unit is half the number of switch ports, that is, 32 GPU servers. On the basis of 32 units as a group, a 10,000-card GPU fully interconnected cluster is constructed through the Core-Spine-Leaf three-layer network.

[0062] Network topology layering description: The mainstream intelligent computing network uses switches with 64 400G ports; Core layer: a total of 256 switches, all ports of each switch are connected to the Spine layer switches through 400G optical cables; Spine layer: each switch is connected to the Core layer switch through an uplink 400G optical cable, and is connected to the Leaf layer switch through a downlink 400G optical cable; Leaf layer: each switch is connected to the Spine layer switch through a 400G optical cable, and is connected to the GPU server through a 400G optical cable; GPU server: each GPU server contains 8 GPU cards, which are connected to the Leaf layer switch through a 400G network card via a 400G optical cable.

[0063] As shown in the above logic, at the spine level, a fully configured switch consists of 4 groups, each with 32 GPUs, which is called a POD here, that is, a total of 128 8-NIC GPU servers and supporting Internet network equipment. A scale of 10,000 GPUs requires 1,250 (10,000 / 8) GPU servers, and 10 PODs need to be built. In the actual configuration and application process, each intelligent computing center will have power restrictions and consider the economic efficiency of computing power. The computing power of 10,000 GPUs is a relatively basic unit, that is, the computing power of the intelligent computing center is generally built in units of 10,000 GPUs. For intelligent computing centers that are multiples of the scale of 10,000 GPUs (such as 20,000 GPUs or more), they can be replicated and expanded with an overall architecture of 10,000 GPUs. Therefore, an intelligent computing center that accommodates Wanka GPUs needs to consider how to effectively place these 10 PODs and the Spine and Leaf layer switches within the PODs in the nine-square grid-shaped private room of the intelligent computing center of the present invention, as well as the Core layer switches that interconnect these 10 PODs and the data storage support required by the entire Wanka cluster, so as to achieve short-distance interconnection of computing power clusters and orderly expansion of supporting resources.

[0064] Since the core room in the middle of the nine-square grid room cluster is only used to place core network equipment and storage equipment, the rooms for placing computing power equipment are the other eight computing power rooms surrounding the core room. The specific placement conditions are: 10 PODs need to be placed in 8 computing power rooms; considering the uniform distribution of large electromechanical systems, the resource distribution of the room cluster needs to be symmetrical (the electromechanical system of the cross-layer architecture is on the left and right sides of each layer, and the electromechanical system of the flat-layer architecture is on the left and right sides of the same layer); the Spine layer of each POD needs to be connected to the Core layer of the entire cluster by the shortest path. It can be obtained that the number of PODs that can be carried by 4 Class A computing power rooms + the number of PODs that can be carried by 4 Class B computing power rooms = 10 PODs; the capacity of Class A computing power rooms and Class B computing power rooms needs to be evenly distributed on the left and right sides of the overall layout; all Class A and Class B computing power rooms need to be around the core room and need to be connected to the core room by the shortest path.

[0065] After the specific configuration, the core network equipment (core layer network Core) and shared storage equipment of the computing power cluster will be placed in the center of the entire resource layout, that is, the core package room. When configuring power resources, 256 switches in the core network require 32 10KW cabinets, and 6 10KW cabinets are required for each 1000P computing power. The total scale of 10000P computing power is expected to be 1MW. The overall resource layout will be mainly based on the nine-square grid design; that is, the three computing power packages in the first row of the nine-square grid are configured with 1.5POD, 1POD and 1.5POD respectively, for a total of 4POD; the computing power packages on both sides of the core package room in the second row are each configured with 1POD, for a total of 2POD; the three computing power packages in the third row are configured with 1.5POD, 1POD and 1.5POD respectively, for a total of 4POD.

[0066] Among them, 0.5 PODs in the second computing power package and 0.5 PODs in the first computing power package are a complete POD, and another 0.5 PODs in the second computing power package and 0.5 PODs in the third computing power package are a complete POD. Similarly, 0.5 PODs in the seventh computing power package and 0.5 PODs in the sixth computing power package are a complete POD, and another 0.5 PODs in the seventh computing power package and 0.5 PODs in the eighth computing power package are a complete POD. The final layout is a mirror-symmetrical arrangement. On the one hand, it is convenient to evenly configure the power supply system supporting the entire intelligent computing center to ensure the average distribution of power resources during the operation of the computing power cluster. On the other hand, the mirror-symmetrical package design can make the interconnection optical cables between the GPU and Leaf layer, Leaf layer and Spine layer required in the construction of each POD, and the optical cables from the Spine layer of each POD to the Core layer of the entire cluster can be purchased and reserved in advance according to a set of standard lengths, without the need for measurement each time.

[0067] When configuring power resources, a POD includes 128 GPU servers (10KW each), 32 Leaf switches and 32 Spine switches (1.2KW each). The IDC resources required for a POD are 128*10KW+64*1.2KW=1356.8KW. In addition, each POD needs to be configured with a certain number of CPU management nodes, and the overall total is expected to be 1.4MW. Among the 8 computing power rooms, 4 of them have a capacity of 1.4MW, and the remaining 4 rooms are 2.1MW. They need to be arranged in a mirror-symmetrical manner, that is, the second computing power room, the fourth computing power room, the fifth computing power room and the seventh computing power room are 1.4MW, and the first computing power room, the third computing power room, the sixth computing power room and the eighth computing power room are 2.1MW.

[0068] In each computing power package room, the cabinet where the Leaf layer switch is located and the GPU server cabinet of the same POD are built nearby. The distance from the cabinet where each Leaf layer switch is located to the cabinet where the Spine switch of the POD is located is controlled within 50 meters. This distance control requirement can be met in one package room or adjacent packages. In order to ensure the shortest path from the Spine to the Core of each POD, horizontal or cross-layer vertical pipe diameters are reserved near the core package room in all POD packages, which can realize the bridge channel with the distance from the Spine to the Core controlled within 40 meters. The location of the Core layer switch, whether in a cross-layer or flat structure, is close to the interconnection channel connected to the package room where the Core layer switch is located, that is, the first computing power package room and the second computing power package room, the second computing power package room and the third computing power package room, the fourth computing power package room and the core package room, the fifth computing power package room and the core package room, the sixth computing power package room and the seventh computing power package room, and the seventh computing power package room and the eighth computing power package room.

[0069] In addition, in each computing power package room, the GPU cabinet is built with 128 10KW or 64 20KW GPU server cabinets (multiples of 32, the same as the previous POD concept), and the remaining number is built as needed for network and management equipment cabinets; based on the symmetrical design, the server cabinet power design takes into account the heterogeneous GPU models, the generation difference of GPUs from the same manufacturer, and the plane area within the module, and designs two power consumption cabinet specifications A and B, among which the power consumption of Class B cabinets = 2 times the power consumption of Class A cabinets, that is, the power consumption of Class A cabinets is 10KW, and the power consumption of Class B cabinets is 20KW. The second computing power package room, the fourth computing power package room, the fifth computing power package room and the seventh computing power package room, which are class A computing power packages, can use class A cabinets, and the first computing power package room, the third computing power package room, the sixth computing power package room and the eighth computing power package room, which are class B computing power packages, can use class B cabinets, so that the number of physical cabinets in each package room is consistent (in this embodiment, the configuration power of the class B computing power package room is actually only 1.5 times that of the class A computing power package room, but because the unit volume power of the class B cabinet is higher, cooling equipment will be configured in the cabinet, and the cooling equipment can occupy a part of the volume of the cabinet, so that the physical number of cabinets in the class A computing power package room and the class B computing power package room are consistent), so as to standardize products by package room, take into account the needs of different generations of servers, and deploy quickly.

[0070] The above embodiments are further elaborations and illustrations of the present invention for ease of understanding, and are not limitations of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent computing center, characterized in that: The invention comprises a cluster of private rooms arranged in three rows and three columns, wherein the cluster of private rooms comprises a bridge network connecting the private rooms; the bridge network comprises: Intra-room bridge: Each private room is equipped with an intra-room bridge to connect the equipment in the room; In-row bridge, connecting the bridges in two adjacent rooms in the same row; The bridge in the same row connects the bridges in two adjacent rooms in the same row; The room cluster includes a core room located in the middle and computing power rooms surrounding the core room; the distribution of the room cluster is symmetrical about the center line of the column where the core room is located.

2. The intelligent computing center according to claim 1, characterized in that: The intelligent computing center includes: Cross-layer architecture, the second row of core rooms are located in the middle layer, and the first and third rows are located in the upper and lower layers adjacent to the middle layer respectively; Flat architecture, all private room clusters including core private rooms are located on the same layer.

3. An intelligent computing center according to claim 1 or 2, characterized in that: The core package room is used to place the core network equipment and storage equipment in the computing power cluster; the computing power package room is used to place the computing power equipment in the computing power cluster; The intra-room bridge frames include a plurality of transverse bridge frames and a plurality of longitudinal bridge frames. The transverse bridge frames and the longitudinal bridge frames are perpendicular to each other, and the transverse bridge frames are parallel to the center line of the row where the core package room is located.

4. The intelligent computing center according to claim 3, characterized in that: In the flat structure, the first row of private rooms includes at least two groups of bridge frames in the same row, and one group of bridge frames in the same row is arranged close to the wall between the first row of private rooms and the second row of private rooms; The second row of private rooms includes at least two groups of bridge frames in the same row, one group of bridge frames in the same row is arranged close to the wall between the first row of private rooms and the second row of private rooms, and the other group of bridge frames in the same row is arranged close to the wall between the third row of private rooms and the second row of private rooms.

5. The intelligent computing center according to claim 3, characterized in that: In the cross-layer structure, the bridge frames in the same row of the private rooms in the first row are arranged close to the wall between the private rooms in the first row and the private rooms in the second row, connecting the bridge frames in the same row in the same horizontal projection position in two adjacent private rooms in the first row; The second row of private rooms includes at least two groups of same-row bridge frames, one group of which is arranged close to the wall between the first row of private rooms and the second row of private rooms, connecting the same-row bridge frames at the same horizontal projection position in two adjacent private rooms in the second row.

6. An intelligent computing center according to claim 4 or 5, characterized in that: Each private room in the private room cluster is provided with a plurality of rows of cabinet groups for placing equipment; The extension direction of each row of cabinet groups is parallel to the transverse bridge, and each transverse bridge is connected to the equipment in one row of cabinet groups.

7. A configuration method for an intelligent computing center, applicable to the intelligent computing center according to any one of claims 1 to 6, characterized in that: include: Calculate the number of computing units PODs that need to be configured based on the computing power requirements of the computing power cluster in the intelligent computing center; The distribution of the room cluster is symmetrical about the center line of the column where the core room is located. The number of computing unit PODs in each computing room is configured based on the preset distribution conditions. Configure the power resources of each private room according to the type and quantity of equipment in each private room.

8. The configuration method of an intelligent computing center according to claim 7, characterized in that: The number of computing power units PODs that need to be configured for the calculation includes: Pre-set the equipment combination of each computing unit POD to determine the computing power of a single computing unit POD; The computing power requirement of the computing power cluster is divided by the computing power of each computing power unit POD, and the number of computing power unit PODs that need to be configured is determined by considering the surplus.

9. A method for configuring an intelligent computing center according to claim 7 or 8, characterized in that: The configuration of the power resources of each private room includes: For the core package room, the power resources to be configured are calculated by summing the power requirements of the core network equipment and the storage equipment. For the computing power package room, the power resources that need to be configured are calculated based on the required power of the computing power equipment and the required power of the associated devices between the computing power equipment and the core network equipment.

10. A configuration method for an intelligent computing center according to claim 9, characterized in that: The preset distribution conditions include: dividing the computing power rooms into Class A computing power rooms adjacent to the core rooms and the remaining Class B computing power rooms; The number of PODs in the same type of computing power rooms is the same; The total number of configured PODs for all types of computing power packages is the number of computing power unit PODs that need to be configured; The path required to connect each POD in the computing power room to the core room is the shortest.

Images