Rackless optical data center architecture for collective communication

Abstract

The application discloses a rackless optical data center architecture for collective communication, which introduces a programmable switch on the basis of an original rackless optical data center architecture based on an optical cross-connection, accelerates a calculation process through the introduced programmable switch, reduces the proportion of cross-rack traffic which increases with the increase of nodes, realizes acceleration of collective communication services, designs an optical cross-connection reconfiguration scheme, and forms new logical racks from traffic generated by rackless optical data center edge servers; through real-time monitoring of traffic demand by a control plane, when traffic communication demand in the network changes or new services are serviced, the programmable switch is controlled in runtime by the control plane, and the traffic of the rackless optical data center is dynamically adjusted. The architecture can reduce communication cost and calculation cost of the rackless optical data center when the rackless optical data center runs collective communication, reduce the completion time of collective communication services, and realize efficient traffic scheduling and service acceleration.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a rackless optical data center architecture for aggregated communication. Background Technology

[0002] Currently, due to the rise of data-intensive applications (such as big data analytics and large-scale language model training), services involving collective communication are beginning to dominate data centers. Collective communication involves a group of processes synchronously iterating through computation and communication phases. During the computation phase, each process completes its own data processing task, thus there is almost no data exchange between processes; however, during the communication phase, a large amount of data traffic with predetermined patterns (such as broadcasting, reverse aggregation, and full interconnection) is generated. Therefore, collective communication further shifts the main bottleneck of data centers from computation to networking, especially at the rack level, requiring greater network reconfigurability. This has led to the introduction of rackless optical data centers (ORDCs)—by inserting optical path switches between servers and top-of-rack switches, breaking down the physical rack boundaries of the data center, avoiding cross-rack service fragmentation, and adapting to dynamic traffic patterns.

[0003] While rackless optical data centers offer significant advantages in reducing cross-rack requirements, the efficiency of collective communication can be greatly improved by introducing in-network computing, even if it is only sparsely implemented. Recent advancements in P4-based programmable data planes (PDPs) have made it possible to directly aggregate and perform arithmetic operations on packet fields within programmable switches, at line speeds (100Gbps or higher). Therefore, in-network computing can offload some computational phases of collective communication to top-of-rack switches (P-ToR switches) based on PDPs, effectively accelerating computation. Taking collective communication in a MapReduce cluster as an example, it includes four phases: 1) Map phase (the master node broadcasts data to worker nodes); 2) Shuffle phase (executed by worker nodes); 3) Upload phase (worker nodes aggregate results and send them to the master node); 4) Reduce phase (the master node summarizes the results from worker nodes). Since the summarization operation in the fourth phase can be implemented using a programmable data plane, in-network computing technology can offload it to a P-ToR switch, allowing the third and fourth phases to run simultaneously, thus effectively accelerating collective communication. Furthermore, since the P-ToR switch only sends the results after on-network computation and aggregation to the master node, it can absorb half of the traffic from the worker nodes to the master node in the third phase. As the number of worker nodes increases, the proportion of traffic absorbed also increases.

[0004] Existing rackless optical data center architectures offer a traffic topology engineering approach for optical cross-connect reconfiguration. This allows for flexible reconfiguration based on communication needs, breaking through physical rack boundaries to form logical racks and transforming cross-rack communication requirements into intra-rack requirements, thereby reducing communication bottlenecks. However, due to the limitation on the number of switch ports, cross-rack traffic cannot be reduced indefinitely. Furthermore, given the burstiness of services, deciding whether to reduce cross-rack traffic for existing tasks by interrupting them when the number of ports is limited becomes a pressing issue. Summary of the Invention

[0005] The purpose of this invention is to provide a rackless optical data center architecture for aggregated communication. This architecture can reduce the communication and computing costs of rackless optical data centers when running aggregated communication, reduce the completion time of aggregated communication services, and achieve efficient traffic scheduling and service acceleration.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] A rackless optical data center architecture for aggregated communication is proposed. Based on the original rackless optical data center architecture based on optical cross-connects, a programmable switch is introduced. This programmable switch accelerates the computing process and reduces the proportion of cross-rack traffic that increases with the number of nodes, thereby accelerating aggregated communication services. Specifically:

[0008] Deploying a P4 pipeline on the introduced programmable switch aggregates the received collective communication Reduce phase packets on the programmable switch without performing aggregation on the master node, thus accelerating the process while reducing the traffic transmitted to the master node. Here, the P4 pipeline refers to the various stages of packet processing in network devices programmed using the P4 language. P4 is a programmable network data plane language used to define the processing flow of network packets.

[0009] Design an optical cross-connect reconfiguration scheme to organize the traffic generated by the rackless optical data center edge server into a new logical rack.

[0010] A control plane is constructed to monitor traffic demand in real time. When the traffic communication demand in the network changes or new services are provided, the programmable switch is controlled at runtime through the control plane to dynamically adjust the traffic of the rackless optical data center and accelerate the aggregated communication services under the condition of limited programmable switch ports.

[0011] As can be seen from the technical solution provided by the present invention, the above architecture can reduce the communication and computing costs of rackless optical data centers when running aggregated communication, reduce the completion time of aggregated communication services, and achieve efficient traffic scheduling and service acceleration. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of a rackless optical data center architecture for aggregated communication provided in an embodiment of the present invention;

[0014] Figure 2 This is a schematic diagram illustrating the data transmission process of the rackless optical data center architecture described in an embodiment of the present invention;

[0015] Figure 3 This is an example diagram of the scheme for reconfiguration via optical cross-connection as described in this invention;

[0016] Figure 4 This is an application example diagram of the rackless optical data center architecture described in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments, and do not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0018] like Figure 1 The diagram shown is a schematic of a rackless optical data center architecture for aggregated communication provided in an embodiment of the present invention. Based on the original rackless optical data center (ORDC) architecture based on optical cross-connects, a programmable switch (P-ToR) is introduced. The introduced programmable switch accelerates the computing process and reduces the proportion of cross-rack traffic that increases with the number of nodes, thereby accelerating aggregated communication services.

[0019] Deploying a P4 pipeline on the introduced programmable switch aggregates the received collective communication Reduce phase packets on the programmable switch, eliminating the need for aggregation on the master node. This accelerates processing while reducing the traffic transmitted to the master node. The P4 pipeline refers to the various stages of packet processing in network devices programmed using the P4 language. These stages can be executed in parallel to improve the efficiency of network packet processing. P4 is a programmable network data plane language used to define the processing flow of network packets.

[0020] Design an optical cross-connect reconfiguration scheme to organize the traffic generated by the rackless optical data center edge server into a new logical rack.

[0021] A control plane is constructed to monitor traffic demand in real time. When network traffic communication demands change or new services are provided, the control plane performs runtime control on the programmable switches to dynamically adjust the traffic in the rackless optical data center, accelerating aggregated communication services despite the limited number of programmable switch ports. Specifically, the control plane periodically collects traffic data from the switches in the data center network; based on the monitored data, a scheduling algorithm is triggered to dynamically adjust the traffic in the rackless optical data center and generate new forwarding rules; the new forwarding rules are distributed to the switches via a runtime protocol; and the switches update their flow tables and change the transmission paths of the traffic according to the new forwarding rules.

[0022] In specific implementation, the deployment of a P4 pipeline on the introduced programmable switch aggregates the received data packets from the collective communication Reduce phase on the programmable switch, eliminating the need for aggregation on the master node. This accelerates the process while reducing the traffic transmitted to the master node. Specifically, this includes:

[0023] This application uses a word counting function within Hadoop MapReduce (Hadoop is an open-source framework for storing and processing large amounts of data; MapReduce is a core component of Hadoop, a programming model and framework for processing large volumes of data, known as the map-reduce model) to calculate the frequency of specific words in the input set. The collective communication for the word counting application occurs in four phases. A DPDK-based packet encoder is programmed to intercept all worker-to-master packets during the third phase of collective communication, Upload, and encodes them into a custom packet format, including JobID, isAgg, key, and value, such as... Figure 2 The diagram shown is a schematic representation of the data transmission process in the rackless optical data center architecture according to an embodiment of the present invention, wherein:

[0024] JobID is used to distinguish different MapReduce clusters; isAgg is a flag used to indicate whether the data packet is received from a worker node (isAgg=0) or is about to be sent to the master node (isAgg=1); key is used to represent a word; value is used to store the number of times the word appears;

[0025] Deploying a P4 pipeline on the introduced programmable switch enables the third phase (Upload) and the fourth phase (Reduce) of aggregated communication to occur simultaneously. When a service completes the Reduce phase and performs computation locally, the traffic of other services about to undergo the Reduce phase is diverted to the rack where the programmable switch is located. The Reduce phase of that service is accelerated through the programmable switch, thereby reusing the resources of the programmable switch in time.

[0026] The specific process of deploying the P4 pipeline on the introduced programmable switch is as follows:

[0027] The P4 pipeline is deployed by communicating with the programmable switch using the control protocol, loading the configuration file into the programmable switch, updating the parser and match-action table, and distributing the generated configuration file to the programmable switch via the control plane.

[0028] like Figure 2 As shown, the specific process of deploying a P4 pipeline for on-network computing on the introduced programmable switch is as follows:

[0029] 1) When a data packet enters the Barefoot hardware switch, the packet header defined by the DPDK-based packet encoder is parsed in the ingress pipeline. The Barefoot hardware switch uses the P4 programming language, which allows users to flexibly configure and program the data according to specific needs.

[0030] 2) In the ingress pipeline, the data packet first undergoes JobID matching. Specifically: the data packet first obtains the index of a register by hashing the JobID and the key, then increments the value in the corresponding index register by register.value, and increments the counter in the index register by register.counter to count the number of data received for the corresponding business.

[0031] 3) Determine if the value has been aggregated in the register. If the aggregation is complete, transmit the data packet to the master node; if the aggregation is not complete, discard the data packet and wait for the data packets from other nodes to arrive.

[0032] 4) The aggregated data packets are sent from the switch's exit point into the link.

[0033] In addition, the aforementioned optical cross-connect reconfiguration scheme organizes the traffic generated by the rackless optical data center edge server into a new logical rack, specifically including:

[0034] Because an optical cross-connection is added between the server and the top-of-rack switch, the physical rack boundary is broken. Therefore, the logical rack can be reorganized by changing the optical cross-connection method. Specifically, when there is a communication requirement between several servers, these servers can be reconfigured to a programmable switch (P-ToR) with a sufficient number of ports by changing the optical cross-connection method to form a new logical rack.

[0035] For example, such as Figure 3 The diagram shown is an example of the reconfiguration scheme via optical cross-connection described in this invention. When there is a communication requirement between the four servers on the left, they are placed under a programmable switch (P-ToR) with a sufficient number of ports to form a rack.

[0036] When the four servers on the right have communication needs, the optical cross-connection method can be changed to form a new logical rack with the programmable switch P-ToR. This can reduce cross-rack traffic and avoid communication bottlenecks caused by cross-rack traffic.

[0037] In practice, since a programmable data plane P4 pipeline is deployed in the programmable switch P-ToR, when applications on the server can be offloaded to the programmable switch P-ToR and the computing process can be accelerated by the hardware switch, this reconfiguration scheme can realize the time reuse of the computing resources of the programmable switch P-ToR, and can more effectively address the cost issue of replacing ordinary switches with programmable switches.

[0038] It is worth noting that the contents not described in detail in the embodiments of the present invention belong to the prior art known to those skilled in the art.

[0039] For example, such as Figure 4 The diagram shown is an application example of the rackless optical data center architecture described in this embodiment of the invention. For the top-of-rack switch, server, and optical cross-connect, a top-of-rack switch controller, a service processor, and an optical cross-connect controller are designed, which are managed through their respective southbound interfaces.

[0040] The top-of-rack switch controller collects traffic statistics from top-of-rack switches (including ordinary top-of-rack switches (ToR) and the programmable switches (P-ToR) introduced in this application) to update the traffic engineering database (TED). Servers in the P4-ORDC pod register their collective communication jobs with the service processor. The service processor checks the JobID database to assign a unique JobID to each job and forwards job information (duration of the computation phase, data size of the communication phase, etc.) to the TED. The service processor also interacts with the on-net computing scheduler to allocate on-net computing resources for the offloadable computation phases in the P-ToR.

[0041] TED analyzes traffic and job information in the data plane, abstracting a three-dimensional (3D) traffic matrix representing data transmission between server pairs over a series of future time periods; then, as... Figure 4 As shown on the right, the topology engineering module acquires a series of optical cross-connect configurations. Each configuration is optimized to group servers that are communicating into the same logical rack, so that collective communication in the corresponding future time period can be accelerated to the maximum extent.

[0042] Finally, the optical cross-connect configuration will be applied to the optical cross-connect according to the optical cross-connect controller's plan. TED will calculate the traffic routing scheme based on each optical cross-connect configuration and implement these schemes in the top-of-rack switch through the top-of-rack switch controller.

[0043] In summary, the architecture (P4-ORDC) described in this embodiment of the invention has the following advantages:

[0044] 1. While maintaining the core operating principles of rackless optical data centers, replacing a small number of ordinary switches with programmable switches deployed with P4 pipelines can effectively reduce the time required for service completion. Most on-grid computing technologies consider programmable switches for full coverage in their architectures. However, due to cost considerations, real-world optical data centers do not replace all top-rack switches with programmable switches. Therefore, when using programmable switches for on-grid computing offloading tasks, cross-rack traffic is inevitably generated, creating communication bottlenecks. Introducing optical cross-connections between servers and top-rack switches can effectively avoid this problem, preventing cross-rack traffic while maximizing the utilization of limited programmable switch resources.

[0045] 2. The rackless optical data center architecture P4-ORDC proposed in this invention is compared with three existing architectures:

[0046] Traditional data center architectures like FatTree lack optical cross-connects, and all top-rack switches are ordinary switches; traditional rackless optical data centers (ORDCs) do not utilize on-grid computing technologies; optical interconnect data centers (P4INC-AOIs) interconnect programmable switches (P-ToRs) and ordinary top-rack switches (ToRs) from the top using optical cross-connects.

[0047] Due to the lack of network reconfiguration capabilities and on-line computing, the traditional data center architecture FatTree provides the longest service completion time; followed by the traditional rackless optical data center ORDC, which does not apply on-line computing technology and only reduces cross-rack traffic; due to the significant acceleration benefits of on-line computing, the service completion time of the optical interconnect data center P4INC-AOI and the P4-ORDC of this application can be greatly reduced, and the P4-RDC of this application can reduce more cross-rack traffic through the reconfiguration of optical cross-connects;

[0048] 3. The rackless optical data center architecture proposed in this invention has significant advantages in addressing the scheduling problems of current data center services, which are mainly based on collective communication services. Most collective communication service scheduling problems start with an optimal placement problem, without considering the dynamic nature of the service and bursty traffic. The P4-ORDC described in this application, due to its flexible scheduling capabilities at the rack edge, has strong compatibility with placement problems. Even if the initial placement is not optimal, the service completion time can be optimized through the reconfiguration of optical cross-connects. Under the P4-ORDC architecture described in this application, the service completion time can be reduced by 27.4% to 43.3%.

[0049] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. A rackless optical data center architecture for aggregated communication, characterized in that, Based on the original rackless optical data center architecture using optical cross-connects, programmable switches are introduced. These switches accelerate the computing process and reduce the proportion of cross-rack traffic that increases with the number of nodes, thereby accelerating aggregated communication services. Deploying a P4 pipeline on the introduced programmable switch aggregates the received collective communication Reduce phase packets on the programmable switch without performing aggregation on the master node, thus accelerating the process while reducing the traffic transmitted to the master node. Here, the P4 pipeline refers to the various stages of packet processing in network devices programmed using the P4 language. P4 is a programmable network data plane language used to define the processing flow of network packets. Design an optical cross-connect reconfiguration scheme to organize the traffic generated by the rackless optical data center edge server into a new logical rack. A control plane is constructed to monitor traffic demand in real time. When the traffic communication demand in the network changes or new services are provided, the programmable switch is controlled at runtime through the control plane to dynamically adjust the traffic of the rackless optical data center and accelerate the aggregated communication services under the condition of limited programmable switch ports.

2. The rackless optical data center architecture for aggregated communication according to claim 1, characterized in that, The deployment of the P4 pipeline on the introduced programmable switch aggregates the received collective communication Reduce phase data packets on the programmable switch without performing aggregation on the master node, thereby accelerating traffic while reducing the amount of data transmitted to the master node. Specifically, this includes: This application uses the Hadoop MapReduce mapping-reduction model to count the occurrences of specific words in the input set. The collective communication for the word counting application occurs in four phases. A DPDK-based packet encoder is programmed to intercept all worker-to-master packets during the third phase, Upload, and encodes them into a custom packet format including JobID, isAgg, key, and value. JobID is used to distinguish different MapReduce clusters; isAgg is a flag that indicates whether the data packet was received from a worker node or is about to be sent to the master node; key is used to represent a word; value is used to store the number of times the word appears. Deploying a P4 pipeline on the introduced programmable switch enables the third phase (Upload) and the fourth phase (Reduce) of aggregated communication to occur simultaneously. When a service completes the Reduce phase and performs computation locally, the traffic of other services about to undergo the Reduce phase is diverted to the rack where the programmable switch is located. The Reduce phase of that service is accelerated through the programmable switch, thereby reusing the resources of the programmable switch in time.

3. The rackless optical data center architecture for aggregated communication according to claim 2, characterized in that, The specific process of deploying the P4 pipeline on the introduced programmable switch is as follows: The P4 pipeline is deployed by communicating with the programmable switch using the control protocol, loading the configuration file into the programmable switch, updating the parser and match-action table, and distributing the generated configuration file to the programmable switch via the control plane.

4. The rackless optical data center architecture for aggregated communication according to claim 2, characterized in that, The process of deploying a P4 pipeline for on-network computing on the introduced programmable switch is as follows: 1) When a data packet enters the Barefoot hardware switch, the packet header defined by the DPDK-based packet encoder is parsed in the ingress pipeline. The Barefoot hardware switch uses the P4 programming language, which allows users to flexibly configure and program the data according to specific needs. 2) In the ingress pipeline, the data packet first undergoes JobID matching. Specifically: the data packet first obtains the index of a register by hashing the JobID and the key, then increments the value in the corresponding index register by register.value, and increments the counter in the index register by register.counter to count the number of data received for the corresponding business. 3) Determine if the value has been aggregated in the register. If the aggregation is complete, transmit the data packet to the master node; if the aggregation is not complete, discard the data packet and wait for the data packets from other nodes to arrive. 4) The aggregated data packets are sent from the switch's exit point into the link.

5. The rackless optical data center architecture for aggregated communication according to claim 1, characterized in that, The process of dynamically adjusting the traffic of the rackless optical data center by controlling the programmable switch through the control plane to accelerate aggregated communication services under the condition of limited programmable switch ports is as follows: Traffic data from switches in the network is collected periodically by controlling the plane. Based on the monitored data, the scheduling algorithm is triggered to dynamically adjust the traffic in the rackless optical data center and generate new forwarding rules; New forwarding rules are distributed to switches in the network via runtime protocols; The switches in the network update the flow tables according to the new forwarding rules, changing the transmission path of the traffic.

6. The rackless optical data center architecture for aggregated communication according to claim 1, characterized in that, The proposed optical cross-connect reconfiguration scheme reassembles the traffic generated by rackless optical data center edge servers into a new logical rack, specifically including: The logical rack can be reorganized by changing the optical cross-connect method. Specifically, when there is a communication requirement between several servers, these servers can be reconfigured to a programmable switch (P-ToR) with a sufficient number of ports to form a new logical rack.

Images