Data transmission method and system based on high-speed network, and computing device cluster

Abstract

The invention discloses a data transmission method and system based on a high-speed network and a computing device cluster, the method is applied to a service grid system based on the high-speed network, the system comprises a gateway, a shared memory pool and a computing node deployed with a service container and a network agent, the gateway, the shared memory pool and the computing node are connected through the high-speed network, and the service container is connected with the network agent. The method comprises: after acquiring first data, a gateway determines an address for sending the first data or a first transmission mode of the first data and a first service container for receiving the data according to container information and a routing rule, and the transmission mode can be any one of a plurality of fast channels supported by the gateway and using an RDMA (Remote Direct Memory Access) technology. And the first network agent connected with the first service container directly acquires the first data. In the above process, data exchange is directly realized between the gateway and the network agent through an intelligent routing rule, an RDMA technology and the like, the network transmission speed can be improved, and computing resources are saved.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a data transmission method and system based on a high-speed network, and a computing device cluster. Background Art

[0002] At present, serverless services have become an important form of service provided by cloud platforms due to their features of free operation and maintenance and rapid development. Each serverless instance in the computing node is connected to a sidecar container, whose network configuration is managed by the control plane and is used to proxy the inbound and outbound traffic of the serverless instance and implement the scheduling of the network traffic of the serverless instance.

[0003] In the network management of serverless service grid, serverless instances cannot communicate directly with gateways or with each other, and independent proxy servers and sidecar containers are required to perform multiple packet unpacking and forwarding operations. Therefore, sidecar containers occupy a large amount of computing resources, take a long time to process data packets, and have slow network transmission speeds. Summary of the invention

[0004] The present application provides a data transmission method, system and computing device cluster based on a high-speed network. In a service grid system based on a high-speed network, gateways and network agents directly implement data interaction through remote direct memory access (RDMA) technology and intelligent routing rules, which can improve network transmission speed and reduce the consumption of computing resources.

[0005] In the first aspect, the present application provides a data transmission method based on a high-speed network, which is applied to a service grid system based on a high-speed network. The system includes a gateway connected through a high-speed network, a shared memory pool, and a computing node deployed with a business container and a network agent, wherein the business container is connected to the network agent. The method includes: after the gateway obtains the first data, the gateway determines the address for sending the first data or the first transmission mode of the first data and the first business container for receiving the data according to the container information and the routing rule, wherein the transmission mode can be any one of a plurality of fast channels supported by the gateway, including the use of RDMA technology. The first network agent connected to the first business container directly obtains the first data, and similarly, the first network agent can support the above-mentioned plurality of fast channels.

[0006] In the above process, the gateway and the network agent interact with data addresses or data through RDMA technology and intelligent routing rules in a service grid system based on a high-speed network, thereby obtaining data stored in a shared memory pool or shared memory in a computing node. There is no need to unpack or encapsulate data packets, nor is there a need to deploy an independent proxy server in the system to process and forward data packets. This can improve network transmission speed and save computing resources.

[0007] In a possible implementation, container information and routing rules are stored in a shared memory pool or in a shared memory of a computing node. Routing rules include transmission rules and transmission mode selection strategies between containers, wherein the transmission rules between containers include one or more of load balancing LB rules, label-based routing rules, time domain identification, memory location, and container / container group binding relationship rules, which are used to determine the next-hop business container, and the transmission mode selection strategy is used to determine the rules and algorithms of the target transmission method according to the data transmission volume or data delay requirements. The above-mentioned container information and routing rules can also be continuously updated according to actual conditions. When the gateway or network agent forwards data, the container information and routing rules in the shared memory pool or shared memory are accessed, or the container information and routing rules are pulled locally to determine the next-hop business container and transmission method.

[0008] In a possible implementation, after the first network agent obtains the first data, the first network agent stores the first processing result generated based on the first data in a shared memory pool or in a shared memory of a computing node including the first network agent. Similar to the step of determining the first business container by the gateway, the first network agent also needs to determine whether there is a next-hop business container based on the container information and routing rules. In the above process, the first network agent stores the first processing result in a shared memory pool or shared memory, so that multiple computing nodes can share a copy of the first processing result, thereby saving memory resources.

[0009] In a possible implementation, when the first network agent determines that there is a next-hop business container according to the container information and the routing rule, the first network agent determines the second business container and the second transmission mode, and sends the address of the first processing result or the first processing result to the second business container through the second transmission mode, wherein the second transmission mode is also one of a plurality of fast channels. The first network agent determines the next-hop business container and the appropriate transmission mode through the intelligent routing rule, which can improve the network transmission performance. Among them, the second transmission mode has the following situations: when the first business container and the second business container are in different computing nodes, that is, when data is exchanged between different computing nodes, the second business container accesses the memory space or shared memory space corresponding to the first business container through the second network agent or the second gateway in the corresponding computing node; when data is accessed based on the shared memory pool, the first network agent or the first gateway stores the first processing result in the shared memory pool and sends the stored address to the second business container; when the first business container and the second business container are in the same computing node and data is shared based on the shared memory, the first network agent caches the data in the local shared memory and sends the cache address to the second business container; when the data transmission volume is very small, for example, when the data transmission volume is at the KB level, the first network agent piggybacks the first processing result in the signaling of establishing a connection with the second business container. The above transmission modes do not require the network agent and the gateway to perform operations such as packet unpacking, and data acquisition can be achieved by transmitting the data address or data, and even the data plane transmission does not need to be established, so the network transmission speed can be improved.

[0010] In a possible implementation, when the first network agent determines that there is no next-hop business container based on the container information and the routing rules, the third transmission mode is determined, and the address of the first processing result or the first processing result is sent to the gateway through the third transmission mode, wherein the third transmission mode is also any one of a plurality of fast channels, which will not be elaborated here. When there is no next-hop business container, the business process ends, and the network agent sends the processing result to the user device through the gateway. The third transmission mode can realize direct data transmission between the gateway and the network agent, which can improve the network transmission speed.

[0011] In a second aspect, the present application provides a service grid system based on a high-speed network, the system comprising a gateway connected via a high-speed network, a shared memory pool, and a computing node deployed with a business container and a network agent, wherein the business container is connected to the network agent. The gateway is used to obtain the first data, wherein the gateway supports multiple fast channels including the use of remote direct data access RDMA technology; according to the container information and the routing rules, the first business container and the first transmission mode are determined, wherein the first transmission mode is any one of the multiple fast channels; the address of the first data or the first data is sent to the first business container via the first transmission mode; the first network agent corresponding to the first business container is used to obtain the first data, wherein the first network agent supports multiple fast channels.

[0012] In a possible implementation, container information and routing rules are stored in a shared memory pool or in a shared memory of a computing node. Routing rules include transmission rules between containers and transmission mode selection strategies, wherein the transmission rules between containers include one or more of load balancing LB rules, label-based routing rules, time domain identification ID, memory location, and container / container group binding relationship rules, which are used to determine the next-hop service container, and the transmission mode selection strategy is used to determine the rules and algorithms of the target transmission mode according to the data transmission volume or data delay requirements.

[0013] In a possible implementation, after obtaining the first data, the first network agent is further used to store a first processing result generated based on the first data in a shared memory pool or in a shared memory of a computing node including the first network agent; and determine whether there is a next-hop business container based on the container information and routing rules.

[0014] In a possible implementation, when it is determined that there is a next-hop business container according to the container information and the routing rule, the first network agent is specifically used to determine the second business container and the second transmission mode, and send the address of the first processing result or the first processing result to the second business container through the second transmission mode, wherein the second transmission mode is any one of a plurality of fast channels.

[0015] In a possible implementation, the first network agent is specifically used to determine a third transmission mode when it is determined according to the container information and the routing rule that there is no next-hop business container, and send the address of the first processing result or the first processing result to the gateway through the third transmission mode, wherein the third transmission mode is any one of a plurality of fast channels.

[0016] In a third aspect, the present application provides a computing device cluster, which includes at least one computing device, each computing device including a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the method provided in the first aspect.

[0017] In a fourth aspect, the present application provides a computer program product comprising instructions, which, when executed by a computing device cluster, enables the computing device cluster to execute the method provided in the first aspect.

[0018] In a fifth aspect, the present application provides a computer-readable storage medium, which includes computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method provided in the first aspect.

[0019] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments.

[0021] Figure 1 It is a structural diagram of a high-speed network-based service grid system provided in an embodiment of the present application;

[0022] Figure 2 is a flow chart of a data transmission method based on a high-speed network provided in an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0025] Figure 5 It is a structural diagram of one or more computing devices connected via a network provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] At present, serverless services have become an important form of service provided by cloud platforms due to their on-demand resource allocation, maintenance-free, and rapid development. The computing nodes of serverless services include business containers and sidecar containers. Business containers are connected to sidecar containers. The network configuration of sidecar containers is managed by the control plane and is used to proxy the inbound and outbound traffic of business containers, thereby scheduling the network traffic of business containers.

[0027] In the network management of the serverless service grid, the business containers in the computing nodes cannot communicate directly with each other, and independent proxy servers and sidecar containers are required to perform operations such as packaging, unpacking, and forwarding. The data request sent by the gateway must also be forwarded to the backend computing node via an independent proxy server. The sidecar container corresponding to the serverless instance in the computing node unpacks the data request forwarded by the proxy, and if there is a next hop, it continues to package and send the data packet to the proxy server, which forwards it to the target computing node.

[0028] In the above process, the sidecar container in the computing node takes up a lot of computing resources for packet unpacking, which leads to long data processing time and slow network transmission speed. In addition, the control plane and the network configuration information sent to the container also take up a lot of storage resources, which increases the cost for the serverless platform that charges by resources used.

[0029] In order to solve the above problems, the present application proposes a data transmission method based on a high-speed network, by deploying gateways and network agents that support high-speed networks such as transmission control protocol / internet protocol (TCP / IP) and remote direct memory access (RDMA) in a disaggregated data center (DDC), so that the network agent in the computing node can access the shared memory pool or the shared memory in the computing node according to the data address, and determine the transmission method according to the routing rules and container information to achieve data interaction. In this process, the transmission of data does not require independent proxy servers and sidecar containers to perform packetization, unpacking and forwarding operations. Data addresses can be exchanged through a high-speed network to obtain data, thereby improving the speed of network transmission and saving computing resources. The shared memory pool in the disaggregated data center can also reduce the demand for storage resources. When the above method is applied to serverless services, the operating cost of the serverless cloud platform can be reduced.

[0030] The present application proposes a high-speed network-based data transmission method that is applied to a high-speed network-based service grid system. Figure 1 As shown, Figure 1It is a structural diagram of a service grid system based on a high-speed network provided in an embodiment of the present application. The service grid system based on a high-speed network includes a gateway 110, a shared memory pool 120, and a computing node 130 deployed with a business container 131 and a network agent 132. The above-mentioned gateway, shared memory pool and computing node are connected via a high-speed network such as RDMA.

[0031] In the specific implementation, the gateway is used to process the inbound and outbound traffic of the system. In addition to supporting traditional sockets and TCP / IP, it can also add multiple fast channels including RDMA technology in a pluggable manner. The gateway can also support more fast channels, which is not specifically limited in this application.

[0032] In a specific implementation, the shared memory pool includes a shared data pool for caching data to be transmitted and a space for storing container information and routing rules, wherein the container information and routing rules are continuously updated according to actual conditions. Routing rules include transmission rules and transmission mode selection strategies between containers, wherein the transmission rules between containers include one or more of load balancing LB rules, label-based routing rules, time domain identification ID, memory location, and container / container group binding relationship rules, which are used to determine the next-hop business container. The transmission rules between containers can also include more or less content, which is not specifically limited in this application. The transmission mode selection strategy is used to determine the rules and algorithms of the target transmission method based on the data transmission volume or data delay requirements. The shared memory pool can also include more or less information, which is not specifically limited in this application.

[0033] In the specific implementation, the business container in the computing node establishes a socket or other connection with the network proxy. The network proxy is extremely lightweight and supports multiple transmission modes, which is used to process the traffic in and out of the connected business container. In the case where the network proxy in the computing node is a container, each network proxy is connected to a business container; in the case where the network proxy in the computing node is a host-level container, each network proxy can be connected to multiple business containers, which is not specifically limited in this application.

[0034] In the specific implementation, for applications that mainly communicate within the computing node, a node gateway 133 can also be deployed in the computing node, so that the network agent is only responsible for the communication within the computing node, and the communication across the computing nodes is handled by the node gateway, which can reduce the resource occupation of the network agent; for applications that mainly communicate between computing nodes, there is no need to deploy a node gateway in the computing node, and the network agent can handle all traffic, thereby avoiding one more hop to increase computing overhead. The computing node can also include a shared memory 134 for storing container information and routing rules, etc. The computing node can also include more or less content, which is not specifically limited in this application.

[0035] In a possible implementation manner, the business container in the computing node in the above process may also be a different instance such as a virtual machine, which is not specifically limited in this application.

[0036] The above-mentioned high-speed network-based service grid system is a service grid solution designed based on a separated data center. The separated data center pools various resources, and various resources are concentrated in the corresponding resource pools for uneven distribution. The resource pools are connected through a high-speed network. The service grid separates service communication and related control functions from business programs, and the network agent performs related operations.

[0037] In the existing service grid solution, since the computing nodes are not connected through a high-speed network and cannot share memory, when network transmission is performed between computing nodes, the sidecar container corresponding to the business container needs to unpack, encapsulate and forward the data packets layer by layer, which occupies a large amount of computing resources and has a slow network transmission speed. Compared with the existing service grid solution, in the service grid system of the high-speed network provided in this application, by utilizing the characteristics of the separated data center, the network agent in the gateway resource pool and the computing node can exchange data addresses through RDMA technology to access the shared memory pool or shared memory to obtain data, which can fully improve the network transmission speed and save computing resources.

[0038] like Figure 2 As shown, Figure 2 is a flowchart of a high-speed network-based data transmission method provided in an embodiment of the present application, the method is applied to Figure 1 In the high-speed network-based service grid system shown, the method includes the following steps.

[0039] Step S210: The gateway acquires the first data, wherein the gateway supports multiple fast channels including utilizing remote direct data access (RDMA) technology.

[0040] The gateway serves as the entrance and exit of the system traffic and receives the first data sent by the device outside the system.

[0041] In a possible implementation, the multiple fast channels supported by the gateway include: when data access is based on a shared memory pool, the gateway stores the data in the shared memory pool and sends the stored address to the target business container; when the data transmission volume is very small, for example, the data transmission volume is at the kilobyte (kB) level, the gateway piggybacks the data in the signaling for establishing a connection with the next-hop business container. The gateway can also support more types of transmission methods, as long as the corresponding fast channel plug-in is installed or enabled as needed, and this application does not make specific limitations on this.

[0042] Step S220: The gateway determines a first service container and a first transmission mode according to the container information and the routing rule, wherein the first transmission mode is any one of a plurality of fast channels.

[0043] After obtaining the first data, the gateway accesses the shared memory pool to obtain container information and routing rules. The gateway determines the next hop target service container according to the container information in each computing node and the transmission rule between containers in the routing rule, determines the first service container, and determines the first transmission mode according to the data transmission volume to be transmitted and the data transmission delay requirements.

[0044] The gateway determines the next-hop business container and transmission method based on the real-time updated container information and routing rules to achieve load balancing and ensure that the business container can work normally. Selecting the appropriate transmission method can also improve network transmission performance.

[0045] In a possible implementation, when the first business container is determined, the gateway establishes a network connection with the first business container. When the first transmission mode is a fast channel for data access based on a shared memory pool, the gateway stores the first data in a shared data pool in the shared memory pool. The gateway stores the first data in the shared data pool to achieve memory sharing, and multiple computing nodes can share a copy of the data, which can save storage resources.

[0046] Step S230: the gateway sends the address of the first data or the first data to the first service container through the first transmission mode.

[0047] When the first transmission mode is a different fast channel, the gateway sends the address of the first data or the first data to the first business container through the first transmission mode. The gateway and the first business container can communicate with each other without the need for the gateway or an independent proxy server to perform operations such as packaging, unpacking, and forwarding, which can improve network transmission speed and save computing resources.

[0048] In a possible implementation, when the first transmission mode is a fast channel for data access based on a shared memory pool, the gateway sends the address of the first data in the shared memory pool to the first service container through the RDMA technology.

[0049] In another possible implementation, when the first transmission mode is to piggyback data with a very small transmission volume through control plane signaling, the gateway entrains the first data in the signaling for establishing a network connection with the first service container.

[0050] In a possible implementation, when the communication between the gateway and the service container involves channel conversion between TCP / IP and RDMA, the gateway has a channel conversion component and can call a local protocol conversion program, etc., which is not specifically limited in this application.

[0051] Step S240: The first network agent obtains first data, wherein the first network agent supports multiple fast channels.

[0052] The first network proxy corresponding to the first business container intercepts the address of the first data or the first data sent by the gateway to the first business container, and accesses the shared memory pool to obtain the first data according to the address of the first data or directly obtains the first data. Compared with the prior art, the first network proxy in the present application can obtain the first data without performing operations such as packet unpacking, which greatly improves the network transmission speed and reduces the computing overhead required by the network proxy.

[0053] In a possible implementation, the multiple fast channels supported by the first network agent include: in the case of data exchange between different computing nodes, the business container accesses the memory space or shared memory space corresponding to the target business container through the network agent or the gateway in the computing node; in the case of data access based on a shared memory pool, the network agent stores the data in the shared memory pool and sends the stored address to the target business container; in the case of data sharing based on shared memory within the same computing node, the network agent caches the data in the local shared memory and sends the cache address to the target business container; when the data transmission volume is very small, for example, the data transmission volume is at the kB level, the network agent piggybacks the data in the signaling of establishing a connection with the next-hop business container.

[0054] The first network agent can also support other transmission modes, and install or enable corresponding transmission mode plug-ins as needed to realize the establishment, management and destruction of the connection mode between the next-hop business container, etc., which are not described in detail in this application. The above transmission modes do not require the network agent to perform operations such as packet unpacking, but only transmit data, and even do not need to establish data plane transmission to achieve data acquisition, which greatly improves the network transmission speed.

[0055] Step S250: the first network agent stores the first processing result generated according to the first data in a shared memory pool or in a shared memory of a computing node including the first network agent.

[0056] After the first network agent obtains the first data, the first business container generates a first processing result according to the first data, and the first network agent stores the first processing result in a shared memory or in a shared memory of a computing node including the first network agent, which is not specifically limited in this application. The first network agent can store the first processing result in a shared memory pool or in a shared memory to achieve data sharing. Through address exchange, other computing nodes, or business containers and gateways of the same computing node can obtain the first processing result through RDMA technology, which can save storage space.

[0057] Step S260: The first network agent determines whether there is a next-hop service container according to the container information and the routing rule. If there is a next-hop service container, the first network agent executes step S270; if there is no next-hop service container, the first network agent executes step S280.

[0058] Step S270: the first network agent determines the second business container and the second transmission mode according to the container information and the routing rule, and sends the address of the first processing result or the first processing result to the second business container through the second transmission mode, wherein the second transmission mode is any one of a plurality of fast channels.

[0059] When the business process is not completed, the first network agent determines the second business container and the second transmission method according to the container information and routing rules, and sends the address of the first processing result or the first processing result to the second business container through the second transmission method. Compared with the business containers of different computing nodes that cannot directly transmit data and are forwarded by the proxy server, the first network agent in this application can establish a connection with the next-hop business container through intelligent routing rules and high-speed networks, and directly transmit data, which can improve network transmission speed.

[0060] In a possible implementation, when the communication between network agents involves TCP / IP and RDMA channel conversion, the network agent has a channel conversion component and can call a local protocol conversion program, etc., which is not specifically limited in this application.

[0061] The first network agent repeats step S270 until it determines that there is no next-hop service container, and then executes step S280.

[0062] Step S280: the first network agent determines a third transmission mode according to the container information and the routing rule, and sends the address of the first processing result or the first processing result to the gateway through the third transmission mode, wherein the third transmission mode is any one of a plurality of fast channels.

[0063] In the absence of a next-hop container, the business process ends, and the first network agent determines the third transmission method based on the container information and the routing rules, and sends the address of the first processing result or the first processing result to the gateway through the third transmission method, so that the gateway obtains the first processing result and sends it to a device outside the system.

[0064] In a specific implementation, when the gateway receives the first data sent by the user device through the traditional TCP / IP protocol, the gateway determines the first service container of the next hop according to the container information stored in the shared memory pool and the transmission rules between containers in the routing rules, and determines the first transmission mode from multiple fast channels supported by the gateway according to the transmission mode selection strategy in the intelligent routing rules. Alternatively, the gateway determines the first transmission mode according to the intelligent routing configuration result sent by the user device.

[0065] When the first transmission mode is data access based on a shared memory pool, the gateway stores the first data in the shared memory pool. After determining the first service container and the first transmission mode, the gateway establishes a network connection with the first service container and sends the address of the first data to the first service container via the first transmission mode.

[0066] After receiving the first data processing request, the first network agent corresponding to the first business container accesses the shared data pool according to the address of the first data through the RDMA technology to obtain the first data. The first business container processes the obtained first data and generates a first processing result. The first network agent determines the second business container and the second transmission mode of the next hop according to the container information and intelligent routing. When the second transmission mode is also based on data access of the shared memory pool, the processed first processing result is stored in the shared data pool, and a connection is established with the second business container, and the address of the first processing result is sent to the second business container through the second transmission mode. The second network agent connected to the second business container also obtains the first processing result in the shared data pool through the RDMA technology, and stores the generated second processing result in the second local memory space.

[0067] The second network agent determines that the third transmission mode is a fast channel for data exchange between different computing nodes. The second network agent exposes the second local memory space to the third business container determined to be deployed in another computing node, so that the third network agent corresponding to the third business container accesses the second local memory space through the RDMA technology to obtain the second processing result. After the third business container processes according to the second processing result, it generates a third processing result with a data volume of KB level. Due to the small data volume, the third network agent determines a suitable fourth transmission mode according to the routing rule, and transmits the third processing result to the determined fourth business container by entraining (piggyback) in the control plane signaling, and the fourth network agent corresponding to the fourth business container sends the third processing result to the gateway by the same signaling entrainment mode, and the gateway sends the third processing result to the user device.

[0068] To sum up, the present application provides a data transmission method based on a high-speed network. In a service grid system based on a high-speed network, a gateway with high-speed network capabilities and a network agent in a computing node utilize the characteristics of a separated data center, utilize RDMA technology and intelligent routing rules, and access a shared memory pool or shared memory by exchanging data addresses, etc. to obtain data, which can improve network transmission speed and save computing resources and storage resources.

[0069] like Figure 3 As shown, Figure 3 is a schematic diagram of a computing device provided in an embodiment of the present application. The computing device can be used as a gateway or computing node in Figure 1 In the system shown, execution Figure 2 The corresponding steps in the data transmission method based on high-speed network are shown. The computing device 300 at least includes: a processor 310, a memory 320, a communication interface 330 and a bus 340. The processor, the memory and the communication interface can communicate through the bus.

[0070] The processor 310 may be composed of at least one general-purpose processor, such as a central processing unit (CPU), or a combination of a CPU and a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. The processor 310 is used to execute the corresponding steps and implement the corresponding functions. In the case where the computing device is a computing node, the computing node includes one or more processors, and the present application does not specifically limit the number of processors.

[0071] The memory 320 may be a volatile memory, such as random access memory (RAM), dynamic RAM (DRAM), static RAM (SRAM), synchronous dynamic RAM (SDRAM), double data rate RAM (DDR), cache, etc. The memory may also include a combination of the above types. The memory 320 stores the gateway or computing node for executing Figure 2 The instructions of the data transmission method based on the high-speed network can also store container information, routing rules, etc., which are not specifically limited in this application.

[0072] The communication interface 330 uses a transceiver module such as but not limited to a network interface card or a transceiver, supports RDMA and TCP / IP protocols, to achieve communication between computing nodes, gateways or shared resource pools, and between gateways and computing nodes or shared resource pools, which is not specifically limited in this application.

[0073] The bus 340 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The bus 340 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 340 may include a path for transmitting information between various components of the computing device (eg, the processor 310, the memory 320, and the communication interface 330).

[0074] Need to explain, Figure 3 This is only one possible implementation of the embodiment of the present application. In actual applications, the computing device may also include more or fewer components, which is not limited here.

[0075] like Figure 4 As shown, the embodiment of the present application also provides a schematic diagram of the structure of a computing device cluster, which includes at least one computing device 300, wherein each computing device can communicate with each other. The memory 320 in one or more computing devices 300 in the computing device cluster may store the same instructions for executing a data transmission method based on a high-speed network provided in the embodiment of the present application.

[0076] In some possible implementations, the memory 320 of one or more computing devices in the computing device cluster may also store partial instructions for executing the above-mentioned data transmission method based on a high-speed network. In other words, a combination of one or more computing devices may jointly execute instructions for executing the above-mentioned data transmission method based on a high-speed network.

[0077] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a high-speed network such as RDMA.

[0078] In a possible implementation, the computing device cluster is integrated into a computing resource pool through virtualization technology, and the cloud platform generates and allocates computing resources on demand to achieve elastic expansion or reduction of computing resources. Figure 4 As shown, the cloud platform can create a cloud server (Elastic Compute Service, ECS) or a cloud application instance on demand in a computing device, and can also create one or more cloud application instances in a cloud server, for example, creating a cloud application instance 1 in cloud server 1. Cloud server 1, cloud server 2 and cloud application instances 1 to 4 can be Figure 1The service container in the computing node, or the network agent, etc., is not specifically limited in this application.

[0079] Figure 5 1 is a schematic diagram of a structure in which one or more computing devices are connected via a network according to an embodiment of the present application. Figure 5 As shown, two computing devices 300A and 300B are connected via a network, which may be a high-speed network such as RDMA. The computing device 300A includes a processor 310A, a memory 320A, a communication interface 330A, and a bus 340A, and the computing device 300B includes a processor 310B, a memory 320B, a communication interface 330B, and a bus 340B. Specifically, one or more computing devices are connected to a network such as RDMA via a communication interface in each computing device.

[0080] In a possible implementation, both computing devices are computing nodes, such as Figure 4 The diagram includes multiple cloud servers and cloud application instances. Cloud application instance 1 in computing device 300A is the first business container, cloud application instance 2 is the first network proxy connected to the first business container, cloud application instance 3 in computing device 300B is the second business container, and cloud application instance 4 is the second network proxy connected to the second business container. Cloud application instance 2 in computing device 300A is used to determine that the next-hop business container is cloud application instance 3 in computing device 300B according to the container information in the memory and the routing rules, and to determine the transmission method for transmitting the processing result generated by cloud application instance 1 in computing device 300A. After that, the address of the processing result is sent to cloud application instance 3 in computing device 300B by using RDMA technology through the determined transmission method, which can realize direct data transmission between computing nodes and improve network transmission speed.

[0081] In a possible implementation, the computing device may also be connected to a shared memory pool integrated with the storage device through a network, store the generated data in the shared memory pool, and utilize the shared memory pool, RDMA technology, and intelligent routing to achieve data interaction between computing devices.

[0082] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes Figure 2 The data transmission method based on high-speed network is shown.

[0083] The present application also provides a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute Figure 2 The data transmission method based on high-speed network is shown.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data transmission method based on a high-speed network, applied to a service grid system based on a high-speed network, the system comprising a gateway connected via a high-speed network, a shared memory pool, and a computing node deployed with a business container and a network agent, wherein: The service container is connected to the network proxy, and the method includes: The gateway acquires first data, wherein the gateway supports multiple fast channels including using remote direct data access (RDMA) technology; The gateway determines a first service container and a first transmission mode according to the container information and the routing rule, wherein the first transmission mode is any one of the multiple fast channels; The gateway sends the address of the first data or the first data to the first service container through the first transmission mode; A first network agent corresponding to the first service container obtains the first data, wherein the first network agent supports the multiple fast channels.

2. The method according to claim 1, characterized in that The container information and the routing rule are stored in the shared memory pool, or in the shared memory of the computing node; The routing rules include transmission rules and transmission mode selection strategies between containers, wherein the transmission rules between containers include one or more of load balancing LB rules, label-based routing rules, time domain identifiers, memory locations, and container / container group binding relationship rules, which are used to determine the next-hop business container, and the transmission mode selection strategy is used to determine the rules and algorithms of the target transmission mode according to the data transmission volume or data delay requirements.

3. The method according to claim 1 or 2, characterized in that: After the first network agent corresponding to the first service container obtains the first data, the method further includes: The first network agent stores a first processing result generated according to the first data in the shared memory pool or in a shared memory of a computing node including the first network agent; The first network agent determines whether there is a next-hop service container according to the container information and the routing rule.

4. The method according to claim 3, characterized in that The first network agent determines whether there is a next-hop service container according to the container information and the routing rule, including: When the first network agent determines that there is the business container with a next hop according to the container information and the routing rule, the first network agent determines a second business container and a second transmission mode, and sends the address of the first processing result or the first processing result to the second business container through the second transmission mode, wherein the second transmission mode is any one of the multiple fast channels.

5. The method according to claim 3, characterized in that: The first network agent determines whether there is a next-hop service container according to the container information and the routing rule, and further includes: When the first network agent determines that there is no next-hop business container according to the container information and the routing rule, the first network agent determines a third transmission mode, and sends the address of the first processing result or the first processing result to the gateway through the third transmission mode, wherein the third transmission mode is any one of the multiple fast channels.

6. A service grid system based on a high-speed network, characterized in that: The system includes a gateway connected via a high-speed network, a shared memory pool, and a computing node deployed with a business container and a network agent, wherein the business container is connected to the network agent: The gateway is used to obtain the first data, wherein the gateway supports multiple fast channels including using remote direct data access RDMA technology; Determine a first service container and a first transmission mode according to the container information and the routing rule, wherein the first transmission mode is any one of the multiple fast channels; Sending the address of the first data or the first data to the first service container through the first transmission mode; The first network agent corresponding to the first service container is used to obtain the first data, wherein the first network agent supports the multiple fast channels.

7. The system according to claim 6, characterized in that The container information and the routing rule are stored in the shared memory pool, or in the shared memory of the computing node; The routing rules include transmission rules and transmission mode selection strategies between containers, wherein the transmission rules between containers include one or more of load balancing LB rules, label-based routing rules, time domain identifiers, memory locations, and container / container group binding relationship rules, which are used to determine the next-hop business container, and the transmission mode selection strategy is used to determine the rules and algorithms of the target transmission mode according to the data transmission volume or data delay requirements.

8. The system according to claim 6 or 7, characterized in that: After acquiring the first data, the first network agent is further used for: storing a first processing result generated according to the first data in the shared memory pool or in a shared memory of a computing node including the first network agent; It is determined whether there is a next-hop service container according to the container information and the routing rule.

9. The system according to claim 8, characterized in that The first network agent is specifically used for: When it is determined that the business container with a next hop exists according to the container information and the routing rule, a second business container and a second transmission mode are determined, and the address of the first processing result or the first processing result is sent to the second business container through the second transmission mode, wherein the second transmission mode is any one of the multiple fast channels.

10. The system according to claim 8, characterized in that The first network agent is specifically used for: When it is determined according to the container information and the routing rule that there is no next-hop business container, a third transmission mode is determined, and the address of the first processing result or the first processing result is sent to the gateway through the third transmission mode, wherein the third transmission mode is any one of the multiple fast channels.

11. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to claims 1 to 5.

12. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster is caused to perform the method according to claims 1 to 5.

13. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, which, when executed by a computing device cluster, perform the method according to claims 1 to 5.

Images