Communication method, system, computer device and storage medium based on data separation

Abstract

The embodiment of the application relates to the computer field, and provides a communication method and system based on data separation, a computer device and a storage medium, the method comprises the following steps: interface configuration is carried out on a converged switching device and each data processing server in a server cluster, and an IP communication control management special interface and a UDP service interface are determined; service data to be processed is acquired from a service communication link based on the converged switching device; IP communication control management data is distributed to a first data processing server through the IP communication control management special interface, and UDP service data is distributed to a second data processing server through the UDP service interface; and data processing is carried out on the distributed IP communication control management data and the UDP service data, so as to send the processed data to the service communication link. The implementation of the method improves the processing efficiency of service data, thereby improving the communication efficiency.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a communication method, system, computer device, and storage medium based on data separation. Background Technology

[0002] In the field of computer technology, User Datagram Protocol / Internet Protocol (UDP / IP) is widely used in communication scenarios that require high bandwidth, low latency, and high concurrency due to its connectionless and lightweight characteristics. Traditional socket communication utilizes operating system interrupts, and data is submitted to the application only after passing through the second and third layers of the kernel protocol stack processing logic.

[0003] To improve data processing efficiency, traditional technologies distribute data to multiple data processing servers via aggregation and switching equipment, utilizing software programs from a data plane development kit for data processing. However, because these software programs lack the network protocol stack functionality of an operating system kernel, the processing efficiency of business data is low, consequently leading to low communication efficiency. Summary of the Invention

[0004] This application provides a data separation-based communication method, system, computer device, and storage medium. More specifically, this application provides a data separation-based communication method, a data separation-based communication system, a computer device, a computer storage medium, and a computer program product, which improves the processing efficiency of business data, thereby improving communication efficiency.

[0005] In a first aspect, embodiments of this application provide a communication method based on data separation, comprising:

[0006] Configure the interfaces of each data processing server in the aggregation switching equipment and server cluster, and determine the dedicated interface for IP communication control and management and the UDP service interface; the data processing server includes a first data processing server and a second data processing server.

[0007] The convergence and switching equipment obtains the service data to be processed from the service communication link; the service data to be processed includes IP communication control and management data and UDP service data;

[0008] According to the access control list rules data pre-configured in the aggregation and switching device, the IP communication control management data is distributed to the first data processing server through the IP communication control management dedicated interface, and the UDP service data is distributed to the second data processing server through the UDP service interface;

[0009] The distributed IP communication control and management data and UDP service data are processed to send the processed data to the service communication link.

[0010] Optionally, in some embodiments of this application, the step of configuring the interfaces of each data processing server in the aggregation switching device and server cluster, and determining the dedicated IP communication control and management interface and the UDP service interface, includes:

[0011] The first data processing server is determined from the server cluster;

[0012] The target interface with preset bandwidth parameters in the first data processing server is identified as the dedicated interface for IP communication control and management.

[0013] Optionally, in some embodiments of this application, the step of configuring the interfaces of each data processing server in the aggregation switching device and server cluster, and determining the dedicated IP communication control and management interface and the UDP service interface, includes:

[0014] The servers in the server cluster other than the first data processing server are identified as the second data processing server;

[0015] The interfaces other than the dedicated IP communication control and management interface in the first data processing server, as well as the interfaces of the second data processing server, are designated as the UDP service interfaces.

[0016] Optionally, in some embodiments of this application, the method further includes:

[0017] The data type of the business data to be processed is determined based on the Ethernet layer type field and the IP layer protocol field corresponding to the data packets in the business data to be processed.

[0018] When the data type is of the first type, the business data to be processed that conforms to the first type is identified as the IP communication control and management data;

[0019] If the data type is of the second type, the business data to be processed that conforms to the second type will be identified as the UDP business data.

[0020] Optionally, in some embodiments of this application, distributing the UDP service data to the second data processing server through the UDP service interface includes:

[0021] Based on traffic characteristic data and hash algorithms, load balancing data distribution rules are determined according to the number of online second data processing servers;

[0022] Based on the load balancing data distribution rules, the UDP service data is distributed to each of the second data processing servers through the UDP service interface.

[0023] Optionally, in some embodiments of this application, the data processing of the distributed IP communication control and management data and UDP service data includes:

[0024] Based on the direct control mechanism of the operating system kernel of the first data processing server, the configuration management is performed using the protocol stack and management tools corresponding to the operating system to process the IP communication control management data.

[0025] Optionally, in some embodiments of this application, the step of processing the distributed IP communication control and management data and UDP service data to send the processed data to the service communication link includes:

[0026] The application overloads the network card driver based on the data plane development kit to receive the UDP service data;

[0027] The UDP service data packets are parsed using the protocol stack to obtain the parsed specific service content data;

[0028] The specific business content data is processed to obtain processed data. Based on the processed data, business response information is constructed and encapsulated to obtain encapsulated data.

[0029] The encapsulated data is sent to the business communication link as the processed data.

[0030] Secondly, embodiments of this application provide a data separation-based communication system with functions corresponding to the data separation-based communication method provided in the first aspect above. These functions can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and these modules can be software and / or hardware.

[0031] In one possible design, the data separation-based communication system includes:

[0032] A convergence switching device, which is used to connect to a service network via a service communication link;

[0033] A first data processing server is used to connect to the aggregation switching device via a dedicated IP communication control and management interface.

[0034] At least one second data processing server is provided, which is used to connect to the aggregation switching device via a UDP service interface.

[0035] In another aspect, this application provides a computer device including at least one connected processor and a memory, wherein the memory is used to store program code, and the processor is used to call the program code in the memory to execute the methods described in the above aspects.

[0036] In another aspect, embodiments of this application provide a computer storage medium including instructions that, when executed on a computer, cause the computer to perform the methods described in the above aspects.

[0037] In another aspect, this application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the methods described in the above aspects.

[0038] The technical solution of this application improves data processing efficiency and network connectivity stability by separating two types of data and distributing them for processing separately, simplifies operation and maintenance processes, enhances the processing capacity of high-volume communication, and improves the processing efficiency and communication efficiency of business data. Attached Figure Description

[0039] Figure 1 This is a flowchart of one embodiment;

[0040] Figure 2 This is a block diagram of a data-separated communication system in one embodiment;

[0041] Figure 3 This is a system topology diagram of a data-separated communication system in one embodiment;

[0042] Figure 4 This is a schematic diagram of the logic principle of a data separation system in one embodiment;

[0043] Figure 5 This is a schematic diagram of the UDP service data processing flow in one embodiment;

[0044] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0045] The terms "first," "second," etc., used in the embodiments of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The division of modules appearing in the embodiments of this application is only a logical division. In actual applications, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not performed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interface, and the indirect coupling or communication connection between modules may be electrical or other similar forms. None of these are limited in the embodiments of this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed among multiple circuit modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of this application.

[0046] Figure 1 This is a flowchart illustrating one embodiment, such as... Figure 1 As shown, the data separation-based communication method provided in this application embodiment is applicable to User Datagram Protocol / Internet Protocol (UDP / IP), and the method includes:

[0047] S1100 configures the interfaces of each data processing server in the aggregation switching equipment and server cluster, and determines the dedicated interface for IP communication control and management and the UDP service interface.

[0048] Among them, aggregation and switching equipment refers to independent network equipment used to access service links and perform data offloading.

[0049] A server cluster refers to a collection of multiple data processing servers used to collaboratively process business data; a server cluster includes multiple data processing servers.

[0050] The data processing server refers to a server used to process IP communication control and management data or UDP service data; the data processing server includes a first data processing server and a second data processing server; the first data processing server refers to a server that receives and processes IP communication control and management data; the second data processing server refers to a server that receives and processes UDP service data.

[0051] More specifically, the first data processing server has one server, and the second data processing server has at least one server; that is, the second data processing server can have multiple servers.

[0052] Among them, the IP communication control and management dedicated interface, also known as the dedicated interface for IP communication control and management, refers to the dedicated physical interface used to receive and transmit IP communication control and management data; the UDP service interface refers to the server interface used to receive and transmit UDP service data.

[0053] S1200 obtains service data to be processed from the service communication link based on the aggregation switching equipment.

[0054] Among them, the business communication link refers to the communication link used to transmit business data to be processed and the processed data; generally, the business communication link is used to connect the business network.

[0055] Among them, the business data to be processed refers to various types of network data that need to be processed and obtained from the business communication link; the business data to be processed includes IP communication control and management data and UDP business data.

[0056] Specifically, IP communication control and management data refers to network data used for control and management such as network addressing and connectivity detection; UDP service data refers to various service-related network data transmitted through the User Datagram Protocol.

[0057] S1300, based on the access control list rules data pre-configured in the aggregation switching device, distributes IP communication control management data to the first data processing server through the dedicated IP communication control management interface, and distributes UDP service data to the second data processing server through the UDP service interface.

[0058] The access control list rule data, also known as ACL rule data, refers to the preset rule data used to distinguish and separate the two types of data.

[0059] S1400 processes the distributed IP communication control and management data and UDP service data to send the processed data to the service communication link.

[0060] Data processing refers to the operations of parsing and processing IP communication control and management data and UDP service data.

[0061] Compared to traditional technologies, in this embodiment, the interfaces of each data processing server in the aggregation switching device and server cluster are first configured to determine the dedicated IP communication control and management interface and the UDP service interface. Then, the service data to be processed is obtained from the service communication link based on the aggregation switching device. According to the access control list rules data pre-configured in the aggregation switching device, the IP communication control and management data is distributed to the first data processing server through the dedicated IP communication control and management interface, and the UDP service data is distributed to the second data processing server through the UDP service interface. Finally, the distributed IP communication control and management data and UDP service data are processed to send the processed data to the service communication link. The technical solution of this embodiment improves data processing efficiency and network connectivity stability by separating the two types of data and distributing them separately, simplifies the operation and maintenance process, and enhances the processing capacity of high-volume communication, thereby improving the processing efficiency and communication efficiency of service data.

[0062] Figure 2 This is a structural block diagram of a data separation-based communication system in one embodiment. This application provides a data separation-based communication system for implementing the data separation-based communication method in any embodiment of this application. The data separation-based communication system includes: a convergence switching device 210, a first data processing server 220, and at least one second data processing server 230; wherein, the convergence switching device is used to connect to a service network 240 via a service communication link; the first data processing server 220 is used to connect to the convergence switching device 210 via an IP communication control and management dedicated interface; and the second data processing server 230 is used to connect to the convergence switching device 210 via a UDP service interface.

[0063] More specifically, Figure 3 This is a system topology diagram of a data-separated communication system in one embodiment. The data-separated communication system of this application proposes an IP communication control and management / UDP service data separation system architecture, the system topology diagram of which is shown below. Figure 3 .

[0064] exist Figure 3 In this context, data processing server 1 refers to the first data processing server, and data processing servers 2 to n refer to the second data processing servers.

[0065] based on Figure 3 The system topology in this embodiment, and the data processing and communication process of the data separation-based communication system are as follows.

[0066] (1) Access all business communication links that need to be performed through aggregation and switching equipment.

[0067] (2) By aggregating the access control list rule data of the switching equipment, the IP communication control management data (such as ARP, ICMP, LACP, etc.) is distributed to the dedicated IP communication control management interface of the first data processing server, and the UDP service data (such as RTP stream, GTP tunnel data, etc.) is distributed in a load-balanced manner to the UDP service interface of the second data processing server (multiple data processing servers).

[0068] Here, ARP stands for Address Resolution Protocol; ICMP stands for Internet Control Message Protocol; LACP stands for Link Aggregation Control Protocol; RTP stands for Real-time Transport Protocol; and GTP stands for GPRS Tunneling Protocol.

[0069] (3) The dedicated IP communication control and management interface corresponding to the first data processing server sets relevant network parameters according to the actual network connection situation. The data is transmitted through the aggregation and switching equipment to realize functions such as network communication addressing, IP bidirectional connectivity detection, and routing optimization, so as to ensure normal communication between UDP service IP and relevant external service IP.

[0070] (4) When an external business IP needs to communicate with the data processing server via UDP, it sends data directly to the UDP business communication IP of the data processing server. The data enters the aggregation switching device via the relevant network. The aggregation switching device distributes the UDP business data to the UDP business interfaces of multiple data processing servers of the second data processing server in a load-balanced manner through the access control list rules.

[0071] (5) Multiple data processing servers of the second data processing server work in parallel. The network card driver of the UDP service interface is overloaded using DPDK to realize the processing functions such as receiving large-volume UDP service data, parsing UDP / IP protocol stack, service processing, UDP / IP protocol stack encapsulation, and forwarding.

[0072] Optionally, in some embodiments of this application, interface configuration is performed on each data processing server in the aggregation switching device and server cluster to determine the dedicated interface for IP communication control and management and the UDP service interface, including: determining a first data processing server from the server cluster; and determining the target interface with preset bandwidth parameters in the first data processing server as the dedicated interface for IP communication control and management.

[0073] Interface configuration refers to the operation of setting the interfaces of the aggregation switching equipment and data processing server.

[0074] The preset bandwidth parameter refers to the pre-set interface bandwidth standard used to filter target interfaces; for example, the preset bandwidth parameter is 1G / 10G; in this case, the target interface refers to the interface in the first data processing server that meets the preset bandwidth parameter; for example, the 1G / 10G network interface.

[0075] In this embodiment, by selecting an interface that meets the preset bandwidth as a dedicated control interface, the stability of IP communication control is improved, the interface configuration process is simplified, and the standardization of system deployment is enhanced, thus adapting to the control data transmission requirements.

[0076] Optionally, in some embodiments of this application, interface configuration is performed on each data processing server in the aggregation switching device and server cluster to determine the IP communication control and management dedicated interface and UDP service interface, including: determining other servers in the server cluster other than the first data processing server as the second data processing server; determining other interfaces in the first data processing server other than the IP communication control and management dedicated interface, and each interface of the second data processing server as UDP service interfaces.

[0077] Among them, other servers refer to servers in the server cluster other than the first data processing server.

[0078] Among them, "other interfaces" refers to the interfaces in the first data processing server other than the dedicated interface for IP communication control and management; "each interface of the second data processing server" refers to all interfaces on the second data processing server that can be used for data transmission.

[0079] In this embodiment, by clearly defining the second data processing server and the UDP service interface, the separation of service and control interfaces is achieved, which improves the standardization of data diversion and enhances the scalability and flexibility of UDP service processing.

[0080] For example, as can be seen from the two embodiments above, the process of configuring the interfaces of each data processing server in the aggregation switching device and server cluster, and determining the dedicated IP communication control and management interface and the UDP service interface, is as follows:

[0081] Because the amount of messages for IP communication control and management is extremely small, 1G or 10G ports can be used based on the existing hardware configuration of the servers. Specifically, one of the data processing servers is designated as the first data processing server, and one of its 1G / 10G network interfaces is used as a dedicated interface for IP communication control and management. The other 10G or 100G network interfaces of the first data processing server are used as interfaces for UDP service data, and the two are physically isolated. The other servers besides the first data processing server are designated as the second data processing servers, and all of their interfaces are used as interfaces for UDP service data.

[0082] More specifically, the first data processing server can be selected based on the hardware configuration. The configuration process is as follows: select the server with the highest hardware configuration in the server cluster as the first data processing server, and install a 1G / 10G network card on the first data processing server as a dedicated network interface for IP communication control and management.

[0083] Optionally, in some embodiments of this application, the method further includes: determining the data type of the service data to be processed based on the Ethernet layer type field and the IP layer protocol field corresponding to the data packet in the service data to be processed; if the data type is a first type, determining the service data to be processed that conforms to the first type as IP communication control and management data; if the data type is a second type, determining the service data to be processed that conforms to the second type as UDP service data.

[0084] Among them, data packets refer to data packets carrying service and control information in network transmission; the type field of the Ethernet layer refers to the field in the Ethernet frame header used to identify the upper layer protocol type; and the protocol field of the IP layer refers to the field in the IP packet header used to identify the transport layer protocol.

[0085] Among them, data type refers to the category used to distinguish the purpose of data, which is divided according to protocol fields; the first type refers to the data type corresponding to IP communication control and management data; the second type refers to the data type corresponding to UDP service data.

[0086] For example, the specific configuration logic of the ACL access control list rule data used in this embodiment is as follows: by using the Ethernet layer type field and the IP layer protocol field of the data packet, it is identified whether the packet belongs to IP communication control management data or UDP service data.

[0087] In this embodiment, data types are determined by Ethernet and IP layer fields, and access control lists are used to accurately identify and distribute data, thereby improving the accuracy of data classification, enhancing the regularity of data processing, and increasing the reliability and efficiency of system data distribution.

[0088] Optionally, in some embodiments of this application, distributing UDP service data to a second data processing server through a UDP service interface includes: determining a load-balanced data distribution rule based on traffic characteristic data and a hash algorithm, according to the number of online second data processing servers; and distributing the UDP service data to each second data processing server through the UDP service interface based on the load-balanced data distribution rule.

[0089] Among them, traffic characteristic data refers to the transmission characteristic information such as address and port carried in the packet used to distinguish data streams; hash algorithm refers to the calculation method based on traffic characteristics to achieve data diversion.

[0090] Among them, the online second data processing server refers to the server that is currently operating normally and can receive and process UDP business data.

[0091] Among them, the data distribution rules for load balancing refer to the basis for traffic distribution to be implemented in a balanced manner to avoid node overload.

[0092] For example, distributing UDP service data to the second data processing server through the UDP service interface specifically includes: relying on the aggregation and switching equipment for load balancing; more specifically, based on traffic characteristics (such as source IP, destination IP, and port), using a hash algorithm, and according to the number of online data processing servers, dynamic load balancing is achieved while ensuring that the data originates from the same source and ends in the same location, that is, a data guarantee mechanism is implemented to consistently distribute data streams from the same source to the same server.

[0093] In this embodiment, distribution rules are generated by combining traffic characteristics with hash algorithms to achieve dynamic load balancing and same-source / same-destination, thereby improving data distribution efficiency and cluster operation stability, and enhancing the performance of handling large volumes of traffic.

[0094] Figure 4 This is a schematic diagram illustrating the logical principle of a data separation system in one embodiment. In one embodiment, for the process of "processing the distributed IP communication control and management data and UDP service data" in step S1400, this application designs a data separation system logic, namely, an IP communication control and management / UDP service data separation system logic, see [link to documentation]. Figure 4 The specific details of the data separation system logic are as follows.

[0095] exist Figure 4On the left, UDP service data is directly written to user-space ring buffers by the network card driven by the DPDK overload using Direct Memory Access (DMA) technology. The DPDK application can directly read and process the data packets at high speed, relying solely on the kernel-level UIO Driver (user-space I / O driver) for hardware access permissions, without kernel protocol stack intervention, thus achieving zero-copy high-speed throughput. Figure 4 On the right, IP communication control and management data are directly entered into the kernel mode by the operating system and processed by the system's native drivers and protocol stack. Users can also directly configure the control plane network interface through tools such as ifconfig and ethtool, which is completely independent of the running state of the DPDK application, ultimately achieving high-speed service processing.

[0096] User mode refers to the restricted execution mode in which applications run. In this mode, applications cannot directly access hardware devices, but can only access limited memory regions and use a limited set of CPU instructions. Kernel mode refers to the privileged mode in which the operating system kernel runs. In this mode, applications can directly manipulate hardware resources, access the entire memory space, and execute all CPU instructions.

[0097] Optionally, in some embodiments of this application, data processing is performed on the distributed IP communication control and management data and UDP service data, including: using the direct control mechanism of the operating system kernel of the first data processing server, and using the corresponding protocol stack and management tools of the operating system for configuration management, so as to process the IP communication control and management data.

[0098] The operating system kernel refers to the system program in the server that manages hardware and resources and performs core scheduling; the direct control mechanism refers to the management method in which the kernel directly takes over the dedicated interface without going through the application layer.

[0099] The protocol stack refers to the multi-layered network protocol set built into the operating system, used to process communication control messages; the management tools refer to the tool programs provided by the system for configuring network parameters and maintaining status.

[0100] Configuration management refers to the operations of setting and maintaining network parameters such as interfaces and routes.

[0101] For example, during the processing of IP communication control and management data, the dedicated IP communication control and management interface is directly controlled by the operating system kernel. It utilizes the operating system's rich protocol stack and flexible management tools for configuration and management. For instance, `ifconfig` can be used to query the status of the dedicated IP communication control and management interface and configure IP addresses, while `ip` commands can be used to configure routes and bridges. Link control messages from IP communication control and management, such as ARP, ICMP, and LACP, enter the dedicated IP communication control and management interface via aggregation switching devices and are directly processed by the operating system's kernel protocol stack. This enables functions such as network communication addressing, IP bidirectional connectivity detection, and route optimization, ensuring normal communication between UDP service IPs and relevant external service IPs.

[0102] Therefore, this embodiment realizes that IP communication is completely controlled by the operating system and isolated from the DPDK application, thereby reducing the complexity of the DPDK application and the workload of operation and maintenance, while significantly improving the stability and processing capacity of the system.

[0103] In this embodiment, the control data is directly managed and processed through the operating system kernel, which improves the stability of communication control and enhances the reliability and efficiency of network configuration.

[0104] Optionally, in some embodiments of this application, data processing is performed on the distributed IP communication control and management data and UDP service data to send the processed data to the service communication link, including: reloading the network card driver based on the data plane development kit application to receive UDP service data; performing protocol stack parsing on the data packets of the UDP service data to obtain the parsed specific service content data; performing service processing on the specific service content data to obtain service-processed data; constructing service response information based on the service-processed data; encapsulating the service response information to obtain encapsulated data; and sending the encapsulated data as processed data to the service communication link.

[0105] Among them, the Data Plane Development Kit (DPDK) application refers to the application that achieves high-speed network data processing by relying on the high-performance development kit; the overloaded network card driver refers to replacing the original network card driver with a dedicated driver to adapt to the operation of high-speed data processing; and the specific business content data refers to the original business transmission data obtained after stripping the protocol header.

[0106] Here, "post-processing data" refers to the intermediate result data obtained after processing by business logic; "business response information" refers to the feedback content information generated in response to business requests.

[0107] Here, encapsulated data refers to network packets that have completed protocol header encapsulation and can be sent directly.

[0108] For example, during the data processing of UDP service data, the application of the data plane development kit based on the data processing server is overloaded with the network card driver. The UDP service data packets are directly stored in memory through DMA technology and transmitted to the application layer software in a zero-copy manner. The DPDK development interface is used to perform high-speed data reception, UDP / IP protocol stack parsing, service processing, UDP / IP protocol stack encapsulation, forwarding, etc.

[0109] Because of the connectionless and lightweight nature of the UDP / IP communication protocol, DPDK applications can easily implement the parsing and encapsulation of the UDP / IP protocol stack. Through aggregation and switching devices, high-volume UDP service data can be evenly distributed to multiple data processing servers, which can then process it in parallel, thereby achieving high-volume, high-speed UDP service throughput.

[0110] Figure 5 This is a schematic diagram illustrating the processing flow of UDP service data in one embodiment. For example... Figure 5 The specific data processing flow for UDP business data is as follows:

[0111] (1) After the DPDK application starts up, it begins to receive data.

[0112] (2) Parse the data packets using the Ethernet layer, IP, and UDP protocol stack.

[0113] (3) Parse out the specific business content of the data packet.

[0114] For example, the 5G communication N3 interface uses the GTP tunneling protocol based on UDP:

[0115] At the Ethernet layer: Record the MAC addresses of the adjacent upstream device and the local device. When forwarding data, the DPDK application needs to re-encapsulate the source MAC and destination MAC. At the IP layer: Record the source IP and destination IP of the data. When forwarding data, the DPDK application needs to re-encapsulate the source IP and destination IP and calculate the new checksum at the IP layer. At the UDP layer: Record the source port and destination port of the data. When forwarding data, the DPDK application needs to re-encapsulate the source port and destination port and calculate the new checksum at the UDP layer.

[0116] (4) Perform business processing.

[0117] For example, operators provide different quality of service guarantees for voice and regular internet traffic based on GTP tunnel protocol data transmitted using UDP; and look up the corresponding IP address based on the domain name in the DNS request message.

[0118] (5) After the business processing is completed, the business response information is constructed and encapsulated in UDP, IP and Ethernet layer protocol stacks.

[0119] (6) Send out the packaged data.

[0120] The forwarding path is as follows: the sending module in the user-mode DPDK application writes the encapsulated data to the UDP service data network interface controller (10G / 100G network card), and then returns to the service communication link through the aggregation switching device.

[0121] In this embodiment, a dedicated application is used to overload the driver to process business data, complete the parsing, processing, encapsulation and transmission, improve the processing efficiency of large-volume data, improve the transmission response effect, and enhance the system's high-speed communication capability.

[0122] The technical research process and other technical details of this application are described below with reference to a specific embodiment.

[0123] In traditional technologies, the UDP / IP communication protocol is widely used in communication scenarios that require high bandwidth, low latency, and high concurrency due to its connectionless and lightweight characteristics. For example, DNS queries are based on UDP to quickly complete domain name resolution; video conferencing and live streaming rely on UDP to transmit audio and video; and core network elements of 4G / 5G communication networks use UDP for data transmission.

[0124] For UDP communication services with processing capabilities of 10Gbps, 100Gbps or higher, traditional socket communication relies on operating system interrupts. Data is processed through the second and third layers of the kernel protocol stack (OSI seven-layer model, such as IP, TCP / UDP parsing) before being submitted to the application. This results in drawbacks such as numerous memory copy operations, high performance consumption, and extended processing time, which obviously cannot meet the requirements of high-volume, high-speed processing.

[0125] For high-speed communication with multiple links and high traffic, the common approach is to connect multiple links through aggregation and switching equipment, aggregate the data, and then distribute it to multiple data processing servers in a load-balanced manner with the same source and destination. The data processing servers are connected to the aggregation and switching equipment through one or more 10G or 100G network interfaces.

[0126] The data processing server utilizes the Data Plane Development Kit (DPDK) to overload the network card driver, directly storing data packets into memory via DMA technology and transferring them to the application layer software in a zero-copy manner. High-speed data processing is then performed using the DPDK interface. However, since DPDK itself lacks the network protocol stack functionality of the Linux operating system kernel, and the development of related protocol stacks is extremely complex, it is neither economical nor practical for applications to develop such functionality themselves.

[0127] To address the network communication addressing and bidirectional IP connectivity functions of DPDK applications, the KNI module needs to be introduced to allow user-mode DPDK applications to interact with the Linux kernel, thereby enabling communication between UDP service IPs and relevant external service IPs.

[0128] In this architecture, the KNI module of the DPDK application is specifically responsible for processing IP communication control and management data packets such as Address Resolution Protocol (ARP) and Internet Control Message Protocol (ICMP), realizing network communication addressing, IP bidirectional connectivity detection, and route optimization. The service processing module of the DPDK application uses zero-copy technology to process UDP service data, including receiving, UDP / IP protocol stack parsing, service processing, UDP / IP protocol stack encapsulation, and forwarding, thereby achieving the coordinated operation of high-volume, high-efficiency data processing and general network functions.

[0129] However, the existing processing method that combines DPDK and KNI has the following drawbacks:

[0130] (1) In order to enable the DPDK application to communicate with the outside world via IP, a kernel virtual network interface managed by DPDK needs to be created. The KNI needs to be initialized in the application and a series of parameter configurations need to be performed.

[0131] (2) Only after the DPDK application is running normally can the user see the virtual network interface (vEth) at the operating system level, and can set IP, routing and other information through user-space network tools (such as ifconfig, ethtool, ip command, etc.), and then perform IP communication control management data processing of address resolution protocol and Internet control message protocol such as network communication addressing, bidirectional connectivity detection, and route optimization.

[0132] (3) If the DPDK application exits, the virtual network interface it created will also be destroyed, and the corresponding IP and routing information will be lost at the same time, causing the outside world to lose contact with the DPDK business IP. Therefore, using this method to communicate with the outside world will be extremely unstable and will easily trigger alarms in sensitive network systems.

[0133] (4) When the application restarts, the corresponding IP, routing and other information need to be reset, which will lead to low work efficiency and increased maintenance difficulty.

[0134] (5) When the application processes data through DPDK, it needs to identify IP communication control management data or UDP service data: if it is IP communication control management data, it needs to be sent to the kernel through the memory queue between user mode and kernel mode; if it is service data, it is distributed to the working core for UDP service data processing.

[0135] (6) In order to send out the relevant kernel responses in a timely manner, the DPDK application needs to continuously poll the KNI memory queue, read the data, and forward it immediately. The different processing of IP communication control and management data and UDP service data not only affects the continuity of data processing, but also increases the complexity of the program and reduces the processing performance of the program.

[0136] (7) In complex network connection scenarios, such as TRUNK, the creation and management of multiple virtual network interfaces will lead to an bloated application architecture, complex configuration, and may even fail to meet business needs.

[0137] (8) Existing technology requires configuring traffic splitting rules according to the dedicated UDP service IP of each server, and the traffic splitting rules are strongly bound to the IP. Because the communication between the UDP service IP and the external IP is controlled by the DPDK application, when using multiple data processing servers, each data processing server must be independently allocated a UDP service IP and the relevant network parameters must be configured. The traffic splitting rules of the aggregation switching device must also ensure that data packets of different UDP service IPs are accurately allocated to the corresponding data processing servers. This solution not only requires the use of multiple IP resources, but also has a large operation and maintenance difficulty and workload.

[0138] Based on this, this application provides a communication method based on data separation, which can also be called a method for high-volume, high-speed UDP protocol communication. Details are as follows.

[0139] The data separation-based communication method of this application adopts the idea of ​​separating IP communication control and management from UDP service data processing. One 1G / 10G network interface of one of the data processing servers is used as a dedicated interface for IP communication control and management, while other 10G or 100G network interfaces are used as interfaces for UDP service data, and the two are physically isolated.

[0140] In the data separation-based communication method of this application, the DPDK application only overloads the network interface driver for UDP service data, while the dedicated network interface for IP communication control and management is still controlled by the operating system kernel.

[0141] Specifically, in terms of configuration, under normal circumstances, if a DPDK application needs to process data from a network interface card, it needs to replace the corresponding network interface driver with a user-mode driver (such as igb_uo or vfio-pci). After the replacement, the original kernel-mode driver of the operating system will no longer work, and the corresponding network interface card control commands will also become ineffective, and the DPDK application will take over completely.

[0142] This application replaces the driver for the network interface of UDP service data with a user-space driver, which is completely taken over by the DPDK application; the driver for the dedicated network interface for IP communication control and management remains unchanged and is still controlled by the operating system. The related IP communication functions are handled by the operating system kernel, and the application does not need to care about this part of the function, thereby reducing the burden on the application and reducing the complexity of the program.

[0143] The data separation-based communication method of this application sets ACL rules through the aggregation switching device to divert IP communication control and management data (such as ICMP, ARP data, etc.) to dedicated interfaces, while UDP service data is diverted to the UDP service data interfaces of multiple data processing servers.

[0144] Specifically, in terms of configuration, the aggregation switching device has multiple ports, enabling it to connect to external network devices and servers. It comes with its own management port and management software. By writing traffic splitting scripts according to the device's syntax rules, different types of data can be distributed to different ports. This means that data coming from the external network can be distributed to the server according to different rules, and data processed by the server can also be forwarded to the external network. After configuration, the aggregation switching device can perform the following processes:

[0145] (1) Based on the Ethernet type and protocol type, the IP communication control and management data (including ARP, LLDP, ICMP, ICMPv6, etc.) are distributed to the dedicated IP communication control and management interface;

[0146] (2) Forward the response data of the operating system to the data packets of IP communication control and management data from the original port;

[0147] (3) Distribute the UDP protocol data load evenly to the UDP service interfaces of multiple data processing servers corresponding to the second data processing server according to the protocol type.

[0148] (4) Forward the UDP service data processed by the data server out of the original port.

[0149] In addition, because the dedicated interface for IP communication control and management is controlled by the operating system, the UDP service IP and routing information will not change with the running state of the DPDK application. At the same time, the rich network protocol stack and network management tools of the operating system can be used to easily configure various network parameters, manage various network connection methods, and realize network communication addressing, IP bidirectional connectivity detection, route optimization, etc.

[0150] Furthermore, regarding the selection of the primary data processing server: to standardize deployment and facilitate operation and maintenance, the following mechanism can be used for selection: the server with the smallest sequence number among online data processing servers 1 to N is designated as the primary data processing server and undertakes IP communication control and management functions. For example, if data processing server 1 is functioning normally, it will undertake this function; otherwise, data processing server 2 will take over, and so on.

[0151] This application can strip DPDK applications of complex network management functions, allowing them to focus on UDP business data processing, reducing application complexity and improving system stability and processing capabilities. When the entire system uses multiple data processing servers for parallel processing, only the dedicated IP communication control and management interface of one data processing server needs to be configured and managed. Therefore, this invention significantly improves system operation and maintenance efficiency and saves IP resources.

[0152] The advantages of the data separation-based communication method in this application are:

[0153] (1) Use aggregation and switching equipment to achieve centralized access of multiple links, and distinguish IP communication control and management data and UDP service data by means of information such as protocol type and port.

[0154] (2) Select one of the multiple data processing servers to configure a dedicated interface for IP communication control and management; other data processing servers do not need to be configured.

[0155] (3) The aggregation and switching equipment distributes IP communication control and management data to the dedicated IP communication control and management interface of the data processing server. The communication control between UDP service IP and external service IP is controlled by the operating system of the server.

[0156] (4) The aggregation and switching equipment distributes UDP service data load evenly to multiple data processing servers.

[0157] (5) Separation of IP communication control and management data and UDP service data is achieved through aggregation and switching equipment; the data processing server achieves physical isolation of the two types of data through physical ports.

[0158] (6) IP communication control and management data processing is separated from the DPDK application and is directly processed by the operating system of the data processing service, ensuring stability, flexibility and efficiency.

[0159] (7) UDP service data can be processed in parallel by multiple data processing servers to achieve high-speed processing of large traffic of UDP service data.

[0160] (8) The DPDK application of the data processing server implements UDP / IP protocol stack parsing and encapsulation.

[0161] The advantages of the data separation-based communication method in this application are as follows: It adopts the concept of separating IP communication control and management from UDP service data processing, achieving physical isolation between the two types of data through physical ports; the IP communication control and management data is directly processed by the operating system, making it more stable, flexible, and efficient; the DPDK application is stripped of the relevant functions of IP communication control and management, focusing only on UDP service data processing, reducing the complexity and development difficulty of the DPDK application, and improving the system's stability and processing capabilities; when using multiple data processing servers for parallel processing, only one data processing server needs to be configured with a dedicated IP communication control and management interface, reducing maintenance difficulty and workload.

[0162] It should be noted that any technical feature in any of the above embodiments provided in this application is also applicable to any of the following embodiments provided in this application, and similar details will not be repeated hereafter.

[0163] In another embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, it includes a processor, memory, input / output interfaces, and a communication interface. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface is connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores relevant data. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. The computer program can be executed by the processor to implement the various methods described in the above embodiments.

[0164] Those skilled in the art will understand that Figure 6 The structure shown is only a block diagram of a part of the structure related to the present application and does not constitute a limitation on the computer device on which the present application is applied. It may also include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements, in order to realize the function of the computer device.

[0165] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0166] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the systems, devices, equipment, modules or units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0167] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, devices, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0168] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0169] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0170] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0171] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium, or a semiconductor medium (e.g., a solid-state drive), etc.

[0172] The technical solutions provided by the embodiments of this application have been described in detail above. Specific examples have been used in the embodiments of this application to illustrate the principles and implementation methods of the embodiments of this application. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.

Claims

1. A communication method based on data separation, characterized in that, The method includes: Configure the interfaces of each data processing server in the aggregation switching equipment and server cluster, and determine the dedicated interface for IP communication control and management and the UDP service interface; the data processing server includes a first data processing server and a second data processing server. The convergence and switching equipment obtains the service data to be processed from the service communication link; the service data to be processed includes IP communication control and management data and UDP service data; According to the access control list rules data pre-configured in the aggregation and switching device, the IP communication control management data is distributed to the first data processing server through the IP communication control management dedicated interface, and the UDP service data is distributed to the second data processing server through the UDP service interface; The distributed IP communication control and management data and UDP service data are processed to send the processed data to the service communication link.

2. The method according to claim 1, characterized in that, The interface configuration for each data processing server in the aggregation switching equipment and server cluster, including determining the dedicated IP communication control and management interface and the UDP service interface, includes: The first data processing server is determined from the server cluster; The target interface with preset bandwidth parameters in the first data processing server is identified as the dedicated interface for IP communication control and management.

3. The method according to claim 2, characterized in that, The interface configuration for each data processing server in the aggregation switching equipment and server cluster, including determining the dedicated IP communication control and management interface and the UDP service interface, includes: The servers in the server cluster other than the first data processing server are identified as the second data processing server; The interfaces other than the dedicated IP communication control and management interface in the first data processing server, as well as the interfaces of the second data processing server, are designated as the UDP service interfaces.

4. The method according to claim 1, characterized in that, The method further includes: The data type of the business data to be processed is determined based on the Ethernet layer type field and the IP layer protocol field corresponding to the data packets in the business data to be processed. When the data type is of the first type, the business data to be processed that conforms to the first type is identified as the IP communication control and management data; If the data type is of the second type, the business data to be processed that conforms to the second type will be identified as the UDP business data.

5. The method according to claim 1, characterized in that, The step of distributing the UDP service data to the second data processing server through the UDP service interface includes: Based on traffic characteristic data and hash algorithms, load balancing data distribution rules are determined according to the number of online second data processing servers; Based on the load balancing data distribution rules, the UDP service data is distributed to each of the second data processing servers through the UDP service interface.

6. The method according to claim 1, characterized in that, The data processing of the distributed IP communication control and management data and UDP service data includes: Based on the direct control mechanism of the operating system kernel of the first data processing server, the configuration management is performed using the protocol stack and management tools corresponding to the operating system to process the IP communication control management data.

7. The method according to claim 1, characterized in that, The step of processing the distributed IP communication control and management data and UDP service data to send the processed data to the service communication link includes: The application overloads the network card driver based on the data plane development kit to receive the UDP service data; The UDP service data packets are parsed using the protocol stack to obtain the parsed specific service content data; The specific business content data is processed to obtain processed data. Based on the processed data, business response information is constructed and encapsulated to obtain encapsulated data. The encapsulated data is sent to the business communication link as the processed data.

8. A communication system based on data separation, characterized in that, The data separation-based communication system is used to implement the data separation-based communication method as described in any one of claims 1 to 7, wherein the data separation-based communication system comprises: A convergence switching device, which is used to connect to a service network via a service communication link; A first data processing server is used to connect to the aggregation switching device via a dedicated IP communication control and management interface. At least one second data processing server is provided, which is used to connect to the aggregation switching device via a UDP service interface.

9. A computer device, characterized in that, The computer device includes: At least one processor and memory; The memory is used to store program code, and the processor is used to call the program code stored in the memory to execute the method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, It includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 7.

Images