Load balancing method, data processing method, data returning method and electronic equipment

Abstract

The invention provides a load balancing method, a data processing method, a data returning method and electronic equipment, and belongs to the technical field of computers. The method comprises the following steps: determining a first processor core, a second processor core and a target data stream based on load information, session information and packet receiving information of a plurality of processor cores at the current moment; modifying a field value of a redirection field in target session information of the target data stream on the first processor core to obtain first session information of the first processor core; adding a first stream forwarding rule in the first processor core; and when determining that the first data packet to be processed at the next moment belongs to the target data stream based on at least one of the first session information and the first stream forwarding rule, migrating the first data packet from the first processor core to the second processor core. According to the method, the data flow on the processor core with high load is migrated to the processor core with low load for processing, so that the performance of the processor is improved, and a packet loss phenomenon is avoided.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a load balancing method, a data processing method, a data migration method and an electronic device. Background Art

[0002] With the development of computer technology, the processors of traffic gateways in data centers are gradually developing towards multi-core. In order to improve the network processing capability and throughput of traffic gateways, the traffic between multiple processor cores needs to be load balanced.

[0003] In the related technology, based on the hardware RSS (Receive Side Scaling) rules and hardware FDIR (Flow Director) rules of the network card in the traffic gateway, the data flow passing through the traffic gateway is evenly distributed to multiple processor cores, and then the received data packets are sent to the processor core corresponding to the data flow for processing.

[0004] However, different data streams carry different services, and the traffic size of different data streams changes dynamically with the change in the number of user visits to the corresponding services. The relevant technology only achieves uniform distribution of traffic at the quantitative level, which is not true load balancing. When the number of user visits to a certain service increases, the traffic of the data stream corresponding to the service increases, and the load of the processor core corresponding to the data stream increases, resulting in a decrease in processor performance and even packet loss. Summary of the invention

[0005] The embodiments of the present application provide a load balancing method, a data processing method, a data migration method and an electronic device, which can improve the performance of the processor and avoid packet loss. The technical solution is as follows:

[0006] In a first aspect, a load balancing method is provided, the method comprising:

[0007] Based on the load information, session information and packet reception information of the multiple processor cores at the current moment, determining the first processor core, the second processor core and the target data flow, wherein the first processor core is a processor core whose current load exceeds a load threshold, the second processor core is a processor core whose current load does not exceed the load threshold, and the target data flow is a data flow whose number of data packets received on the first processor core at the current moment exceeds the number threshold;

[0008] Modify a field value of a redirection field in target session information of the target data flow on the first processor core to obtain first session information of the first processor core, where the first session information is used to instruct the first processor core to migrate a data packet belonging to the target data flow being processed to the second processor core;

[0009] Adding a first flow forwarding rule in the first processor core, wherein the first flow forwarding rule represents a correspondence between a target hash value of the target data flow and a core identifier of the second processor core, and is used to instruct the first processor core to migrate received data packets belonging to the target data flow to the second processor core, wherein the target hash value is calculated based on the five-tuple information of the target data flow;

[0010] When it is determined based on at least one of the first session information and the first flow forwarding rule that a first data packet to be processed at the next moment belongs to the target data flow, the first data packet is migrated from the first processor core to the second processor core.

[0011] In a second aspect, a data processing method is provided. The method is based on the load balancing method in the first aspect, and the method includes:

[0012] Creating the same session information on the second processor core according to the target session information of the target data flow on the first processor core;

[0013] Modify a field value of a redirection field in target session information of the target data flow on the second processor core to obtain second session information of the second processor core, where the second session information is used to instruct the second processor core to process a received data packet belonging to the target data flow;

[0014] When the first data packet migrated from the first processor core is received, the first data packet is processed based on the second processor core and the second session information.

[0015] In a third aspect, a data migration method is provided. The method is based on the data processing method in the second aspect, and the method includes:

[0016] When it is detected that the target data flow meets the relocation condition, modify the field value of the redirection field in the second session information to obtain the third session information of the second processor core, wherein the third session information is used to instruct the second processor core to relocate the data packets belonging to the target data flow to the first processor core;

[0017] Adding a second flow forwarding rule in the second processor core, wherein the second flow forwarding rule represents a correspondence between the target hash value and the core identifier of the first processor core, and is used to instruct the second processor core to migrate a data packet belonging to the target data flow to the first processor core;

[0018] When it is determined based on at least one of the third session information and the second flow forwarding rule that the third data packet to be processed by the second processor core at the next moment belongs to the target data flow, the third data packet is migrated back from the second processor core to the first processor core.

[0019] In a fourth aspect, a data processing method is provided. The method is based on the data migration method in the third aspect, and the method includes:

[0020] When it is detected that the target data stream meets the relocation condition, synchronizing the state of the target data stream in the first session information based on the current state of the target data stream indicated by the second session information;

[0021] Modify a field value of a redirection field in the first session information of state synchronization to obtain fourth session information of the first processor core, wherein the fourth session information is used to instruct the first processor core to process a received data packet belonging to the target data flow;

[0022] Deleting the first flow forwarding rule on the first processor core;

[0023] When a third data packet returned from the second processor core is received, the third data packet is processed based on the first processor core and the fourth session information.

[0024] In a fifth aspect, an electronic device is provided, comprising a processor and a memory; the memory stores at least one program code; the at least one program code is used to be called and executed by the processor to implement the load balancing method described in the first aspect, or the data processing method described in the second aspect, or the data migration method described in the third aspect, or the data processing method described in the fourth aspect.

[0025] In the sixth aspect, a computer-readable storage medium is provided, in which at least one computer program is stored. When the at least one computer program is executed by a processor, it can implement the load balancing method described in the first aspect, or the data processing method described in the second aspect, or the data migration method described in the third aspect, or the data processing method described in the fourth aspect.

[0026] In the seventh aspect, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, it can implement the load balancing method described in the first aspect, or the data processing method described in the second aspect, or the data migration method described in the third aspect, or the data processing method described in the fourth aspect.

[0027] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0028] In the process of data processing based on a multi-core processor, the load conditions of multiple processor cores are monitored. When it is monitored that the current load on the first processor core exceeds the load threshold, the data flow on the first processor core is migrated to the second processor core with a smaller load to reduce the load on the first processor core, improve the performance of the first processor core, and achieve load balancing between multiple processor cores. Usually, each processor core needs to process data packets of multiple data flows, and the data flow that causes the current load of the processor core to exceed the load threshold is generally a data flow with a large number of data packets. Therefore, when the current load of the first processor core exceeds the load threshold, the target data flow with a large number of data packets can be selected to migrate to the second processor core, thereby effectively alleviating the load of the first processor core. In order to achieve the migration of the target data flow between the first processor core and the second processor core, the target session information of the target data flow can be created on the second processor core, so that the second processor core can process the received data packets belonging to the target data flow. At the same time, the field value of the redirection field in the target session information of the target data stream on the first processor core and the second processor core can also be modified, so that the first processor core can migrate the data packet in the process of processing the data packet belonging to the target data stream to the second processor core based on the target session information. At the same time, a flow forwarding rule can also be added to the first processor core, so that when the data packet belonging to the target data stream is received, the data packet can be migrated to the second processor core. The embodiment of the present application can not only realize the migration of data streams between different processor cores, but also can reduce the load of the processor core when the current load on any processor core is large, improve the performance of the processor core, avoid packet loss, and realize load balancing between multiple processor cores. Furthermore, when the current load of the processor core is reduced and the processor core has sufficient resources to process the received data packets, the migration of the target data stream is supported, thereby reducing the resource consumption caused by the migration of the data stream. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 It is an architecture diagram of an electronic device for implementing data flow migration between processor cores provided by an embodiment of the present application;

[0031] Figure 2 It is an overall flow chart of a load balancing method and a data processing method provided in an embodiment of the present application;

[0032] Figure 3 It is a structural diagram of session information provided by an embodiment of the present application;

[0033] Figure 4 It is a preparation flow chart for implementing migration of a data packet from a source processor core to a destination processor core provided by an embodiment of the present application;

[0034] Figure 5 is a schematic diagram of the entire process of processing a data packet to be migrated provided by an embodiment of the present application;

[0035] Figure 6 It is an overall flow chart of a data migration method and a data processing method provided by an embodiment of the present application;

[0036] Figure 7 It is a preparation flow chart for implementing the migration of a data packet from a destination processor core to a source processor core provided by an embodiment of the present application;

[0037] Figure 8 It is a schematic diagram of the entire process of a source processor processing a relocated data packet provided by an embodiment of the present application;

[0038] Fig. 9 It is a schematic diagram of the entire process of a source processor processing a data packet provided by an embodiment of the present application;

[0039] Fig.10 This is a flowchart of processing session information in an abnormal state provided by an embodiment of the present application;

[0040] Fig.11 It is a structural diagram of a load balancing device provided in an embodiment of the present application;

[0041] Fig.12 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0042] Fig.13 It is a structural schematic diagram of a data migration device provided in an embodiment of the present application;

[0043] Fig.14 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0044] Fig.15 A structural block diagram of an electronic device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0045] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0046] It can be understood that the terms "each", "multiple", and "any" used in the embodiments of the present application include two or more, each refers to each of the corresponding multiple, and any refers to any one of the corresponding multiple. For example, the multiple words include 10 words, and each word refers to each of the 10 words, and any word refers to any one of the 10 words.

[0047] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0048] Before executing the embodiments of the present application, the terms involved in the embodiments of the present application are first explained.

[0049] RSS (Receive Side Scaling) is a traffic distribution technology that can distribute data streams to multiple processor cores for processing, thereby improving the system's network processing capabilities and throughput. When the network card receives a data packet, it calculates a hash value based on the five-tuple information of the data packet (including the source IP (Internet Protocol) address, destination IP address, source port, destination port and transport layer protocol). The network card maps the hash value to the receive queue of a processor core and copies the data packet from the network card's receive queue to the receive queue of the corresponding processor core through the DMA controller (direct memory access).

[0050] FDIR (Flow Director) is a network card rule that can filter incoming data flows according to preset rules and import qualified data flows into the receiving queue of the specified processor core, thereby achieving more sophisticated flow control and processing.

[0051] DPDK (Data Plane Development Kit) is a collection of function libraries and drivers for fast data packet processing, which can greatly improve data processing performance and throughput, and improve the work efficiency of data plane applications.

[0052] ICC (Inter-Core Communication) technology is used to implement communication between processor cores.

[0053] With the development of computer technology, the traffic gateway of the data center can evenly distribute all data flows flowing through the traffic gateway to multiple processor cores of the traffic gateway based on the hardware RSS and hardware FDIR rules of the network card, and then when receiving data packets belonging to different data flows, the received data packets can be sent to the corresponding processor core for processing. Although this method can achieve load balancing between multiple processor cores, this load balancing is only a balance at the level of data flow quantity, not a real load balancing. Generally speaking, different data flows carry different services, and the flow size of data flows of different services changes with the change of user visits. During the peak period of a certain service, the user visits are large, and the number of data packets of the data flow corresponding to the service is large, resulting in a large current load of the processor core corresponding to the data flow and poor performance. In addition, the same processor core generally needs to process multiple data flows, and the cache space of the processor core is limited. When the number of data packets of a certain data flow increases, the data packets of other data flows may not be fully stored, resulting in packet loss in the processor core. In order to facilitate the management of received and sent data packets, each processor core includes two mbuf queues, one is the mbuf receive queue and the other is the mbuf send queue. The received data packets will be stored in the mbuf receive queue. When the data packets are processed, the data packets are obtained from the mbuf receive queue. The processed data packets will be stored in the mbuf send queue. When the sending conditions are met, the data packets in the mbuf send queue will be transmitted to the network card for sending.

[0054] In order to improve the performance of the processor core and avoid packet loss due to heavy load on the processor core, the embodiment of the present application simulates software RSS rules and software FDIR rules (i.e., stream forwarding rules) on the processor core based on DPDK and ICC, allowing data streams to be migrated from one processor core to another, thereby realizing the migration of data streams between different processor cores. Under this premise, on the one hand, by real-time monitoring of the current load of each processor core, when the current load of any processor core is large, the data stream with a large load on the processor core is dynamically transferred to the processor core with a smaller current load, thereby avoiding a single processor core from being overloaded and improving the overall utilization of the processor core.

[0055] On the other hand, when the traffic of the migrated data flow decreases, it is allowed to migrate the data flow back from the destination processor core to the source processor core, thereby returning the data flow to the message sending and receiving process of the hardware RSS and hardware FDIR, reducing the ICC performance overhead in the implementation of software RSS and software FDIR.

[0056] On the other hand, three mbuf queues are set for each processor core, namely, an mbuf common receiving queue (i.e., the first receiving queue described in the embodiment of the present application), an mbuf priority receiving queue (i.e., the second receiving queue described in the embodiment of the present application), and an mbuf common sending queue (i.e., the sending queue described in the embodiment of the present application). When a data stream migrates between different processor cores, the data packets belonging to this data stream are copied from the mbuf common queue of the source processor core to the mbuf priority queue of the destination processor core through ICC. When the destination processor core processes the data packets, it first obtains the data packets from the mbuf priority queue for processing, thereby avoiding the problem of out-of-order data packets when the data stream migrates between processor cores.

[0057] Figure 1 The architecture diagram of an electronic device for implementing data flow migration between processor cores provided by an embodiment of the present application is shown. Figure 1 The components of the architecture of the electronic device include: a load monitor, multiple CPU cores, a session hash table on each CPU core, a DPDK user-mode driver, three mbuf queues on each CPU core, and a network card taken over by DPDK. The above multiple components are divided into hardware-level and software-level elements, wherein the hardware-level elements include multiple CPU cores and network cards, and the software-level elements include a load monitor, a session hash table on each CPU core, a DPDK user-mode driver, and three mbuf queues on each CPU core. The above multiple elements will be introduced below.

[0058] Among them, the load monitor is used to periodically collect packet processing information, session information, load information, etc. on each CPU core, and based on the load information and session information on each CPU core, select a source CPU core with a higher load and a destination CPU core with a lower load from multiple CPU cores, and based on the packet processing information of each data flow on the source CPU core, select the target data flow that needs to be migrated from the source CPU core.

[0059] The CPU core is used to process received data packets.

[0060] Each CPU core stores a session hash table, which includes session information of each data flow that the CPU core is responsible for processing. The session information includes the connection status of the corresponding data flow and a redirection field, etc.

[0061] The DPDK user-state driver can implement software RSS and software FDIR rules, which are implemented based on the flow redirection hash table on each CPU core and the inter-core communication ICC mechanism. That is, the DPDK user-state driver carries the flow redirection hash table of each CPU core, and the flow redirection hash table represents the correspondence between the hash value of the data flow that needs to be migrated on the corresponding CPU core and the destination CPU core identifier. The hash value of the data flow that needs to be migrated is obtained by hashing the five-tuple information of the data flow. The data in the flow redirection hash table can be stored in a key-value format, where the hash value of the data flow that needs to be migrated is the key and the destination CPU core identifier is the value.

[0062] Each CPU forwarding core includes three mbuf queues, namely, mbuf normal receive queue, mbuf normal send queue and mbuf priority receive queue. Among them, mbuf normal receive queue is used to store data packets received from the network card, mbuf normal send queue is used to store data packets processed by the CPU core, and mbuf priority receive queue is used to store data packets of data streams migrated from other CPU cores. When the CPU core needs to obtain data packets, it detects whether the mbuf priority receive queue is empty based on the DPDK user-state driver. If the mbuf priority receive queue is not empty, it means that a data stream has migrated to the current CPU core. In order to avoid the problem of data packet disorder during data stream migration, the CPU core prioritizes the data packets in the mbuf priority queue. When the mbuf priority receive queue is empty, the DPDK user-state driver obtains data packets from the mbuf normal receive queue for processing.

[0063] The network card is located at the bottom layer of the system and can implement hardware RSS rules and hardware FDIR rules. Generally, when the network card receives a data packet, it first matches the hardware FDIR rules at the hardware level. If the hardware FDIR rules are not matched, the data stream to which the data packet belongs is determined based on the five-tuple information of the data packet, and then the CPU core corresponding to the data packet is determined based on the mapping relationship between the data stream and the CPU core, and then an interrupt request is sent to the CPU core. When the interrupt request is received, the CPU core sends a notification message to the DMA controller. In response to the received notification message, the DMA controller copies the data packet from the network card's receive queue to the mbuf normal receive queue of the CPU core.

[0064] The present application embodiment provides a load balancing method and a data processing method. Taking an electronic device having multiple processor cores executing the present application embodiment as an example, see Figure 2 , the method flow provided in the embodiment of the present application includes:

[0065] 201. Determine a first processor core, a second processor core, and a target data flow based on current load information, session information, and packet reception information of multiple processor cores.

[0066] The load information includes utilization, etc. The session information includes the number of sessions established based on each communication protocol. For example, if the communication protocols used are TCP (Transmission Control Protocol) and UDP (User Datagram Protocol), the session information includes the number of TCP sessions and the number of UDP sessions, etc. The packet reception information includes the number of packets received per second (Packets per second, referred to as PPS), etc. The first processor core is a processor core whose current load exceeds the load threshold. It is a processor core allocated to the target data flow based on the hardware RSS rule and the hardware FDIR rule of the network card, that is, the source processor core of the target data flow. The target data flow is a data flow on which the number of packets received on the first processor core at the current moment exceeds the number threshold. The second processor core is a processor core whose current load does not exceed the load threshold. It is a processor core to which the target data flow needs to be migrated based on the inter-core migration mechanism, that is, the destination processor core of the target data flow.

[0067] Specifically, based on the load information, session information and packet reception information of the multiple processor cores at the current moment, determining the first processor core, the second processor core and the target data flow includes the following steps:

[0068] 2011. Based on the utilization rate of each processor core at the current moment and the number of sessions established by each communication protocol, calculate the current load of each processor core.

[0069] In the embodiment of the present application, for each communication protocol, a corresponding weight value is set in advance for the session established by each communication protocol, and different weight values ​​are set for the utilization of the processor core at the current moment and different sessions according to the degree of influence of the utilization and the session on the current load of the processor core. Then, based on the weight value set for the session established by each communication protocol, the load associated with the session is calculated, and then based on the utilization of the processor core and the weights set for different sessions, the current load of the processor core is calculated.

[0070] Assume that the current load of the processor core is expressed as core_overload, then core_overload = the utilization of the processor core at the current moment * M + (the number of sessions established by the processor core based on communication protocol 1 * W 1 +Number of sessions established by the processor core based on communication protocol 2*W 2 +…+Number of sessions established by the processor core based on communication protocol n*Wn )*(1–M). Where M represents the weight value set for the utilization of the processor core at the current moment, and the value of M is greater than 0 and less than 1; 1-M represents the weight value set for the session; W 1 , W 2 , …, W n Represents the weight values ​​set for communication protocol 1, communication protocol 2, ..., communication protocol n.

[0071] Assuming that the communication protocols are TCP and UDP, the current load of the processor core core_overload = the utilization of the processor core at the current moment * M + (the number of sessions established by the processor core based on TCP * W 1 +Number of sessions established by the processor core based on UDP*W 2 )*(1–M).

[0072] It should be noted that the weight values ​​set above are not fixed, but are dynamically adjusted as the data processing process progresses. When the weight values ​​set make the current loads of multiple processor cores discrete, the weight values ​​set are adjusted so that the current loads of each processor core are relatively concentrated.

[0073] 2012. Calculate the average load of the processor cores based on the number of processor cores, the total utilization of the multiple processor cores, and the total number of sessions established by each communication protocol.

[0074] Assume that the average load of the processor core is expressed as avg_overload, and the number of processor cores is T, then avg_overload = [the utilization of all processor cores at the current moment * M + (the total number of sessions established by all processor cores based on communication protocol 1 * W 1 +The total number of sessions established by all processor cores based on communication protocol 2*W 2 +…+The total number of sessions established by all processor cores based on communication protocol n*W n )*(1–M)] / T.

[0075] Assuming that the communication protocols are TCP and UDP, the average load of the processor core core_overload = the utilization of all processor cores at the current moment * M + (the total number of sessions established by all processor cores based on TCP * W 1 +The total number of sessions established by all processor cores based on UDP*W 2 )*(1–M).

[0076] 2013. Take K times of the average load as the load threshold.

[0077] Wherein, K is an integer greater than 1. Since the load information, session information and packet reception information of each processor core at different times are not fixed, the current load of each processor core at different times changes dynamically, and accordingly, the average load of the processor core also changes dynamically, and the load threshold calculated based on the average load of the processor core also changes dynamically.

[0078] 2014. Any processor core whose current load is greater than the load threshold is used as the first processor core, and any processor core whose current load is less than the load threshold is used as the second processor core.

[0079] The current load of each processor core is compared with the load threshold. When the current load of any processor core is greater than the load threshold, the processor core is used as the first processor. When the current load of any processor core is less than the load threshold, the processor core is used as the second processor core.

[0080] In another implementation, the current load of multiple processor cores may be sorted in descending order based on the current load of each processor core, and then a preset number of processor cores whose current load exceeds the load threshold may be used as the first processor core. The preset number may be 1, 2, etc. For example, if the preset number is 1, the processor core with the largest load at the current moment may be used as the first processor core, and the processor core with the smallest load at the current moment may be used as the second processor core.

[0081] 2015. The data flow whose number of data packets received per second on the first processor core is greater than the number threshold is taken as the target data flow.

[0082] For each first processor core, the number of packets received per second (PPS) of each data flow on each first processor core is obtained, and then the number of packets received per second of each data flow is compared with a quantity threshold, and a data flow whose number of packets received per second is greater than the quantity threshold is taken as a target data flow. The quantity threshold can be determined based on data of packets of each data flow received at a historical moment.

[0083] In another possible implementation, each data flow may be sorted in descending order of the number of data packets received per second, and the data flow with the largest number of data packets received per second is used as the target data flow.

[0084] 202. Create the same session information on the second processor core according to the target session information of the target data flow on the first processor core.

[0085] In order to process the received data packets, it is necessary to create and store session information of each processed data stream on each processor core, and the session information is used to indicate the connection status of the corresponding data stream. In order to enable the second processor core to process the data packets belonging to the target data stream after the target data stream is migrated, it is necessary to create session information of the target data stream on the second processor core. Specifically, when creating, the target session information of the target data stream on the first processor core can be queried, and then the same session information, i.e., the target session information, can be created on the second processor core.

[0086] Optionally, since the target session information created on the second processor core depends on the target session information on the first processor core, in order to avoid the target session information on the first processor core and the second processor core being out of sync, thereby affecting the processing of the target data stream, the embodiment of the present application will also lock the target session information on the first processor core after querying the target session information on the first processor core, thereby avoiding operations on the target session information on the first processor core and ensuring the consistency of the target session information on the first processor core and the second processor core.

[0087] 203. Modify the field value of the redirection field in the target session information of the target data flow on the first processor core and the second processor core to obtain the first session information of the first processor core and the second session information of the second processor core.

[0088] In the embodiment of the present application, the session structure of the session information includes an incoming flow field, a return flow field, and a redirection field, etc. The incoming flow field and the return flow are used to indicate different directions of the data flow. The incoming flow is the data flow from the user to the data center, and the return flow is the data flow from the data center to the user. The incoming flow field is represented by origin_flow, and the return flow field is represented by reply_flow. The redirection field includes a redirection identifier, a source core identifier, and a destination core identifier, etc. Among them, the redirection identifier is used to indicate whether the data flow has been migrated between cores, and is represented by redirect_flag; the source core identifier represents the core identifier of the processor core selected by the data flow according to the hardware RSS rule of the network card, and is represented by origin_core; the destination core identifier represents the core identifier of the destination processor core to which the data flow migrates, and is represented by target_core. Figure 3 A diagram showing the session structure of session information, see Figure 3 The session information of each data flow includes: the inbound flow field origin_flow, the return flow field reply_flow, the redirection identifier redirect_flag, the source core identifier origin_core and the destination core identifier target_core.

[0089] Specifically, modifying the field value of the redirection field in the target session information on the first processor core includes: modifying the field value of the redirection identifier in the target session information on the first processor core to a first value, modifying the field value of the source core identifier to the core identifier of the first processor core, and modifying the field value of the destination core identifier to the core identifier of the second processor core, to obtain the first session information of the first processor core. The first value is used to indicate that the corresponding data flow is to be migrated, and in the embodiment of the present application, the first value may be 1. The first session information is used to instruct the first processor core to migrate the data packets belonging to the target data flow being processed to the second processor core.

[0090] Modifying the field value of the redirection field in the target session information on the second processor core includes: modifying the field value of the redirection identifier in the target session information on the second processor core to a second value, modifying the field value of the source core identifier to the core identifier of the first processor core, and modifying the field value of the destination core identifier to the core identifier of the second processor core, to obtain the second session information of the second processor core. The second value is used to indicate that the corresponding data flow is not to be migrated, and in the embodiment of the present application, the second value can be 0. The second session information is used to instruct the second processor core to process the received data packets belonging to the target data flow.

[0091] 204. Add a first flow forwarding rule in the first processor core.

[0092] Among them, the first stream forwarding rule characterizes the correspondence between the target hash value of the target data stream and the core identifier of the second processor core, and is used to instruct the first processor core to migrate the received data packets belonging to the target data stream to the second processor core. The target hash value is calculated based on the five-tuple information of the target data stream. Considering that the same data stream includes an inbound stream and a return stream, the five-tuple information of the inbound stream and the return stream is different, and the calculated target hash value is different. In order to refine the management of the data stream on the processor core, the first stream forwarding rule is refined into an inbound stream forwarding rule and a return stream forwarding rule for different flow directions of the data stream. Among them, the inbound stream forwarding rule, that is, the software RSS rule described in the embodiment of the present application, is used to indicate the correspondence between the target hash value of the inbound target data stream and the core identifier of the second processor core; the return stream forwarding rule, that is, the software FDIR rule described in the embodiment of the present application, is used to indicate the correspondence between the target hash value of the return target data stream and the core identifier of the second processor core. Both the inbound stream rule and the return stream rule are stored in the form of key-value. Specifically, the inbound flow rule uses the target hash value of the inbound target data flow as the key and the core identifier of the second processor core as the value; the return flow rule uses the target hash value of the return target data flow as the key and the core identifier of the second processor core as the value.

[0093] Furthermore, to facilitate the management of flow forwarding rules on each processor core, a redirection flow hash table can be constructed for each processor core, and then the first flow forwarding rules (including inbound flow forwarding rules and return flow forwarding rules) of the data flow to be migrated on each processor core are stored in the corresponding flow redirection hash table.

[0094] Furthermore, after adding the flow forwarding rule in the first processor core, the embodiment of the present application also unlocks the target session information on the first processor core, so that the processing process of the target data flow can proceed normally.

[0095] The above-mentioned step 203 can migrate the data packets belonging to the target data flow to the destination processor core during the processing of the data packets belonging to the target data flow by modifying the redirection field value of the target session information on the first processor core, and the first flow forwarding rule issued by step 204 can migrate the data packets belonging to the target data flow to the destination processor core before processing the data packets belonging to the target data flow, thereby being able to migrate the data packets belonging to the target data flow to the destination processor core at different times.

[0096] The above steps 203 and 204 are actually the preparation process for the target data flow migration. Figure 4 Provide detailed explanation.

[0097] See also Figure 4 In the scenario of data flow migration between cores, select the data flow with the highest PPS on the CPU core N with the highest load as the data flow to be migrated, and select the CPU core M with low load as the destination CPU core. Search for the session record (i.e., session information) corresponding to the data flow to be migrated on the source CPU core, and lock the session record. Then, modify the session record on the source CPU core, specifically, set redirect_flag to 1, origin_core to N, and target_core to M. Create a session record on the destination CPU core, and modify the session record on the destination CPU core, specifically, set redirect_flag to 0, origin_core to N, and target_core to M. Then, send the software RSS rule (i.e., inbound flow forwarding rule) and software FDIR rule (i.e., return flow forwarding rule) to the redirect_flow hash table (i.e., redirect flow hash table) of the source CPU core, and then unlock the session record on the source CPU core.

[0098] 205. When it is determined based on at least one of the first session information and the first flow forwarding rule that the first data packet to be processed at the next moment belongs to the target data flow, migrate the first data packet from the first processor core to the second processor core.

[0099] In the embodiment of the present application, each processor core includes three queues, namely, a first receiving queue (i.e., mbuf normal receiving queue), a second receiving queue (i.e., mbuf priority receiving queue) and a sending queue (i.e., mbuf normal sending queue). Among them, the first receiving queue is used to store data packets copied from the receiving queue of the network card, the second receiving queue is used to store data packets migrated from other processor cores, and the sending queue is used to store data packets processed by the processor core.

[0100] When the processing time of the data packet is reached by the first processor core at the next moment, the first data packet to be processed at the next moment is obtained from the first receiving queue or the second receiving queue of the first processor core, and then it is determined whether the first data packet belongs to the target data flow. When it is determined that the first data packet belongs to the target data flow, the first data packet is migrated from the first processor core to the second processor core.

[0101] Specifically, when obtaining a first data packet and determining whether the first data packet belongs to a target data flow, the following method may be used: the DPDK user-state driver corresponding to the first processor core detects whether the second receive queue of the first processor core is empty. When the second receive queue of the first processor core is not empty, the DPDK user-state driver returns the data packet in the second receive queue to the upper-layer DPDK application (i.e., the first processor core); when the second receive queue of the first processor core is empty, indicating that there are no data packets migrated by other processor cores, the first data packet is obtained from the first receive queue of the first processor core, and then the first hash value of the first data packet is calculated according to the five-tuple information of the first data packet, and the redirection flow hash table is searched according to the first hash value. When the first hash value hits the target hash value in the first flow forwarding rule, it is determined that the first data packet belongs to the target data flow; when the first hash value does not hit the hash value in the redirection flow hash table, the first data packet is returned to the upper-layer DPDK application, and the upper-layer DPDK application searches the session hash table (flow hash table) on the first processor core according to the first hash value, and finds the session entry (session entry) according to the session pointer in the flow entry (flow entry). entry), that is, the first session information, and based on the redirection identifier (redirect_flag) in the first session information, determine whether the session has inter-core migration. When the field value of the redirection identifier in the first session information is a first value, it is determined that the session has inter-core migration, and the first data packet belongs to the target data stream.

[0102] Furthermore, if the redirection identifier in the first session information is not the first value, the data packet is processed according to the type of the session entry. For example, if the type of the session entry is the fullNAT type, the source IP, source port, destination IP, and destination port of the data packet are changed at the same time; if the type of the session entry is the NAT type, the destination IP and destination port of the data packet are changed. In addition, it is possible to insert a PPV2 header, a TCP option field, etc. into the message.

[0103] Specifically, migrating the first data packet from the first processor core to the second processor core includes: if the first hash value of the first data packet hits the target hash value, then based on the correspondence between the target hash value and the second processor core in the first flow forwarding rule, determining that the second processor core is the destination core, and then migrating the first data packet to the second receiving queue of the second processor core through ICC. If the first hash value of the first data packet does not hit the target hash value, but it is determined through the redirection identifier in the first session information that the session has undergone inter-core migration, then according to the core identifier of the destination core in the first session information, the first data packet is migrated to the second receiving queue of the second processor core through ICC.

[0104] Compared with traditional DPDK applications, each CPU forwarding core has only one mbuf receiving queue and mbuf sending queue. The embodiment of the present application adds an mbuf priority receiving queue for each CPU core. When the CPU core obtains a data packet from the receiving queue, it first checks whether the mbuf priority receiving queue is empty. If it is not empty, it means that a data flow has migrated to the current CPU core. Each CPU core first processes the data packets in the mbuf priority queue, and then processes the data packets in the mbuf common receiving queue, avoiding the problem of data packet disorder during the migration of data flow between CPU cores.

[0105] 206. Process the first data packet based on the second processor core and the second session information.

[0106] Considering that each processor core has two receiving queues, the processor core has different processing priorities for data packets in the two receiving queues. The data packets in the second receiving queue are data packets migrated from other processor cores. In order to avoid data packets being out of order during the migration process, the data packets in the second receiving queue need to be processed first. In view of this, when calling the second processor core for data processing, it can be first detected whether the second receiving queue of the second processor core is empty. When the second receiving queue of the second processor core is not empty, the first data packet is obtained from the second receiving queue of the second processor core, and the address information of the first data packet is modified according to the second session information to obtain the second data packet, and then the second data packet is added to the sending queue of the second processor core. When the sending condition of the second data packet is met, the second data packet is copied from the sending queue of the second processor core to the sending queue of the network card, so that the network card sends the second data packet. Of course, if the second receiving queue of the second processor core is empty, a data packet is obtained from the first receiving queue of the second processor core and the data packet is processed. Specifically, according to the second session information, the address information of the first data packet is modified to obtain the second data packet, including: according to the type of the second session information, the first data packet is processed. For example, if the type of the second session information is FullNAT, the source IP, source port, destination IP, and destination port of the first data packet are changed at the same time; if the type of the second session information is NAT, the destination IP and destination port of the first data packet are changed.

[0107] Regarding the migration and processing of the first data packet, the following will be combined with Figure 5 Describe in detail.

[0108] See also Figure 5, the network card receives a data packet of any data stream, and according to the hardware FDIR rule or the hardware RSS rule, copies the data packet to the mbuf normal receive queue of the source CPU core through the DMA controller. The source CPU core checks whether the mbuf priority receive queue is empty. When the mbuf priority receive queue is empty, a data packet is obtained from the mbuf normal receive queue, and then the hash value of the data packet is calculated according to the five-tuple information of the data packet, and then the redirect_flow hash table is searched on the source CPU core. If the hash value hits any record in the redirect_flow hash table, the data packet is copied to the mbuf priority queue of the destination CPU core corresponding to the value through ICC according to the value of the hit record. The destination CPU core processes the data packet, and then the source CPU core continues to process the next data packet in the mbuf normal queue. If the hash value does not hit all records in the redirect_flow hash table, the session record (i.e., session information) of the data packet is queried. If the redirect_flag in the session record is 1, the data packet is copied to the mbuf priority receive queue of the CPU core specified by target_core through ICC. The CPU core processes the data packet, and then the source CPU core continues to process the next data packet in the mbuf normal queue.

[0109] Before processing a data packet, the destination CPU core checks whether the mbuf priority receiving queue is empty. If it is not empty, it means that the data packets of other processor cores have been migrated to the current CPU core, and the destination CPU core first processes the data packets in the mbuf priority queue. Specifically, the destination CPU core obtains the session record of the data packet, processes the data packet according to the session record, and then puts the processed data packet into the mbuf general sending queue. Then, the DMA controller copies the data packet in the mbuf general sending queue of the destination CPU forwarding core to the network card sending queue, and the network card sends the data packet.

[0110] Considering that the traffic of each data flow changes dynamically, the traffic increases suddenly during peak hours and decreases during off-peak hours. When the traffic increases suddenly, the processor core can be prevented from being overloaded by migrating the data flow between cores. However, the migration of data flow between cores inevitably redirects the data between different processor cores through the ICC mechanism, and copies the data packets from the mbuf normal receive queue of the source core to the mbuf priority receive queue of the destination core. These two processes will bring additional processor performance overhead and memory bandwidth resource usage. In particular, memory access in cross-chip scenarios for inter-core communication will bring greater performance overhead due to L2Cache misses. In order to reduce the performance overhead of data flow migration between cores, the embodiment of the present application supports migrating the target data flow from the second processor core to the first processor core when it is detected that the target data flow meets the migration conditions. See. Figure 6 The embodiment of the present application provides a data migration method and a data processing method. Taking an electronic device having multiple processor cores executing the embodiment of the present application as an example, the method flow provided by the embodiment of the present application includes:

[0111] 601. When it is detected that the target data stream meets the relocation condition, the state of the target data stream in the first session information is synchronized based on the current state of the target data stream indicated by the second session information.

[0112] Among them, the re-migration condition refers to the condition that needs to be met when the migrated data stream migrates back to the source core. The re-migration condition includes: the current load of the source core is less than the load threshold, and the PPS of the data stream at the current moment is less than S times (S is greater than 0 and less than 1) of the most recent maximum PPS, and the PPS of the data stream counted each time for L consecutive times (L is greater than 1) is less than the PPS counted last time. When it is detected that the target data stream meets the re-migration condition, the second session information of the target data stream on the second processor core is searched, and the current state of the target data stream indicated by the second session information is synchronized to the first session information, so that the state of the session information of the target data stream on the first processor core and the second processor core is synchronized.

[0113] Optionally, to ensure that the status of session information of the target data stream on the first processor core and the second processor core can be synchronized, the session information of the target data stream on the first processor core and the second processor core may be locked before synchronizing the status of the target data stream in the first session information.

[0114] 602. Modify the field value of the redirection field in the first session information and the second session information of the state synchronization to obtain the fourth session information of the first processor core and the third session information of the second processor core.

[0115] Since the redirection identifier in the redirection field is used to indicate whether to migrate the data flow, when the current load of the first processor core is large, the present application can migrate the data packets belonging to the target data flow on the first processor core to the second processor core by modifying the field value of the redirection identifier in the target session information on the first processor core to the first value and modifying the redirection identifier in the target session information on the second processor core to the second value. When the flow of the target data flow decreases and needs to be migrated back, in order to prevent the first processor core from migrating the received data packets belonging to the target data flow to the second processor core, and at the same time enable the data packets belonging to the target data flow that have been migrated on the second processor core to be migrated back to the first processor core, the field value of the redirection identifier in the state-synchronized first session information can be modified to the second value to obtain the fourth session information, which is used to instruct the first processor core to process the received data packets belonging to the target data flow. The field value of the redirection identifier in the second session information is modified to the third value to obtain the third session information, which is used to instruct the second processor core to migrate the data packets belonging to the target data flow back to the first processor core. Among them, the third value is used to indicate the corresponding data flow to be migrated back, which can be 2.

[0116] In addition, since the source processor core and the destination processor core corresponding to the target data flow have not changed, when modifying the redirection field, there is no need to modify the source core identifier and the destination core identifier.

[0117] 603. Delete the first flow forwarding rule on the first processor core, and add a second flow forwarding rule in the second processor core.

[0118] When the target data stream is migrated back to the first processor core for processing, in order to avoid the target data stream migrating back and forth between the first processor core and the second processor core, the first stream forwarding rule on the first processor core can be deleted. In this way, after receiving a data packet belonging to the target data stream, the first processor core will no longer migrate the data packet to the second processor core based on the first stream forwarding rule.

[0119] Furthermore, considering that some data packets belonging to the target data flow may be stored in the second receiving queue of the second processor core, in order to avoid the loss of these packets, after deleting the first flow forwarding rule on the first processor core, it is also necessary to send the second flow forwarding rule to the second processor core, which represents the correspondence between the target hash value and the core identifier of the first processor core, and is used to instruct the second processor core to migrate the data packets belonging to the target data flow to the first processor core. Based on the second flow forwarding rule, the data packets belonging to the target data flow on the second processor core can be redirected back to the first processor core for processing. Similar to the first flow forwarding rule, the second flow forwarding rule also includes an inbound flow rule and a return flow rule, wherein the inbound flow forwarding rule is used to indicate the correspondence between the target hash value of the inbound target data flow and the core identifier of the first processor core; the return flow forwarding rule is used to indicate the correspondence between the target hash value of the return target data flow and the core identifier of the first processor core. Both the inbound flow rule and the return flow rule are stored in key-value form. Specifically, the inbound flow rule uses the target hash value of the inbound target data flow as the key and the core identifier of the first processor core as the value; the return flow rule uses the target hash value of the return target data flow as the key and the core identifier of the first processor core as the value.

[0120] The method provided by the embodiment of the present application deletes the first stream forwarding rule when the target data stream meets the conditions for re-migration, so that the data packets of the target data stream can be re-imported into the first receiving queue of the first processor core based on the hardware RSS rule and hardware FDIR rule of the network card, and processed on the first processor core, thereby avoiding the additional overhead caused by the migration process, and also avoiding the overhead caused by querying the first stream forwarding rule. In particular, during the process of switching back between cores of the target data stream, by first deleting the first stream forwarding rule on the first processor core, and then adding the second stream forwarding rule on the second processor core, it is avoided that the data packets are cyclically copied between the first processor core and the second processor core, which causes unnecessary performance loss.

[0121] Furthermore, after the second flow forwarding rule is added in the second processor core, the session information of the target data flow on the first processor core and the second processor core will be unlocked, so that the first processor core and the second processor core can process the target data flow based on the session information.

[0122] The above target data stream migration process will be combined with Figure 7 Describe in detail.

[0123] See also Figure 7In the data flow migration scenario, the data flow that needs to be migrated is determined based on the migration conditions, and the source CPU core N and the destination CPU core M are determined based on the migrated data flow. Then, the session records of the data flow on the source CPU core and the destination CPU core are searched, and the session records on the source CPU core and the destination CPU core are locked. Next, the current status information of the session record on the destination CPU core is synchronized to the session record in the source CPU core. Then, the redirection field in the session record on the source CPU core is modified, and the redirection_flag is set to 0, and the redirection field in the session record on the destination CPU core is modified, and the redirection_flag is set to 2, and then the corresponding RSS rules and FDIR rules in the flow redirection hash table in the source CPU core are deleted, and then the RSS rules and FDIR rules are sent to the flow redirection hash table in the destination CPU core, and finally the session records on the source CPU core and the destination CPU core are unlocked.

[0124] 604. When it is determined based on the third session information and at least one of the second flow forwarding rules that the third data packet to be processed by the second processor core at the next moment belongs to the target data flow, migrate the third data packet from the second processor core to the first processor core.

[0125] Specifically, based on the third session information and at least one of the second flow forwarding rules, when determining that the third data packet to be processed by the second processor core at the next moment belongs to the target data flow, the following method can be used: the DPDK user-mode driver corresponding to the second processor core detects whether the second receiving queue of the second processor core is empty. When the second receiving queue of the second processor core is not empty, the third data packet is obtained from the second receiving queue of the second processor core, and then the second hash value of the third data packet is calculated according to the five-tuple information of the third data packet, and the second hash value is matched with the redirection flow hash table stored in the second processor core. When the second hash value hits the target hash value in the second flow forwarding rule, it is determined that the third data packet belongs to the target data flow. Further, in response to the third data packet belonging to the target data flow, according to the second flow forwarding rule, the destination core is determined to be the first processor core, and then the third data packet is copied to the second receiving queue of the first processor core.

[0126] In another embodiment of the present application, when the second hash value does not hit any record in the redirection flow hash table, the DPDK user-mode driver corresponding to the second processor core returns the third data packet to the DPDK upper-layer application (i.e., the second processor core). The DPDK upper-layer application searches the session hash table (flow hash table) on the second processor core based on the second hash value of the third data packet, and finds the session entry, i.e., the third session information, based on the session pointer in the flow entry, and then queries the redirection identifier (redirect_flag) in the third session information to determine whether the session has undergone inter-core migration. When the field value of the redirection identifier in the fourth session information is the third numerical value, it is determined that the session has undergone inter-core migration, and the third data packet belongs to the target data stream and needs to be migrated back to the first processor core. Among them, the third numerical value is used to indicate the re-migration processing of the corresponding data stream. Further, based on the source core identifier in the fourth session information, the third data packet is copied to the second receive queue of the first processor core.

[0127] 605. Process the third data packet based on the first processor core and the fourth session information.

[0128] After the target data stream is migrated back, based on the first processor core and the fourth session information, the processing process of the third data packet is as follows: detecting whether the second receiving queue of the first processor core is empty, when the second receiving queue of the first processor core is not empty, obtaining the third data packet from the second receiving queue of the first processor core, modifying the address information of the third data packet according to the fourth session information, obtaining the fourth data packet, and then adding the fourth data packet to the sending queue of the first processor core; when the sending condition of the fourth data packet is met, copying the fourth data packet from the sending queue of the first processor core to the sending queue of the network card, so that the network card sends the fourth data packet.

[0129] In another embodiment of the present application, when the second receiving queue of the first processor core is empty, the fifth data packet to be processed at the next moment is obtained from the first receiving queue of the first processor core, and the third hash value of the fifth data packet is calculated based on the five-tuple information of the fifth data packet. When the third hash value does not hit the target hash value, the session hash table on the first processor core is searched according to the third hash value, and the session entry, that is, the third session information, is found according to the session pointer in the flow entry. When the field value of the redirection identifier in the third session information is the second value, the address information of the fifth data packet is modified according to the third session information to obtain a sixth data packet, and then the sixth data packet is added to the sending queue of the first processor core. When the sending condition of the sixth data packet is met, the sixth data packet is copied from the sending queue of the first processor core to the sending queue of the network card, so that the network card sends the sixth data packet.

[0130] The above data flow migration and processing process will be combined with Figure 7 and Figure 8 Describe in detail.

[0131] See also Figure 8 , the session structure of the returned data flow in the source CPU core is: redirect_flag is 1, origin_core is N, target_core is M, and the session structure of the returned data flow in the destination CPU core is: redirect_flag is 0, origin_core is N, and target_core is M. In the data flow return scenario, the destination CPU core checks whether the mbuf priority queue is empty. When the mbuf priority queue is not empty, it obtains the data packet from the mbuf priority queue, and calculates the hash value of the data packet based on the five-tuple information of the data packet, and then queries the redirect_flow hash table based on the hash value. If the hash value hits the software RSS or software FDIR rule, the data packet is copied to the mbuf priority receiving queue of the source CPU core through ICC according to the hit software RSS or software FDIR rule; when the hash value does not hit the software RSS or software FDIR rule, the session record of the data flow to which the data packet belongs is searched. If the redirect_flag value corresponding to the session record is 2, the data packet is copied to the mbuf priority receiving queue of the source CPU core through ICC. The source CPU core checks whether the mbuf priority receive queue is empty. If the mbuf priority receive queue is not empty, it obtains the data packet from the mbuf priority queue and searches for the session record corresponding to the data stream to which the data packet belongs. Then, it processes the data packet according to the session record and transfers the processed data packet to the mbuf common send queue. The DMA controller copies the data packet in the mbuf common send queue in the source CPU core to the network card send queue, and the network card sends the data packet.

[0132] See also Fig. 9In the scenario where the data flow is migrated back from the destination CPU core to the source CPU core, when the network card receives the data packet of the data flow, it copies the data packet to the mbuf general receiving queue of the source CPU forwarding core according to the hardware FDIR rule or hardware RSS rule of the network card. Since the software RSS and software FDIR rules in the redirect_flow hash table in the source CPU core are deleted, the data packet of the data flow will no longer hit the software RSS and software FDIR rules. The source CPU core checks whether the mbuf priority receiving queue is empty. If the mbuf priority receiving queue is empty, it obtains the data packet from the mbuf general receiving queue and processes the data packet; if the mbuf priority receiving queue is not empty, it prioritizes the data packet in the mbuf priority receiving queue. When the source CPU core processes the data packet, it obtains the session record of the data flow to which the data packet belongs. After processing the data packet according to the session record, the data packet is transferred to the mbuf general sending queue. The DMA controller copies the data packet in the mbuf general sending queue in the source CPU forwarding core to the network card sending queue through the DMA controller, and the network card sends the data packet out.

[0133] In an embodiment of the present application, taking into account that the state of the session information of the migrated data stream on the destination processor core may change, such as session information timeout, and the front-end and back-end disconnecting by sending a reset message, for these scenarios that terminate the life cycle of the session information, it is necessary to correctly handle the session information of the data stream migration to avoid residual session information on the source processor core and the destination processor core.

[0134] Specifically, when the state of the target session information is detected to be abnormal, the target session information is deleted. Deleting the target session information includes: checking whether the destination core identifier in the target session information on the first processor core is empty; when the destination core identifier is empty, deleting the target session information on the first processor core; when the destination core identifier is not empty, deleting the target session information and forwarding rule information on the first processor core, and deleting the target session information and forwarding rule information on the second processor core.

[0135] The above session information deletion process will be combined with Fig.10 Describe in detail.

[0136] 1. Search for session records on the current CPU core and lock the session records on the current CPU core.

[0137] 2. Check whether the target_core of the session record is empty. If it is not empty, jump to step 3; otherwise, jump to step 7;

[0138] 3. According to the target_core field of the session record, find and delete the session record on the target CPU core;

[0139] 4. According to the target_core field of the session record, find and delete the software rss and software fdir rules in the redirect_flow hash table on the target CPU core;

[0140] 5. Find and delete the session record on the source CPU core according to the origin_core field of the session record;

[0141] 6. According to the origin_core field of the session record, find and delete the software rss and software fdir rules in the redirect_flowhash table on the source CPU core, and clear the session process.

[0142] 7. Delete the session record on the current CPU core and the session clearing process ends.

[0143] The above embodiment is described by taking multiple processor cores all participating in the data processing process and setting three mbuf queues for each processor core as an example. Optionally, some processor cores can be selected from the multiple processor cores in advance as idle processor cores. These idle processor cores will not participate in the data processing process under normal circumstances. When the current load of a processor core is large, the data flow on the processor core with a large load is migrated to the idle processor core for processing. In this case, all processor cores do not need three mbuf queues, only two mbuf queues are needed, namely, mbuf receiving queue and mbuf sending queue. When performing data flow migration, software RSS rules and software FDIR rules are sent to the source processor core, and the data flow is migrated from the source processor core to the idle processor core for processing; when performing data flow back migration, the software RSS rules and software FDIR rules on the processor core are deleted, and the software RSS rules and software FDIR rules are sent to the idle processor core, and the data flow is back migrated from the idle processor core to the source processor core for processing.

[0144] The method provided in the embodiment of the present application has the following beneficial effects:

[0145] On the one hand, through the migration of data flows between cores, when the system is running, data flows with large traffic are dynamically transferred from CPU cores with excessive load to CPU cores with lower load for processing, thus avoiding high CPU usage of a single core and improving overall CPU utilization;

[0146] On the other hand, when the data flow rate decreases, migrating the data flow back to the source CPU core can reduce the ICC performance overhead of software RSS and software FDIR;

[0147] On the other hand, the mbuf priority queue on each CPU core is used to avoid the problem of data packet disorder when the data flow migrates between CPU cores;

[0148] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0149] Please refer to Fig.11 , which shows a schematic diagram of the structure of a load balancing device provided in an embodiment of the present application. The device can be implemented by software, hardware, or a combination of both, and becomes all or part of an electronic device. The device includes:

[0150] The determination module 1101 is used to determine the first processor core, the second processor core and the target data flow based on the load information, session information and packet reception information of the multiple processor cores at the current moment, wherein the first processor core is a processor core whose current load exceeds the load threshold, the second processor core is a processor core whose current load does not exceed the load threshold, and the target data flow is a data flow whose number of data packets received at the first processor core at the current moment exceeds the number threshold;

[0151] A modification module 1102 is used to modify a field value of a redirection field in target session information of a target data flow on a first processor core, and obtain first session information of the first processor core, where the first session information is used to instruct the first processor core to migrate a data packet belonging to the target data flow being processed to a second processor core;

[0152] An adding module 1103 is used to add a first flow forwarding rule in the first processor core, where the first flow forwarding rule represents the correspondence between the target hash value of the target data flow and the core identifier of the second processor core, and is used to instruct the first processor core to migrate the received data packets belonging to the target data flow to the second processor core, where the target hash value is calculated according to the five-tuple information of the target data flow;

[0153] The migration module 1104 is configured to migrate the first data packet from the first processor core to the second processor core when it is determined that the first data packet to be processed at the next moment belongs to the target data flow based on at least one of the first session information and the first flow forwarding rule.

[0154] Please refer to Fig.12 , which shows a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. The device is based on Fig.11The load balancing device shown in the figure can be implemented by software, hardware or a combination of the two, and becomes the whole or part of the electronic device. The device includes:

[0155] A creation module 1201, configured to create the same session information on the second processor core according to the target session information of the target data stream on the first processor core;

[0156] A modification module 1202 is used to modify a field value of a redirection field in the target session information of the target data flow on the second processor core to obtain second session information of the second processor core, where the second session information is used to instruct the second processor core to process a received data packet belonging to the target data flow;

[0157] The processing module 1203 is configured to process the first data packet migrated from the first processor core based on the second processor core and the second session information when receiving the first data packet.

[0158] Please refer to Fig.13 , which shows a schematic diagram of the structure of a data migration device provided by an embodiment of the present application. The device is based on Fig.12 The data processing device shown in the figure can be implemented by software, hardware or a combination of both, and becomes the whole or part of the electronic device. The device includes:

[0159] The modification module 1301 is used to modify the field value of the redirection field in the second session information when detecting that the target data flow meets the relocation condition, and obtain the third session information of the second processor core, wherein the third session information is used to instruct the second processor core to relocate the data packets belonging to the target data flow to the first processor core;

[0160] An adding module 1302 is used to add a second flow forwarding rule in the second processor core, where the second flow forwarding rule represents the correspondence between the target hash value and the core identifier of the first processor core, and is used to instruct the second processor core to migrate the data packets belonging to the target data flow to the first processor core;

[0161] The processing module 1303 is configured to migrate the third data packet from the second processor core back to the first processor core when it is determined based on at least one of the third session information and the second flow forwarding rule that the third data packet to be processed by the second processor core at the next moment belongs to the target data flow.

[0162] Please refer to Fig.14 , which shows a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. The device is based on Fig.13 The data migration device shown in the figure can be implemented by software, hardware or a combination of both, and becomes the whole or part of the electronic device. The device includes:

[0163] The synchronization module 1401 is used to synchronize the state of the target data stream in the first session information based on the current state of the target data stream indicated by the second session information when it is detected that the target data stream meets the relocation condition;

[0164] The modification module 1402 is used to modify the field value of the redirection field in the first session information of the state synchronization to obtain the fourth session information of the first processor core, where the fourth session information is used to instruct the first processor core to process the received data packet belonging to the target data flow;

[0165] A deleting module 1403, configured to delete the first flow forwarding rule on the first processor core;

[0166] The processing module 1404 is configured to process the third data packet based on the first processor core and the fourth session information when receiving the third data packet returned from the second processor core.

[0167] Fig.15 The structure block diagram of an electronic device 1500 provided by an exemplary embodiment of the present application is shown. Generally, the electronic device 1500 includes: a processor 1501 and a memory 1502 .

[0168] Processor 1501 can be implemented in at least one of the following hardware forms: DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1501 may also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state; the coprocessor being a low-power processor for processing data in a standby state. In some embodiments, processor 1501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, processor 1501 may also include an artificial intelligence processor, which is used to process computing operations related to machine learning.

[0169] The memory 1502 may include one or more computer-readable storage media, which may be non-temporary computer-readable storage media, for example, the non-temporary computer-readable storage media may be CD-ROM (Compact Disc Read-Only Memory), ROM, RAM (Random Access Memory), magnetic tape, floppy disk, optical data storage device, etc. The computer-readable storage medium stores at least one computer program, which can implement the above-mentioned load balancing method, data processing method, or data migration method when executed.

[0170] Of course, the above electronic device may also include other components, such as input / output interface, communication component, etc. The input / output interface provides an interface between the processor and the peripheral interface module, and the above peripheral interface module may be an output device, an input device, etc. The communication component is configured to facilitate wired or wireless communication between the electronic device and other devices.

[0171] Those skilled in the art will understand that Fig.15 The structure shown in the figure does not constitute a limitation on the electronic device 1500, and may include more or less components than those shown in the figure, or combine certain components, or adopt a different component arrangement.

[0172] An embodiment of the present application provides a computer-readable storage medium, in which at least one computer program is stored. When the at least one computer program is executed by a processor, the above-mentioned load balancing method, or the data processing method, or the data migration method can be implemented.

[0173] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it can implement the above-mentioned load balancing method, or data processing method, or data migration method.

[0174] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0175] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A load balancing method, characterized in that: The method comprises: Based on the load information, session information and packet reception information of the multiple processor cores at the current moment, determining the first processor core, the second processor core and the target data flow, wherein the first processor core is a processor core whose current load exceeds a load threshold, the second processor core is a processor core whose current load does not exceed the load threshold, and the target data flow is a data flow whose number of data packets received on the first processor core at the current moment exceeds the number threshold; Modify a field value of a redirection field in target session information of the target data flow on the first processor core to obtain first session information of the first processor core, where the first session information is used to instruct the first processor core to migrate a data packet belonging to the target data flow being processed to the second processor core; Adding a first flow forwarding rule in the first processor core, wherein the first flow forwarding rule represents a correspondence between a target hash value of the target data flow and a core identifier of the second processor core, and is used to instruct the first processor core to migrate received data packets belonging to the target data flow to the second processor core, wherein the target hash value is calculated based on the five-tuple information of the target data flow; When it is determined based on at least one of the first session information and the first flow forwarding rule that a first data packet to be processed at the next moment belongs to the target data flow, the first data packet is migrated from the first processor core to the second processor core.

2. The method according to claim 1, characterized in that The redirection field includes a redirection identifier, a source core identifier, and a destination core identifier, and the modifying of the field value of the redirection field in the target session information of the target data flow on the first processor core includes: The field value of the redirection identifier in the target session information on the first processor core is modified to a first value, the field value of the source core identifier is modified to the core identifier of the first processor core, and the field value of the destination core identifier is modified to the core identifier of the second processor core, wherein the first value is used to indicate migration processing of the corresponding data stream.

3. The method according to claim 1, characterized in that Each processor core includes a first receiving queue and a second receiving queue, wherein the first receiving queue is used to store data packets copied from the receiving queue of the network card, and the second receiving queue is used to store data packets migrated from other processor cores, and when it is determined based on at least one of the first session information and the first flow forwarding rule that the first data packet to be processed at the next moment belongs to the target data flow, before migrating the first data packet from the first processor core to the second processor core, the method further includes: detecting whether a second receiving queue of the first processor core is empty; When the second receiving queue of the first processor core is empty, acquiring the first data packet from the first receiving queue of the first processor core; Calculate a first hash value of the first data packet according to the five-tuple information of the first data packet; When the first hash value hits the target hash value, it is determined that the first data packet belongs to the target data flow.

4. The method according to claim 3, characterized in that The redirection field includes a redirection identifier, and the method further includes: When the first hash value does not hit the target hash value, querying the redirection identifier in the first session information; When the field value of the redirection identifier in the first session information is a first value, it is determined that the first data packet belongs to the target data flow, and the first value is used to indicate to perform migration processing on the corresponding data flow.

5. A data processing method, characterized in that: The method is based on the load balancing method according to any one of claims 1 to 4, and the method comprises: Creating the same session information on the second processor core according to the target session information of the target data flow on the first processor core; Modify a field value of a redirection field in target session information of the target data flow on the second processor core to obtain second session information of the second processor core, where the second session information is used to instruct the second processor core to process a received data packet belonging to the target data flow; When the first data packet migrated from the first processor core is received, the first data packet is processed based on the second processor core and the second session information.

6. The method according to claim 5, characterized in that The step of modifying the field value of the redirection field in the target session information of the target data flow on the second processor core to obtain the second session information of the second processor core includes: The field value of the redirection identifier in the target session information on the second processor core is modified to a second value, the field value of the source core identifier is modified to the core identifier of the first processor core, and the field value of the destination core identifier is modified to the core identifier of the second processor core, wherein the second value is used to indicate that no migration processing is performed on the corresponding data stream.

7. The method according to claim 5, characterized in that Each processor core further includes a sending queue, and based on the second processor core and the second session information, processes the first data packet, including: detecting whether a second receiving queue of the second processor core is empty; When the second receiving queue of the second processor core is not empty, acquiring the first data packet from the second receiving queue of the second processor core; Modify the address information of the first data packet according to the second session information to obtain a second data packet; adding the second data packet to a sending queue of the second processor core; When the sending condition of the second data packet is met, the second data packet is copied from the sending queue of the second processor core to the sending queue of the network card, so that the network card sends the second data packet.

8. A data migration method, characterized in that: The method is based on the data processing method according to any one of claims 5 to 7, and the method comprises: When it is detected that the target data flow meets the relocation condition, modify the field value of the redirection field in the second session information to obtain the third session information of the second processor core, wherein the third session information is used to instruct the second processor core to relocate the data packets belonging to the target data flow to the first processor core; Adding a second flow forwarding rule in the second processor core, wherein the second flow forwarding rule represents a correspondence between the target hash value and the core identifier of the first processor core, and is used to instruct the second processor core to migrate a data packet belonging to the target data flow to the first processor core; When it is determined based on at least one of the third session information and the second flow forwarding rule that the third data packet to be processed by the second processor core at the next moment belongs to the target data flow, the third data packet is migrated back from the second processor core to the first processor core.

9. The method according to claim 8, characterized in that The modifying the field value of the redirection field in the second session information to obtain the third session information of the second processor core includes: The field value of the redirection identifier in the second session information is modified to a third value to obtain the third session information, where the third value is used to indicate to perform migration processing on the corresponding data flow.

10. The method according to claim 8, characterized in that Before determining, based on at least one of the third session information and the second flow forwarding rule, that a third data packet to be processed by the second processor core at the next moment belongs to the target data flow and migrating the third data packet from the second processor core to the first processor core, the method further includes: detecting whether a second receiving queue of the second processor core is empty; When the second receiving queue of the second processor core is not empty, acquiring the third data packet from the second receiving queue of the second processor core; Calculate a second hash value of the third data packet according to the five-tuple information of the third data packet; When the second hash value hits the target hash value, determining that the third data packet belongs to the target data flow; The migrating the third data packet from the second processor core to the first processor core comprises: The third data packet is copied to the second receiving queue of the first processor core.

11. The method according to claim 10, characterized in that The method further comprises: When the second hash value does not hit the target hash value, querying the redirection identifier in the third session information; When the field value of the redirection identifier in the third session information is a third value, it is determined that the third data packet belongs to the target data flow, and the third value is used to indicate that the corresponding data flow is to be relocated; The migrating the third data packet from the second processor core to the first processor core comprises: The third data packet is copied to the second receiving queue of the first processor core.

12. A data processing method, characterized in that: The method is based on the data migration method according to any one of claims 8 to 11, and the method comprises: When it is detected that the target data stream meets the relocation condition, synchronizing the state of the target data stream in the first session information based on the current state of the target data stream indicated by the second session information; Modify a field value of a redirection field in the first session information of state synchronization to obtain fourth session information of the first processor core, wherein the fourth session information is used to instruct the first processor core to process a received data packet belonging to the target data flow; Deleting the first flow forwarding rule on the first processor core; When a third data packet returned from the second processor core is received, the third data packet is processed based on the first processor core and the fourth session information.

13. An electronic device, characterized in that: It comprises a processor and a memory; the memory stores at least one program code; the at least one program code is used to be called and executed by the processor to implement the load balancing method as described in any one of claims 1 to 4, or the data processing method as described in any one of claims 5 to 7, or the data migration method as described in any one of claims 8 to 11, or the data processing method as described in claim 12.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and when the at least one computer program is executed by the processor, it can implement the load balancing method described in any one of claims 1 to 4, or the data processing method described in any one of claims 5 to 7, or the data migration method described in any one of claims 8 to 11, or the data processing method described in claim 12.

Images