Packet transmission method and related device

Abstract

Disclosed in embodiments of the present application are a packet transmission method and a related device. A sending device performs ESP encapsulation on a first data stream to be sent, to obtain a first ESP packet, wherein the first data stream falls within an encryption scope of the first ESP packet, that is, payload and IP information of the first data stream are encrypted and hidden. The sending device generates a first hash value on the basis of the triple of the first data stream and designs a first ESP header of the first ESP packet, so that the first ESP header can carry the first hash value. The first ESP header does not fall within the encryption scope of the first ESP packet, and therefore, a receiving device can allocate the first ESP packet to a corresponding CPU core on the basis the first hash value in the first ESP header upon receiving the first ESP packet. On the basis of such a design method, the receiving device can distribute received ESP packets to corresponding CPU cores, thereby achieving multi-core payload balancing and enabling the overall performance not to be limited by a single-core performance bottleneck.

Description

A message transmission method and related device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on March 28, 2024, with application number 202410374258.1 and invention name “A message transmission method and related device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of communications, and in particular to a message transmission method and related devices. Background Art

[0003] In a Software-Defined Wide-Area Network (SD-WAN), branches, headquarters, and the cloud communicate via overlay tunnels established by edge gateways. Overlay tunnels typically use Encapsulating Security Payload (ESP) encryption to ensure data security. Both the Internet Protocol (IP) information and payload data of the service flow are encrypted and hidden within the tunnel.

[0004] Edge gateways typically communicate with each other through a single overlay tunnel, with multiple service flows aggregated and transmitted over the same overlay tunnel. Due to ESP encryption, the peer edge gateway cannot read the IP information for each service flow from the packets transmitted over the overlay tunnel. Consequently, it cannot hash multiple service flows to multiple cores for separate processing, and can only process multiple service flows on a single core. This results in overall device performance being limited by the performance bottleneck of a single core. When the service flow bandwidth exceeds the performance of a single core, packet loss occurs, resulting in reduced service quality. Summary of the Invention

[0005] The embodiment of the present application provides a message transmission method and related devices, and the receiving device can distribute each received ESP message to the corresponding central processing unit (CPU) core, achieving multi-core load balancing, and the overall performance is not limited by the performance bottleneck of a single core.

[0006] In a first aspect, an embodiment of the present application provides a message transmission method, which is applied to a sending device. Specifically, the sending device performs an Encapsulating Security Payload (ESP) encapsulation on a first data stream to obtain a first ESP message. The sending device generates a first hash value based on a triplet of the first data stream, and carries the first hash value in a first ESP header of the first ESP message. Furthermore, the sending device sends the first ESP message to a receiving device.

[0007] In this embodiment, the first data stream is within the encryption scope of the first ESP message, that is, the payload and IP information of the first data stream are encrypted and hidden. Therefore, the embodiment of the present application designs the first ESP header of the first ESP message so that the first ESP header can carry the first hash value generated by the triplet of the first data stream. Since the first ESP header is not within the encryption scope of the first ESP message, after receiving the first ESP message, the receiving device can distribute the first ESP message to the corresponding CPU core based on the first hash value in the first ESP header. Based on this design, the receiving device can distribute each received ESP message to the corresponding CPU core, achieving multi-core load balancing, and the overall performance is not limited by the performance bottleneck of a single core.

[0008] In some possible implementations, the method further includes: the sending device performing ESP encapsulation on the second data stream to obtain a second ESP message. The second ESP header in the second ESP message includes a second hash value generated from the triplet of the second data stream. Furthermore, the sending device sends the second ESP message to the receiving device. In this implementation, because the sending device generates the hash value based on the triplet of each data stream, the receiving device can effectively distinguish ESP messages encapsulating different data streams using the hash value. Each CPU core only needs to process the corresponding ESP message, rather than all ESP messages. This reduces the processing overhead of each CPU core and improves overall processing efficiency and forwarding performance.

[0009] In some possible implementations, the sequence number field in the ESP header is mainly used for anti-replay function. In the embodiment of the present application, the anti-replay function can be turned off, and the first hash value is carried by the sequence number field in the first ESP header, which has good practicality.

[0010] In some possible implementations, before the sending device performs ESP encapsulation on the first data stream to obtain the first ESP message, the method further includes: the sending device sending the first message to the receiving device, and receiving a second message from the receiving device. The first message includes first indication information, the first indication information being used to indicate that the sending device supports carrying a first hash value in the first ESP header, and the second message includes second indication information, the second indication information being used to indicate that the receiving device supports allocating the first ESP message to a corresponding CPU core based on the first hash value. In other words, before the sending device sends the ESP message to the receiving device in the manner designed in this application, it will first conduct capability negotiation with the receiving device to ensure that both parties support the ESP message transmission method designed in this application.

[0011] In some possible implementations, the first message and the second message are Border Gateway Protocol (BGP), which is convenient for adapting to existing standard protocols.

[0012] In some possible implementations, the attribute or extended community attribute in the first message includes first indication information, and the attribute or extended community attribute in the second message includes second indication information, which is conducive to improving the feasibility of this solution.

[0013] In some possible implementations, the first message and the second message are Internet Key Exchange (IKE) messages, which facilitates adaptation to existing standard protocols. The IKE header in the first message includes the first indication information, and the IKE header in the second message includes the second indication information.

[0014] In some possible implementations, the next payload field of the IKE header in the first message includes the first indication information, and the next payload field of the IKE header in the second message includes the second indication information, which helps to improve the feasibility of this solution.

[0015] In some possible implementations, the first indication information includes a first type length value (TLV), and the second indication information includes a second TLV. The first TLV is used to indicate that the sending device supports carrying the first hash value in the first ESP header, and the second TLV is used to indicate that the receiving device supports assigning the first ESP message to the corresponding CPU core based on the first hash value. This implementation uses TLVs to indicate capabilities, thereby increasing the flexibility of the implementation of this solution.

[0016] In some possible implementations, the first ESP header includes third indication information, where the third indication information is used to indicate that the first ESP header includes the first hash value, so that the receiving device can parse the first ESP header according to the third indication information to obtain the first hash value.

[0017] In some possible implementations, the Security Parameters Index (SPI) field in the first ESP header is used to carry the third indication information, which has good practicality.

[0018] In some possible implementations, the first hash value is generated according to the quintuple of the first data stream, which enriches the implementation of this solution.

[0019] In some possible implementations, the sending device sending the first ESP message to the receiving device includes: the sending device sending the first ESP message to the receiving device through an encrypted tunnel, thereby effectively ensuring the security of data transmission in an insecure network environment.

[0020] In some possible implementations, the encrypted tunnel is an overlay encrypted tunnel. The overlay encrypted tunnel includes but is not limited to an Internet Protocol Security (IPsec) tunnel, a Generic Routing Encapsulation (GRE) tunnel, and a Virtual Extensible Local Area Network (VXLAN) tunnel, which can be well adapted to existing message transmission scenarios.

[0021] In some possible implementations, the sending device and the receiving device are endpoint devices of the encryption tunnel, and the intermediate device is only responsible for forwarding messages, which is conducive to improving message transmission efficiency.

[0022] In a second aspect, embodiments of the present application provide a message transmission method, which is applied to a receiving device. Specifically, the receiving device receives a first ESP message sent by a sending device. The first ESP message is obtained by the sending device performing ESP encapsulation on a first data stream. The first ESP header in the first ESP message includes a first hash value, which is generated based on a triplet of the first data stream. Furthermore, the receiving device distributes the first ESP message to a first CPU core of the receiving device based on the first hash value.

[0023] In some possible implementations, the method further includes: the receiving device receiving a second ESP message sent by the sending device. The second ESP message is obtained by the sending device performing ESP encapsulation on the second data stream, and the second ESP header in the second ESP message includes a second hash value generated based on a triplet of the second data stream. The receiving device distributes the second ESP message to a second CPU core of the receiving device based on the second hash value.

[0024] In some possible implementations, the sequence number field in the first ESP header includes a first hash value.

[0025] In some possible implementations, before the receiving device receives the first ESP message sent by the sending device, the method further includes: the receiving device receiving the first message sent by the sending device, the first message including first indication information, the first indication information being used to indicate that the sending device supports carrying the first hash value in the first ESP header. The receiving device sending a second message to the sending device, the second message including second indication information, the second indication information being used to indicate that the receiving device supports allocating the first ESP message to the first CPU core based on the first hash value.

[0026] In some possible implementations, the first message and the second message are BGP messages.

[0027] In some possible implementations, the attribute or extended community attribute in the first message includes first indication information, and the attribute or extended community attribute in the second message includes second indication information.

[0028] In some possible implementations, the first message and the second message are IKE messages, the IKE header in the first message includes the first indication information, and the IKE header in the second message includes the second indication information.

[0029] In some possible implementations, the next payload field of the IKE header in the first message includes first indication information, and the next payload field of the IKE header in the second message includes second indication information.

[0030] In some possible implementations, the first indication information includes a first TLV, and the second indication information includes a second TLV, the first TLV is used to indicate that the sending device supports carrying the first hash value in the first ESP header, and the second TLV is used to indicate that the receiving device supports allocating the first ESP message to the corresponding CPU core according to the first hash value.

[0031] In some possible implementations, the first ESP header includes third indication information, where the third indication information is used to indicate that the first ESP header includes the first hash value.

[0032] In some possible implementations, the SPI field in the first ESP header includes third indication information.

[0033] In some possible implementations, the first hash value is generated according to the quintuple of the first data stream.

[0034] In some possible implementations, the receiving device receiving the first message sent by the sending device includes: the receiving device receiving the first ESP message sent by the sending device through an encrypted tunnel.

[0035] In some possible implementations, the encrypted tunnel is an overlay encrypted tunnel, and the overlay encrypted tunnel includes but is not limited to an IPsec tunnel, a GRE tunnel, and a VXLAN tunnel.

[0036] In some possible implementations, the sending device and the receiving device are endpoint devices of an encrypted tunnel.

[0037] In a third aspect, embodiments of the present application provide a sending device comprising: a processing unit and a transceiver unit. The processing unit is configured to perform ESP encapsulation on a first data stream to obtain a first ESP message, wherein a first ESP header in the first ESP message includes a first hash value generated based on a triplet of the first data stream. The transceiver unit is configured to send the first ESP message to a receiving device.

[0038] In some possible implementations, the processing unit is further configured to cause the sending device to perform ESP encapsulation on the second data stream to obtain a second ESP message, wherein the second ESP header in the second ESP message includes a second hash value generated by a triplet of the second data stream. The transceiver unit is further configured to send the second ESP message to the receiving device.

[0039] In some possible implementations, the sequence number field in the first ESP header includes a first hash value.

[0040] In some possible implementations, before the sending device performs ESP encapsulation on the first data stream to obtain a first ESP message, the transceiver unit is further configured to send the first message to the receiving device, the first message including first indication information indicating that the sending device supports carrying the first hash value in the first ESP header. The transceiver unit is further configured to receive a second message from the receiving device, the second message including second indication information indicating that the receiving device supports allocating the first ESP message to a corresponding CPU core based on the first hash value.

[0041] In some possible implementations, the first message and the second message are BGP messages.

[0042] In some possible implementations, the attribute or extended community attribute in the first message includes first indication information, and the attribute or extended community attribute in the second message includes second indication information.

[0043] In some possible implementations, the first message and the second message are IKE messages, the IKE header in the first message includes the first indication information, and the IKE header in the second message includes the second indication information.

[0044] In some possible implementations, the next payload field of the IKE header in the first message includes first indication information, and the next payload field of the IKE header in the second message includes second indication information.

[0045] In some possible implementations, the first indication information includes a first TLV, and the second indication information includes a second TLV, the first TLV is used to indicate that the sending device supports carrying the first hash value in the first ESP header, and the second TLV is used to indicate that the receiving device supports allocating the first ESP message to the corresponding CPU core according to the first hash value.

[0046] In some possible implementations, the first ESP header includes third indication information, where the third indication information is used to indicate that the first ESP header includes the first hash value.

[0047] In some possible implementations, the SPI field in the first ESP header includes third indication information.

[0048] In some possible implementations, the first hash value is generated according to the quintuple of the first data stream.

[0049] In some possible implementations, the transceiver unit is specifically configured to send the first ESP message to the receiving device through an encrypted tunnel.

[0050] In some possible implementations, the encrypted tunnel is an IPsec tunnel.

[0051] In some possible implementations, the sending device and the receiving device are endpoint devices of an encrypted tunnel.

[0052] In a fourth aspect, an embodiment of the present application provides a receiving device, comprising: a processing unit and a transceiver unit. The transceiver unit is configured to receive, by the receiving device, a first ESP message sent by a sending device, wherein the first ESP message is obtained by the sending device performing ESP encapsulation on a first data stream, and a first ESP header in the first ESP message includes a first hash value, which is generated based on a triplet of the first data stream. The processing unit is configured to distribute the first ESP message to a first CPU core of the receiving device based on the first hash value.

[0053] In some possible implementations, the transceiver unit is further configured to receive a second ESP message sent by the sending device, wherein the second ESP message is obtained by the sending device performing ESP encapsulation on the second data stream, and a second ESP header in the second ESP message includes a second hash value generated based on a triplet of the second data stream. The processing unit is further configured to distribute the second ESP message to a second CPU core of the receiving device based on the second hash value.

[0054] In some possible implementations, the sequence number field in the first ESP header includes a first hash value.

[0055] In some possible implementations, before the receiving device receives the first ESP message sent by the sending device, the transceiver unit is further configured to receive the first message sent by the sending device, the first message including first indication information, the first indication information being used to indicate that the sending device supports carrying the first hash value in the first ESP header. The transceiver unit is further configured to send a second message to the sending device, the second message including second indication information being used to indicate that the receiving device supports allocating the first ESP message to the first CPU core based on the first hash value.

[0056] In some possible implementations, the first message and the second message are BGP messages.

[0057] In some possible implementations, the attribute or extended community attribute in the first message includes first indication information, and the attribute or extended community attribute in the second message includes second indication information.

[0058] In some possible implementations, the first message and the second message are IKE messages, the IKE header in the first message includes the first indication information, and the IKE header in the second message includes the second indication information.

[0059] In some possible implementations, the next payload field of the IKE header in the first message includes first indication information, and the next payload field of the IKE header in the second message includes second indication information.

[0060] In some possible implementations, the first indication information includes a first TLV, and the second indication information includes a second TLV, the first TLV is used to indicate that the sending device supports carrying the first hash value in the first ESP header, and the second TLV is used to indicate that the receiving device supports allocating the first ESP message to the corresponding CPU core according to the first hash value.

[0061] In some possible implementations, the first ESP header includes third indication information, where the third indication information is used to indicate that the first ESP header includes the first hash value.

[0062] In some possible implementations, the SPI field in the first ESP header includes third indication information.

[0063] In some possible implementations, the first hash value is generated according to the quintuple of the first data stream.

[0064] In some possible implementations, the transceiver unit is specifically configured to receive the first ESP message sent by the sending device through an encrypted tunnel.

[0065] In some possible implementations, the encrypted tunnel is an IPsec tunnel.

[0066] In some possible implementations, the sending device and the receiving device are endpoint devices of an encrypted tunnel.

[0067] In the fifth aspect, an embodiment of the present application provides a sending device, which includes at least one processor and an interface, the at least one processor and the interface are interconnected, the interface is used to receive and send signals, and the at least one processor is used to execute the method described in any embodiment of the first aspect.

[0068] In a sixth aspect, an embodiment of the present application provides a receiving device, comprising at least one processor and an interface, wherein the at least one processor is interconnected with the interface. The at least one processor includes, but is not limited to, a CPU, a network processor (NP), and a built-in processor of a network card, wherein the CPU includes multiple CPU cores. The interface is used to receive and send signals, and the at least one processor is used to execute the method described in any embodiment of the second aspect. In one possible scenario, the built-in processor of the network card is used to distribute the received ESP message to the corresponding CPU core. In another possible scenario, the network processor is used to distribute the received ESP message to the corresponding CPU core.

[0069] It should be noted that the sending device and receiving device described in the third to sixth aspects above can specifically be network devices such as switches and routers, or can be a component of a network device, such as a single board or line card on the network device, or can be a functional module on the network device, or can be a chip used to implement the embodiments of the present application, and the specific details are not limited here. When the sending device and receiving device are chips, the interface in the sending device and receiving device can be an interface circuit in the chip, and the processor in the sending device and receiving device can be a processing circuit in the chip for performing processing operations.

[0070] In a seventh aspect, the present application provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed by a computer, the method described in any one of the embodiments of the first and second aspects is implemented.

[0071] In an eighth aspect, the present application provides a computer program product, which includes program instructions. When the computer program product is executed, it is used to implement the method introduced in any embodiment of the first and second aspects above.

[0072] In this embodiment of the present application, a transmitting device performs ESP encapsulation on a first data stream to be transmitted to obtain a first ESP message. The first data stream is within the encryption scope of the first ESP message, meaning that the payload and IP information of the first data stream are encrypted and hidden. The transmitting device generates a first hash value based on the triplet or quintuple of the first data stream and designs the first ESP header of the first ESP message so that it can carry the first hash value. Since the first ESP header is not within the encryption scope of the first ESP message, upon receiving the first ESP message, the receiving device can distribute the first ESP message to the corresponding CPU core based on the first hash value in the first ESP header. Based on this design, the receiving device can distribute each received ESP message to the corresponding CPU core, achieving multi-core load balancing and ensuring that overall performance is not limited by the performance bottleneck of a single core. Furthermore, since the hash value is generated based on the triplet or quintuple of each data stream, the hash value can effectively distinguish ESP messages encapsulating different data streams. Each CPU core only needs to process the corresponding ESP message, rather than all ESP messages. This reduces the processing overhead of each CPU core and improves overall processing efficiency and forwarding performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] FIG1 is a schematic diagram of a network architecture used in an embodiment of the present application;

[0074] FIG2 is a schematic diagram of another network architecture used in an embodiment of the present application;

[0075] FIG3 is a flow chart of a message transmission method according to an embodiment of the present application;

[0076] FIG4 is a schematic diagram of the structure of an ESP message in an embodiment of the present application;

[0077] FIG5 is a schematic structural diagram of an ESP head in an embodiment of the present application;

[0078] FIG6 is a schematic diagram of an application scenario in which a receiving device performs ESP message distribution in an embodiment of the present application;

[0079] FIG7 is another flow chart of a message transmission method according to an embodiment of the present application;

[0080] FIG8 is a schematic structural diagram of a sending device according to an embodiment of the present application;

[0081] FIG9 is a schematic structural diagram of a receiving device according to an embodiment of the present application;

[0082] FIG10 is a schematic structural diagram of another sending device according to an embodiment of the present application;

[0083] FIG11 is a schematic structural diagram of another receiving device according to an embodiment of the present application;

[0084] FIG12 is a schematic structural diagram of a communication system in an embodiment of the present application. DETAILED DESCRIPTION

[0085] Embodiments of the present application provide a message transmission method and related apparatus. A sending device generates a first hash value based on a triplet or quintuple of a first data stream and designs a first ESP header of a first ESP message so that the first ESP header can carry the first hash value. After receiving the first ESP message, a receiving device can distribute the first ESP message to a corresponding CPU core based on the first hash value in the first ESP header. Based on this design, the receiving device can distribute each received ESP message to the corresponding CPU core, achieving multi-core load balancing and ensuring that overall performance is not limited by the performance bottleneck of a single core.

[0086] It should be noted that the terms "first", "second", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0087] To facilitate understanding, the technical terms involved in the embodiments of the present application are first explained below.

[0088] (1) Internet Protocol Security (IPsec): IPsec is a collection of protocols and services that provide security for IP networks. It is a commonly used technology in virtual private networks (VPNs). Because IP packets themselves do not integrate any security features, IP data packets transmitted over public networks such as the Internet may be at risk of being forged, stolen, or tampered with. VPN is a technology that establishes a private network on a public network. It is called a virtual network mainly because the two nodes in a VPN do not use end-to-end physical links like traditional private networks. Instead, they are logical networks built on public networks such as the Internet, and user data is transmitted through logical links.

[0089] Figure 1 is a schematic diagram of a network architecture used in an embodiment of the present application. As shown in Figure 1, two communicating parties establish an IPsec tunnel through IPsec. IP data packets are encrypted and transmitted through the IPsec tunnel, effectively ensuring the security of data transmission in insecure network environments such as the Internet. IPsec VPN protects point-to-point communications. Through IPsec VPN, secure tunnel connections can be established between hosts, between hosts and network security gateways, or between network security gateways (such as routers and firewalls).

[0090] (2) Internet Key Exchange (IKE): IKE is an important protocol related to IPsec, mainly used for Security Association (SA) negotiation and key management.

[0091] (3) Encapsulating Security Payload (ESP): ESP is an important protocol related to IPsec. In addition to authenticating the origin and integrity of IP packets, ESP also encrypts data. ESP adds an ESP header (also referred to as the ESP header) to the end of the standard IP header of each data packet, and also adds an ESP trailer and ESP authentication data to the end of the packet.

[0092] (4) Border Gateway Protocol (BGP): BGP is a routing protocol used to exchange Network Layer Reachability Information (NLRI) between routing domains. Because different administrative agencies control their respective routing domains, routing domains are often referred to as Autonomous Systems (AS). The current Internet is a large network consisting of multiple interconnected autonomous systems. BGP, as the de facto external routing protocol standard for the Internet, is widely used among Internet Service Providers (ISPs).

[0093] (5) Software Defined Wide Area Network (SD-WAN): SD-WAN connects an enterprise's branches, headquarters, and cloud networks, allowing applications to select the optimal transmission path across different hybrid links, providing a high-quality cloud experience. Hybrid links include Multiprotocol Label Switching (MPLS) and the Internet.

[0094] (6) Overlay network: An overlay network is one or more virtual logical networks built on the same underlying network using network virtualization technology. Although different overlay networks share the equipment and lines in the underlying network, the services in the overlay network are decoupled from the physical networking and interconnection technologies in the underlying network. The multi-instance overlay network can serve different services of the same tenant (such as multiple departments) as well as different tenants. It is the core networking technology used in solutions such as SD-WAN and data centers.

[0095] Figure 2 is a schematic diagram of another network architecture used in an embodiment of the present application. As shown in Figure 2, branches, headquarters, and cloud networks in the SD-WAN interconnect and communicate through overlay tunnels established by edge devices. Overlay tunnels typically use ESP encryption to ensure data security. The IP header information and payload of the data stream are encrypted and hidden within the inner layer of the tunnel.

[0096] The message transmission method provided in the embodiment of the present application is described in detail below. Figure 3 is a flow chart of the message transmission method in the embodiment of the present application. As shown in Figure 3, the message transmission method includes the following steps. It should be understood that the sending device and the receiving device in the embodiment of the present application are named based on the data flow direction, and do not limit the functions of the devices. For example, the sending device may also have a receiving function, and the receiving device may also have a sending function. The present application does not limit the specific types of the sending device and the receiving device. For example, they may be routers, switches, edge devices, etc.

[0097] 101. A sending device performs ESP encapsulation on a first data stream to obtain a first ESP message.

[0098] In an embodiment of the present application, the sending device must first perform ESP encapsulation on the first data stream to be sent to obtain a first ESP message to achieve encryption of the first data stream. ESP encapsulation here can also be referred to as ESP encryption. Unlike traditional ESP messages, the sending device generates a first hash value (hash) based on the triplet or quintuple of the first data stream, and carries the first hash value through the first ESP header of the first ESP message. The hash value here can also be referred to as a hash value. It should be understood that the first data stream includes multiple messages with the same triplet, wherein the triplet includes a source Internet Protocol (IP) address and a destination IP address, as well as one of a source port or a destination port. The quintuple includes a triplet, for example, a quintuple includes a source IP address, a destination IP address, a source port, a destination port, and a protocol identifier.

[0099] The following further introduces the ESP message. For more details about the ESP message, please refer to the protocol RFC4303.

[0100] Figure 4 is a schematic diagram of the structure of an ESP message in an embodiment of the present application. As shown in Figure 4, the ESP message includes an outer IP header, an ESP header, a raw IP header, a transport layer protocol header, data, an ESP trailer, and ESP authentication data. The transport layer protocol header includes, but is not limited to, a Transmission Control Protocol (TCP) header and a User Datagram Protocol (UDP) header. It should be understood that the data stream's triplet information is located in the raw IP header, while the data stream's quintuple information is distributed across the raw IP header and the transport layer protocol header.

[0101] As can be seen, since the ESP header is not within the encryption range of the ESP message, after receiving the first ESP message, the receiving device can distribute the first ESP message to the corresponding CPU core based on the first hash value in the first ESP header. This eliminates the need for the receiving device to decapsulate the first ESP message before distributing the first data stream to the corresponding CPU core. This avoids the complex operation of first decapsulating the first ESP message through the CPU core and then redistributing the first data stream.

[0102] Figure 5 is a schematic diagram of the structure of the ESP header in an embodiment of the present application. As shown in Figure 5, the ESP header includes the following fields: a Security Parameters Index (SPI) and a Sequence Number. The Sequence Number field in the ESP header is primarily used for anti-replay functionality. In an embodiment of the present application, the anti-replay functionality can be disabled, and the first hash value can be carried by the Sequence Number field in the first ESP header. Furthermore, the first ESP header also carries indication information, which indicates that the Sequence Number field in the first ESP header carries the first hash value. In one possible implementation, the indication information is carried by the SPI field in the first ESP header. For example, if the high-order bit 31 of the SPI field is 1, it can indicate that the Sequence Number field in the first ESP header carries the first hash value. If the high-order bit 31 of the SPI field is 0, it can indicate that the first ESP header retains the original Sequence Number field.

[0103] 102. The sending device sends a first ESP message to the receiving device.

[0104] Specifically, the sending device sends the first ESP message to the receiving device through an encrypted tunnel. For example, the encrypted tunnel can be an overlay encrypted tunnel. Overlay encrypted tunnels include but are not limited to IPsec tunnels, generic routing encapsulation (GRE) tunnels, and virtual extensible local area network (VXLAN) tunnels. In one possible scenario, the sending device and the receiving device are the two endpoint devices of the encrypted tunnel.

[0105] 103. The receiving device distributes the first ESP message to the first CPU core according to the first hash value in the first ESP header of the first ESP message.

[0106] Specifically, after receiving the first ESP message from the sending device, the receiving device parses the first ESP header of the first ESP message to obtain a first hash value, and then distributes the first ESP message to the first CPU core of the receiving device based on the first hash value. Furthermore, the first CPU core can perform ESP decapsulation and forwarding of the first data stream on the first ESP message. It should be understood that the receiving device can pre-configure the correspondence between the hash value carried by the ESP header and its own CPU core. For example, the receiving side scaling (RSS) network card hash module of the receiving device learns that the serial number field of the first ESP header carries the first hash value by identifying the indication information carried by the SPI field in the first ESP header, and then searches for the pre-configured correspondence based on the first hash value and distributes the first ESP message to the corresponding first CPU core.

[0107] 104. The sending device performs ESP encapsulation on the second data stream to obtain a second ESP message.

[0108] Specifically, the sending device performs ESP encapsulation on the second data stream to be transmitted to obtain a second ESP message, thereby encrypting the second data stream. The sending device generates a second hash value based on the triplet or quintuple of the second data stream and carries the second hash value in a second ESP header of the second ESP message. It should be understood that the specific implementation of step 104 is similar to that of step 101 above. The specific implementation of carrying the second hash value in the second ESP header can be found in the description of step 101 above and will not be further elaborated here.

[0109] 105. The sending device sends a second ESP message to the receiving device.

[0110] Specifically, the sending device sends the second ESP message to the receiving device through an encrypted tunnel. For example, the encrypted tunnel may be an Overlay encrypted tunnel.

[0111] 106. The receiving device distributes the second ESP message to the second CPU core according to the second hash value in the second ESP header of the second ESP message.

[0112] Specifically, after receiving the second ESP message from the sending device, the receiving device parses the second ESP header of the second ESP message to obtain a second hash value, and then distributes the second ESP message to the second CPU core of the receiving device based on the second hash value. It should be understood that step 106 is similar to the specific implementation of step 103 above. The specific implementation of the receiving device distributing the second ESP message to the second CPU core based on the second hash value can be referred to the description of step 103 above and will not be repeated here.

[0113] It should be noted that the embodiments of the present application do not limit the specific number of CPU cores in the receiving device and the specific correspondence between the hash value and the CPU core. For example, if the number of CPU cores is greater than or equal to the number of data streams to be processed, the hash value generated according to each data stream has a unique corresponding CPU core. For another example, if the number of CPU cores is less than the number of data streams to be processed, each CPU core can correspond to multiple hash values ​​generated according to different data streams. A possible application scenario is described below.

[0114] FIG6 is a schematic diagram of an application scenario in which a receiving device performs ESP message distribution in an embodiment of the present application. As shown in FIG6 , the receiving device receives a total of eight ESP messages, ESP message 1 to ESP message 8, and the receiving device includes four CPU cores, CPU core 1 to CPU core 4. Each data stream encapsulated in an ESP message has a corresponding hash value. The receiving device is pre-configured with a correspondence between the four CPU cores and the eight hash values. For example, CPU core 1 corresponds to hash value 1 and hash value 2, CPU core 2 corresponds to hash value 3 and hash value 4, CPU core 3 corresponds to hash value 5 and hash value 6, and CPU core 4 corresponds to hash value 7 and hash value 8. Specifically, the RSS network card hash module of the receiving device parses the ESP header of each received ESP message to obtain a hash value, and then searches the pre-configured correspondence based on the hash value to allocate ESP message 1 and ESP message 2 to CPU core 1, allocate ESP message 3 and ESP message 4 to CPU core 2, allocate ESP message 5 and ESP message 6 to CPU core 3, and allocate ESP message 7 and ESP message 8 to CPU core 4.

[0115] It should be noted that in actual applications, the sending and receiving devices can also conduct capability negotiation before transmitting ESP messages to determine whether both parties support hash distribution of ESP messages using the hash value carried in the ESP header. This negotiation process is described in detail below. It should be understood that this describes capability negotiation between the sending and receiving devices. Additionally, capabilities can be manually configured for the sending and receiving devices through the network controller.

[0116] Figure 7 is another flow chart of the message transmission method in the embodiment of the present application. As shown in Figure 7, the message transmission method includes the following steps.

[0117] 107. The sending device sends a first message to the receiving device.

[0118] In an embodiment of the present application, a sending device first sends a first message to a receiving device, wherein the first message includes first indication information, which is used to indicate that the sending device supports carrying a hash value in the ESP header of the ESP message. For example, the first indication information includes a bit. If the value of this bit is 0, it indicates that the sending device does not support carrying a hash value in the ESP header of the ESP message. If the value of this bit is 1, it indicates that the sending device supports carrying a hash value in the ESP header of the ESP message. It should be understood that the hash value is specifically generated based on the triple or quintuple of the data stream encapsulated in the ESP message. For the specific implementation of carrying a hash value in the ESP header, please refer to the relevant description of step 101 in the embodiment shown in Figure 3 above, and will not be repeated here.

[0119] 108. The receiving device sends a second message to the sending device.

[0120] In an embodiment of the present application, after the receiving device receives the first message from the sending device, the receiving device sends a second message to the sending device. The second message is used to indicate that the receiving device supports allocating the received ESP message to the corresponding CPU core based on the hash value carried by the ESP header. For example, the second indication information includes a bit. If the value of the bit is 0, it indicates that the receiving device does not support allocating the received ESP message to the corresponding CPU core based on the hash value carried by the ESP header. If the value of the bit is 1, it indicates that the receiving device supports allocating the received ESP message to the corresponding CPU core based on the hash value carried by the ESP header. It should be understood that after the sending device receives the second message sent by the receiving device, it can be determined that the two parties have completed the capability negotiation, and then the sending device starts to send the ESP message to the receiving device, and the receiving device allocates the received ESP message to the corresponding CPU core based on the hash value carried by the ESP header. For specific operations, please refer to the introduction of steps 101 to 106 in the embodiment shown in Figure 3 above, and will not be repeated here.

[0121] Optionally, the first message and the second message may include a type-length value (TLV), that is, whether the above-mentioned capability is supported is indicated by adding a TLV in the first message and the second message. For example, the first indication information includes a first TLV, and the second indication information includes a second TLV. The first field in the first TLV is used to indicate whether the sending device supports carrying a hash value in the ESP header of the ESP message. The second field in the second TLV is used to indicate whether the receiving device supports allocating the received ESP message to the corresponding CPU core based on the hash value carried in the ESP header.

[0122] It should be noted that the sending device and the receiving device may implement the above-mentioned capability negotiation in a variety of different ways. Several possible implementations are described below.

[0123] In one possible implementation, both the first message and the second message adopt the format of a BGP message. For example, the first message includes a first attribute, and the first indication information is carried in the first attribute; the second message includes a second attribute, and the second indication information is carried in the second attribute. For another example, the first message includes a first extended community attribute, and the first indication information is carried in the first extended community attribute; the second message includes a second extended community attribute, and the second indication information is carried in the second extended community attribute. Optionally, a first TLV can be added to the first attribute or the first extended community attribute, and a second TLV can be added to the second attribute or the second extended community attribute.

[0124] In another possible implementation, both the first and second messages utilize a message format based on the IPsec protocol. For example, both the first and second messages utilize an IKE message format. The IKE header in the first message includes first indication information, and the IKE header in the second message includes second indication information. Optionally, the next payload field of the IKE header in the first message includes the first indication information, and the next payload field of the IKE header in the second message includes the second indication information. Bits 128-255 of the next payload field of the IKE header can be used to add custom attribute negotiation. As an example, the 128th bit of the next payload field of the IKE header can be used to define whether both parties support hash distribution of ESP messages using the hash value carried in the ESP header. For example, if the 128th bit of the next payload field of the IKE header is 0, this capability is not supported. If the 128th bit of the next payload field of the IKE header is 1, this capability is supported. It should be understood that for more detailed information regarding negotiation via IKE messages, reference can be made to the protocol RFC4306.

[0125] As can be seen from the above description, the sending device performs ESP encapsulation on the first data stream to be transmitted to obtain a first ESP message. The first data stream is within the encryption scope of the first ESP message, meaning that the payload and IP information of the first data stream are encrypted and concealed. The sending device generates a first hash value based on the triplet or quintuple of the first data stream and designs the first ESP header of the first ESP message so that it can carry the first hash value. Since the first ESP header is not within the encryption scope of the first ESP message, the receiving device can distribute the first ESP message to the corresponding CPU core based on the first hash value in the first ESP header after receiving the first ESP message. Based on this design, the receiving device can distribute each received ESP message to the corresponding CPU core, achieving multi-core load balancing and eliminating the overall performance bottleneck of a single core. Furthermore, because the hash value is generated based on the triplet or quintuple of each data stream, the hash value can effectively distinguish ESP messages encapsulating different data streams. Each CPU core only needs to process the corresponding ESP message, rather than all ESP messages. This reduces the processing overhead of each CPU core and improves overall processing efficiency and forwarding performance.

[0126] Figure 8 is a schematic diagram of the structure of a sending device in an embodiment of the present application. As shown in Figure 8, the sending device includes a processing unit 201 and a transceiver unit 202. The processing unit 201 is used to perform the operations of steps 101 and 104 above, and the transceiver unit is used to perform the operations of steps 102, 105, 107 and 108 above. It should be understood that the sending device provided in the embodiment of the present application can also be implemented in other ways. For example, the unit division in the above-mentioned sending device is only a logical functional division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system. In addition, the functional units in the various embodiments of the present application can be integrated into one processing unit, or can be independent physical units, or two or more functional units can be integrated into one processing unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0127] Figure 9 is a schematic diagram of the structure of a receiving device in an embodiment of the present application. As shown in Figure 9, the receiving device includes a processing unit 301 and a transceiver unit 302. The processing unit 301 is used to perform the operations of step 103 and step 106 above, and the transceiver unit is used to perform the operations of step 102, step 105, step 107 and step 108 above. It should be understood that the receiving device provided in the embodiment of the present application can also be implemented in other ways. For example, the unit division in the above-mentioned sending device is only a logical functional division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system. In addition, the functional units in the various embodiments of the present application can be integrated into one processing unit, or can be independent physical units, or two or more functional units can be integrated into one processing unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0128] Figure 10 is a schematic diagram of the structure of another sending device in an embodiment of the present application. As shown in Figure 10, the sending device includes at least one processor 401 (such as 401a, 401b and 401c) and an interface 402, and at least one processor 401 and the interface 402 are interconnected via a line. For example, at least one processor 401 includes but is not limited to a CPU 401a and a network processor (NP) 401b. The interface 402 is used to receive signals from other devices outside the sending device or send signals to other devices outside the sending device. For example, the interface 402 can be integrated in a network card, or the interface 402 can also be set independently of the network card. Among them, the network card can also be referred to as a network interface card (NIC), and the at least one processor 401 can also include a built-in processor 401c of the network card, and the built-in processor 401c of the network card is used to control the network card and communicate with other processors such as the CPU 401a and the network processor 401b. Optionally, the sending device can also include a memory 403, wherein the memory 403 is used to store program instructions and data.

[0129] In one possible implementation, the CPU 401a may be used to implement control plane operations. For example, the capability negotiation operation in the embodiment of the present application is performed by the CPU 401a. The CPU 401a may also be used to implement data forwarding plane operations. For example, the encapsulation and transmission of ESP messages in the embodiment of the present application may be performed by the CPU 401a. The network processor 401b is primarily used to implement data forwarding plane operations. For example, the encapsulation and transmission of ESP messages in the embodiment of the present application may also be performed by the network processor 401b.

[0130] Figure 11 is a schematic diagram of the structure of another receiving device in an embodiment of the present application. As shown in Figure 11, the receiving device includes at least one processor 501 (such as 501a, 501b and 501c) and an interface 502, and at least one processor 501 and the interface 502 are interconnected via a line. For example, at least one processor 501 includes but is not limited to a CPU 501a and a network processor 501b. The interface 502 is used to receive signals from other devices outside the receiving device or send signals to other devices outside the receiving device. For example, the interface 502 can be integrated into the network card, or the interface 502 can also be set independently of the network card. Among them, at least one processor 501 can also include a built-in processor 501c of the network card, and the built-in processor 501c of the network card is used to control the network card and communicate with other processors such as the CPU 501a and the network processor 501b. Optionally, the receiving device may also include a memory 503, wherein the memory 503 is used to store program instructions and data.

[0131] In one possible implementation, CPU 501a can be used to implement control plane operations. For example, the capability negotiation operation in the embodiment of the present application is performed by CPU 501a. In the embodiment of the present application, CPU 501a includes multiple CPU cores. As an example, the built-in processor 501c of the network card is used to parse the ESP header of a received ESP message and send the ESP message to the corresponding CPU core in CPU 501a. As another example, the network processor 501b is used to parse the ESP header of a received ESP message and send the ESP message to the corresponding CPU core.

[0132] It should be noted that the transmitting device and receiving device in the embodiments of the present application can specifically be a network device such as a switch or router, or a component of a network device, such as a single board or line card on the network device, or a functional module on the network device, or a chip used to implement the embodiments of the present application, and the specific details are not limited here. When the transmitting device and receiving device are chips, the interface in the transmitting device and receiving device can be an interface circuit in the chip, and the processor in the transmitting device and receiving device can be a processing circuit in the chip for performing processing operations.

[0133] An embodiment of the present application further provides a computer-readable storage medium, comprising a program or instructions, which, when executed on a computer, enables the computer to execute the method described in the above method embodiment.

[0134] It should be understood that the processors mentioned in the embodiments of the present application can be implemented by hardware or software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor that is implemented by reading software code stored in a memory. The memory can exist independently and be connected to the processor, or the memory can be integrated with the processor.

[0135] As an example, the processor in the embodiments of the present application may be a central processing unit (CPU) or a network processor (NP), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor may be a microprocessor, any conventional processor, or a processing circuit that implements a specific function.

[0136] In the embodiments of the present application, the memory may be a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a mobile hard disk, a CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may also be an integral part of the processor. The processor and the storage medium may be located in an ASIC. In addition, the ASIC may be located in a network device or a terminal device. Of course, the processor and the storage medium may also exist in a network device or a terminal device as discrete components.

[0137] In the above embodiments, all or part of them can be implemented by software, hardware, firmware or any combination thereof.

[0138] When implemented using hardware, the data processing method provided in the embodiments of the present application may be implemented without reading software code or instructions. For example, it may be implemented by a CPU, DSP, ASIC, FPGA, other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0139] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a terminal device, or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it may also be an optical medium, such as a digital versatile disc (DVD); it may also be a semiconductor medium, such as a solid state disk (SSD).

[0140] Figure 12 is a schematic diagram of the structure of a communication system according to an embodiment of the present application. As shown in Figure 12, the communication system includes a transmitting device 601 and a receiving device 602. The transmitting device 601 is used to perform the operations of the transmitting device in the embodiment shown in Figure 3 or Figure 7, and the receiving device 602 is used to perform the operations of the receiving device in the embodiment shown in Figure 3 or Figure 7.

[0141] Finally, it should be noted that the above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A message transmission method, characterized in that: include: The sending device performs Encapsulation Security Payload (ESP) encapsulation on the first data stream to obtain a first ESP message, wherein a first ESP header in the first ESP message includes a first hash value, and the first hash value is generated according to a triplet of the first data stream; The sending device sends the first ESP message to the receiving device.

2. The method according to claim 1, characterized in that The method further comprises: The sending device performs ESP encapsulation on the second data stream to obtain a second ESP message, wherein the second ESP header in the second ESP message includes a second hash value, and the second hash value is generated by the triplet of the second data stream; The sending device sends the second ESP message to the receiving device.

3. The method according to claim 1 or 2, characterized in that The sequence number field in the first ESP header includes the first hash value.

4. The method according to any one of claims 1 to 3, characterized in that Before the sending device performs ESP encapsulation on the first data stream to obtain the first ESP message, the method further includes: The sending device sends a first message to the receiving device, where the first message includes first indication information, where the first indication information is used to indicate that the sending device supports carrying the first hash value in the first ESP header; The sending device receives a second message from the receiving device, where the second message includes second indication information, and the second indication information is used to indicate that the receiving device supports allocating the first ESP message to a corresponding central processing unit (CPU) core according to the first hash value.

5. The method according to any one of claims 1 to 4, characterized in that The first ESP header includes third indication information, where the third indication information is used to indicate that the first ESP header includes the first hash value.

6. The method according to any one of claims 1 to 5, characterized in that The sending device sending the first ESP message to the receiving device includes: The sending device sends the first ESP message to the receiving device through an encrypted tunnel.

7. The method according to claim 6, characterized in that The encrypted tunnel is an encrypted tunnel superimposed with an Overlay, and the Overlay encrypted tunnel includes one of an Internet Security Protocol IPsec tunnel, a Generic Routing Encapsulation GRE tunnel, and a Virtual Extended Local Area Network VXLAN tunnel.

8. A message transmission method, characterized in that: include: The receiving device receives a first Encapsulating Security Payload (ESP) message sent by the sending device, wherein the first ESP message is obtained by the sending device performing ESP encapsulation on a first data stream, and a first ESP header in the first ESP message includes a first hash value, and the first hash value is generated based on a triplet of the first data stream; The receiving device distributes the first ESP message to a first CPU core of the receiving device according to the first hash value.

9. The method according to claim 8, characterized in that The method further comprises: The receiving device receives a second ESP message sent by the sending device, wherein the second ESP message is obtained by the sending device performing ESP encapsulation on a second data stream, and a second ESP header in the second ESP message includes a second hash value, and the second hash value is generated based on a triplet of the second data stream; The receiving device distributes the second ESP message to the second CPU core of the receiving device according to the second hash value.

10. The method according to claim 8 or 9, characterized in that The sequence number field in the first ESP header includes the first hash value.

11. The method according to any one of claims 8 to 10, characterized in that Before the receiving device receives the first ESP message sent by the sending device, the method further includes: The receiving device receives a first message sent by the sending device, where the first message includes first indication information, where the first indication information is used to indicate that the sending device supports carrying the first hash value in the first ESP header; The receiving device sends a second message to the sending device, where the second message includes second indication information, and the second indication information is used to indicate that the receiving device supports allocating the first ESP message to the first CPU core according to the first hash value.

12. The method according to any one of claims 8 to 11, characterized in that The first ESP header includes third indication information, where the third indication information is used to indicate that the first ESP header includes the first hash value.

13. The method according to any one of claims 8 to 12, characterized in that The receiving device receiving the first message sent by the sending device includes: The receiving device receives the first ESP message sent by the sending device through an encrypted tunnel.

14. The method according to claim 13, characterized in that The encrypted tunnel is an encrypted tunnel superimposed with an Overlay, and the Overlay encrypted tunnel includes one of an Internet Security Protocol IPsec tunnel, a Generic Routing Encapsulation GRE tunnel, and a Virtual Extended Local Area Network VXLAN tunnel.

15. A sending device, characterized in that: include: processing unit and transceiver unit; The processing unit is configured to: perform Encapsulation Security Payload (ESP) encapsulation on the first data stream to obtain a first ESP message, wherein a first ESP header in the first ESP message includes a first hash value, and the first hash value is generated according to a triplet of the first data stream; The transceiver unit is configured to send the first ESP message to a receiving device.

16. A receiving device, characterized in that: include: processing unit and transceiver unit; The transceiver unit is configured to: receive, by a receiving device, a first Encapsulating Security Payload (ESP) message sent by a sending device, wherein the first ESP message is obtained by the sending device performing ESP encapsulation on a first data stream, and a first ESP header in the first ESP message includes a first hash value, and the first hash value is generated based on a triplet of the first data stream; The processing unit is configured to distribute the first ESP message to a first CPU core of the receiving device according to the first hash value.

17. A sending device, characterized in that: include: The sending device includes at least one processor and an interface, the at least one processor and the interface are connected to each other, the interface is used to receive and send signals, and the at least one processor is used to execute the method according to any one of claims 1 to 7.

18. A receiving device, characterized in that: include: The receiving device includes at least one processor and an interface, the at least one processor and the interface are interconnected, the central processing unit CPU in the at least one processor includes multiple CPU cores, the interface is used to receive and send signals, and the at least one processor is used to execute the method as described in any one of claims 8 to 14.

19. A communication system, characterized in that: include: A sending device and a receiving device, wherein the sending device is used to execute the method according to any one of claims 1 to 7, and the receiving device is used to execute the method according to any one of claims 8 to 14.

20. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed by a computer, the method according to any one of claims 1 to 14 is implemented.

21. A computer program product, characterized in that A computer program is stored, and when the computer program is executed by a computer, the method according to any one of claims 1 to 14 is performed.

Images