Techniques for mitigating NIC KTLS denial of service attacks

Abstract

In various embodiments, the kernel uses the NIC to communicate encrypted data over the TCP connection. The kernel causes the NIC to encrypt the first data according to a hardware-based transport layer security (TLS) offload mode to generate a first transport control protocol (TCP) packet. The kernel computes first legacy information in response to the first TCP packet failing to reach the first device over the first TCP connection; the kernel assigns the first TCP connection to a software-based TLS offload mode, rather than a hardware-based TLS offload mode, based on the first legacy information. The kernel then encrypts the second data according to the software-based TLS offload mode to generate a second TCP packet, and then the kernel causes the NIC to transmit the second TCP packet to the first device over the first TCP connection.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. patent application serial number 17 / 878,757, filed on August 1, 2022, which is hereby incorporated by reference herein. Technical Field

[0003] Various embodiments relate generally to computer networking and network security, and more specifically, to techniques for mitigating NIC kTLS denial of service attacks. Background Art

[0004] A typical streaming media service provides access to a library of media files that can be viewed on a range of different client devices. In many implementations, to access a given media file, a client application executing on a client device opens a Transmission Control Protocol (TCP) connection with a media delivery application executing on a server device. A TCP connection provides reliable and ordered bidirectional data delivery using TCP packets. Notably, in order to provide reliable data delivery, TCP implementations are configured to detect lost TCP packets and then retransmit those TCP packets once detected.

[0005] After the TCP connection has been opened, the client application will often also establish a transport layer security (TLS) session with the media delivery application. A TLS session enables end-to-end secure data delivery over a TCP connection while protecting user privacy. To deliver data during a TLS session, the data is encrypted and optionally authenticated to generate a TLS record. The TLS record is then divided into TCP segments, and each TCP segment is encapsulated in a different TCP packet for transmission over the associated TCP connection.

[0006] In one approach to generating TLS records, data is encrypted using a kernel running on a processor of a server device. The kernel first performs one or more direct memory access (DMA) operations to copy data from a non-volatile disk storage device to a volatile "system" memory assigned to an associated processor. The kernel then reads the data from the system memory and divides the data into different data fragments. The kernel encrypts and optionally authenticates each data fragment to generate a corresponding TLS record. After generating the TLS record, the kernel writes the TLS record back to the system memory, divides the TLS record into an arbitrary number of TCP segments, and encapsulates each TCP segment in a different TCP packet. A network interface controller (NIC) copies each TCP packet from the system memory for transmission over an associated TCP connection.

[0007] One disadvantage of this approach (referred to herein as "software-based TLS offloading" and "software kernel TLS (kTLS)") is that generating and encrypting TLS records can consume significant processing resources and system memory bandwidth. As used herein, "system memory bandwidth" refers to the rate at which a processor or peripheral device can access system memory using DMA. As a result, software kTLS can significantly reduce the overall performance of a server device, thereby reducing the amount of network traffic that can be served by the server device.

[0008] In another approach to generating TLS records, TLS data encryption is offloaded from the processor to the NIC, which is capable of encrypting and optionally authenticating data while transmitting TCP packets over a TCP connection. In some implementations, the kernel generates a "plaintext" TLS record in which unencrypted data is encapsulated, rather than encrypted data. After generating the plaintext TLS record, the kernel writes the plaintext TLS record back to system memory, divides the plaintext TLS record into an arbitrary number of plaintext TCP segments, and encapsulates each plaintext TCP segment in a different plaintext TCP packet. The NIC copies each plaintext TCP packet from system memory via DMA, encrypts the unencrypted data encapsulated in the plaintext TCP segment corresponding to the plaintext TCP packet to generate a TCP packet, and then transmits the TCP packet over an associated TCP connection. Importantly, the amount of processing resources and system memory bandwidth consumed by this particular approach (referred to herein as "hardware-based TLS offload," "NIC kTLS," and "hardware inline kTLS") can be significantly reduced compared to the amount of processing resources and system memory bandwidth consumed by a software kTLS approach.

[0009] However, one disadvantage of NIC kTLS is that in order to retransmit a lost TCP packet, the NIC needs to read from system memory the portion of the plaintext TLS record up to and including the encrypted and encapsulated plaintext TCP segment in the lost TCP packet in order to recreate the correct encryption state. Since the loss of a single TCP packet may cause the NIC to read and re-encrypt the entire plaintext TLS record, NIC kTLS is susceptible to a particular type of denial-of-service (DoS) attack in which an attacker repeatedly forges lost TCP packets across multiple TLS sessions that correspond to TCP segments near or at the end of the TLS record. This type of attack effectively forces the NIC to re-read and re-encrypt the entire plaintext TLS record, which may significantly reduce the bus bandwidth between the NIC and system memory.

[0010] As discussed above, what is needed in the art is a more efficient technique for generating TLS records when transmitting TCP packets over a TCP connection. Summary of the invention

[0011] One embodiment of the present invention records a computer-implemented method for encrypting data for transmission, the method comprising causing a network interface controller (NIC) to encrypt first data according to a hardware-based Transport Layer Security (TLS) offload mode to generate a first Transmission Control Protocol (TCP) packet; in response to the first TCP packet failing to reach a first device through a first TCP connection, calculating first repetition accounting information; based on the first repetition accounting information, assigning the first TCP connection to a software-based TLS offload mode instead of a hardware-based TLS offload mode; encrypting second data according to the software-based TLS offload mode to generate a second TCP packet; and causing the NIC to transmit the second TCP packet to the first device through the first TCP connection.

[0012] At least one technical advantage of the disclosed technology relative to the prior art is that, using the disclosed technology, TLS records can be efficiently generated when TCP packets are transmitted over a TCP connection, while also mitigating the risk of DoS attacks. In this regard, because TLS data encryption is offloaded from software to the NIC, the amount of processing resources and system memory bandwidth consumed using the disclosed technology can be reduced relative to software kTLS unless a maximum retransmission threshold is exceeded. In addition, because TLS data encryption is automatically transferred from the NIC to software when the maximum retransmission threshold is exceeded, the overall risk of DoS attacks is reduced relative to NIC kTLS. These technical advantages provide one or more technical advances relative to prior art solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In the following description, many specific details are recorded to provide a more thorough understanding of various embodiments. However, it will be clear to those skilled in the art that the inventive concept can be implemented without one or more of these specific details. For illustrative purposes, multiple instances of similar objects are represented by a number identifying the object and by (one or more) bracketed alphanumeric characters identifying the instance, where necessary.

[0014] In order that the above-mentioned features of various embodiments may be understood in detail, a more particular description of the inventive concept briefly summarized above may be made by reference to various embodiments, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings illustrate only typical embodiments of the inventive concept and are therefore not to be considered as limiting the scope in any way and that there are other equally effective embodiments.

[0015] Figure 1 is a conceptual illustration of a system configured to implement one or more aspects of various embodiments;

[0016] Figure 2 It is described according to various embodiments Figure 1 An illustration of an exemplary pseudo-code for a transfer accounting function of

[0017] Figure 3 is a flow chart of method steps for encrypting data for transmission over a TCP connection according to various embodiments. DETAILED DESCRIPTION

[0018] In the following description, numerous specific details are set forth to provide a more thorough understanding of various embodiments. However, it will be apparent to one skilled in the art that the inventive concept can also be implemented without one or more of these specific details.

[0019] In order to enable a library of media files to be efficiently delivered to different client devices under a variety of network conditions, a typical streaming media service stores multiple pre-generated encoded versions of each available media file on a content delivery network (CDN). In order to play a media file on a client device, a client application executed on the client device opens a TCP connection with a server application executed on a server in the CDN. In order to provide reliable data delivery, a TCP implementation is configured to transmit data via TCP packets, detect lost TCP packets, and retransmit data associated with the lost TCP packets once detected.

[0020] After the TCP connection has been opened, the client application will often also establish a TLS session with the server application. In order to provide end-to-end secure delivery of data over the TCP connection while protecting user privacy during the TLS session, the data is encrypted and optionally authenticated to generate a TLS record. The TLS record is then divided into TCP segments, and each TCP segment is encapsulated in a different TCP packet for transmission over the associated TCP connection.

[0021] In a software kTLS scheme that encrypts data for transmission over a TCP connection, a kernel running on a processor of a server device copies data from a non-volatile disk storage device to a volatile system memory. Subsequently, the kernel reads the data from the system memory, encrypts the data to generate encrypted data, writes the encrypted data back to the system memory, and generates one or more TLS records that collectively specify the encrypted data. For each TLS record, the kernel generates one or more TCP packets. The NIC copies each TCP packet from the system memory for transmission over an associated TCP connection. One disadvantage of software kTLS is that generating TLS records may consume a significant amount of processing resources and system memory bandwidth. Therefore, software kTLS may significantly reduce the overall performance of the server device, thereby reducing the amount of network traffic that the server device can serve.

[0022] In a NIC kTLS scheme that encrypts data for transmission over a TCP connection, TLS data encryption is offloaded from a processor to a NIC, which is capable of encrypting and optionally authenticating the data while transmitting TCP packets over the TCP connection. In some implementations, a kernel running on the processor copies data from a disk storage device to a system memory and generates one or more plaintext TLS records that collectively specify the data, and for each plaintext TLS record, the kernel generates one or more plaintext TCP packets, each packet encapsulating a different plaintext TCP segment of the plaintext TLS record. The NIC reads each plaintext TCP packet from the system memory, encrypts the plaintext TCP segments encapsulated in the plaintext TCP packet to generate a TCP packet, and then transmits the TCP packet over an associated TCP connection. Importantly, the amount of processing resources and system memory bandwidth consumed by the NIC kTLS can be significantly reduced compared to the amount of processing resources and system memory bandwidth consumed by software kTLS.

[0023] However, one disadvantage of NIC kTLS is that in order to retransmit a lost TCP packet, the NIC needs to read from system memory the portion of the plaintext TLS record up to and including the encrypted and encapsulated plaintext TCP segment in the lost TCP packet in order to recreate the correct encryption state. Since the loss of a single TCP packet may cause the NIC to read and re-encrypt the entire plaintext TLS record, NIC kTLS may exacerbate the negative impact of lossy data TCP connections on memory bandwidth and is vulnerable to "fake packet loss" DoS attacks. In a fake packet loss DoS attack, an attacker repeatedly fakes lost TCP packets across multiple TLS sessions that correspond to TCP segments near or at the end of a TLS record.

[0024] More specifically, in NIC kTLS, the bus bandwidth between the NIC and system memory may be significantly reduced because the NIC is often forced to reread and re-encrypt multiple TCP segments to retransmit a single lost TCP packet. For example, if each TLS record is split into sixteen TCP segments, then in order to retransmit the data, the NIC must reread and re-encrypt sixteen times the amount of data originally carried by the lost TCP packet. A typical lossy TCP connection may reduce the bandwidth between the NIC and system memory by an average of eight times, and a DoS attack may reduce the bandwidth between the NIC and system memory by a factor of sixteen, relative to just rereading and re-encrypting the single TCP segment actually carried by the lost TCP packet. Thus, an attacker can launch a 16:1 DoS amplification attack across multiple TLS sessions, which effectively reduces a 200 Gigabit per second (Gbs) NIC to a 12.5 Gb / s NIC.

[0025] However, using the disclosed techniques, if the kernel is offloading TLS data encryption to the NIC for a TLS session and the corresponding TCP connection has an excessively high retransmission rate, the kernel stops offloading TLS data encryption to the NIC for the TLS session. In some embodiments, a configurable NIC kTLS maximum packet-based retransmission rate and / or a configurable NIC kTLS maximum byte-based retransmission rate is set to a value indicative of a DoS attack of spurious TCP packet loss and / or a TCP connection with excessive packet loss.

[0026] As part of initializing each TLS session, a kernel running on a processor in a server device sets the TLS mode of the TLS session to NIC kTLS mode. While the TLS session is in NIC kTLS mode, the kernel offloads encryption of TLS data sent within the TLS session to the associated NIC. Whenever the server device sends data over a TCP connection within a TLS session in NIC kTLS mode, the kernel automatically updates any amount of tracking data associated with the TCP connection and the TLS session. Specifically, in some embodiments, the kernel updates a count of TCP packets used to transmit data over the TCP connection and / or a total number of data bytes transmitted over the TCP connection.

[0027] Whenever a server device retransmits data over a TCP connection within a TLS session in NIC kTLS mode, the kernel automatically updates any amount of tracking data associated with both the TCP connection and the TLS session, and then calculates one or more retransmission statistics to determine whether to continue to offload TLS data encryption to the NIC for that TLS session. More specifically, in some embodiments, the kernel updates a count of TCP packets used to retransmit data over the TCP connection and / or a total number of bytes of data retransmitted over the TCP connection. The kernel then calculates a packet-based retransmission rate for the TLS session and / or a byte-based retransmission rate for the TLS session based on the tracking data associated with the TLS session.

[0028] If the packet-based retransmission rate of a TLS session exceeds the NIC kTLS maximum packet-based retransmission rate, and / or the byte-based retransmission rate of a TLS session exceeds the NIC kTLS maximum byte-based retransmission rate, the kernel sets the TLS mode of the TLS session to software kTLS mode instead of NIC kTLS mode. When a TLS session is in software kTLS mode, the kernel performs TLS data encryption for transmissions within the TLS session.

[0029] At least one technical advantage of the disclosed technology relative to the prior art is that, using the disclosed technology, TLS records can be efficiently generated when TCP packets are transmitted over a TCP connection, while also mitigating the risk of DoS attacks and optionally reducing the negative impact of lossy TCP connections. In this regard, because the kernel offloads TLS data encryption from the kernel to the NIC, the amount of processing resources and system memory bandwidth consumed can be reduced relative to software kTLS unless a maximum retransmission threshold is exceeded. In addition, because the kernel automatically transfers TLS data encryption from the NIC to the kernel when the maximum retransmission threshold is exceeded, the overall risk of DoS attacks and optionally the negative impact of lossy TCP connections can be reduced relative to NIC kTLS. These technical advantages provide one or more technical advances relative to prior art solutions.

[0030] System Overview

[0031] In the following description, numerous specific details are set forth to provide a more thorough understanding of various embodiments. However, it will be apparent to one skilled in the art that the inventive concept can also be implemented without one or more of these specific details.

[0032] Figure 1 A conceptual diagram of a system 100 configured to implement one or more aspects of various embodiments is illustrated. As shown, in some embodiments, system 100 includes, but is not limited to, server device 110, client device 190, and network 102. In the same or other embodiments, system 100 may include, but is not limited to, any number of other server devices, any number of other client devices, any number of other networks, or any combination thereof.

[0033] It will be appreciated that the core architecture described herein, including the connection topology of the system 100 and the server device 110, is illustrative and that many variations and modifications are possible without departing from the broader spirit and scope of the present invention. Figure 1 One or more of the components shown in the drawings may not be present, and / or any number of other components may be present, and the techniques described herein modified accordingly.

[0034] Any number of components of system 100 may be distributed across multiple geographic locations or implemented in any combination in one or more cloud computing environments (i.e., packaged shared resources, software, data, etc.). In some embodiments, server device 110, any number of other server devices, client device 190, any number of other server devices, or any combination thereof may be implemented in a cloud computing environment, as part of any other distributed computing environment, or in a standalone manner.

[0035] In some embodiments, client device 190 can be any type of device that can execute instructions, supports TLS and TCP, and can communicate with server device 110 via network 102. In the same or other embodiments, network 102 is the Internet. In some embodiments, client device 190 is a user device or is integrated into a user device. Some examples of user devices include, but are not limited to, desktop computers, laptop computers, smart phones, smart TVs, game consoles, tablet devices, and the like.

[0036] In some embodiments, server device 110 can be any type of device that is capable of executing instructions, supports kTLS and TCP, and is capable of communicating with client device 190 over network 102. In the same or other embodiments, server device 110 can communicate with any number of other client devices over network 102 and / or any number of other networks.

[0037] As shown, in some embodiments, server device 110 includes, but is not limited to, processor 112, system memory 116, memory bus 114, disk storage 106, and NIC 180. In the same or other embodiments, server device 110 may include, but is not limited to, any number of other processors, any number and / or type of other memory, any number and / or type of other storage, any number and / or type of other NICs, or any combination thereof.

[0038] The processor 112 may be any instruction execution system, device, or apparatus capable of executing instructions. For example, the processor 112 may include a central processing unit, a graphics processing unit, a controller, a microcontroller, a state machine, or any combination thereof. The system memory 116 of the server device 110 stores content, such as a kernel, any number of software applications, and data, for use by the processor 112 of the server device 110. In some embodiments, the system memory 116 may include, but is not limited to, any number and / or type of volatile memory capable of storing data and software applications, such as random access memory (RAM).

[0039] In some embodiments, server device 110 may include, but is not limited to, any number and / or type of other processors instead of or in addition to processor 112. In the same or other embodiments, server device 110 may include, but is not limited to, any number and / or type of other memory instead of or in addition to system memory 116. In some embodiments, server device 110 and any number and / or type of other devices (e.g., other server devices) may provide any number of multi-processing environments in any technically feasible manner.

[0040] Disk storage 106 provides non-volatile storage for applications and data accessible by server device 110. In some embodiments, disk storage 106 includes, but is not limited to, any number and / or type of fixed or removable hard disk drives, solid-state drives, flash memory devices, compact disk read-only memory, digital versatile disk read-only memory, Blu-ray, high-definition digital versatile disk, or other magnetic, optical, or solid-state storage devices.

[0041] NIC 180 may be any type of hardware component that enables server device 110 to connect to client device 190 over network 102 and supports at least NIC kTLS mode and software kTLS mode for transmission. Software kTLS mode is also commonly referred to as "software-based TLS offload mode," "software encryption mode," and "software kTLS." NIC kTLS mode is also commonly referred to as "hardware-based TLS offload mode," "inline hardware kTLS offload mode," and NIC kTLS.

[0042] As referred to herein, when operating in software kTLS mode, kernel 130 resident in system memory 116 and executing on processor 112 performs TLS data encryption. In contrast, when operating in transmit mode in NIC kTLS mode, kernel 130 offloads TLS data encryption and optionally any number and / or type of other TLS or TCP operations to NIC 180. Some examples of other TLS or TCP operations that kernel 130 may offload to NIC 180 include, but are not limited to, checksum calculations for authentication and TCP segmentation.

[0043] As described in more detail below, to support NIC kTLS mode for transmission in some embodiments, NIC 180 performs TLS data encryption and any number and / or type of other offloaded TLS or TCP operations "inline," or when NIC 180 transmits TCP packets over a TCP connection. NIC 180 may be installed on server device 110 in any technically feasible manner.

[0044] The processor 112, disk storage device 106, and NIC 180 are each connected to the system memory 116 in any technically feasible manner. As shown, in some embodiments, the processor 112 is connected to the system memory 116 via, but not limited to, a memory bus 114. In the same or other embodiments, the disk storage device 106, the NIC 180, and the system memory 116 are interconnected via, but not limited to, an input / output (I / O) bus 108. The memory bus 114 and the I / O bus 108 can be implemented using any technically appropriate protocol, including but not limited to Peripheral Component Interconnect Express (PCIe), Accelerated Graphics Port (AGP), HyperTransport, or any other bus or point-to-point communication protocol known in the art.

[0045] In some embodiments, instead of or in addition to the memory bus 114 and / or the I / O bus 108, the processor 112, the disk storage device 106, and the NIC 180 may each be connected to the system memory 116 via, but not limited to, any number and / or type of other buses, any number and / or type of controllers (e.g., memory controllers, I / O controllers), any number and / or type of bridges, any number and / or type of switches, or any combination thereof. For example, in some embodiments, the memory bus 114 and the I / O bus 108 are replaced with any number of PCIe switches and PCIe buses that interconnect the processor 112, the disk storage device 106, the NIC 180, and the system memory 116.

[0046] Although not shown, in some embodiments, server device 110 includes, but is not limited to, any number of direct memory access (DMA) controllers that enable NIC 180, disk storage device 106, any number of other components of server device 110, or any combination thereof, to read from and write to system memory 116 without involvement of processor 112. For example, in some embodiments, a DMA controller included in NIC 180 enables NIC 180 to read from and write to system memory 116 via DMA. In the same or other embodiments, a DMA controller included in disk storage device 106 enables disk storage device 106 to read from and write to system memory 116 via DMA.

[0047] In general, server device 110 is configured to provide any number and / or type of client applications executing on any number and / or type of client devices, via any number of TLS sessions executing over any number of TCP connections, access to data stored in disk storage 106. For example, in some embodiments, server device 110 provides streaming access to encoded video present in disk storage 106 via TLS sessions associated with TCP connections so that the corresponding media files can be viewed on a variety of different client devices.

[0048] In some embodiments, each TLS session is associated with a transmit side of a TCP connection or a receive side of a TCP connection for server device 110. In the same or other embodiments, server device 110 provides data to a client device via a TLS session associated with the transmit side of a corresponding TCP connection. For purposes of illustration, a TLS session associated with the transmit side of a TCP connection is also referred to herein as a "TLS transmit session."

[0049] As previously described herein, some conventional server devices implement software kTLS to deliver data to client devices. In a typical implementation of software kTLS, the kernel copies data from a disk storage device to a system memory via DMA and then reads the data from the system memory. The kernel encrypts the data and optionally authenticates the data to generate one or more TLS records. After generating the TLS record, the kernel writes the TLS record back to the system memory and generates one or more TCP packets, each of which encapsulates a different TCP segment of the TLS record. The NIC copies each TCP packet from the system memory via DMA and then transmits the TCP packet over an associated TCP connection. One disadvantage of a software kTLS software kernel is that encrypting data may consume a large amount of processing resources and system memory bandwidth. Therefore, software kTLS may significantly reduce the overall performance of the server device, thereby reducing the amount of network traffic that the server device can serve.

[0050] As also previously described herein, some conventional server devices implement NIC kTLS to deliver data to client devices. In a typical implementation of NIC kTLS, the kernel copies data from disk storage to system memory via DMA and generates one or more plaintext TLS records, each record encapsulating a different segment of the data. After generating the plaintext TLS record, the kernel writes the plaintext TLS record back to system memory and generates one or more plaintext TCP packets, each packet encapsulating a different plaintext TCP segment of the TLS record. The NIC copies each plaintext TCP packet from system memory via DMA, encrypts the unencrypted data encapsulated in the plaintext TCP segment corresponding to the plaintext TCP packet to generate a TCP packet, and then transmits the TCP packet over an associated TCP connection.

[0051] However, one disadvantage of NIC kTLS is that in order to retransmit a lost TCP packet, the NIC needs to read from system memory the portion of the plaintext TLS record up to and including the encrypted and encapsulated plaintext TCP segments in the lost TCP packet in order to recreate the correct encryption state. Therefore, NIC kTLS is vulnerable to a particular type of denial of service (DoS) attack in which an attacker repeatedly forges lost TCP packets to force the NIC to reread and re-encrypt the entire plaintext TLS record, which may significantly reduce the bus bandwidth between the NIC and system memory.

[0052] Offload TLS data encryption from the NIC to the kernel to mitigate excessive retransmission rates

[0053] To address the above issues, in some embodiments, the kernel 130 implements a "temporary NIC encryption" scheme to implement the kTLS protocol. In temporary NIC encryption kTLS, the kernel 130 initially offloads the TLS data encryption for each new TLS transport session to the NIC associated with the TLS transport session. For each data retransmission or "re-send" within a TLS session whose TLS data encryption has been offloaded to the NIC, the kernel calculates one or more retransmission statistics to determine whether the TLS session is associated with an excessively high retransmission rate. If the kernel 130 determines that the TLS session is associated with an excessively high retransmission rate, the kernel 130 stops offloading TLS data encryption to the NIC for the TLS session. In some embodiments, after the kernel 130 stops offloading TLS data encryption to the NIC for the TLS session, the kernel 130 performs TLS data encryption on data associated with new transmissions or "sends" within the TLS session.

[0054] Advantageously, because the kernel 130 offloads the TLS data encryption of the TLS session to the NIC, the amount of processing resources and system memory bandwidth consumed to serve the same amount of traffic relative to software kTLS can be reduced, unless the kernel 130 determines that the TLS session is associated with an excessively high retransmission rate. Conversely, the amount of traffic that the server device 110 can serve can be increased relative to software kTLS. Furthermore, because the kernel 130 automatically transfers the TLS data encryption of the TLS session from the NIC to the software when it determines that the TLS session is associated with an excessively high retransmission rate, the overall risk of DoS attacks is reduced relative to traditional NIC kTLS.

[0055] Importantly, in some embodiments, a relatively poor TCP connection may be associated with an excessively high retransmission rate. Advantageously, for TLS sessions associated with a poor TCP connection and, therefore, an excessively high retransmission rate, the kernel 130 may automatically stop offloading TLS data encryption to the NIC. Thus, the kernel 130 may mitigate the negative impact of a relatively poor TCP connection between a client device and a server device 110 on the amount of network traffic that can be served by other client devices and / or the server device 110.

[0056] For purposes of illustration, some techniques, including some ad hoc NIC encryption techniques, are described herein in the context of providing access to data stored in disk storage 106 to client devices 190 within system 100 using kTLS, NIC kTLS mode, software kTLS mode, and exemplary implementations of TCP via an exemplary TLS session performed on the transport side of an exemplary TCP connection. It should be noted, however, that the functions and techniques described herein are intended to be illustrative rather than limiting, and that changes may be made thereto without departing from the broader spirit and scope of the invention.

[0057] In particular, many changes and modifications may be made to the architecture, including the connection topology and functionality of the system 100 and the server device 110, and corresponding modifications to the techniques described herein will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. For example, in some embodiments, the techniques described herein may be modified to enable any number of client applications executing on any number of client devices to access data stored in any number and / or type of memory associated with any number and / or type of server applications executing on any number of server devices via any number of TLS sessions operating on any number of TCP connections.

[0058] In some embodiments, server device 110, any number of other server devices, client device 190, any number of other client devices, or any combination thereof may perform any number of kTLS implementations and any number of TCP implementations, and the techniques described herein may be modified accordingly without departing from the scope and spirit of the described embodiments.

[0059] In general, the techniques described herein may be modified to automatically disable offloading of TLS data encryption, and optionally any number and / or type of other TLS and / or TCP operations, to one or more NICs for each of any number and / or type of TLS sessions based on any number and / or type of data associated with transmissions and / or retransmissions within the TLS sessions without departing from the scope and spirit of the described embodiments.

[0060] As shown, in some embodiments, server application 120, user space TLS library 122, and kernel 130 reside in system memory 116 of server device 110 and execute on processor 112 of server device 110. In some embodiments, server application 120 may be any software application that executes in user space and transmits data stored in disk storage 106 to a client application executing on client device 190 via TLS and TCP. In the same or other embodiments, user space TLS library 122 establishes and manages TLS session(s) associated with any number of software applications executing on processor 112 of server device 110, including server application 120.

[0061] Although not shown, in some embodiments, the client application executed on the client device 190 opens a TCP connection with the server application 120. The TCP connection uses TCP packets to provide reliable and orderly bidirectional data transmission. It is worth noting that in order to provide reliable data transmission, the TCP implementation is configured to detect lost TCP packets, and then resend these TCP packets once detected. After the TCP connection has been opened, the client application establishes at least one TLS session with the server application 120. In some embodiments, the server application 120 uses the user space TLS library 122 to complete the TLS handshake in the user space, and establishes a TLS / TCP socket, which allows the server application 120 to send data to the client application within the TLS session and through the TCP connection. Therefore, the TLS session is associated with the transmission side of the TCP connection for the server device 110. As shown in the figure, in some embodiments, the user space TLS library 122 can interact with the kernel 130.

[0062] In some embodiments, the kernel 130 implements, but is not limited to, any number and / or type of TLS functionality, any number and / or type of kTLS functionality, any number and / or type of TCP functionality, and may configure and control temporary NIC encryption for each TLS transport session in any technically feasible manner. Some examples of operations that the kernel 130 may perform to configure and control temporary NIC encryption include, but are not limited to, offloading TLS data encryption for a TLS transport session, stopping offloading TLS data encryption for a TLS transport session, performing any number and / or type of operations to calculate any number and / or type of retransmission statistics information, and determining that a TLS transport session is associated with an excessively high retransmission rate.

[0063] The kernel 130 may include, but is not limited to, any number and / or type of data (including any number and / or type of metadata), any number and / or type of functions, any number and / or type of drivers, and any number and / or type of other software organized in any technically feasible manner. As shown, in some embodiments, the kernel 130 includes, but is not limited to, TLS / TCP metadata 150, kTLS send file function (fn) 142, resend context function 146, TCP packet send function 166, transmit accounting function 144, disable NIC kTLS function 148, and NIC driver 168.

[0064] In some embodiments, as the kernel 130 processes data transmitted within each TLS session and through each TCP connection, the kernel 130 stores any amount and / or type of data associated with the TLS records and TCP packets. In the same or other embodiments, the kernel 130 determines and / or maintains any amount and / or type of TLS / TCP metadata 150 to configure and control each TCP connection, each TLS session, and temporary NIC encryption for each TLS session. In some embodiments, the TLS / TCP metadata 150 includes, but is not limited to, any amount and / or type of shared TLS metadata applicable to each TLS session, and different session / connection states for any number of TLS sessions and associated TCP connections.

[0065] In some embodiments, the shared TLS metadata includes, but is not limited to, one or more thresholds, limits, or other metadata that the kernel 130 uses to determine whether each TLS session whose TLS data encryption is offloaded to the NIC is associated with an excessively high retransmission rate. The kernel 130, any number of software applications, any number of components of the server device 110, or any combination thereof, may determine and optionally update the shared TLS metadata in any technically feasible manner.

[0066] As follows Figure 2In more detail, in some embodiments, the TLS / TCP metadata 150 includes, but is not limited to, a NIC kTLS threshold set. In some embodiments, the NIC kTLS threshold set includes, but is not limited to, one or more thresholds that the kernel 130 enforces for each TLS session where TLS data encryption is offloaded to the NIC. Specifically, in some embodiments, the NIC kTLS threshold set includes, but is not limited to, one or more "maximum retransmission thresholds," which are different configurable definitions of excessive retransmission rates for temporary NIC encryption.

[0067] For example, in some embodiments, the NIC kTLS threshold set ( Figure 1 NIC kTLS maximum byte retransmission percentage refers to the maximum percentage of data bytes retransmitted out of the total number of data bytes transmitted or retransmitted within a TLS session that the kernel 130 allows when TLS data encryption for the TLS session is offloaded to the NIC. The maximum packet retransmission percentage via NIC kTLS refers to the maximum percentage of TCP packets used to retransmit data within a TLS session out of the total number of TCP packets used to transmit or retransmit data within a TLS session that the kernel 130 allows when TLS data encryption for the TLS session is offloaded to the NIC.

[0068] In some embodiments, kernel 130 and / or one or more other components of server device 110 may determine and optionally update any number and / or type of data included in the NIC kTLS threshold set in any technically feasible manner. For example, in some embodiments, one or more maximum retransmission thresholds are set to values ​​indicative of a DoS attack with spoofed packet loss in order to mitigate the risk of a DoS attack when implementing NIC kTLS. In the same or other embodiments, one or more maximum retransmission thresholds are set to values ​​indicative of an excessively lossy connection in order to mitigate any negative impact on other better connections and the overall performance of server device 110. In some embodiments, the NIC kTLS threshold set is replaced or supplemented with any number and / or type of constraints or any other type of data that can be used to define excessive retransmissions globally, per session, per connection, at any other granularity, or any combination thereof in any technically feasible manner.

[0069] For each TLS session, the associated session / connection state may include, but is not limited to, any amount and / or type of data associated with implementing the TLS session, a TCP connection associated with the TLS session, software kTLS, NIC kTLS, temporary NIC encryption kTLS, or any combination thereof. For example, in some embodiments, the NIC kTLS threshold set is replaced or supplemented with a different NIC kTLS threshold set that is included in the session / connection state of the corresponding TLS session.

[0070] In some embodiments, the session connection state of a TLS transport session includes but is not limited to TLS mode ( Figure 1 ) and the delivery tracking dataset ( Figure 1 ). In the same or other embodiments, the TLS mode may be a NIC kTLS mode or a software kTLS mode. In some embodiments, as an alternative or in addition to setting the TLS mode, the kernel 130 may assign the TLS session to the NIC kTLS mode or the software kTLS mode in any technically feasible manner. In some embodiments, as an alternative or in addition to assigning the TLS session to the NIC kTLS mode or the software kTLS mode, the kernel 130 may assign the associated TCP connection and / or the transmission side of the associated TCP connection to the NIC kTLS mode or the software kTLS mode, respectively.

[0071] In some embodiments, the transmission tracking data set includes, but is not limited to, any number and / or type of data that tracks any number of aspects of retransmissions and optionally transmissions within a TLS session and over an associated TCP connection. Figure 2 Described in more detail, in some embodiments, the transmission tracking data set associated with the TCP connection includes, but is not limited to, a count of TCP packets used to transmit data over the TCP connection via the NIC kTLS, a total number of data bytes transmitted over the TCP connection, a count of TCP packets used to retransmit data over the TCP connection via the NIC kTLS, and a total number of data bytes retransmitted over the TCP connection.

[0072] The kernel 130 and / or one or more other components of the server device 110 may initialize and optionally maintain TLS modes, transport tracking data sets, and any number (including none) and / or types of other session connection states for any number of TLS sessions in any technically feasible manner. Notably, in some embodiments, as part of initializing the session connection state for a new TLS transport session, the kernel 130 sets the TLS mode to the NIC kTLS mode to configure the TLS transport session to initially offload TLS data encryption to the NIC.

[0073] In some embodiments, kTLS send file function 142 combines a send file system call with kTLS to transfer data from disk storage device 106 to a client application over a TCP connection without copying the data to user space memory (not shown) in system memory 116. In some embodiments, in response to a kTLS send file function call associated with a TLS session in NIC kTLS mode, kTLS send file function 142 generates and submits one or more plaintext TCP packets to NIC 180 via TCP packet send function 166 and NIC driver 168. In the same or other embodiments, in response to a kTLS send file function call associated with a TLS session in software kTLS mode, kTLS send file function 142 generates and submits one or more TCP packets to NIC 180 via TCP packet send function 166 and NIC driver 168.

[0074] In some embodiments, the resend context function 146 is associated with a TLS session in NIC kTLS mode and enables the NIC 180 to correctly re-encrypt the "target" TCP segment carried in encrypted form by the lost TCP packet. More specifically, in the same or other embodiments, the resend context function 146 identifies the target plaintext TCP packet encapsulating the target TCP segment and the target TLS record to which the target TCP segment belongs. In some embodiments, the resend context function 146 then generates an encryption context that specifies (but is not limited to) the TCP segments that constitute the target TLS record, up to and including the target TCP segment. In some other embodiments, the resend context function 146 generates an encryption context that specifies (but is not limited to) the TCP segments that constitute the target TLS record, up to but not including the target TCP segment. In some embodiments, the resend context function 146 associates the encryption context with the target plaintext TCP packet in any technically feasible manner. In the same or other embodiments, the resend context function 146 resubmits the target plaintext TCP packet to the NIC 180 along with the encryption context via the TCP packet send function 166 and the NIC driver 168.

[0075] In some embodiments, for each transmission and each retransmission over the TCP connection, the TCP packet sending function 166 submits (but is not limited to) any number of TCP packets and / or any number of plaintext TCP packets, which are optionally associated with encryption contexts, to the NIC driver 168. The TCP packet sending function 166 can submit the TCP packets, plaintext TCP packets, encryption contexts, and any amount and / or type of related data to the NIC driver 168 in any technically feasible manner. In the same or other embodiments, for each transmission and each retransmission over the TCP connection, the TCP packet sending function 166 automatically executes the transmission accounting function 144.

[0076] Among many other things, the transport accounting function 144 updates any number and / or type of state data and / or tracking data for the TCP connections, updates any number and / or type of state data and / or tracking data for any TLS sessions associated with the TCP connections, and performs any number and / or type of checks and / or operations to enforce any number and / or type of constraints. In some embodiments, the transport accounting function 144 updates any number and / or type of state data and / or tracking data for any number of TCP connections, any number and / or type of state data and / or tracking data for any number of TLS sessions, any number and / or type of other data, or any combination thereof.

[0077] It is worth noting that in some embodiments, for each transmission and each retransmission, the transmit accounting function 144 updates the transmit tracking data set. In addition, for each retransmission associated with a TLS session in NIC kTLS mode, the transmit accounting function 144 calculates one or more retransmission statistics based on the transmit tracking data set to determine whether any retransmission statistics exceed the associated NIC kTLS transmission threshold. If any retransmission statistics exceed the associated NIC kTLS transmission threshold, the transmit accounting function 144 prohibits offloading TLS data encryption to the NIC 180 for the TLS session.

[0078] In some embodiments, the kernel 130 may update any amount and / or type of tracking data, calculate any amount and / or type of retracing statistics information, and perform any number and / or type of operations based on the tracking data, the retracing statistics information, any number and / or type of thresholds, limits, or other constraints, or any combination thereof, to determine whether to prohibit offloading TLS data encryption to the NIC 180 for a particular TLS session.

[0079] In some embodiments, as part of sending TCP packets over a TCP connection within a TLS session in NIC kTLS mode, kernel 130 updates any amount and / or type of tracking data associated with both the TLS session and the TCP connection in any technically feasible manner. For example, in some embodiments, kernel 130 updates at least one of a count of TCP packets used to transmit data over the TCP connection or a total number of data bytes transmitted over the TCP connection based on the TCP packets.

[0080] In the same or other embodiments, as part of resending TCP packets over a TCP connection within a TLS session in NIC kTLS mode, kernel 130 updates any amount and / or type of tracking data, calculates any amount and / or type of retransmission statistics, and determines in any technically feasible manner whether to disable NIC kTLS for the TLS session, thereby disabling the TCP connection. For example, in some embodiments, kernel 130 updates at least one of a count of TCP packets used to retransmit data over a TCP connection or a total number of data bytes retransmitted over a TCP connection based on TCP packets.

[0081] In the same or other embodiments, the kernel 130 calculates at least one retransmission statistic (e.g., a ratio or percentage) for the TLS session based on at least one of a count of TCP packets used to transmit data over the TCP connection, a total number of data bytes transmitted over the TCP connection, a count of TCP packets used to retransmit data over the TCP connection, or a total number of data bytes retransmitted over the TCP connection. In some embodiments, the kernel 130 compares the at least one retransmission statistic to at least one retransmission threshold to determine whether to disable the NIC kTLS mode for the TLS session.

[0082] The transport accounting function 144 may disable offloading of TLS data encryption to the NIC 180 for the TLS session in any technically feasible manner. As shown, in some embodiments, the transport accounting function 144 calls the disable NIC kTLS function 148 to disable offloading of TLS data encryption to the NIC 180 for the TLS session. Figure 2 Example pseudo code for the transfer accounting function 144 implemented in some embodiments is described in more detail.

[0083] Disabling the NIC kTLS function 148 may disable offloading of TLS data encryption to the NIC 180 for the TLS session in any technically feasible manner, and in some embodiments, disabling the NIC kTLS function 148 sets the TLS mode of the TLS session to software kTLS mode rather than NIC kTLS mode. Thus, the NIC 180 performs TLS data encryption for any previously initiated transmissions and any associated retransmissions within the TLS session, while the kernel 130 performs TLS data encryption for any newly initiated transmissions and any associated retransmissions within the TLS session.

[0084] In some embodiments, NIC driver 168 may be any type of software that manages the operation of NIC 180 for kernel 130, data transfer between disk storage device 106 and NIC 180, metadata transfer between kernel 130 and NIC 180, or any combination thereof.

[0085] For illustration, the circles labeled 1a-1e depict an exemplary sequence of events associated with generating and sending TCP packet 174(1) to client device 190 within an exemplary TLS session when the TLS mode of the exemplary TLS session is NIC kTLS mode. As depicted by the circle labeled 1a, server application 120 invokes kTLS send file function 142 via user space TLS library 122 to send application data 170(1) stored in disk storage 106 to client device 190 within the exemplary TLS session. In response, kernel 130 executes kTLS send file function 142.

[0086] More specifically, in some embodiments, kernel 130 executes a send file system call to copy application data 170(1) from disk storage 106 to system memory 116 via DMA (depicted by the circle labeled 1b). In the same or other embodiments, kernel 130 segments application data 170(1) into one or more plaintext segments and generates a plaintext TLS record 162 that specifies, but is not limited to, the location of a first plaintext segment of application data 170(1) in system memory 116. For purposes of illustration, while kernel 130 may generate any number of additional plaintext TLS records specifying any remaining plaintext segments of application data 170(1), the additional plaintext TLS records are neither depicted nor described herein.

[0087] In some embodiments, kernel 130 segments plaintext TLS record 162 into one or more plaintext TCP segments (not shown) and generates plaintext TCP packet 164 that specifies, but is not limited to, a first plaintext TCP segment of plaintext TLS record 162. For purposes of illustration, while kernel 130 may generate any number of other plaintext TCP packets specifying any remaining plaintext TCP segments of plaintext TLS record 162, the other plaintext TCP segments are neither depicted nor described herein.

[0088] As depicted by the circle labeled 1c, in some embodiments, TCP packet send function 166 relays plaintext TCP packet 164 to NIC 180 via NIC driver 168. In some embodiments, in response to receiving plaintext TCP packet 164, NIC 180 reads a corresponding portion of application data 170(1) from system memory 116 via DMA. As depicted by the first instance of the circle labeled 1d, NIC 180 encrypts and optionally authenticates a first plaintext TCP segment of a corresponding portion of application data 170(1) to generate TCP packet 174(1).

[0089] As depicted by the second example of the circle labeled 1d, in some embodiments, TCP packet sending function 166 executes transmit accounting function 144 to account for sending TCP packet 174(1) over the associated exemplary TCP connection within the exemplary TLS session. In the same or other embodiments, transmit accounting function 144 updates (but is not limited to) any number of items included in a transmit tracking data set associated with the exemplary TLS session. As depicted by the circle labeled 1e, NIC 180 transmits TCP packet 174(1) to client device 190 over the exemplary TCP connection.

[0090] For illustration, the circles labeled 2a-2f depict an exemplary sequence of events that triggers kernel 130 to change the TLS mode of the exemplary TLS session from NIC kTLS mode to software kTLS mode. Although not shown, the TCP packet carrying the last TCP segment of the first fragment of application data 170(1) fails to reach client device 190 over the exemplary TCP connection and is therefore a lost TCP packet. In response, as depicted by the circle labeled 2a, in some embodiments, NIC driver 168 calls TCP resend context function 146 to obtain the plaintext TCP packet 164 and encryption context 182 corresponding to the lost TCP packet.

[0091] As previously described herein, in some embodiments, plaintext TCP packet 164 encapsulates the last plaintext TCP segment of the first plaintext segment of application data 170(1). Thus, in some embodiments, TCP resend context function 146 determines encryption context 182 that specifies (but is not limited to) the first plaintext segment of application data 170(1). As depicted by the circle labeled 2b, in some embodiments, TCP resend context function 146 resubmits plaintext TCP packet 164 along with encryption context 182 to TCP packet send function 166.

[0092] In the same or other embodiments, TCP packet send function 166 resubmits plaintext TCP packet 164 along with encryption context 182 to NIC 180 via NIC driver 168. In response, NIC 180 in some embodiments reads a first plaintext segment of application data 170(1) from system memory 116 via DMA. NIC 180 encrypts and optionally authenticates the first plaintext segment of application data 170(1) as depicted by the first instance of circle 2c labeled 2c to generate TCP packet 174(2).

[0093] As depicted by the second example of the circle labeled 2c, in some embodiments, TCP packet sending function 166 executes transmit accounting function 144 to account for retransmitting the last fragment of application data 170(1) to client device 190 via TCP packet 174(2) over the exemplary TCP connection within the exemplary TLS session. In the same or other embodiments, transmit accounting function 144 updates, but is not limited to, any number of items included in a transmit tracking data set associated with the exemplary TLS session. Transmit accounting function 144 calculates one or more retransmission statistics based on the transmit tracking data set associated with the exemplary TLS session. For illustration, the transmit accounting function determines that at least one retransmission statistic has exceeded an associated NICkTLS transmit threshold.

[0094] As depicted by the circle labeled 2d, in some embodiments, the transmit accounting function 144 calls the disable NIC kTLS function 148 to disable offloading of TLS data encryption to the NIC 180 for the exemplary TLS session. Although not shown, in some embodiments, the disable NIC kTLS function 148 changes the TLS mode of the exemplary TLS session from the NIC kTLS mode to the software kTLS mode. As depicted by the circle labeled 2e, the NIC 180 transmits the TCP packet 174(2) to the client device 190 over the exemplary TCP connection.

[0095] For illustration, the circles labeled 3a-3g depict an exemplary sequence of events associated with generating and sending TCP packet 174(3) to client device 190 within an exemplary TLS session when the TLS mode of the exemplary TLS session is software kTLS. As depicted by the circle labeled 3a, server application 120 invokes kTLS send file function 142 via user space TLS library 122 to send application data 170(2) stored in disk storage 106 to client device 190 within the exemplary TLS session. In response, kernel 130 executes kTLS send file function 142.

[0096] More specifically, in some embodiments, kernel 130 executes a send file system call to copy application data 170(2) from disk storage 106 to system memory 116 via DMA (depicted by the circle labeled 3b). As depicted by circle number 3c, in the same or other embodiments, kernel 130 reads application data 170(2) from system memory 116. Kernel 130 encrypts application data 170(2) to generate encrypted application data (not shown). Kernel 130 writes the encrypted application data back to system memory 116 as depicted by the circle labeled 3d.

[0097] As shown, in some embodiments, kernel 130 segments the encrypted application data into one or more segments and generates a TLS record 172 that specifies, but is not limited to, the location of a first segment of the encrypted application data in system memory 116. For purposes of illustration, while kernel 130 may generate any number of other TLS records specifying any remaining segments of the encrypted application data, other TLS records are neither depicted nor described herein.

[0098] In some embodiments, kernel 130 segments TLS record 172 into one or more TCP segments (not shown) and generates TCP packet 174(3) that specifies, but is not limited to, a first TCP segment of TLS record 172. For purposes of illustration, although kernel 130 may generate any number of other TCP packets specifying any remaining TCP segments of TLS record 172, the other TCP segments are neither depicted nor described herein. As depicted by the circle labeled 3e, in some embodiments, TCP packet send function 166 relays TCP packet 174(3) to NIC 180 via NIC driver 168.

[0099] As depicted by the circle labeled 3f, in some embodiments, TCP packet sending function 166 executes transmit accounting function 144 to account for sending TCP packet 174(3) over the exemplary TCP connection within the exemplary TLS session. In the same or other embodiments, transmit accounting function 144 updates (but is not limited to) any number of items included in a transmit tracking data set associated with the exemplary TLS session.

[0100] In some embodiments, in response to receiving TCP packet 174(3), NIC 180 reads the encrypted application data portion corresponding to the first TCP segment of TLS record 172 from system memory 116 via DMA and transmits TCP packet 174(3) to client device 190 (depicted by a circle labeled 3g) over an exemplary TCP connection.

[0101] Detecting excessive transfer rate in TLS sessions in NIC KTLS mode

[0102] Figure 2 It is described according to various embodiments Figure 1 More specifically, Figure 2 Depicted is exemplary transfer accounting pseudo code 260, which is described in some embodiments to achieve Figure 1 An example of pseudo code for a portion of the functionality of the transfer accounting function 144 is shown below. To illustrate, Figure 2 Also depicted is a portion of TLS / TCP metadata 150 implemented in some embodiments. Many modifications and variations of the functionality of TLS / TCP metadata 150 and transport accounting function 144 will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

[0103] As shown, in some embodiments, the TLS / TCP metadata 150 includes, but is not limited to, a NIC kTLS threshold set 210 and a session / connection state 220 of an exemplary TLS session and an associated exemplary TCP connection. Figure 1 As described, NIC kTLS threshold set 210 is an example of shared TLS / TCP metadata. As indicated by the ellipsis, in the same or other embodiments, TLS / TCP metadata 150 may include, but is not limited to, any number and / or type of other shared metadata, and / or any number of other session / connection states for any number of TLS sessions and associated TCP connections.

[0104] In some embodiments, the NIC kTLS threshold set 210 includes, but is not limited to, a configurable NIC kTLS maximum byte retransmission percentage and a configurable maximum packet retransmission percentage via NIC kTLS. The NIC kTLS maximum byte retransmission percentage refers to the maximum percentage of data bytes retransmitted out of the total number of data bytes transmitted or retransmitted within a TLS session that the kernel 130 allows when TLS data encryption for the TLS session is offloaded to the NIC. The maximum packet retransmission percentage via NIC kTLS refers to the maximum percentage of TCP packets that the kernel 130 allows for retransmitting data within a TLS session via NIC kTLS when TLS data encryption for the TLS session is offloaded to the NIC. For illustration, as depicted in italics, the NIC kTLS maximum byte retransmission percentage and the maximum packet retransmission percentage via NIC kTLS are also referred to herein as nic_ktls_max_byte_resend_percent and max_nic_ktls_packet_resend_percent, respectively.

[0105] Kernel 130 and / or any other component of server device 110 may determine NIC kTLS threshold set 210 in any technically feasible manner. For example, in some embodiments, NIC kTLS maximum byte retransmission percentage and maximum packet retransmission percentage via NIC kTLS, any number of other data included in NIC kTLS threshold set 210, or any combination thereof are set via a system control mechanism that enables a user (e.g., an administrator) to provide data to kernel 130. In the same or other embodiments, a user may configure the system control mechanism in any technically feasible manner.

[0106] As shown, in some embodiments, session / connection state 220 includes, but is not limited to, a TLS mode 230 of an exemplary TLS session and a transport tracking data set 240 associated with the exemplary TLS session and / or the exemplary TCP connection. As depicted by the ellipsis, in the same or other embodiments, session / connection state 220 may include, but is not limited to, any number and / or type of other data associated with the exemplary TLS session and / or the exemplary TCP connection. In some embodiments, TLS mode 230 may be a NIC kTLS mode or a software kTLS mode. As depicted by italics, for illustration, NIC kTLS mode and software kTLS mode are also referred to herein as nic_ktls and sw_ktls, respectively.

[0107] In some embodiments, the transmission tracking data set 240 includes, but is not limited to, any number and / or type of status data associated with an exemplary TCP connection. In the same or other embodiments, the transmission tracking data set for each TCP connection includes, but is not limited to, the total number of data bytes transmitted over the TCP connection, the count of TCP packets used to transmit data over the TCP connection, the total number of data bytes retransmitted over the TCP connection, and the count of TCP packets used to retransmit data over the TCP connection. As depicted in italics, for illustration, the total number of data bytes transmitted over the TCP connection, the count of TCP packets used to transmit data over the TCP connection, the total number of data bytes retransmitted over the TCP connection, and the count of TCP packets used to retransmit data over the TCP connection are also referred to herein as send_byte, nic_ktls_send_packet, resendbyte, and nic_ktls_resend_packet, respectively.

[0108] In some embodiments, any number of portions of TLS / TCP metadata 150, session / connection state, and delivery tracking data set 240 may be stored within and / or distributed among any number and / or types of other data structures and / or components in any technically feasible manner. For example, in some embodiments, delivery tracking data set 240 is implemented via counters stored in a TCP control block (not shown).

[0109] In some embodiments, kernel 130 initializes TLS mode 230 to NIC kTLS to configure the exemplary TLS session to initially offload TLS data encryption to NIC 180. Kernel 130 may initialize a transmission tracking data set 240 in any technically feasible manner. For each transmission and each retransmission over the exemplary TCP connection, kernel 130 executes transmission accounting pseudo code 260 to update transmission tracking data set 240 and determine whether the exemplary TLS session and / or the exemplary TCP connection exceeds any thresholds included in NIC kTLS threshold set 210.

[0110] In the same or other embodiments, kernel 130 changes TLS mode 230 to software kTLS if kernel 130 determines that the exemplary TLS session and / or the exemplary TCP connection exceeds any threshold included in NIC kTLS threshold set 210. When TLS mode 230 is software kTLS, kernel 130 ensures that data was originally transmitted over the exemplary TCP connection in accordance with the software kTLS protocol and that data is retransmitted over the exemplary TCP connection in accordance with the kTLS protocol that was originally used to transmit over the exemplary TCP connection.

[0111] For illustration, exemplary transport accounting pseudo code 260 is described in the context of an exemplary TLS session and an exemplary TCP connection. As shown, the transport accounting pseudo code 260 includes, but is not limited to, byte-based tracking pseudo code 270, byte-based NIC TLS threshold enforcement pseudo code 280, and NIC kTLS packet tracking and threshold enforcement pseudo code 290.

[0112] According to the byte-based tracking pseudo code 270, in some embodiments, for each retransmission made through the exemplary TCP connection, the kernel 130 increments resend_byte by the number of data bytes associated with the retransmission. In the same or other embodiments, for each transmission made through the exemplary TCP connection, the kernel 130 increments send_byte by the number of data bytes associated with the transmission.

[0113] In some embodiments, for each retransmission performed over the exemplary TCP connection when the TLS mode 230 is in the NIC kTLS mode, the kernel 130 performs byte-based NIC TLS threshold enforcement, as described in the byte-based NIC TLS threshold enforcement pseudo code 280. According to the byte-based NIC TLS threshold enforcement pseudo code 280, the kernel 130 sets a variable, denoted as "resend_byte_percent", equal to 100*resend_byte / (resend_byte+send_byte). If the resend_byte_percent exceeds the nic_ktls_max_byte_resend_percent, the kernel 130 calls the disable NIC kTLS function 148 to change the TLS mode 230 from the NIC kTLS mode to the software kTLS mode.

[0114] In some embodiments, for each send and each resend over the exemplary TCP connection when the TLS mode 230 is the NIC kTLS mode, if the resend_byte_percent does not exceed the nic_ktls_max_byte_resend_percent, the kernel 130 performs NIC kTLS packet tracking and threshold enforcement, as described in the NIC kTLS packet tracking and threshold enforcement pseudo code 290. Following the NIC kTLS packet tracking and threshold enforcement pseudo code 290, in some embodiments, for each send over the exemplary TCP connection, the kernel 130 increments send_nic_ktls_packet by one.

[0115] In the same or other embodiments, in accordance with the NIC kTLS packet tracking and threshold execution pseudo code 290, for each resend performed over the exemplary TCP connection, the kernel 130 increments resend_nic_ktls_packet by 1 and calculates a variable denoted as "resend_nic_ktls_packet_percent". More specifically, the kernel 130 sets resend_nic_ktls_packet_percent equal to 100*nic_ktls_resend_packet / (nic_ktls_resend_packet+nic_ktls_send packet). If resend_nic_ktls_packet_percent exceeds max_nic_ktls_packet_resend_percent, the kernel 130 calls the disable NIC kTLS function 148 to change the TLS mode 230 from the NIC kTLS mode to the software kTLS mode.

[0116] Figure 3 is a flow chart of method steps for encrypting data for transmission over a TCP connection according to various embodiments. Figure 1 and Figure 2 Although the method steps are described with reference to a system, those skilled in the art will understand that any system configured to implement the method steps in any order falls within the scope of the present invention.

[0117] As shown, the method 300 begins at step 302, where the kernel 130 sets the TLS mode of the newly established TLS session to the NIC kTLS mode and initializes a transmission tracking data set associated with the TLS session. At step 304, within the TLS session, the kernel 130 processes transmissions according to the NIC kTLS mode while maintaining the transmission tracking data set until a lost TCP packet is detected or the TLS session is completed. At step 306, if the server device 110 determines that the TLS session has been completed, the method 300 terminates.

[0118] However, if at step 306, server device 110 determines that the TLS session is not complete, method 300 proceeds to step 308. At step 308, server device 110 resends the data carried by the lost TCP packets according to the NIC kTLS mode, and kernel 130 updates the delivery tracking data set accordingly.

[0119] At step 310, the kernel 130 determines whether at least one NIC kTLS threshold has been exceeded based on the transmit tracking data set. At step 312, if the kernel 130 determines that any (one or more) NIC kTLS thresholds have not been exceeded, the method 300 returns to step 304, where the server device 110 processes transmissions according to the NIC kTLS mode within the TLS session while maintaining the transmit tracking data set until a lost TCP packet is detected or the TLS session is determined to be complete.

[0120] However, if at step 312, the kernel 130 determines that at least one NIC kTLS threshold has been exceeded, the method 300 proceeds directly to step 314. At step 314, the kernel 130 switches the TLS mode of the TLS session from the NIC kTLS mode to the software kTLS mode. At step 316, within the TLS session, the server device 110 processes previously queued transmissions according to the NIC kTLS mode, processes lost TCP packets according to the associated kTLS mode, and processes new transmissions according to the software kTLS mode, while maintaining the transmission tracking data set 240 until it is determined that the TLS session is complete. The method 300 then terminates.

[0121] Note that for purposes of illustration, method 300 describes method steps with reference to a single TLS session that enables a server application 120 executing on a server device to securely deliver data to a client application executing on a client device over a single TCP connection. Those skilled in the art will recognize that user space TLS library 122, kernel 130, any number of NIC drivers, and any number of NICs can collaborate to establish any number of TLS sessions that operate by connecting any number of server applications to corresponding TCP connections of any number of client applications.

[0122] For each TLS session, the kernel 130 initializes a different TLS mode to the NIC kTLS mode, maintains an independent data set for transmission tracking, and independently switches the TLS mode to the software kTLS mode upon detecting that any NIC kTLS threshold is exceeded. Thus, at any given time, each of the server device 110, the user space TLS library 122, the kernel 130, any number of NIC drivers, and any number of NICs can be associated with zero or more TLS sessions in the NIC kTLS mode and zero or more TLS sessions in the software kTLS mode.

[0123] In summary, the disclosed techniques can be used to mitigate the risk of DoS attacks when implementing NIC kTLS. In some embodiments, a configurable NIC kTLS maximum byte retransmission percentage and a configurable maximum packet retransmission percentage via NIC kTLS are set to values ​​indicative of a DoS attack with forged TCP packet loss. The NIC kTLS maximum byte retransmission percentage and the maximum packet retransmission percentage via NIC kTLS are two examples of NIC kTLS thresholds. More specifically, the NIC kTLS maximum byte retransmission percentage and the maximum packet retransmission percentage via NIC kTLS are two examples of maximum retransmission thresholds associated with a NIC kTLS mode.

[0124] A kernel executing on a processor in a server device initially sets the kTLS mode for a TLS session to NIC kTLS mode. Within a TLS session, the kernel, NIC driver, and NIC implement end-to-end secure delivery of application data from a server device to a client device via an associated TCP connection. When the TLS session is in NIC kTLS mode, the kernel offloads encryption of the data to the NIC, which transmits TCP packets encapsulating the encrypted data from a server application executing on the server device to a client application executing on the client device via an associated TCP connection. Notably, the kernel automatically performs a "transmit accounting" function on each send and each resend within a TLS session. Among many other things, the transmit accounting function updates the state data of the TCP connection stored in the TCP control block and performs any number and / or type of checks and / or operations to enforce any number and / or type of constraints.

[0125] To facilitate DoS attack mitigation, the TCP control block for each TCP connection includes, but is not limited to, counters that store: a count of TCP packets used to transmit data over the TCP connection via the NIC kTLS, a total number of data bytes transmitted over the TCP connection, a count of TCP packets used to retransmit data over the TCP connection via the NIC kTLS, and a total number of data bytes retransmitted over the TCP connection.

[0126] For each send associated with the TCP connection, the transmit accounting function automatically updates a counter storing the total number of data bytes transmitted over the TCP connection. For each retransmission associated with the TCP connection, the transmit accounting function automatically updates a counter storing the total number of data bytes retransmitted over the TCP connection. For each send associated with the TCP connection made via the NIC kTLS, transmit accounting automatically updates a counter storing a count of TCP packets used to transmit data over the TCP connection via the NIC kTLS.

[0127] For each retransmission associated with the TCP connection via the NIC kTLS, the transmit accounting function automatically updates a counter storing a count of TCP packets used to retransmit data over the TCP connection via the NIC kTLS, calculates a byte retransmission percentage, and calculates a packet retransmission percentage via the NIC kTLS. If the byte retransmission percentage exceeds the NIC kTLS maximum byte retransmission percentage and / or the packet retransmission percentage via the NIC kTLS exceeds the maximum packet retransmission percentage via the NIC kTLS, the transmit accounting function sets the TLS mode for the TLS session to software kTLS mode instead of NIC kTLS mode. After the kernel switches the TLS session from the NIC kTLS mode to software kTLS mode, the NIC performs TLS data encryption for any previously initiated transmissions and any associated retransmissions, while the kernel performs TLS data encryption for any newly initiated transmissions and any associated retransmissions.

[0128] At least one technical advantage of the disclosed technology relative to the prior art is that, using the disclosed technology, TLS records can be efficiently generated when TCP packets are transmitted over a TCP connection, while also mitigating the risk of DoS attacks. In this regard, because TLS data encryption is offloaded from software to the NIC, the amount of processing resources and system memory bandwidth consumed using the disclosed technology can be reduced relative to software kTLS unless a maximum retransmission threshold is exceeded. In addition, because TLS data encryption is automatically transferred from the NIC to software when the maximum retransmission threshold is exceeded, the overall risk of DoS attacks is reduced relative to NIC kTLS. These technical advantages provide one or more technical advances relative to prior art solutions.

[0129] 1. In some embodiments, a computer-implemented method for encrypting data for transmission includes causing a network interface controller (NIC) to encrypt first data according to a hardware-based Transport Layer Security (TLS) offload mode to generate a first Transmission Control Protocol (TCP) packet; in response to the first TCP packet failing to reach a first device through a first TCP connection, calculating first repetition accounting information; based on the first repetition accounting information, assigning the first TCP connection to a software-based TLS offload mode instead of the hardware-based TLS offload mode; encrypting second data according to the software-based TLS offload mode to generate a second TCP packet; and causing the NIC to transmit the second TCP packet to the first device through the first TCP connection.

[0130] 2. A computer-implemented method as described in clause 1, wherein calculating the first retransmission statistics information includes calculating at least one of a ratio or a percentage based on a total number of data bytes transmitted through the first TCP connection and a total number of data bytes retransmitted through the first TCP connection.

[0131] 3. A computer-implemented method as described in clause 1 or 2, wherein the first repetition counting information is calculated based on a count of TCP packets used to transmit data over the first TCP connection.

[0132] 4. A computer-implemented method as described in any of clauses 1-3, wherein assigning the first TCP connection to the software-based TLS offload mode instead of the hardware-based TLS offload mode includes: comparing the first retransmission counting information with a retransmission threshold, and determining to disable the hardware-based TLS offload mode for the first TCP connection.

[0133] 5. The computer-implemented method of any of clauses 1-4, further comprising initially assigning the first TCP connection to the hardware-based TLS offload mode.

[0134] 6. The computer-implemented method as described in any one of clauses 1-5 further includes causing the NIC to re-encrypt the first data according to the hardware-based TLS offload mode to generate a third TCP packet in response to the first TCP packet failing to reach the first device through the first TCP connection.

[0135] 7. A computer-implemented method as described in any of clauses 1-6, wherein, while the first TCP connection is assigned to the software-based TLS offload mode, one or more TCP packets are transmitted according to the hardware-based TLS offload mode assigned to the second TCP connection.

[0136] 8. A computer-implemented method as described in any of clauses 1-7, wherein assigning the first TCP connection to the software-based TLS offload mode instead of the hardware-based TLS offload mode includes: assigning a TLS session associated with a transmitting side of the first TCP connection to the software-based TLS offload mode instead of the hardware-based TLS offload mode.

[0137] 9. The computer-implemented method as described in any one of clauses 1-8 further includes, before calculating the first repetition counting information, updating at least one of the following items: a count of TCP packets used to transmit data through the first TCP connection, or a total number of data bytes transmitted through the first TCP connection based on the first TCP packets.

[0138] 10. The computer-implemented method of any of clauses 1-9 further includes, before using the NIC to encrypt the first data to generate the first TCP packet, determining that a TLS session associated with a transmitting side of the first TCP connection is assigned to the hardware-based TLS offload mode.

[0139] 11. In some embodiments, one or more non-transitory computer-readable media include instructions that, when executed by one or more processors, cause the one or more processors to encrypt data for transmission by performing the following steps: causing a network interface controller (NIC) to encrypt first data according to a hardware-based Transport Layer Security (TLS) offload mode to generate a first Transmission Control Protocol (TCP) packet; in response to the first TCP packet failing to reach the first device through the first TCP connection, calculating first repetition accounting information; based on the first repetition accounting information, assigning the first TCP connection to a software-based TLS offload mode instead of the hardware-based TLS offload mode; encrypting second data according to the software-based TLS offload mode to generate a second TCP packet; and causing the NIC to transmit the second TCP packet to the first device through the first TCP connection.

[0140] 12. The one or more non-transitory computer-readable media of clause 11, wherein the first repetition counting information is calculated based on a total number of data bytes transmitted over the first TCP connection.

[0141] 13. One or more non-transitory computer-readable media as described in clauses 11 or 12, wherein calculating the first retransmission measurement information includes: calculating a ratio or percentage based on a first count of TCP packets used to transmit data through the first TCP connection and a second count of TCP packets used to retransmit data through the first TCP connection.

[0142] 14. One or more non-transitory computer-readable media as described in any of clauses 11-13, wherein assigning the first TCP connection to the software-based TLS offload mode instead of the hardware-based TLS offload mode includes: comparing the first retransmission counting information with a retransmission threshold, and determining to disable the hardware-based TLS offload mode for the first TCP connection.

[0143] 15. The one or more non-transitory computer-readable media of any of clauses 11-14, further comprising initially assigning the first TCP connection to the hardware-based TLS offload mode.

[0144] 16. One or more non-transitory computer-readable media as described in any of clauses 11-15, further comprising causing the NIC to re-encrypt the first data according to the hardware-based TLS offload mode to generate a third TCP packet in response to the first TCP packet failing to reach the first device through the first TCP connection.

[0145] 17. One or more non-transitory computer-readable media as described in any of clauses 11-16, wherein, while the first TCP connection is assigned to the software-based TLS offload mode, one or more TCP packets are transmitted according to the hardware-based TLS offload mode assigned to the second TCP connection.

[0146] 18. One or more non-transitory computer-readable media as described in any of clauses 11-17, further comprising, before calculating the first retransmission counting information, updating at least one of a count of TCP packets used to retransmit data via the first TCP connection or a total number of data bytes retransmitted via the first TCP connection to reflect the retransmission of encrypted data to the first device via the first TCP connection.

[0147] 19. One or more non-transitory computer-readable media as described in any of clauses 11-18, further comprising, before using the NIC to encrypt the first data to generate the first TCP packet, determining that a TLS session associated with a transmitting side of the first TCP connection is assigned to the hardware-based TLS offload mode.

[0148] 20. In some embodiments, a system includes one or more memories storing instructions and one or more processors coupled to the one or more memories, and when executing the instructions, the one or more processors perform the following steps: causing a network interface controller (NIC) to encrypt first data according to a hardware-based transport layer security (TLS) offload mode to generate a first transmission control protocol (TCP) packet; in response to the first TCP packet failing to reach a first device through a first TCP connection, calculating first repetition accounting information; based on the first repetition accounting information, assigning the first TCP connection to a software-based TLS offload mode instead of the hardware-based TLS offload mode; encrypting second data according to the software-based TLS offload mode to generate a second TCP packet; and causing the NIC to transmit the second TCP packet to the first device through the first TCP connection.

[0149] Any and all combinations of any claim elements recited in any claim and / or any elements described in the present application, in any manner, are within the scope and protection of the contemplated invention.

[0150] The description of the various embodiments is given for purposes of illustration, but is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

[0151] Aspects of the embodiments herein may be embodied as systems, methods, or computer program products. Thus, aspects of the present disclosure may take the form of fully hardware embodiments, fully software embodiments (including firmware, resident software, microcode, etc.), or combined software and hardware embodiments, which may all be collectively referred to herein as "modules" or "systems." Additionally, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer-readable media having computer-readable program code embodied thereon.

[0152] Any combination of one or more computer-readable media may be utilized. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer-readable storage media will include the following: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium may be any tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment.

[0153] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiments of the present disclosure. It will be understood that each square frame of the flowchart and / or block diagram and the combination of each square frame in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine. When the instruction is executed via the processor of a computer or other programmable data processing device, the function / action specified in one or more square frames of the flowchart and / or block diagram can be realized. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable gate array.

[0154] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a fragment or a part of a code, and the module, a fragment or a part of the code includes one or more executable instructions for implementing (one or more) specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, depending on the functions involved, the two boxes shown in succession may actually be executed substantially simultaneously, or the boxes may sometimes be executed in reverse order. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0155] While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the disclosure may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.

Claims

1. A computer-implemented method for encrypting data for transmission, the method comprising: causing a network interface controller (NIC) to encrypt the first data according to a hardware-based transport layer security (TLS) offload mode to generate a first transmission control protocol (TCP) packet; In response to the first TCP packet failing to reach the first device through the first TCP connection, calculating first repetition counting information; assigning the first TCP connection to a software-based TLS offload mode instead of the hardware-based TLS offload mode based on the first recurrence accounting information; encrypting the second data according to the software-based TLS offload mode to generate a second TCP packet; and The NIC transmits the second TCP packet to the first device through the first TCP connection.

2. The computer-implemented method of claim 1, wherein: Calculating the first retransmission statistics information includes calculating at least one of a ratio or a percentage based on a total number of data bytes transmitted over the first TCP connection and a total number of data bytes retransmitted over the first TCP connection.

3. The computer-implemented method of claim 1, wherein: The first repetition counting information is calculated based on a count of TCP packets used to transmit data through the first TCP connection.

4. The computer-implemented method of claim 1, wherein: Assigning the first TCP connection to the software-based TLS offload mode instead of the hardware-based TLS offload mode includes comparing the first retransmission counting information to a retransmission threshold and determining that the hardware-based TLS offload mode is to be disabled for the first TCP connection.

5. The computer-implemented method of claim 1, further comprising initially assigning the first TCP connection to the hardware-based TLS offload mode.

6. The computer-implemented method of claim 1 further comprises causing the NIC to re-encrypt the first data according to the hardware-based TLS offload mode to generate a third TCP packet in response to the first TCP packet failing to reach the first device through the first TCP connection.

7. The computer-implemented method of claim 1, wherein: While the first TCP connection is assigned to the software-based TLS offload mode, one or more TCP packets are transmitted according to the hardware-based TLS offload mode assigned to a second TCP connection.

8. The computer-implemented method of claim 1, wherein: Assigning the first TCP connection to the software-based TLS offload mode instead of the hardware-based TLS offload mode includes assigning a TLS session associated with a transmitting side of the first TCP connection to the software-based TLS offload mode instead of the hardware-based TLS offload mode.

9. The computer-implemented method of claim 1 further comprises, before calculating the first repetition counting information, updating at least one of: a count of TCP packets used to transmit data over the first TCP connection, or a total number of data bytes transmitted over the first TCP connection based on the first TCP packets.

10. The computer-implemented method of claim 1 further comprises, prior to using the NIC to encrypt first data to generate the first TCP packet, determining that a TLS session associated with a transmit side of the first TCP connection is assigned to the hardware-based TLS offload mode.

11. One or more non-transitory computer-readable media comprising instructions that, when executed by one or more processors, cause the one or more processors to encrypt data for transmission by performing the following steps: causing a network interface controller (NIC) to encrypt the first data according to a hardware-based transport layer security (TLS) offload mode to generate a first transmission control protocol (TCP) packet; In response to the first TCP packet failing to reach the first device through the first TCP connection, calculating first repetition counting information; assigning the first TCP connection to a software-based TLS offload mode instead of the hardware-based TLS offload mode based on the first recurrence accounting information; encrypting the second data according to the software-based TLS offload mode to generate a second TCP packet; and The NIC transmits the second TCP packet to the first device through the first TCP connection.

12. The one or more non-transitory computer readable media of claim 11, wherein: The first repetition counting information is calculated based on a total number of data bytes transmitted through the first TCP connection.

13. The one or more non-transitory computer readable media of claim 11, wherein: Calculating the first retransmission statistics information includes calculating a ratio or percentage based on a first count of TCP packets used to transmit data through the first TCP connection and a second count of TCP packets used to retransmit data through the first TCP connection.

14. The one or more non-transitory computer readable media of claim 11, wherein: Assigning the first TCP connection to the software-based TLS offload mode instead of the hardware-based TLS offload mode includes comparing the first retransmission counting information to a retransmission threshold and determining that the hardware-based TLS offload mode is to be disabled for the first TCP connection.

15. The one or more non-transitory computer-readable media of claim 11, further comprising initially assigning the first TCP connection to the hardware-based TLS offload mode.

16. One or more non-transitory computer-readable media as described in claim 11 also include causing the NIC to re-encrypt the first data according to the hardware-based TLS offload mode to generate a third TCP packet in response to the first TCP packet failing to reach the first device through the first TCP connection.

17. The one or more non-transitory computer readable media of claim 11, wherein: While the first TCP connection is assigned to the software-based TLS offload mode, one or more TCP packets are transmitted according to the hardware-based TLS offload mode assigned to a second TCP connection.

18. One or more non-transitory computer-readable media as described in claim 11 further include, before calculating the first retransmission statistics information, updating at least one of a count of TCP packets used to retransmit data through the first TCP connection or a total number of data bytes retransmitted through the first TCP connection to reflect the retransmission of encrypted data to the first device through the first TCP connection.

19. One or more non-transitory computer-readable media as described in claim 11 further comprising, before using the NIC to encrypt the first data to generate the first TCP packet, determining that a TLS session associated with a transmitting side of the first TCP connection is assigned to the hardware-based TLS offload mode.

20. A system comprising: one or more memories for storing instructions; as well as One or more processors coupled to the one or more memories, wherein when executing the instructions, the one or more processors perform the following steps: causing a network interface controller (NIC) to encrypt the first data according to a hardware-based transport layer security (TLS) offload mode to generate a first transmission control protocol (TCP) packet; In response to the first TCP packet failing to reach the first device through the first TCP connection, calculating first repetition counting information; assigning the first TCP connection to a software-based TLS offload mode instead of the hardware-based TLS offload mode based on the first recurrence accounting information; encrypting the second data according to the software-based TLS offload mode to generate a second TCP packet; and The NIC transmits the second TCP packet to the first device through the first TCP connection.

Images