Network card device, network data transmission method, apparatus, system, device and medium

Abstract

The application discloses a network card device, a network data transmission method, device, system, equipment and medium, and is applied to the technical field of Internet. The network card device comprises a programmable device, a network platform is built based on a memory expansion method, and the network platform comprises a network interface, a target memory and a network component. The programmable device is connected with the network through the network interface, and data interaction is carried out with a host through the memory expansion method. The target memory of the programmable device is set as the expansion memory of the host based on the memory expansion method, and the target memory is used for storing the received target network data belonging to the network card processing message type or the to-be-sent network data of the host. The network component adopts a data transmission mode matched with the data message type of the to-be-transmitted data for data transmission and reception, and the data message type is determined according to the data type and the data length of the to-be-transmitted data. The application can solve the problem of low network processing efficiency in the related art, and effectively improve the transmission efficiency of network data.

Description

Technical Field

[0001] This invention relates to the field of Internet technology, and in particular to a network interface card (NIC), a network data transmission method, apparatus, system, electronic device, and readable storage medium. Background Technology

[0002] Network data refers to data transmitted over the Internet and other network environments. It can be structured, organized data information, raw data without any processing, or audio and video data that are published and received.

[0003] During network data transmission and reception, relevant technologies utilize the network card's DMA (Direct Memory Access) module for data transfer operations. For network data transmission, the DMA module sends data from the host memory to the receiving device's buffer; for network data reception, the DMA module sends data from the sending device's buffer to the host memory. However, in some application scenarios, such as when some data does not require host processing, using the DMA module for network data transfer operations is inefficient, resulting in low overall network data transmission efficiency.

[0004] Therefore, improving the efficiency of network data transmission is a technical problem that needs to be solved by professionals in this field. Summary of the Invention

[0005] This invention provides a network interface card (NIC) device, a network data transmission method, apparatus, system, electronic device, and readable storage medium, which can effectively improve the transmission efficiency of network data.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] The first aspect of the present invention provides a network device, including a programmable device, wherein the programmable device is based on a memory expansion method to build a network platform, and includes a network interface, a target memory and network components;

[0008] The programmable device connects to the network via the network interface and interacts with the host via the memory expansion method; the host sets the target memory of the programmable device as the host's extended memory based on the memory expansion method; the network component transmits data using a data transmission method that matches the data packet type of the data to be transmitted.

[0009] The data to be transmitted is either the received target network data or the network data to be sent by the host; the data packet type is determined according to the data type and data length of the data to be transmitted; the extended memory is used to store the data to be transmitted that belongs to the network card processing packet type.

[0010] In a first exemplary embodiment, the network component includes a media access control layer module, a direct memory access module, a buffer, and a packet splitter;

[0011] The media access control layer module is used to store target network data received from the network into the buffer, and is also used to send the network data to be sent in the buffer to the network.

[0012] The packet splitter is used to identify the data packet type of the target network data and determine the corresponding storage location according to the data packet type of the target network data; it is also used to store the network data to be sent, which is read from the target memory or from the host memory through the direct memory access module, into the transmission buffer; wherein the storage location includes the host memory and the target memory.

[0013] In a second exemplary embodiment, the host's central processing unit accesses the target memory via memory copying; the network interface card processes packet types including data packet types with a data length less than a length threshold;

[0014] If the data to be transmitted is the target network data, and the data length of the data to be transmitted is less than the length threshold, the network component stores the data to be transmitted in the target memory and sends a message data read instruction to the host;

[0015] If the data to be transmitted is the network data to be sent, and the data length of the data to be transmitted is less than the length threshold, the host moves the data to be transmitted to the target memory through the memory copy method.

[0016] In a third exemplary embodiment, the network component includes a direct memory access module, and the host's central processing unit accesses the target memory via direct memory access; the network interface card (NIC) processes packet types that do not include data packet types with a data length greater than a length threshold.

[0017] If the data to be transmitted is the target network data, and the data length of the data to be transmitted is greater than the length threshold, the network component moves the data to be transmitted to the host's memory through the direct memory access module;

[0018] If the data to be transmitted is the network data to be sent, and the data length of the data to be transmitted is greater than the length threshold, the host stores the data to be transmitted in the host's memory and sends a message data read instruction to the network component.

[0019] In a fourth exemplary embodiment, the programmable device employs a field-programmable gate array and is built on a network platform based on high-speed computing interconnect technology;

[0020] The host computer configures the target memory of the field-programmable gate array as the extended memory through the high-speed computing interconnect technology.

[0021] The data to be transmitted is the target network data. The field-programmable gate array (FPGA) identifies the data packet type of the data to be transmitted. If the data to be transmitted belongs to a target preset data type and / or the data length of the data to be transmitted is less than a length threshold, then the data to be transmitted is stored in the target memory. Alternatively, if the data to be transmitted is the network data to be sent, the host identifies the data packet type of the data to be transmitted. If the data to be transmitted belongs to a target preset data type and / or the data length of the data to be transmitted is less than a length threshold, then the data to be transmitted is stored in the target memory of the FPGA via memory copy.

[0022] A second aspect of the present invention provides a network data transmission method, comprising:

[0023] Get the data to be transmitted;

[0024] Based on the network interface card device as described in any of the preceding items, a matching data transmission method is determined according to the data packet type of the data to be transmitted, so as to transmit the data to be transmitted over the network according to the data transmission method.

[0025] In a first exemplary embodiment, the data to be transmitted is target network data received from the network, and the network interface card (NIC) device identifies the data packet type of the data to be transmitted; determining the matching data transmission method based on the data packet type of the data to be transmitted includes:

[0026] If the data packet type of the data to be transmitted belongs to the network card processing packet type, then the data to be transmitted is stored in the target memory of the network card device, and a packet data read instruction is sent to the host;

[0027] If the data packet type of the data to be transmitted belongs to the host processing packet type, then the data to be transmitted is stored in the host's memory through the direct memory access module.

[0028] In a second exemplary embodiment, determining the matching data transmission method based on the data packet type of the data to be transmitted includes:

[0029] If the data type of the data to be transmitted is a preset data type, then the data packet type of the data to be transmitted belongs to the network interface card (NIC) processing packet type; wherein, the preset data type is data that is not stored in the host's memory;

[0030] If the length of the data to be transmitted is less than the length threshold, then the data packet type of the data to be transmitted belongs to the network card processing packet type.

[0031] In a third exemplary implementation, the host accesses the target memory via memory copying, including:

[0032] If the data to be transmitted belongs to a first preset data type, the host reads the data to be transmitted and discards it directly;

[0033] If the data to be transmitted belongs to a second preset data type, the host reads the data to be transmitted and forwards the data to be transmitted;

[0034] If the length of the data to be transmitted is less than the length threshold, the host calls the memory copy function to copy the data to be transmitted into the host's memory, so that the host can access the target memory through memory copy.

[0035] In a fourth exemplary implementation, the host accesses the target memory via memory copying, including:

[0036] If the data to be transmitted is cyclic redundancy check code data, the host reads the data to be transmitted and discards it directly.

[0037] If the data to be transmitted is a message to be forwarded, the host reads the data to be transmitted and forwards the data to the target address.

[0038] In the fifth exemplary embodiment, before determining the matching data transmission method based on the data packet type of the data to be transmitted, the method further includes:

[0039] Data of different lengths were transmitted using both memory copy and direct memory access methods, and the corresponding transmission times were recorded.

[0040] The transmission time required for transmitting the same data length using the memory copy method and the direct memory access method is compared and analyzed to determine the data length corresponding to the same transmission time for the memory copy method and the direct memory access method, which is then used as a length threshold.

[0041] In the sixth exemplary embodiment, before determining the matching data transmission method based on the data packet type of the data to be transmitted, the method further includes:

[0042] When a length threshold configuration instruction is received from the user, the length threshold is obtained by parsing the length threshold configuration instruction and stored in the target location.

[0043] In the seventh exemplary embodiment, the data to be transmitted is network data to be sent by the host, the host identifies the data packet type of the data to be transmitted, and the step of determining the matching data transmission method according to the data packet type of the data to be transmitted includes:

[0044] If the data packet type of the data to be transmitted belongs to the network card processing packet type, then the data to be transmitted is stored in the target memory of the network card device, and a packet data read instruction is sent to the network card device;

[0045] If the data packet type of the data to be transmitted belongs to the host processing packet type, then the data to be transmitted is stored in the host's memory, and a packet data read instruction is sent to the network card device.

[0046] In the eighth exemplary embodiment, determining the matching data transmission method based on the data packet type of the data to be transmitted includes:

[0047] A storage data index is pre-built in the target memory of the network interface card device; the storage data index is used to record the storage location information of received and sent packets.

[0048] Update the stored data index information based on the data packet type of the data to be transmitted;

[0049] Based on the stored data index information and the sending end information of the data to be transmitted, the data transmission method corresponding to the data to be transmitted is determined.

[0050] In the ninth exemplary embodiment, the data to be transmitted is target network data, and determining the data transmission method corresponding to the data to be transmitted based on the stored data index information and the sending end information of the data to be transmitted includes:

[0051] A network interface card (NIC) packet receiving queue is pre-built in the target memory of the NIC device, and a host packet receiving queue is built in the host's memory.

[0052] The queue pointers of the receiver producers corresponding to the network interface card (NIC) packet receiving queue and the host packet receiving queue are pre-set to their respective initialization positions; the receiver producers include buffer producers and direct memory access module producers;

[0053] Based on the receiving queue containing the stored data index information, if the data to be transmitted belongs to the network interface card (NIC) processing message type, the data to be transmitted is stored in the pointer position of the buffer producer in the NIC message receiving queue, and the pointer of the buffer producer is moved down; if the data to be transmitted belongs to the host processing message type, the data to be transmitted is stored in the pointer position of the direct memory access module (DMI) producer in the host message receiving queue, and the pointer of the DMI producer is moved down.

[0054] In a tenth exemplary embodiment, after transmitting the data to be transmitted over the network according to the data transmission method, the method further includes:

[0055] The queue pointers of the receiving consumers corresponding to the network interface card packet receiving queue and the host packet receiving queue are pre-set to their respective initialization positions; the receiving consumer is the packet data sending and receiving thread of the host; the pointer position of the receiving consumer is less than the pointer position of the receiving producer;

[0056] The host determines the storage location of the storage queue corresponding to the data to be transmitted according to the receiving queue, reads the data to be transmitted according to the storage location of the data to be transmitted, and moves the pointer of the receiving consumer down.

[0057] In the eleventh exemplary embodiment, the data to be transmitted is network data to be sent, and determining the data transmission method corresponding to the data to be transmitted based on the stored data index information and the sending end information of the data to be transmitted includes:

[0058] A network interface card (NIC) packet sending queue is pre-built in the target memory of the NIC device, and a host packet sending queue is built in the host's memory.

[0059] The queue pointers of the sending producers corresponding to the network interface card packet receiving queue and the host packet receiving queue are pre-set to their respective initialization positions; the sending producer is the host's packet data sending and receiving thread;

[0060] The queue pointer of the sending producer corresponding to the sending queue storing the data index information is pre-set to the corresponding initialization position, and when the sending queue is updated, the pointer position of the sending producer is moved down;

[0061] Based on the sending queue containing the stored data index information, if the data to be transmitted belongs to the network card processing message type, then the data to be transmitted is stored in the network card message receiving queue, and the pointer of the sending producer is moved down; if the data to be transmitted belongs to the host processing message type, then the data to be transmitted is stored in the host message receiving queue, and the pointer of the sending producer is moved down.

[0062] In the twelfth exemplary embodiment, after transmitting the data to be transmitted over the network according to the data transmission method, the method further includes:

[0063] The queue pointers of the sending consumers corresponding to the network interface card packet sending queue and the host packet sending queue are pre-set to their respective initialization positions; the sending consumers are buffer consumers and direct memory access module consumers; the pointer position of the sending consumer is less than the pointer position of the sending producer;

[0064] Based on the sending queue of the stored data index information, if the data to be stored belongs to the network card processing message type, then the data to be stored is read from the network card message sending queue, and the pointer of the buffer consumer is moved down; if the data to be stored belongs to the host processing message type, then the data to be stored is read from the host message sending queue, and the pointer of the direct memory access module consumer is moved down.

[0065] A third aspect of the present invention provides a network data transmission apparatus, comprising:

[0066] The data acquisition module is used to acquire the data to be transmitted.

[0067] A network transmission module is used to determine a matching data transmission method based on the data packet type of the data to be transmitted, according to the network interface card device as described in any of the preceding items, so as to transmit the data to be transmitted over the network according to the data transmission method.

[0068] A fourth aspect of the present invention also provides an electronic device including a processor, the processor being configured to implement the steps of the network data transmission method as described in any of the preceding claims when executing a computer program stored in a memory.

[0069] A fifth aspect of the present invention also provides a readable storage medium storing a computer program that, when executed by a processor, implements the steps of the network data transmission method as described in any of the preceding claims.

[0070] A sixth aspect of the present invention provides a network data transmission system, including a host, a network interface card (NIC) device as described in the preceding item, and a processor;

[0071] The processor is used to implement the network data transmission method as described in any of the preceding claims when executing a computer program stored in the memory.

[0072] The advantage of the technical solution provided by this invention is that it utilizes programmable devices and memory expansion technology to realize the function of a high-performance network card. The host configures a block of memory in the programmable device as its own extended memory through memory expansion technology. This extended memory can be used to store network data of a pre-specified message type, instead of storing it all in the host's memory. For some network data that does not require host processing or for which data transfer using the direct memory access module is inefficient, it can be directly stored in the extended memory. That is, the optimal data transmission method is determined according to the message type of the data to be transmitted, thereby effectively improving the transmission efficiency of network data.

[0073] Furthermore, the present invention also provides a corresponding network data transmission method, implementation device, system, electronic device, and readable storage medium for network interface card (NIC) devices, further making the NIC devices more practical and feasible. The network data transmission method, device, electronic device, system, and readable storage medium have corresponding advantages.

[0074] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit the invention. Attached Figure Description

[0075] To more clearly illustrate the technical solutions of the present invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1 A schematic diagram of the data reception process of a network interface card (NIC) device in the related technology provided by this invention;

[0077] Figure 2 This invention provides a schematic diagram of the data transmission process of a network interface card (NIC) device in the related technology.

[0078] Figure 3 A schematic diagram of the structural framework of a specific embodiment of the network interface card device provided by the present invention;

[0079] Figure 4 A schematic diagram of the structural framework of another specific embodiment of the network interface card device provided by the present invention;

[0080] Figure 5 A schematic diagram illustrating host access to extended memory provided by the present invention;

[0081] Figure 6 A flowchart illustrating a network data transmission method provided by the present invention;

[0082] Figure 7 A schematic diagram of the structural framework of the network transmission system provided by the present invention in an exemplary embodiment;

[0083] Figure 8 The present invention provides a basis for Figure 7 A schematic diagram of the message data receiving process;

[0084] Figure 9 The present invention provides a basis for Figure 7 A schematic diagram of the host data reading process;

[0085] Figure 10 The present invention provides a basis for Figure 7 A schematic diagram of the message data transmission process;

[0086] Figure 11 The present invention provides a basis for Figure 7 A schematic diagram of the host data distribution process;

[0087] Figure 12 A structural diagram of a specific embodiment of the network data transmission device provided by the present invention;

[0088] Figure 13 A structural diagram of a specific embodiment of the electronic device provided by the present invention;

[0089] Figure 14 This is a structural diagram of another specific embodiment of the network data transmission system provided by the present invention. Detailed Implementation

[0090] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. In this specification, the terms "first," "second," "third," "fourth," etc., in the description, claims, and the aforementioned drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. The term "exemplary" means "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior to or better than other embodiments.

[0091] Understandably, a network interface card (NIC) is a piece of computer hardware that enables a computer host to communicate on a computer network. Traditionally, NICs were plugged into the host bus as expansion cards. However, due to their low price and the widespread availability of the Ethernet standard, computer hosts can now integrate network interfaces on the motherboard. These motherboards either integrate Ethernet functionality in the motherboard chipset or connect an inexpensive NIC to the host motherboard via the PCI-Express (peripheral component interconnect express, high-speed serial computer expansion bus standard) bus.

[0092] For a network transmission system that connects the network card to the host via a PCI-Express bus, the host performs pre-initialization: it allocates two memory spaces in its local memory, namely memory space 1 and memory space 2. These two memory spaces are used to store received data packets and sent data packets, respectively. After initialization, network data packet sending and receiving operations are performed. The corresponding data packet receiving process is as follows: Figure 1 As shown, the process includes: 1. The network interface card (NIC) receives network data through the network port and caches the data in the MAC (Media Access Control) module; 2. The MAC module sends a data transfer operation to the DMA (Direct Memory Access) module to perform the data transfer; 3. The DMA module moves the received packet data to the host's first memory block and then notifies the host's packet data transmission and reception threads to read the corresponding data. Thus, the packet data transmission and reception threads can retrieve the corresponding packet data from the first memory block. The corresponding data packet transmission process is as follows: Figure 2 As shown, the process may include: 1. The message data transmission and reception thread stores the message data to be sent in the second memory block and sends a data read command to the network card. 2. The DMA module of the network card retrieves the data to be sent from the second memory block of the host and stores it in the network card's buffer. 3. The MAC module retrieves the data to be sent from the buffer and begins the data transmission process. 4. The network port completes the data transmission process.

[0093] Combination Figure 1 and Figure 2It is known that related technologies use DMA modules to perform data transfer operations during both network data reception and transmission. Sending involves transferring data from the host's memory to the network card's buffer, while receiving involves transferring data from the network card's buffer to the host's memory. In network transmission, some network data that does not require host processing, such as CRC (Cyclic Redundancy Check) error data, does not need to be transferred to host memory and can be processed and discarded on the network card. Some packet data that needs to be forwarded also does not need to be copied to host memory and can be processed directly in the network card's memory. Furthermore, based on the inherent characteristics of DMA modules, they are more suitable for transferring large amounts of data. For some data packets that are too short, using DMA modules for data transfer is inefficient. Therefore, it is evident that related technologies using network cards for network data transmission are inefficient and cannot meet the practical needs of users.

[0094] Therefore, this invention utilizes programmable devices and memory expansion technology to achieve the functionality of a high-performance network card. The host computer configures a block of memory from the programmable device as its own usable extended memory through memory expansion technology. For network data that does not require host processing or where data transfer via direct memory access (DMA) is inefficient, it can be directly stored in this extended memory, eliminating the need to store all data to the host computer via DMA, thus effectively improving network data transmission efficiency. After introducing the technical solution of this invention, various non-limiting embodiments are described in detail below. To better illustrate this invention, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this invention can be implemented without these specific details. In other examples, methods, means, components, and circuits well known to those skilled in the art are not described in detail to highlight the main points of this invention.

[0095] Please see first. Figure 3 , Figure 3 This is a schematic diagram of the structural framework of the network interface card (NIC) device provided in this embodiment under an exemplary implementation. This embodiment may include the following:

[0096] The network interface card (NIC) device may include a programmable device 31, which builds a network platform based on a memory expansion method. The network platform built by the programmable device 31 may include a network interface 310, a target memory 311, and a network component 312.

[0097] In this embodiment, the programmable device 31 connects to the network via a network interface 310. This network interface 310 can be pre-compiled to enable the transmission and reception of network data; that is, it can both receive target network data sent by the network and send network data to be sent by the host to the network. The entire process may include: receiving data packets from the network through a physical layer interface (such as an Ethernet interface); the received data packets are demodulated and decoded by the physical layer, converted into digital signals, and then the digital signals are passed to the receive buffer. The data packets in the receive buffer are transmitted to the data processing unit. The data processing unit parses the data packets and extracts information such as the destination address, source address, and protocol type. According to the protocol type, the data processing unit passes the data packets to the corresponding protocol processing unit, such as the TCP / IP (Transmission Control Protocol / Internet Protocol) protocol stack. The protocol processing unit performs protocol parsing and processing on the data packets, such as the layered processing of the TCP / IP protocol stack. The protocol processing unit processes the data packets according to protocol rules, such as checking checksums, fragmentation and reassembly, error detection, etc., and can also perform some advanced functions such as packet filtering and flow control. After protocol processing, the data packets are transmitted to the send buffer. The data packets in the transmit buffer are transferred to the transmit data unit. The transmit data unit converts the data packets into digital signals and sends them to the network through the physical layer interface.

[0098] In this embodiment, the programmable device 31 is inserted into the host as a network interface card (NIC). The host can interact with the programmable device 31 via a memory expansion method. Simultaneously, when the host initializes and configures the programmable device 31, it can set the target memory 311 of the programmable device 31 as the host's extended memory based on the memory expansion method, allowing the host to read and write to this extended memory. The programmable device 31 can be any programmable device capable of handling large amounts of complex data transmission, and the memory expansion method can be any memory expansion technology compatible with the programmable device 31 and the host; this does not affect the implementation of the present invention. The network component 312 is a set of hardware for transmitting and receiving data. It can use a data transmission method matching the data packet type of the data to be transmitted. In this embodiment, the data to be transmitted is target network data received from the network or network data received from the host that the host wants to send to the network. The data packet type is determined according to the data type and length of the data to be transmitted; that is, different data types correspond to different data transmission methods, and different data lengths correspond to different data transmission methods. The data transmission method in this embodiment refers to whether the data received from the data sender is stored in the host's memory or in the target memory (i.e., extended memory) of the programmable device 31. In other words, there are two transmission methods: one is to store the data received from the data sender in the host's memory, and the other is to store the data received from the data sender in the extended memory of the programmable device 31. Whether to store it in the host's memory or the extended memory is determined by the data type and length of the data to be transmitted. The data packet type can be predefined and assigned a unique identifier for identification. For ease of description, this embodiment defines the data packet type corresponding to storing the data received from the data sender in the extended memory of the programmable device 31 as a network interface card (NIC) processing packet type. The data to be transmitted corresponding to this NIC processing packet type either does not require host processing or is transferred to host memory via a DMA module, which is time-consuming. Therefore, to improve network data transmission efficiency, it is directly stored in the extended memory, allowing the host to access the data in the extended memory and perform corresponding processing.

[0099] As can be seen from the above, this embodiment utilizes programmable devices and memory expansion technology to realize the function of a high-performance network card. The host configures a block of memory in the programmable device as its own extended memory through memory expansion technology. This extended memory can be used to store network data of a pre-specified message type, instead of storing it all in the host's memory. For some network data that does not require host processing or for which data transfer using the direct memory access module is inefficient, it can be directly stored in the extended memory. That is, the optimal data transmission method is determined according to the message type of the data to be transmitted, thereby effectively improving the transmission efficiency of network data.

[0100] The above embodiments do not limit the structure of the network component 312 of the programmable device 31. This embodiment also provides an exemplary implementation, such as... Figure 4 As shown, it may include the following:

[0101] Network component 312 may include a media access control layer module 3120, a direct memory access module 3121, a buffer 3122, and a packet splitter 3123. The media access control layer module 3120 consists of a hardware controller and a MAC communication protocol. The direct memory access module 3121 enables efficient data transmission within the computer system without the intervention of the central processing unit or data copying. The buffer 3122 is a cache space of the programmable device 31. The packet splitter 3123 is a device that stores a computer program used to identify data packet types and distribute data to be transmitted.

[0102] In this embodiment, the media access control layer module 3120 is used to store target network data received from the network into a buffer 3122, and also to send the network data to be sent in the buffer 3122 to the network. The packet splitter 3123 is used to identify the data packet type of the target network data and determine the corresponding storage location according to the data packet type of the target network data; the storage location includes the host's memory and the target memory. The packet splitter 3123 is also used to store the network data to be sent, read from the target memory or from the host's memory through the direct memory access module 3121, into the delivery buffer 3122.

[0103] Based on this, the host can access the extended memory on the programmable device 31 in two ways, such as Figure 5As shown, the host's central processing unit (CPU) can access the target memory via memory copying. For example, it can access extended memory by calling memory copy functions such as `memcpy` (the memory copy function used in C and C++). For instance, for network interface cards (NICs) handling data packets whose length is less than a certain threshold, the memory copying process includes: if the data to be transmitted is target network data and its length is less than the threshold, the network component stores the data to be transmitted in the target memory and sends a data read command to the host; if the data to be transmitted is network data to be sent and its length is less than the threshold, the host moves the data to be transmitted to the target memory via memory copying. Another access method is to move data using the DAM module, i.e., accessing the target memory via direct memory access. For example, the network component includes a direct memory access module. For network interface cards (NICs), the packet types processed do not include data packets with a length greater than a length threshold. The implementation process of direct memory access to target memory may include: if the data to be transmitted is target network data and the data length of the data to be transmitted is greater than the length threshold, the network component moves the data to be transmitted to the host's memory through the direct memory access module; if the data to be transmitted is network data to be sent and the data length of the data to be transmitted is greater than the length threshold, the host stores the data to be transmitted in the host's memory and sends a packet data read instruction to the network component. Therefore, the two access methods provided in this embodiment are suitable for data of different lengths. Memory copying (such as the memcpy function) is more efficient for short data, while DMA moving is more efficient for long data. As can be seen from the above, the network component in this embodiment can achieve efficient transmission of network data to be sent from the host and target network data from the network through a MAC module, a DMA module, a buffer, and a packet splitter.

[0104] It is understandable that FPGAs (Field-Programmable Gate Arrays) can be programmed to implement various logical functions. This flexibility and reconfigurability allows FPGAs to be customized and optimized according to user needs, leading to their widespread application. Network interface cards (NICs) need to handle large amounts of data flow, including packet reception, processing, and transmission. FPGAs have parallel processing capabilities and can achieve high-speed data processing and transmission by designing efficient data paths and parallel computing units. In other words, FPGAs have high-speed data processing capabilities and can support the data processing capabilities required by NICs. FPGAs can be programmed and configured according to specific needs to implement various logical functions. For NICs, the logic for data processing and transmission can be customized and optimized according to different network protocols and requirements to improve performance and efficiency. FPGAs can support various interfaces and protocols, such as Ethernet and PCIe, through hardware design and programming. Furthermore, by designing adapters and interface circuits, FPGAs can connect and communicate with computer systems and networks. Therefore, FPGAs can implement the interfaces and protocols required by NICs. Based on this, as an exemplary implementation, the aforementioned programmable device 31 can be an FPGA. Furthermore, CXL (Compute Express Link) provides high-bandwidth, low-latency connectivity and can be used to connect FPGAs with processors, accelerators, storage devices, etc. Therefore, this embodiment can combine it with an FPGA to implement network card functionality. Correspondingly, the programmable device 31 can also include a CXL module. The CXL module interconnects the FPGA with the CXL, enabling the host to expand the memory of the programmable device 31, achieving high-performance network card functionality, supporting high-speed data transmission and processing. The host configures the FPGA's target memory as extended memory through the CXL module. For the data reception process, i.e., when the data to be transmitted is the target network data, the FPGA identifies the data packet type of the data to be transmitted. If the data to be transmitted belongs to the target preset data type and / or the data length of the data to be transmitted is less than the length threshold, then the data to be transmitted is stored in the target memory. For the data transmission process, that is, the data to be transmitted is the network data to be sent by the host, the host identifies the data packet type of the data to be transmitted. If the data to be transmitted belongs to the target preset data type and / or the data length of the data to be transmitted is less than the length threshold, the host stores the data to be transmitted in the target memory of the FPGA through memory copying. That is, the host directly stores the data to be transmitted to the network in its extended memory, rather than storing it in its local memory.

[0105] Furthermore, considering that CXL's Type 3 (third generation) devices are a set of memory modules providing persistent, volatile, or composable memory, supporting the .io (Input / Output) and .mem (memory) protocols. The CXL .io protocol can be used for device discovery, enumeration, error reporting, and management, allowing the device to be used for other I / O-specific applications. In the CXL .mem protocol, the host's CPU acts as the master device, sending requests, and the device responds as a slave device. Therefore, the Type 3 device can be used as a memory extender for the host because there is no cache; the host's CPU does not need to snoop on the device cache and can use the streamlined .mem protocol. Thus, the host can expand memory by configuring the CXL to TPYE3 mode.

[0106] As can be seen from the above, this embodiment utilizes the flexibility and customizability of FPGA to program and optimize according to specific needs, and utilizes CXL to provide high-speed interconnection and interface support to achieve high-bandwidth, low-latency connection with other devices. Thus, combining FPGA and CXL can achieve high-performance network card functionality.

[0107] Furthermore, based on the network interface card (NIC) device provided in the above embodiments, this invention also provides an implementation process for data transmission and reception based on the NIC device, i.e., a network data transmission method. Please refer to [link to relevant documentation]. Figure 6 , Figure 6 This is a flowchart illustrating a network data transmission method provided in this embodiment. This embodiment may include the following:

[0108] S601: Obtain the data to be transmitted.

[0109] The data to be transmitted can be target network data received from the network or network data to be sent from the host.

[0110] S602: Determine the matching data transmission method based on the data packet type of the data to be transmitted, and then transmit the data to be transmitted over the network according to the data transmission method.

[0111] This embodiment is based on the network interface card (NIC) device as described in the previous embodiment, which determines the corresponding data transmission method for the data to be transmitted. Data transmission includes a data sending process and a data receiving process. Specifically, if the data to be transmitted is target network data received from the network, the data transmission is a data receiving process, which means storing the target network data at a location accessed by the host, which can be the host's memory or the target memory of a programmable device, i.e., the host's extended memory. If the data to be transmitted is network data to be sent from the host, the data transmission is a data sending process, which means reading the data to be transmitted from the host's memory or extended memory and then sending it to the network through the NIC device's network interface.

[0112] In the technical solution provided in this embodiment, the function of a high-performance network card is realized by using programmable devices and memory expansion technology. The host configures a block of memory in the programmable device as its own extended memory through memory expansion technology. This extended memory can be used to store network data of a pre-specified message type, instead of storing it all in the host's memory. For some network data that does not require host processing or for which data transfer using the direct memory access module is inefficient, it can be directly stored in the extended memory. That is, the optimal data transmission method is determined according to the message type of the data to be transmitted, thereby effectively improving the transmission efficiency of network data.

[0113] In the above embodiments, no limitations are made on the entire process of determining the matching data transmission method based on the data packet type of the data to be transmitted. The present invention also provides an exemplary implementation method, which may include the following:

[0114] In this embodiment, the data type of the message to be transmitted is defined as the network card processing message type and the host processing message type. The network card processing message type is the data message type that does not need to be stored in the host's memory, and the host processing message type is the data message type that needs to be stored in the host's memory.

[0115] For example, to further improve practicality, the network interface card (NIC) processing message types in this embodiment may include two types: preset data types and data length less than a length threshold. All others are host processing message types. Correspondingly, the process for identifying the data type of the data to be transmitted can be as follows: if the data type of the data to be transmitted is a preset data type, then the data message type of the data to be transmitted belongs to the NIC processing message type; wherein, the preset data type is data not stored in the host's memory; if the data length of the data to be transmitted is less than the length threshold, then the data message type of the data to be transmitted belongs to the NIC processing message type. Further, the preset data types can be divided into two categories: one is data requiring subsequent processing, and the other is data not requiring subsequent processing. The host processor processes these two types of data as follows: if the data to be transmitted belongs to the first preset data type, the host reads the data to be transmitted and directly discards it; the first preset data type may be, for example, cyclic redundancy check (CRC) data. Correspondingly, if the data to be transmitted is CRC data, the host reads the data to be transmitted and directly discards it. If the data to be transmitted belongs to the second preset data type, the host reads the data to be transmitted and forwards it. The second preset data type could be, for example, a packet to be forwarded. Correspondingly, if the data to be transmitted is a packet to be forwarded, the host reads the data and forwards it to the target address. For network interface card (NIC) processing packet types where the data length is less than a length threshold, if the data length of the data to be transmitted is less than the length threshold, the host calls a memory copy function to copy the data to be transmitted into the host's memory.

[0116] As can be seen from the above, in this embodiment, when a received packet contains a CRC error, the packet is stored in the network interface card (NIC) device, thus eliminating the need for a DMA module to move the erroneous data to the host memory. When the received data packet is forwarded data, it is also stored in the NIC device for easy forwarding. For short data packets during transmission and reception, the host directly handles the data movement, eliminating the need for a DMA module, thereby effectively improving network data transmission efficiency.

[0117] Regarding the length threshold in the above embodiments, as a simple implementation method, it can be preset based on prior knowledge or a default value such as 1KB can be set and stored locally. That is, when a length threshold configuration instruction is received from the user, the length threshold is obtained by parsing the length threshold configuration instruction and stored in the target location. However, this method is not accurate and is not conducive to the efficient transmission of network data.

[0118] As a more accurate implementation, the length threshold can be determined through convergence testing: Data transmission of different lengths is performed using both memory copy and direct memory access methods, and the corresponding transmission times are recorded. The transmission times required for transmitting the same length of test data using both methods are compared and analyzed to determine the data length corresponding to the same transmission time, which serves as the length threshold. For ease of description, the method where the host's CPU moves data from extended memory to host memory by calling a memory copy function can be defined as memory copy, and the method where data is moved from the network interface card's buffer to host memory via a DMA module can be defined as direct memory access. During convergence testing, when the length of the speed-tested data transmission equals the length threshold, the DMA operation (i.e., direct memory access) and the memory copy operation (i.e., memory copy) take the same amount of time. When the length exceeds the length threshold, the data transfer operation using direct memory access takes less time; when the length is less than the length threshold, the operation using memory copy takes less time. The testing process may include: The test data transfer length is 1KB. Timing begins, and the test data is moved using both direct memory access and memory copy methods. Timing ends when the data transfer is successful. The required transfer time for each method is compared. If the direct access method takes longer, the test data transfer length needs to be increased; if the memory copy method takes longer, the test data transfer length needs to be decreased. This convergence test is performed to find the data length for which both methods take the same amount of time, and this length is used as the length threshold.

[0119] Therefore, it can be seen that this embodiment can maximize the efficiency of network data transmission by determining the data transfer operation method based on the length threshold.

[0120] The data reception process, where the data to be transmitted is target network data received from the network, involves determining the matching data transmission method based on the data packet type of the data to be transmitted. This process includes: the network interface card (NIC) receiving the target network data from the network and using its packet analyzer to identify the data packet type of the data to be transmitted. If the data packet type belongs to the NIC's processing type, the data to be transmitted is stored in the NIC's target memory (i.e., the host's extended memory), and a data read command is sent to the host. For data that needs to be copied to the host's memory, the host uses a memory copy method, such as calling a memory copy function, to access the target memory and copy the corresponding data to the host's memory. For data that does not need to be copied to the host, the host can directly read the corresponding data and perform the appropriate operations according to the data type. If the data packet type belongs to the host's processing type, the data to be transmitted can be stored in the host's memory through the NIC's direct memory access module.

[0121] For the data transmission process, i.e., the data to be transmitted is network data sent from the host to the network, the host identifies the data packet type of the data to be transmitted. If the data packet type belongs to the network interface card (NIC) processing type, the data to be transmitted is stored in the target memory of the NIC device, i.e., the host's extended memory, and a data packet read command is sent to the NIC device's packet divider. The packet divider then reads the data to be transmitted and stores it in a buffer. The buffer, based on the MAC module, sends it to the network through the network interface. If the data packet type belongs to the host processing type, the data to be transmitted is stored in the host's memory, and a data packet read command is sent to the NIC device. The packet divider then reads the data to be transmitted through the DMA module and stores it in a buffer. The buffer, based on the MAC module, sends it to the network through the network interface.

[0122] Therefore, this embodiment determines whether the data to be transmitted is stored in the host memory or the host's extended memory based on the different data types of the message during the network data transmission and reception process, thereby achieving data transmission with maximum efficiency.

[0123] Furthermore, to improve network data transmission efficiency, based on the above embodiments, before data packets are sent and received, the host of this invention can perform initialization configuration on the network interface card (NIC). The initialization process may include: the host allocating memory for a host packet receiving queue storing packets received by the host and a host packet sending queue storing packets sent by the host. On the extended memory, the host can allocate space for the NIC packet receiving queue storing packets received by the NIC and the NIC packet sending queue storing packets to be sent by the NIC. On the extended memory, the host can allocate space for storing data index information.

[0124] Based on the above configuration process, an exemplary method for determining the data transmission method corresponding to the data to be transmitted is as follows: First, a data storage index is pre-built in the target memory of the network interface card (NIC) device; second, the data storage index is updated based on the data packet type of the data to be transmitted; third, the data transmission method corresponding to the data to be transmitted is determined according to the data storage index and the sending end information of the data to be transmitted. In this embodiment, the data storage index can be used to record the storage location information of received and sent packets. For example, the storage location information can be represented by a packet descriptor. Correspondingly, the data storage index may include a sending queue and a receiving queue. The sending queue is used to record sent packet descriptors, and the receiving queue is used to record received packet descriptors. The sending end information indicates whether the data to be transmitted is data sent by the host or data from the network. Once the data source is determined, it can be confirmed whether the data is being sent or received. Combined with the data storage location recorded in the data storage index, it can be determined whether the data to be transmitted is stored in the host's memory or in the host's extended memory.

[0125] The implementation of reading network data during the receiving process in determining the data transmission method corresponding to the data to be transmitted can be as follows: The data to be transmitted is target network data. A network interface card (NIC) packet receiving queue is pre-built in the target memory of the NIC device, and a host packet receiving queue is pre-built in the host's memory. The queue pointers of the receiving producers corresponding to the NIC packet receiving queue and the host packet receiving queue are pre-set to their respective initialization positions. The receiving producers include buffer producers and direct memory access module producers. Based on the receiving queue storing data index information, if the data to be transmitted belongs to the NIC processing packet type, the data to be transmitted is stored in the pointer position of the buffer producer in the NIC packet receiving queue, and the pointer of the buffer producer is moved down. If the data to be transmitted belongs to the host processing packet type, the data to be transmitted is stored in the pointer position of the direct memory access module producer in the host packet receiving queue, and the pointer of the direct memory access module producer is moved down. After the data to be transmitted is stored in the corresponding location, in order to facilitate data reading and storage, the movement trajectory can be recorded based on the position of the pointer, so as to know the starting position next. The process of the host reading the data to be transmitted can be as follows: the queue pointers of the receiving consumers corresponding to the network card packet receiving queue and the host packet receiving queue are pre-set to the corresponding initial positions; the receiving consumer is the host's packet data sending and receiving thread; the pointer position of the receiving consumer is less than the pointer position of the receiving producer; the host determines the storage position of the storage queue corresponding to the data to be transmitted according to the receiving queue, reads the data to be transmitted according to the storage position of the data to be transmitted, and moves the pointer of the receiving consumer down.

[0126] The implementation of reading network data during the transmission process, which involves determining the data transmission method corresponding to the data to be transmitted, can be as follows: The data to be transmitted is the network data to be sent. A network interface card (NIC) packet sending queue is pre-built in the target memory of the NIC device, and a host packet sending queue is pre-built in the host's memory. The queue pointers of the sending producers corresponding to the NIC packet receiving queue and the host packet receiving queue are pre-set to their respective initialization positions. The sending producer is the host's packet data transmission and reception thread. The queue pointer of the sending producer corresponding to the sending queue storing data index information is pre-set to its respective initialization position, and the pointer position of the sending producer is moved down when updating the sending queue. Based on the sending queue storing data index information, if the data to be transmitted belongs to the NIC processing packet type, the data to be transmitted is stored in the NIC packet receiving queue, and the pointer of the sending producer is moved down. If the data to be transmitted belongs to the host processing packet type, the data to be transmitted is stored in the host packet receiving queue, and the pointer of the sending producer is moved down. After the data to be transmitted is stored in the corresponding location, to facilitate data reading and storage, the movement trajectory can be recorded based on the pointer position to determine the starting position for the next transmission. The process of the network interface card (NIC) device reading the data to be transmitted can be as follows: The queue pointers of the sending consumers corresponding to the NIC packet sending queue and the host packet sending queue are pre-set to the corresponding initialization positions; the sending consumers are buffer consumers and direct memory access module consumers; the pointer position of the sending consumer is lower than the pointer position of the sending producer; based on the sending queue containing the stored data index information, if the data to be stored belongs to the NIC processing packet type, the data to be stored is read from the NIC packet sending queue, and the pointer of the buffer consumer is moved down; if the data to be stored belongs to the host processing packet type, the data to be stored is read from the host packet sending queue, and the pointer of the direct memory access module consumer is moved down.

[0127] To enable those skilled in the art to better understand the technical solution of this invention, this invention also provides an exemplary embodiment, which may include the following:

[0128] like Figure 7As shown, the network interface card (NIC) device in this embodiment is an FPGA+CXL. The FPGA's network components include a buffer, a DMA module, a packet splitter, and a MAC module. The host configures the CXL to TPYE3 mode for memory expansion. The functional module that implements memory expansion is the CXL module. The host's memory includes a host packet receive queue and a host packet send queue. The host's expanded memory includes the NIC packet receive queue, the NIC packet send queue, and stored data index information. The stored data index information includes a send queue and a receive queue. Initialize the host packet receive queue, host packet send queue, network interface card (NIC) packet receive queue, NIC packet send queue, send queue storing data index information, and receive queue to ensure that each queue stores the corresponding data sequentially. For example, the host packet receive queue stores the first, second, fifth, seventh, and Nth frame received data sequentially; the host packet send queue stores the first, second, sixth, seventh, and Nth frame sent data sequentially; the NIC packet receive queue stores the third, fourth, sixth, eighth, and Nth frame received data sequentially; and the NIC packet send queue stores the third, fourth, fifth, eighth, and Nth frame sent data sequentially. The send and receive queues record the location of the data to be transmitted. The first frame received data is stored in the host packet receive queue, which can be represented by the character 1: RX-H; the second frame received data is stored in the host packet receive queue, which can be represented by the character 2: RX-H; the third frame received data is stored in the network card packet receive queue, which can be represented by the character 3: RX-C; the fourth frame received data is stored in the network card packet receive queue, which can be represented by the character 4: RX-C; the fifth frame received data is stored in the host packet receive queue, which can be represented by the character 5: RX-H; the sixth frame received data is stored in the network card packet receive queue, which can be represented by the character 6: RX-C; the seventh frame received data is stored in the host packet receive queue, which can be represented by the character 7: RX-H; the eighth frame received data is stored in the network card packet receive queue, which can be represented by the character 8: RX-C; the Nth frame received data can be stored in either the network card packet receive queue or the host packet queue, which can be represented by the character N: RX-N.The transmission queues are as follows: the first frame transmits data to the host packet transmission queue (character 1: TX-H); the second frame transmits data to the host packet transmission queue (character 2: TX-H); the third frame transmits data to the network interface card (NIC) packet transmission queue (character 3: TX-C); the fourth frame transmits data to the NIC packet transmission queue (character 4: TX-C); the fifth frame transmits data to the NIC packet transmission queue (character 5: TX-C); the sixth frame transmits data to the host packet transmission queue (character 6: TX-H); the seventh frame transmits data to the host packet transmission queue (character 7: TX-H); the eighth frame transmits data to the NIC packet transmission queue (character 8: TX-C); and the Nth frame transmits data that can be stored in either the NIC packet transmission queue or the host packet transmission queue (character N: TX-N).

[0129] based on Figure 7 The network data transmission system shown first identifies the data type of the data to be transmitted when it is target network data from the network. Different data types are then classified and processed. The packet segmenter's processing method for the data packet type is as follows: If the data to be transmitted is CRC data, the packet is stored locally on the FPGA. The host's packet data transmission thread can discard this erroneous packet. If the data to be transmitted is a forwarding packet set by the host's packet data transmission thread, the packet is stored locally on the FPGA. The host's packet data transmission thread can forward it directly. If the data length of the data to be transmitted is less than the length threshold, the packet is stored locally on the FPGA. The host's packet data transmission thread program performs data transfer operations through a memory copy function. The packet data receiving process is as follows: Figure 8 As shown, the following processes may be included:

[0130] 1. The MAC module stores the data to be transmitted in the FPGA's buffer.

[0131] 2. The packet splitter determines the data type of the message to be transmitted and marks the corresponding data type.

[0132] 3. The packet divider updates the receive queue in extended memory according to the data type of the message to be transmitted, for example... Figure 7 As shown, message 1 is stored in the host memory, message 2 is stored in the host memory, and message 3 is stored in the local memory.

[0133] 4. The packet splitter uses the DMA module to store the cached data to be transmitted into the host message receiving queue.

[0134] 5. The packet splitter stores the cached data to be transmitted in the network card's packet receive queue in the local extended memory and notifies the host to read the packet data.

[0135] 6. The host's data transmission and reception thread reads the send queue list in the extended memory of the network card device and finds the location of the corresponding queue where the corresponding packet exists.

[0136] 7. The host reads packets from the host packet receive queue in its local memory, and also reads packets from the memory-based network interface card (NIC) packet receive queue in the extended memory of the NIC device. The packet data transmission and reception thread of the host performs the following operations to acquire packets: Figure 9 As shown:

[0137] The host's packet transmission and reception process initializes the receive queue, setting the producer and consumer queue pointers to their initial positions. The Direct Memory Access (DMI) module producer stores data in its pointer position (i.e., the host's packet receive queue), and then moves its DMI module producer pointer down. The buffer producer stores data in its pointer position (i.e., the network interface card (NIC) packet receive queue), and then moves its buffer producer pointer down. Upon receiving a notification from the NIC device, the host's packet transmission and reception thread (acting as a consumer) first reads the packet's location from the receive queue in extended memory. For example, if packet 1 is located in the host's packet transmission queue in host memory, the host's packet transmission and reception thread reads the data from the host's packet transmission queue and moves its corresponding pointer down. The consumer's pointer is always smaller than the producer's pointer, and after reaching the tail of the transmission queue, the pointers are looped back to their initial positions.

[0138] based on Figure 7 The network data transmission system shown categorizes the data type of the data to be transmitted from the host when the data to be transmitted is network data to be sent. If the data length is greater than a length threshold, the data is stored in the host's packet sending queue and subsequently moved using a DMA module. If the data length is less than the length threshold, it is directly copied from memory to the packet sending queue of the extended memory network card. The packet data transmission process is as follows: Figure 10 As shown, the following processes may be included:

[0139] 1. The host's message data transmission and reception thread classifies the data packets to be sent and updates the transmission queue in extended memory. For example, packet 1 is stored in the host message transmission queue in the host memory.

[0140] 2. Classify the packets and place them into the corresponding queues. Packets exceeding the length threshold are stored in the host's packet sending queue. For packets shorter than the length threshold, the host's packet data transmission and reception thread copies them to the network interface card packet sending queue in extended memory.

[0141] 3. The host notifies the packet splitter to retrieve the data packets to be sent. The packet splitter reads the sending queue and finds the corresponding packet according to the pointer in the sending queue.

[0142] 4. The packet splitter stores the packets in the network card's packet sending queue into the buffer, ready for subsequent sending.

[0143] 5. The packet splitter uses the DMA module to transfer packets in the host packet sending queue to the buffer, ready for subsequent transmission.

[0144] 6. Send the message in the buffer.

[0145] The operation flow of the host's message data sending and receiving thread sending messages is as follows: Figure 11 As shown, the message data transmission thread initializes the sending queue to point the producer and consumer queue pointers to the initial position of the sending queue. The host's message data transmission thread, acting as the producer, writes the message position information to the sending queue in extended memory and moves the pointer down. Different types of messages are stored in the corresponding host message sending queue and network card message sending queue, and the corresponding pointers are moved down. The DMA module (Direct Memory Access module) consumer is notified to move the message data to the buffer and move the pointer down. The host directly stores the message data in the buffer and moves the pointer down in the buffer consumer's buffer.

[0146] As shown above, this embodiment uses CXL to extend the host's memory and creates network interface card (NIC) packet sending and receiving queues to store data packets. Sending and receiving queues are created in the extended memory as index information so that the packet sending and receiving programs can locate the corresponding data packets. Based on the length threshold for DMA data transfer between the host and FPGA, obtained from tests under different device environments, packets longer than the length threshold are transferred using DMA, while packets shorter than the length threshold are copied from the host, further improving network data storage efficiency. During the FPGA's data packet reception process, the packet segmenter stores CRC error packets, host-set forwarding packets, and packets shorter than the length threshold in the FPGA's extended memory. The host's packet sending and receiving threads can then access the data packets through this extended memory, further improving network data processing efficiency. During the data packet transmission process, the host's packet data transmission and reception thread determines the packet storage location based on the packet length. Data packets longer than the length threshold are stored in the host packet transmission queue in local memory, while data packets shorter than the length threshold are directly copied to the network card packet transmission queue in extended memory, thereby effectively improving the overall network data transmission efficiency.

[0147] This invention also provides a corresponding apparatus for the network data transmission method, further enhancing the practicality of the method. The apparatus can be described from both a functional module perspective and a hardware perspective. The network data transmission apparatus provided by this invention is described below. This apparatus is used to implement the network data transmission method provided by this invention. In this embodiment, the network data transmission apparatus may include or be divided into one or more program modules. These program modules are stored in a storage medium and executed by one or more processors to complete the network data transmission method disclosed in Embodiment 1. The program module referred to in this embodiment is a series of computer program instruction segments capable of performing a specific function, which is more suitable than the program itself for describing the execution process of the network data transmission apparatus in the storage medium. The following description will specifically introduce the functions of each program module in this embodiment. The network data transmission apparatus described below and the network data transmission method described above can be referred to in correspondence.

[0148] From the perspective of functional modules, see Figure 12 , Figure 12 This is a structural diagram of the network data transmission device provided in this embodiment under one specific implementation. The device may include:

[0149] Data acquisition module 121 is used to acquire data to be transmitted;

[0150] The network transmission module 122 is used to determine the matching data transmission method based on the data packet type of the data to be transmitted, based on the network card device, so as to transmit the data to be transmitted over the network according to the data transmission method.

[0151] For example, in some embodiments of this example, the network transmission module 122 described above can also be used for:

[0152] The data to be transmitted is the target network data received from the network. The network interface card (NIC) device identifies the data packet type of the data to be transmitted. If the data packet type belongs to the NIC processing packet type, the data to be transmitted is stored in the target memory of the NIC device, and a packet data read instruction is sent to the host so that the host can access the target memory through memory copying. If the data packet type belongs to the host processing packet type, the data to be transmitted is stored in the host's memory through the direct memory access module so that the host can access the target memory through direct memory access.

[0153] As an exemplary implementation of the above embodiments, the network transmission module 122 may further be used for:

[0154] If the data type of the data to be transmitted is a preset data type, then the data packet type of the data to be transmitted belongs to the network card processing packet type; whereby the preset data type is data that is not stored in the host's memory;

[0155] If the length of the data to be transmitted is less than the length threshold, then the data packet type of the data to be transmitted belongs to the network card processing packet type.

[0156] As another exemplary implementation of the above embodiments, the network transmission module 122 may further be used for:

[0157] If the data to be transmitted belongs to the first preset data type, the host reads the data to be transmitted and discards it directly;

[0158] If the data to be transmitted belongs to the second preset data type, the host reads the data to be transmitted and forwards it.

[0159] If the length of the data to be transmitted is less than the length threshold, the host calls the memory copy function to copy the data to be transmitted into the host's memory.

[0160] As an exemplary implementation of the above embodiments, the network transmission module 122 may further be used for:

[0161] If the data to be transmitted is cyclic redundancy check code data, the host reads the data to be transmitted and discards it directly.

[0162] If the data to be transmitted is a forwarding message, the host reads the data to be transmitted and forwards it to the target address.

[0163] For example, in some other embodiments of this example, the network transmission module 122 described above can also be used for:

[0164] Data of different lengths were transmitted using both memory copy and direct memory access methods, and the corresponding transmission times were recorded.

[0165] A comparative analysis was conducted on the transmission time required for transmitting the same data length using memory copy and direct memory access methods. The corresponding data length when the transmission time required for memory copy and direct memory access methods is the same was determined and used as a length threshold.

[0166] As another implementation method parallel to the above embodiments, the network transmission module 122 can also be used for:

[0167] When a length threshold configuration instruction is received from the user, the length threshold is obtained by parsing the instruction and then stored in the target location.

[0168] For example, in some further embodiments of this example, the network transmission module 122 described above can also be used for:

[0169] The data to be transmitted is the network data to be sent by the host, and the host identifies the data packet type of the data to be transmitted;

[0170] If the data packet type to be transmitted belongs to the network card processing packet type, the data to be transmitted is stored in the target memory of the network card device by memory copying, and a packet data read command is sent to the network card device;

[0171] If the data packet type to be transmitted belongs to the host processing packet type, the data to be transmitted is stored in the host's memory, and a packet data read command is sent to the network card device to read the corresponding data through direct access.

[0172] For example, in some further embodiments of this example, the network transmission module 122 described above can also be used for:

[0173] Pre-build a storage data index in the target memory of the network interface card (NIC) device; the storage data index is used to record the storage location information of received and sent packets;

[0174] Update the stored data index information based on the data packet type of the data to be transmitted;

[0175] Based on the stored data index information and the sender information of the data to be transmitted, determine the data transmission method corresponding to the data to be transmitted.

[0176] As an exemplary implementation of the above embodiments, the network transmission module 122 may further be used for:

[0177] A network interface card (NIC) packet receiving queue is pre-built in the target memory of the NIC device, and a host packet receiving queue is built in the host's memory.

[0178] The queue pointers of the receiver producers corresponding to the network interface card (NIC) packet receive queue and the host packet receive queue are pre-set to their respective initialization positions; the receiver producers include buffer producers and direct memory access module producers;

[0179] The data to be transmitted is target network data. Based on the receiving queue storing data index information, if the data to be transmitted belongs to the network card processing message type, the data to be transmitted is stored in the pointer position of the buffer producer in the network card message receiving queue, and the pointer of the buffer producer is moved down; if the data to be transmitted belongs to the host processing message type, the data to be transmitted is stored in the pointer position of the direct memory access module producer in the host message receiving queue, and the pointer of the direct memory access module producer is moved down.

[0180] As an exemplary implementation of the above embodiments, the network transmission module 122 may further be used for:

[0181] Beforehand, the queue pointers of the receiving consumers corresponding to the network interface card packet receiving queue and the host packet receiving queue are respectively set to the corresponding initial positions; the receiving consumers are the packet data sending and receiving threads of the host; the pointer position of the receiving consumer is less than the pointer position of the receiving producer;

[0182] The host determines the storage location of the storage queue corresponding to the data to be transmitted based on the receiving queue, reads the data to be transmitted based on the storage location of the data to be transmitted, and moves the pointer of the receiving consumer down.

[0183] As another exemplary implementation of the above embodiments, the network transmission module 122 may further be used for:

[0184] A network interface card (NIC) packet sending queue is pre-built in the target memory of the NIC device, and a host packet sending queue is built in the host's memory.

[0185] In advance, the queue pointers of the sending producers corresponding to the network card packet receiving queue and the host packet receiving queue are respectively set to the corresponding initialization positions; the sending producer is the host's packet data sending and receiving thread;

[0186] The queue pointer of the sending producer corresponding to the sending queue storing data index information is pre-set to the corresponding initial position, and the pointer position of the sending producer is moved down when the sending queue is updated;

[0187] Based on the sending queue containing the stored data index information, if the data to be transmitted belongs to the network card processing message type, the data to be transmitted is stored in the network card message receiving queue, and the sending producer's pointer is moved down; if the data to be transmitted belongs to the host processing message type, the data to be transmitted is stored in the host message receiving queue, and the sending producer's pointer is moved down.

[0188] As an exemplary implementation of the above embodiments, the network transmission module 122 may further be used for:

[0189] The queue pointers of the sending consumers corresponding to the network card packet sending queue and the host packet sending queue are pre-set to their respective initialization positions; the sending consumers are buffer consumers and direct memory access module consumers; the pointer position of the sending consumer is less than the pointer position of the sending producer;

[0190] Based on the sending queue of stored data index information, if the data to be stored belongs to the network card processing message type, then the data to be stored is read from the network card message sending queue and the pointer of the buffer consumer is moved down; if the data to be stored belongs to the host processing message type, then the data to be stored is read from the host message sending queue and the pointer of the direct memory access module consumer is moved down.

[0191] The functions of each functional module of the network data transmission device in this embodiment can be specifically implemented according to the methods in the above method embodiments. The specific implementation process can be referred to the relevant descriptions in the above method embodiments, which will not be repeated here.

[0192] As can be seen from the above, this embodiment can effectively improve the transmission efficiency of network data.

[0193] The network data transmission device mentioned above is described from the perspective of functional modules. This invention also provides an electronic device described from a hardware perspective, such as... Figure 13 As shown. The electronic device includes a memory 130 for storing computer programs; and a processor 131 for executing the computer programs to implement the network data transmission method steps mentioned in any of the above embodiments. In some embodiments, the electronic device may further include a display screen 132, an input / output interface 133, a communication interface 134, a power supply 135, a communication bus 136, and sensors 137 that perform various functions.

[0194] The processor 131 may include one or more processing cores, which may be a controller, microcontroller, microprocessor, or artificial intelligence processor for processing computational operations related to machine learning. The memory 130 may include one or more computer-readable storage media, high-speed random access memory, and non-volatile memory, and may be an internal storage unit or an external storage device. The memory 130 may store application software installed on the electronic device and various types of data, such as code for programs executing network data transmission methods, and may also temporarily store data that has been output or will be output. In this embodiment, the memory 130 is at least used to store a computer program 1301 that, after being loaded and executed by the processor 131, can implement the relevant steps of the network data transmission method disclosed in any of the previous embodiments. The resources stored may also include an operating system 1302 and data 1303. The operating system 1302 may include Windows, Unix, Linux, etc. The data 1303 may include, but is not limited to, data corresponding to the network data transmission results.

[0195] It is understood that if the network data transmission method in the above embodiments is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of the present invention. Based on this, the present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the network data transmission method in any of the above embodiments.

[0196] Finally, this invention also provides a network data transmission system, please refer to [link / reference]. Figure 14 , Figure 14 A schematic diagram of the structural framework of a network data transmission system in an exemplary implementation may include the following:

[0197] The network data transmission system of this embodiment may include a host 141, a network interface card (NIC) device 142, and a processor 131. The host 141 may be any small to medium-sized single server, a data center server, a personal computer, etc., capable of connecting to an external NIC device. The NIC device 142 may be any NIC device described in any of the above embodiments. The processor 131 may be used to execute a computer program stored in the memory to implement the method steps described in any of the network data transmission method embodiments.

[0198] The functions of each functional module of the network data transmission system in this embodiment of the invention can be specifically implemented according to the methods in the above method embodiments. The specific implementation process can be referred to the relevant descriptions in the above method embodiments, which will not be repeated here.

[0199] As can be seen from the above, this embodiment can effectively improve the network data transmission efficiency of the host by deploying the aforementioned network card device on the host.

[0200] The foregoing has provided a detailed description of the network interface card (NIC) device, network data transmission method, apparatus, system, electronic device, and readable storage medium provided by the present invention. Several improvements and modifications can be made to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A network card device, characterized by The programmable device is used to build a network platform based on a memory expansion method, which includes a network interface, target memory, and network components. The programmable device connects to the network via the network interface and interacts with the host via the memory expansion method; the host sets the target memory of the programmable device as the host's extended memory based on the memory expansion method; the network component transmits data using a data transmission method that matches the data packet type of the data to be transmitted. The data to be transmitted is either received target network data or network data to be sent by the host; the data packet type is determined according to the data type and data length of the data to be transmitted; the extended memory is used to store the data to be transmitted that belongs to the network card processing packet type. The host's central processing unit accesses the target memory via memory copy; the network interface card (NIC) processes message types including data message types with a data length less than a length threshold; if the data to be transmitted is the target network data and the data length of the data to be transmitted is less than the length threshold, the network component stores the data to be transmitted in the target memory and sends a message data read instruction to the host; if the data to be transmitted is the network data to be sent and the data length of the data to be transmitted is less than the length threshold, the host moves the data to be transmitted to the target memory via memory copy.

2. The network interface card (NIC) device according to claim 1, characterized in that, The network components include a media access control layer module, a direct memory access module, a buffer, and a packet splitter; The media access control layer module is used to store target network data received from the network into the buffer, and is also used to send the network data to be sent in the buffer to the network. The packet splitter is used to identify the data packet type of the target network data and determine the corresponding storage location according to the data packet type of the target network data; it is also used to store the network data to be sent, which is read from the target memory or from the host memory through the direct memory access module, into the transmission buffer; wherein the storage location includes the host memory and the target memory.

3. The network interface card (NIC) device according to claim 1, characterized in that, The network component includes a direct memory access module, and the host's central processing unit accesses the target memory via direct memory access; the network interface card (NIC) does not process data packet types whose data length exceeds a length threshold. If the data to be transmitted is the target network data, and the data length of the data to be transmitted is greater than the length threshold, the network component moves the data to be transmitted to the host's memory through the direct memory access module; If the data to be transmitted is the network data to be sent, and the data length of the data to be transmitted is greater than the length threshold, the host stores the data to be transmitted in the host's memory and sends a message data read instruction to the network component.

4. The network interface card (NIC) device according to any one of claims 1 to 3, characterized in that, The programmable device employs a field-programmable gate array and is built on a network platform based on high-speed computing interconnect technology; The host computer configures the target memory of the field-programmable gate array as the extended memory through the high-speed computing interconnect technology. The data to be transmitted is the target network data. The field-programmable gate array identifies the data packet type of the data to be transmitted. If the data to be transmitted belongs to the target preset data type and / or the data length of the data to be transmitted is less than the length threshold, then the data to be transmitted is stored in the target memory. The data to be transmitted is the network data to be sent. The host identifies the data packet type of the data to be transmitted. If the data to be transmitted belongs to a target preset data type and / or the data length of the data to be transmitted is less than a length threshold, the data to be transmitted is stored in the target memory of the field-programmable gate array by memory copying.

5. A network data transmission method, characterized in that, include: Get the data to be transmitted; Based on the network interface card device as described in any one of claims 1 to 4, a matching data transmission method is determined according to the data packet type of the data to be transmitted, so as to transmit the data to be transmitted over the network according to the data transmission method.

6. The network data transmission method according to claim 5, characterized in that, The data to be transmitted is target network data received from the network, and the network interface card (NIC) device identifies the data packet type of the data to be transmitted. The step of determining the matching data transmission method based on the data packet type of the data to be transmitted includes: If the data packet type of the data to be transmitted belongs to the network card processing packet type, then the data to be transmitted is stored in the target memory of the network card device, and a packet data read instruction is sent to the host; If the data packet type of the data to be transmitted belongs to the host processing packet type, then the data to be transmitted is stored in the host's memory through the direct memory access module.

7. The network data transmission method according to claim 5, characterized in that, The step of determining the matching data transmission method based on the data packet type of the data to be transmitted includes: If the data type of the data to be transmitted is a preset data type, then the data packet type of the data to be transmitted belongs to the network interface card (NIC) processing packet type; wherein, the preset data type is a data type that is not stored in the host's memory; If the length of the data to be transmitted is less than the length threshold, then the data packet type of the data to be transmitted belongs to the network card processing packet type.

8. The network data transmission method according to claim 7, characterized in that, The host accesses the target memory via memory copy, including: If the data to be transmitted belongs to a first preset data type, the host reads the data to be transmitted and discards it directly; the first preset data type is a data type that does not require processing by the host. If the data to be transmitted belongs to a second preset data type, the host reads the data to be transmitted and forwards it; the second preset data type is a data type that requires subsequent processing by the host. If the length of the data to be transmitted is less than the length threshold, the host calls the memory copy function to copy the data to be transmitted into the host's memory, so that the host can access the target memory through memory copy.

9. The network data transmission method according to claim 8, characterized in that, The host accesses the target memory via memory copy, including: If the data to be transmitted is cyclic redundancy check code data, the host reads the data to be transmitted and discards it directly. If the data to be transmitted is a message to be forwarded, the host reads the data to be transmitted and forwards the data to the target address.

10. The network data transmission method according to claim 5, characterized in that, Before determining the matching data transmission method based on the data packet type of the data to be transmitted, the method further includes: Data of different lengths were transmitted using both memory copy and direct memory access methods, and the corresponding transmission times were recorded. The transmission time required for transmitting the same data length using the memory copy method and the direct memory access method is compared and analyzed to determine the data length corresponding to the same transmission time for the memory copy method and the direct memory access method, which is then used as a length threshold.

11. The network data transmission method according to claim 5, characterized in that, Before determining the matching data transmission method based on the data packet type of the data to be transmitted, the method further includes: When a length threshold configuration instruction is received from the user, the length threshold is obtained by parsing the length threshold configuration instruction and stored in the target location.

12. The network data transmission method according to claim 5, characterized in that, The data to be transmitted is network data to be sent by the host. The host identifies the data packet type of the data to be transmitted. Determining the matching data transmission method based on the data packet type of the data to be transmitted includes: If the data packet type of the data to be transmitted belongs to the network card processing packet type, then the data to be transmitted is stored in the target memory of the network card device, and a packet data read instruction is sent to the network card device; If the data packet type of the data to be transmitted belongs to the host processing packet type, then the data to be transmitted is stored in the host's memory, and a packet data read instruction is sent to the network card device.

13. The network data transmission method according to claim 5, characterized in that, The step of determining the matching data transmission method based on the data packet type of the data to be transmitted includes: A storage data index is pre-built in the target memory of the network interface card device; the storage data index is used to record the storage location information of received and sent packets. Update the stored data index information based on the data packet type of the data to be transmitted; Based on the stored data index information and the sending end information of the data to be transmitted, the data transmission method corresponding to the data to be transmitted is determined.

14. The network data transmission method according to claim 13, characterized in that, The data to be transmitted is target network data. Determining the data transmission method corresponding to the data to be transmitted based on the stored data index information and the sending end information of the data to be transmitted includes: A network interface card (NIC) packet receiving queue is pre-built in the target memory of the NIC device, and a host packet receiving queue is built in the host's memory. The queue pointers of the receiver producers corresponding to the network interface card (NIC) packet receiving queue and the host packet receiving queue are pre-set to their respective initialization positions; the receiver producers include buffer producers and direct memory access module producers; Based on the receiving queue containing the stored data index information, if the data to be transmitted belongs to the network interface card (NIC) processing message type, the data to be transmitted is stored in the pointer position of the buffer producer in the NIC message receiving queue, and the pointer of the buffer producer is moved down; if the data to be transmitted belongs to the host processing message type, the data to be transmitted is stored in the pointer position of the direct memory access module (DMI) producer in the host message receiving queue, and the pointer of the DMI producer is moved down.

15. The network data transmission method according to claim 14, characterized in that, After transmitting the data to be transmitted over the network according to the data transmission method, the method further includes: The queue pointers of the receiving consumers corresponding to the network interface card packet receiving queue and the host packet receiving queue are pre-set to their respective initialization positions; the receiving consumer is the packet data sending and receiving thread of the host; the pointer position of the receiving consumer is less than the pointer position of the receiving producer; The host determines the storage location of the storage queue corresponding to the data to be transmitted according to the receiving queue, reads the data to be transmitted according to the storage location of the data to be transmitted, and moves the pointer of the receiving consumer down.

16. The network data transmission method according to claim 13, characterized in that, The data to be transmitted is network data to be sent. Determining the data transmission method corresponding to the data to be transmitted based on the stored data index information and the sending end information of the data to be transmitted includes: A network interface card (NIC) packet sending queue is pre-built in the target memory of the NIC device, and a host packet sending queue is built in the host's memory. The queue pointers of the sending producers corresponding to the network interface card packet receiving queue and the host packet receiving queue are pre-set to their respective initialization positions; the sending producer is the host's packet data sending and receiving thread; The queue pointer of the sending producer corresponding to the sending queue storing the data index information is pre-set to the corresponding initialization position, and when the sending queue is updated, the pointer position of the sending producer is moved down; Based on the sending queue containing the stored data index information, if the data to be transmitted belongs to the network card processing message type, then the data to be transmitted is stored in the network card message receiving queue, and the pointer of the sending producer is moved down; if the data to be transmitted belongs to the host processing message type, then the data to be transmitted is stored in the host message receiving queue, and the pointer of the sending producer is moved down.

17. The network data transmission method according to claim 16, characterized in that, After transmitting the data to be transmitted over the network according to the data transmission method, the method further includes: The queue pointers of the sending consumers corresponding to the network interface card packet sending queue and the host packet sending queue are pre-set to their respective initialization positions; the sending consumers are buffer consumers and direct memory access module consumers; the pointer position of the sending consumer is less than the pointer position of the sending producer; Based on the sending queue of the stored data index information, if the data to be transmitted belongs to the network card processing message type, then the data to be transmitted is read from the network card message sending queue, and the pointer of the buffer consumer is moved down; if the data to be transmitted belongs to the host processing message type, then the data to be transmitted is read from the host message sending queue, and the pointer of the direct memory access module consumer is moved down.

18. A network data transmission device, characterized in that, include: The data acquisition module is used to acquire the data to be transmitted. A network transmission module is configured to determine a matching data transmission method based on the data packet type of the data to be transmitted, according to the network interface card device as described in any one of claims 1 to 4, so as to transmit the data to be transmitted over the network according to the data transmission method.

19. An electronic device, characterized in that, It includes a processor and a memory, the processor being configured to implement the steps of the network data transmission method as described in any one of claims 5 to 17 when executing a computer program stored in the memory.

20. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the steps of the network data transmission method as described in any one of claims 5 to 17.

21. A network data transmission system, characterized in that, Includes a host, a network interface card (NIC) device as described in any one of claims 1 to 4, and a processor; The processor is used to implement the steps of the network data transmission method as described in any one of claims 5 to 17 when executing a computer program stored in the memory.

Images