OS network data processing architecture, processing method and storage medium

Abstract

The invention provides an OS network data processing architecture and processing method.The processing architecture comprises a network card, a cache and an application program, the cache comprises a first buffer area used for temporarily storing data packets, and the first buffer area is a part of storage area of the cache and used for storing the data packets received by the network card; the first buffer area further comprises a control structure and a data packet storage area and is used for managing data packets of the first buffer area; the application program is used for directly accessing the data packet in the data packet storage area in the first buffer area according to the control structure of the first buffer area. According to the invention, the problem of multiple duplication of data related to the memory of the operating system and CPU scheduling is solved, and high-speed and low-delay network data packet temporary storage and processing are realized.

Description

Technical Field

[0001] The present invention relates to the field of OS network data processing, and in particular to an OS network data processing architecture and method, and a simulation architecture generation method thereof. Background Art

[0002] In recent years, with the development of Internet of Things technology, various IoT smart terminals have been widely used in many fields, such as smart home and smart industry. These smart terminals face new business scenarios of device networking and massive real-time network data processing. Compared with the traditional terminal equipment, the source of operation data is mainly the local hard disk, and the source of data operated by smart terminals in new business scenarios is mainly the communication interface. As the basic software installed on the smart terminal, the operating system (OS) faces new challenges. In the face of rich business scenarios and massive real-time network data processing requirements, how can OS make full use of processing resources and communication resources to achieve high-speed and low-latency traffic processing. In the traditional network data processing mechanism, data packets need to be transmitted from the network card to the memory and then to the input buffer queue. This process consumes a lot of CPU cycles, squeezing out the time for further processing of data packets. In the transmission process, multiple copies of data cause a large number of memory accesses, which greatly limits the efficiency of receiving data packets and affects the performance of network data processing. From the perspective of data characteristics, the network data processed in the relevant application scenarios of dedicated IoT smart terminals has the characteristics of strong real-time, small data packets, sparse data packet arrival density, and simple data processing. For example, some in-vehicle applications require the transmission of control instructions, which have a small amount of data but very high latency requirements.

[0003] To solve the above problems, the Netmap mechanism (a high-performance network I / O framework) can modify the kernel, design a new lightweight data structure, pre-allocate linear, fixed-size shared data packet buffers, and avoid dynamic allocation and release. Application threads can directly access the data packet buffer, avoiding unnecessary memory copying. DPDK (Data Plane Development Kit) uses the Linux system's UIO technology (Userspace I / O, I / O technology running in user space) to open up the transmission channel between kernel space and user space, bypass the Linux kernel protocol stack, shield hard interrupts, and use active polling in user mode to process packet reception and transmission. Many of the above technologies such as running in the kernel and bypassing the network protocol stack rely on specific hardware features, need to be bound to specific devices, and even provide unprotected access to hardware, making many application scenarios in real network environments unable to meet the implementation requirements of network data processing mechanisms, and have poor portability, which limits the application and development of network data processing mechanisms.

[0004] Under the background of the existing 5G, artificial intelligence Internet of Things (AIoT), the emergence of rich IoT terminals and big data, considering the data characteristics of IoT smart terminals, there is an urgent need for an OS network data processing technology for IoT smart terminals to solve the problems of multiple data replication in traditional network data processing mechanisms and the particularity of application devices. Summary of the invention

[0005] To solve the above problems, the present invention proposes an OS network data processing architecture and method, and a simulation architecture generation method thereof. In order to solve the problem of multiple data copies, the data packets are directly transferred from the network card to the cache, and the application can directly access the buffer, which reduces the number of copies and avoids the switching between kernel state and user state. In order to solve the particularity of application devices, an open source network simulator is used as a technical platform for experiments, which can solve the bottleneck problem of specific hardware limitations to adapt to more application scenarios.

[0006] According to a first aspect, the present invention provides an OS network data processing architecture, including a network card, a cache and an application, the cache including a first buffer for temporarily storing data packets, the first buffer being a partial storage area of ​​the cache, for storing the data packets received by the network card, the first buffer also including a control structure and a data packet storage area for managing the data packets in the first buffer; the application is used to directly access the data packets in the data packet storage area in the first buffer according to the control structure of the first buffer.

[0007] Preferably, the control structure includes a global control structure and a sub-control structure, the global control structure is used to manage all the sub-control structures and space blocks, each sub-control structure manages a data packet; the data packet storage area includes a number of space blocks of the same size, and the space blocks are used to store data packets.

[0008] Preferably, the global control structure includes multiple control elements, and manages the idle and used status of the sub-control structure according to the control elements; wherein the control elements include: the total number of sub-control structures, the number of available sub-control structures, the number of used sub-control structures, the number of free space blocks, the total number of space blocks, the sub-control structure area address offset, the next sub-control structure sequence number to be used, the next sub-control structure sequence number to be polled, the space block first address, the next space block to be filled, and the next space block to be processed; the sub-control structure includes the current sub-control structure sequence number, the current data packet size managed by the sub-control structure, and the key position of the data packet in the space block; wherein the key position includes the position of the data packet header in the space block, the position where the data part of the data packet starts, the position where the data part of the data packet ends, and the position of the end part of the entire data packet.

[0009] Preferably, it also includes a memory; the memory includes a second buffer, which is a partial storage area of ​​the memory, and when the available space in the first buffer is insufficient, the second buffer stores the data packets sent by the network card; the second buffer also includes a control structure and a data packet storage area for managing the data packets in the second buffer; the application is also used to access the data packets in the data packet storage area in the second buffer according to the control structure in the second buffer.

[0010] According to the second aspect, the present invention provides an OS network data processing method based on any architecture in the first aspect, comprising: when the data packet arrives at the network card, the data packet is transmitted from the input queue of the network card to the first buffer; after the data packet arrives at the first buffer, interrupt processing is triggered, and the application accesses the data packet in the first buffer in a polling manner.

[0011] Preferably, the transmitting of the data packet from the input queue of the network card to the first buffer includes: obtaining the size of the data packet, querying whether the number of available sub-control structures is greater than 0: if the number of available sub-control structures is greater than 0, obtaining the sub-control structure corresponding to the next sub-control structure number to be used, and storing the data packet in a space block by the sub-control structure; if the number of available sub-control structures is less than or equal to 0, it indicates that there is no available sub-control structure at present, and the data packet is discarded.

[0012] Preferably, if the number of available sub-control structures is greater than 0, the sub-control structure corresponding to the next sub-control structure number to be used is obtained, and the sub-control structure stores the data packet in the space block, further including: if the number of available sub-control structures is greater than 0, the number of space blocks occupied by the data packet is calculated according to the size of the space block and the size of the data packet, and it is determined whether the number is less than or equal to the number of free space blocks: if so, the number of available sub-control structures is reduced by one, the number of used sub-control structures is increased by one, and the corresponding sub-control structure is determined according to the sub-control structure area address offset and the next sub-control structure number to be used, the position of the data packet header in the sub-control structure points to the next space block to be filled, and the data packet is stored in the corresponding number of space blocks according to the position of the data packet header, and the next sub-control structure number to be used and the next space block to be filled are updated; if not, it means that the current number of free space blocks is insufficient, and the data packet is discarded.

[0013] Preferably, the application accesses the data packets in the first buffer in a polling manner, including: querying whether the number of used sub-control structures is greater than 0: if so, determining the current sub-control structure that manages the data packet based on the next sub-control structure sequence number to be polled and the sub-control structure area address offset, and the application processes the data packet corresponding to the sub-control structure; if not, it indicates that there is currently no data packet that needs to be processed by the application.

[0014] Preferably, after the application processes the data packet corresponding to the sub-control structure, it also includes: updating the control element in the global control structure; wherein the control element includes the sequence number of the next sub-control structure to be polled and the next space block to be processed, the number of free space blocks and the number of available sub-control structures.

[0015] Preferably, after the application accesses the data packets in the first buffer in a polling manner, it also includes: the data packets accessed by the application are marked as replaceable, and the new data packets sent from the network card cover the space in the first buffer occupied by the data packets marked as replaceable.

[0016] Preferably, it also includes: when the data packet arrives at the network card, if the available space in the current first buffer is insufficient, the data packet is transmitted from the input queue of the network card to the second buffer, and the interrupt processing is triggered after the data packet arrives at the second buffer, and the application accesses the data packet in the second buffer in a polling manner.

[0017] According to a third aspect, the present invention provides a storage medium, wherein the storage medium stores computer executable instructions, and when the computer executable instructions are loaded and executed by a processor, the steps of any method described in the second aspect are implemented.

[0018] The OS network data of the present invention can be directly transmitted from the network card to the buffer of the user space, and the application program directly accesses the data packets in the buffer, which solves the problem of multiple data copying related to the operating system memory and CPU scheduling, and realizes the temporary storage and processing of network data packets with high speed and low latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a storage structure design of a first buffer according to an embodiment of the present invention;

[0020] Figure 2 A schematic diagram of the structural relationship between a control structure and a data packet storage area according to an embodiment of the present invention;

[0021] Figure 3 A schematic diagram of an OS network data processing method for an IoT smart terminal proposed according to an embodiment of the present invention;

[0022] Figure 4 A flowchart of allocating data packets to space blocks after they arrive at the first buffer according to an embodiment of the present invention;

[0023] Figure 5 A flow chart of data packet processing and space block recovery by an application according to an embodiment of the present invention;

[0024] Figure 6 It is a schematic diagram of an OS network data processing method with a second buffer added according to an embodiment of the present invention;

[0025] Figure 7 A schematic diagram of a simulation architecture of a simulator-based DUCA architecture according to an embodiment of the present invention;

[0026] Figure 8 A simulation workflow diagram of a DUCA architecture based on an NS-3 simulator according to an embodiment of the present invention;

[0027] Fig. 9 The figure is a schematic diagram of simulation results on the NS-3 simulator according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The specific embodiments of the present invention will be described in detail below. It should be noted that the embodiments herein are only used for illustration and are not intended to limit the present invention. In the following description, a large number of specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to those of ordinary skill in the art that these specific details are not necessarily used to implement the present invention. In other examples, in order to avoid confusing the present invention, known procedures, materials or methods are not specifically described.

[0029] Traditional network data processing mechanisms rely on memory. The efficiency of memory greatly restricts processing efficiency. There are problems such as multiple data replication, memory competition between processes, memory competition between processor cores, high memory allocation overhead, and interrupt frequency adjustment. The problem of multiple data replication is a key factor restricting performance. For example, the NAPI mechanism and the BPS mechanism both require memory to temporarily store buffer packets. When a packet arrives, it must first be stored in the input buffer queue of the memory, and then sent to the kernel protocol stack for processing through interrupts or polling according to the specific processing mechanism, and finally sent to the user space for application access. In addition, in a real network environment, some network data processing technologies rely on specific devices or hardware features, which are difficult to implement in real scenarios and have problems such as poor portability.

[0030] After research, the inventor proposed an OS network data processing architecture, implementation method and simulation architecture for the IoT smart terminal OS, targeting the problem of multiple data replication in traditional network data processing mechanisms and the particularity of application devices.

[0031] First, in order to solve the problem of multiple data replication, according to one embodiment of the present invention, an OS network data processing architecture for IoT smart terminals (i.e., direct user space cache access architecture, Direct User space Cache Access, DUCA) is proposed. The architecture includes a network card, a cache, and an application, wherein the cache is used to pre-set a portion of its storage area as a first buffer, and the first buffer is used to temporarily store data packets; when the data packet arrives at the network card, the network card directly transfers the data packet to the first buffer for processing, so that the application can directly access the data packet in the first buffer. Since the cache space is relatively small compared to the memory, the data processing of IoT smart terminals is simple and has higher requirements for real-time performance, a portion of the storage area is allocated as the first buffer by static pre-allocation, and a lightweight storage structure is designed in the first buffer to manage the first buffer. The storage structure design of the first buffer is as follows: Figure 1As shown, it includes a control structure and a data packet storage area, wherein the control structure includes a global control structure and a sub-control structure, the global control structure is used to manage all sub-control structures and space blocks, each sub-control structure manages a data packet, so the sub-control structure can also be called a data packet control structure; the data packet storage area includes several space blocks of the same size, and the space blocks are used to store data packets.

[0032] According to an embodiment of the present invention, the control elements and meanings included in the global control structure are shown in Table 1:

[0033] Table 1 Control elements and their meanings

[0034]

[0035]

[0036] According to an embodiment of the present invention, the elements and meanings contained in the sub-control structure are shown in Table 2:

[0037] Table 2 Sub-control structure elements and their meanings

[0038] Element Name Element Meaning sequence Current sub-control structure number size Current packet size head Points to the location of the packet header data Points to the start of the data section tail Points to the end of the data section end Points to the end of the entire data packet

[0039] According to one embodiment of the present invention, the structural relationship between the control structure and the data packet storage area is as follows: Figure 2 As shown, the global control structure includes control elements such as the total number of sub-control structures total_count, the next sub-control structure sequence number next_to_use to be used, and the sub-control structure area address offset offset (see Table 1 for details), and manages the idle and used status of the sub-control structure based on these elements to determine which sub-control structure manages the data packet and allocate the data packet. The sub-control structure elements in the sub-control structure (such as Table 2) include the current sub-control structure sequence number sequence, the current data packet size managed by the sub-control structure, and the key position of the data packet in the space block; wherein the key position includes the position of the header of the data packet in the space block head, the position of the data part of the data packet starting data, the position of the data part of the data packet ending tail, and the position of the end part of the entire data packet end.

[0040] In some embodiments, the architecture also includes a memory, which is used to pre-select a portion of the storage area therein as a second buffer; when the available space in the first buffer is insufficient, the network card transfers the data packet to the second buffer, and the second buffer also includes a control structure and a data packet storage area, which are used to manage the data packets in the second buffer; so that the application can access the data packets in the data packet storage area in the second buffer according to the control structure in the second buffer.

[0041] The existing network data processing mechanism is overly dependent on memory, which brings problems such as multiple copies of related data. Most of the data processed by applications on dedicated smart terminals is relatively sparse. By directly transferring the data arriving at the network card to the cache in the user space and processing it in time, these problems can be shielded to a large extent, thereby significantly improving the performance of network data processing.

[0042] According to an embodiment of the present invention, a data processing method for IoT smart terminals is proposed for the above OS network data processing architecture. Figure 3 As shown, the OS network data processing method, that is, the workflow of the DUCA architecture, includes: selecting a portion of the storage area in the cache in advance to temporarily store the data packet, and the storage area is recorded as the first buffer; when the data packet arrives at the network card, it is temporarily stored in the input queue of the network card, and the data packet is transmitted from the input queue of the network card to the first buffer; in some embodiments, the process of allocating the data packet to the space block after arriving at the first buffer is as shown in Figure 4 As shown, after the data packet arrives, its size size is obtained, and it is queried whether the number of available sub-control structures is greater than 0: if the number of available sub-control structures is greater than 0, the number of space blocks occupied by the data packet is calculated according to the size of the space block and the size of the data packet, and it is determined whether the number number is less than or equal to the number of free space blocks empty_c: if so, the number of available sub-control structures is reduced by one, the number of used sub-control structures is increased by one, and the corresponding sub-control structure is determined according to the sub-control structure area address offset offset and the next sub-control structure sequence number next_to_use to be used, the head data in the sub-control structure points to the next space block next_c_alloc to be filled, and the data is stored in the corresponding number of space blocks according to the head data, the next sub-control structure sequence number next_to_use to be used is updated according to formula (1), and the next space block next_c_alloc to be filled is updated according to formula (2); if not, it means that the current number of free space blocks is insufficient, and the data packet is discarded; if the number of available sub-control structures is less than or equal to 0, it means that there is no available sub-control structure at present, and the data packet is discarded. The interrupt processing is triggered, and the application accesses the data packets in the first buffer in a polling manner. In some embodiments, the application processes the data packets and the space block recovery process is as follows: Figure 5As shown, check whether the number of used sub-control structures used_count is greater than 0: if so, determine the current sub-control structure that manages the data packet according to the next sub-control structure sequence number next_to_clean to be polled and the sub-control structure area address offset offset, and the application processes the data packet corresponding to the sub-control structure, and updates the elements in the global control structure according to equations (3) to (6), including the next sub-control structure sequence number next_to_clean to be polled and the next space block next_c_fetch to be processed, the number of free space blocks empty_c and the number of available sub-control structures empty_count; if not, it means that there is currently no data packet that needs to be processed by the application.

[0043] m=(n+1)%total_count (1)

[0044] p=(q+number)%cache_count (2)

[0045] a=(b+1)%total_count (3)

[0046] c=(d+number)%cache_count (4)

[0047] e=f+number (5)

[0048] g=h+1 (6)

[0049] Among them, m represents next_to_use after update, n represents next_to_use before update, p represents next_c_alloc after update, q represents next_c_alloc before update, a represents next_to_clean after update, b represents next_to_clean before update, c represents next_c_fetch after update, d represents next_c_fetch before update, e represents empty_c after update, f represents empty_c before update, g represents empty_count after update, and h represents empty_count before update.

[0050] In some embodiments, the data packet accessed by the application is marked as replaceable, and the new data packet can cover the space of the first buffer occupied by the replaceable data packet.

[0051] In some embodiments, Figure 6As shown, a portion of the storage area can also be selected in advance in the memory as the second buffer; when the available space in the first buffer is insufficient, the data packet is transferred from the input queue of the network card to the second buffer. The method of transferring and storing the data packet to the second buffer is the same as the aforementioned method of storing the data packet to the first buffer, which is not described in detail here. The interrupt processing is triggered, and the application accesses the data packet in the second buffer in a polling manner. The method of the application accessing the data packet in the second buffer is the same as the aforementioned application processing of the data packet in the first buffer and the space block recovery process, which is not described in detail here.

[0052] After the data packet arrives at the network card, the data packet of the embodiment of the present invention can be directly sent to the cache and processed in time, which can shield the problem of the existing network data processing mechanism being overly dependent on the memory to a large extent and solve the problem of multiple data replication related to the operating system memory and CPU scheduling. The preset second buffer can solve the problem of insufficient storage space in the first buffer when the data packet throughput is large.

[0053] Aiming at the particularity of application devices in the traditional network data processing mechanism, according to one embodiment of the present invention, a simulation architecture and generation method of a DUCA architecture based on a simulator based on an OS network data processing architecture are proposed, wherein the simulator includes nodes, network devices, channels, applications, protocol stacks, and data packet structures; and also includes a processing simulation module and a hardware resource management module. The processing function simulation unit includes several subunits with different processing functions, including a network card processing subunit, an interrupt processing subunit, a polling processing subunit, a data replication processing subunit, a protocol stack processing subunit, and an application processing subunit; the network card processing unit is used to transfer the data packet to the data replication processing unit; the interrupt processing unit is used to simulate the interrupt processing process triggered after the data packet is transferred to the buffer; the polling processing unit is used to simulate the process of polling and reading the data packet from the buffer after the interrupt processing is completed; the data replication unit is used to simulate the process of data packet transmission from the network card to the memory, or the process of data packet transmission from the network card to the buffer, or the process of data packet transmission from the memory to the buffer; the protocol stack processing unit is used to simulate the processing process of the protocol stack; and the application processing unit is used to simulate the processing process of the application. According to the working principle of the DUCA architecture, a network card processing subunit, an interrupt processing subunit, a polling processing subunit, a data replication processing subunit, and an application processing subunit are selected from the subunits for combination, and the functions in the selected subunits are set based on the functions of the built-in functions in the simulator to generate a simulation model. According to the working principle of the DUCA architecture, the hardware resources required by the DUCA architecture in the data processing process are determined, including a network card, a processor, a cache, and a bus; in some embodiments, the required hardware resources also include memory. The hardware resource management module provides hardware resources and their corresponding hardware parameters for the simulation model of the DUCA architecture. A simulation instance is generated based on the simulation model, the hardware resources, and the corresponding hardware parameters; the hardware resources, the network device, the application, the data packet structure, the protocol stack, and the simulation instance are added to the node, and the nodes are connected with the channel to form a simulation model as follows. Figure 7 The simulation architecture of the simulator-based DUCA architecture is shown.

[0054] The NS-3 simulator is a widely used network simulation platform for simulating data packet transmission and processing in a real environment. According to an embodiment of the present invention, the NS-3 simulator is used as an example of a technical platform for experimental verification. Based on the proposed simulation architecture of the DUCA architecture based on the simulator, the DUCA architecture is modeled using its processing subunits with different functions, and the data processing flow of the DUCA architecture is simulated. The working process of the processing subunits is as follows: Figure 8 As shown below:

[0055] (1) The network card processing subunit reads the data packets in the first queue and then applies for network card resources. The hardware resource management module provides network card resources. After obtaining the network card resources, the data packets are transferred to the second queue to release the network card resources.

[0056] (2) The first interrupt processing subunit applies for cache resources, and the hardware resource management module provides cache resources. After obtaining the cache resources, the data copy processing subunit copies the data packet from the network card to the cache (i.e., simulates transferring the data packet from the input queue of the network card to the first buffer), and then transfers it to the third queue. If the cache resources are insufficient (i.e., the hardware resource management module cannot provide more cache resources), the data packet is transferred to the fourth queue.

[0057] (3) The second interrupt processing subunit reads the data packet in the third queue, applies for CPU resources, obtains the cache resources provided by the hardware resource management module, and then performs interrupt processing. After processing, the data packet is transferred to the fifth queue and the CPU resources are released.

[0058] (4) The polling processing subunit reads the data packets in the fifth queue, applies for CPU resources, obtains the cache resources provided by the hardware resource management module, and then performs polling processing. After the polling processing is completed, the data packets are transferred to the sixth queue and the CPU resources are released.

[0059] (5) The application processing subunit reads the data packets in the sixth queue, applies for CPU resources, obtains the cache resources provided by the hardware resource management module, and then processes the data packets. After completion, the CPU resources and cache resources are released.

[0060] (6) The third interrupt processing subunit reads the data packet in the fourth queue, then applies for cache resources or memory resources, and after obtaining the resources, transmits the data packet to the seventh queue, and then transmits the data packet to the second interrupt processing subunit for processing, and executes steps (3)-(5). In some embodiments, if there is no cache resource temporarily, the request is continued until the cache resource is obtained, and the data packet is transmitted to the seventh queue; in some embodiments, if there is no cache resource temporarily, a request is made to obtain memory resources, and the data packet is transmitted to the seventh queue.

[0061] According to the above design, the experiment was conducted on the NS-3 simulator. By setting the packet size to 64B and the sender throughput to 524288bit / s, the running results on the NS-3 simulator are as follows: Fig. 9 As shown, the OS network data processing method proposed in the present invention can realize the temporary storage and processing of high-speed and low-latency network data packets.

[0062] The NS-3 simulator is used as the experimental technology platform to design the simulation architecture of the DUCA architecture. The processing flow of the DUCA architecture is modularized into processing units with different functions, and each processing unit processes the data packet in turn. The NS-3 simulator is universal. The DUCA architecture is simulated based on the NS-3 platform, so that the processing mechanism is not limited to specific hardware, which solves the application difficulties of the network data processing mechanism brought about by the construction of the actual physical experimental environment, expands the application scope of the DUCA architecture, and makes the DUCA architecture also have a certain universality, which has a good application in the digital twin scenario.

[0063] The various embodiments of this specification are described in a progressive manner, and each embodiment describes the differences from other embodiments or implementations. The same or similar parts between the various embodiments of the present invention can be referred to each other, and the implementation principles and technical effects produced around the inventive concept can be referred to each other, which will not be repeated here. In the absence of conflict, the various embodiments or implementations of the present invention can be combined with each other.

[0064] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0065] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0066] Computer readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. Computer readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0067] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An OS network data processing architecture, including a network card, a cache and an application, characterized in that: The cache includes a first buffer for temporarily storing data packets, the first buffer is a partial storage area of ​​the cache, and is used to store the data packets received by the network card, and the first buffer also includes a control structure and a data packet storage area, and is used to manage the data packets in the first buffer; The application is used to directly access the data packet in the data packet storage area in the first buffer according to the control structure of the first buffer.

2. The architecture according to claim 1, characterized in that The control structure includes a global control structure and a sub-control structure, the global control structure is used to manage all the sub-control structures and space blocks, and each sub-control structure manages a data packet; The data packet storage area includes a plurality of space blocks of the same size, and the space blocks are used to store data packets.

3. The architecture according to claim 2, characterized in that: The global control structure includes a plurality of control elements, and manages the idle and used conditions of the sub-control structures according to the control elements; wherein the control elements include: the total number of sub-control structures, the number of available sub-control structures, the number of used sub-control structures, the number of free space blocks, the total number of space blocks, the sub-control structure area address offset, the next sub-control structure sequence number to be used, the next sub-control structure sequence number to be polled, the space block head address, the next space block to be filled, and the next space block to be processed; The sub-control structure includes the current sub-control structure serial number, the current data packet size managed by the sub-control structure, and the key position of the data packet in the space block; wherein the key position includes the position of the data packet header in the space block, the position where the data part of the data packet starts, the position where the data part of the data packet ends, and the position of the end part of the entire data packet.

4. The architecture according to claim 1, characterized in that: It also includes memory; The memory includes a second buffer, which is a partial storage area of ​​the memory. When the available space in the first buffer is insufficient, the second buffer stores the data packets sent by the network card; the second buffer also includes a control structure and a data packet storage area for managing the data packets in the second buffer; The application is also used to access the data packet in the data packet storage area in the second buffer according to the control structure in the second buffer.

5. An OS network data processing method based on any OS network data processing architecture in claims 1 to 4, characterized in that: include: When the data packet arrives at the network card, the data packet is transmitted from the input queue of the network card to the first buffer; After the data packet arrives at the first buffer, an interrupt process is triggered, and the application accesses the data packet in the first buffer in a polling process.

6. The method according to claim 5, characterized in that The step of transmitting the data packet from the input queue of the network card to the first buffer includes: Get the size of the data packet and check whether the number of available sub-control structures is greater than 0: If the number of available sub-control structures is greater than 0, the sub-control structure corresponding to the sub-control structure sequence number to be used next is obtained, and the sub-control structure stores the data packet in the space block; If the number of available sub-control structures is less than or equal to 0, it means that there is no available sub-control structure currently, and the data packet is discarded.

7. The method according to claim 6, characterized in that If the number of available sub-control structures is greater than 0, the sub-control structure corresponding to the next sub-control structure sequence number to be used is obtained, and the sub-control structure stores the data packet in the space block, further comprising: If the number of available sub-control structures is greater than 0, the number of space blocks occupied by the data packet is calculated based on the size of the space block and the size of the data packet, and Determine whether the number is less than or equal to the number of free space blocks: If so, the number of available sub-control structures is reduced by one, the number of used sub-control structures is increased by one, and the corresponding sub-control structure is determined according to the sub-control structure area address offset and the next sub-control structure sequence number to be used, the position of the data packet header in the sub-control structure points to the next space block to be filled, and the data packet is stored in the corresponding number of space blocks according to the position of the data packet header, and the next sub-control structure sequence number to be used and the next space block to be filled are updated; If not, it means that the current number of free space blocks is insufficient, and the data packet is discarded.

8. The method according to claim 5, characterized in that The application accesses the data packet in the first buffer in a polling manner, including: Check whether the number of used sub-control structures is greater than 0: If yes, the current sub-control structure managing the data packet is determined according to the sub-control structure sequence number to be polled next and the sub-control structure area address offset, and the application processes the data packet corresponding to the sub-control structure; If not, it means that there is currently no data packet that needs to be processed by the application.

9. The method according to claim 8, characterized in that After the application processes the data packet corresponding to the sub-control structure, the application further includes: Update the control elements in the global control structure; wherein the control elements include the sequence number of the next sub-control structure to be polled and the next space block to be processed, the number of free space blocks and the number of available sub-control structures.

10. The method according to claim 5, characterized in that After the application accesses the data packet in the first buffer in a polling manner, the method further includes: The data packet accessed by the application is marked as replaceable, and a new data packet sent from the network card covers the space of the first buffer occupied by the data packet marked as replaceable.

11. The method according to claim 5, characterized in that Also includes: When the data packet arrives at the network card and the available space in the first buffer is insufficient, The data packet is transmitted from the input queue of the network card to the second buffer. After the data packet arrives at the second buffer, an interrupt process is triggered, and the application accesses the data packet in the second buffer in a polling process.

12. A method for generating a simulation architecture, wherein the simulation architecture is used to simulate the processing architecture of any one of claims 1 to 4 based on a simulator; the simulator includes nodes, network devices, channels, applications, protocol stacks, and data packet structures, characterized in that: It also includes a hardware resource management module and a processing function simulation module; The method comprises: The hardware resource management module determines hardware resources according to any one of the processing architectures of claims 1-4, and provides hardware resources and corresponding hardware parameters for the simulation architecture; The processing function simulation unit includes a plurality of sub-units with different processing functions, and the sub-units corresponding to the processing architecture are selected from the sub-units and combined to generate a simulation model; a simulation instance is generated according to the simulation model, the hardware resources and the corresponding hardware parameters; The hardware resources, the network equipment, the application program, the data packet structure, the protocol stack and the simulation instance are added to the node, and the nodes are connected by the channel to form the simulation architecture.

13. The method according to claim 12, characterized in that The subunit corresponding to the processing architecture includes: A network card processing subunit, an interrupt processing subunit, a polling processing subunit, a data replication processing subunit and an application processing subunit.

14. The method according to claim 12, characterized in that The hardware resources include: network card, processor, cache, bus and memory.

15. The method according to claim 12, characterized in that The simulator is an NS-3 simulator.

16. A storage medium, characterized in that: The storage medium stores computer executable instructions, and when the computer executable instructions are loaded and executed by the processor, the steps of the method as claimed in any one of claims 5 to 11 are implemented.

Images