Packet processing computations with pre-allocated memory functionality
The PmemMalloc library provides persistence and high-performance enumeration support in network devices, solving the problem of persistence and enumeration deficiency of existing malloc libraries in network devices, and achieving efficient and flexible memory management and data processing.
Patent Information
- Application Number
- CN202380035724.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-05-24
- Filing Date
- 2023-03-30
- Publication Date
- 2024-12-03
AI Technical Summary
The existing malloc library has insufficient persistence and enumeration support in network devices, resulting in inefficiency, inaccuracy and inflexibility, affecting data processing and memory management.
Introduced the PmemMalloc library to provide persistence, enumeration and debugging. By pre-allocating, partitioning and fixed shared memory blocks, it can process parallel across multiple CPU cores, provide high-performance enumeration and delay services, avoid OS interactions, and adopt asynchronous Programming a model to improve CPU utilization.
It significantly improves the computing efficiency, accuracy and flexibility of the memory allocation system, improves data processing throughput, reduces memory locking and CPU resource consumption, and achieves more efficient CPU utilization and memory management.
Smart Images

Figure CN119072682A_ABST
Abstract
Description
Background Art
[0001] In recent years, significant progress has been seen in the hardware and software platforms that implement network devices such as those found in cloud computing systems. For example, network devices typically provide functions or services by receiving and processing incoming data packets. In some cases, one or more network applications (or simply referred to as "applications") on the network device process the incoming data packets. In many cases, these applications need to request memory from the network device to process the incoming data packets. Currently, applications like these rely on existing memory allocation function ("malloc") libraries, such as Glibc, jemalloc, and tcmalloc to allocate memory for packet data processing. However, existing malloc libraries face several technical disadvantages that lead to inefficient, inaccurate, and inflexible operations.
[0002] In detail, existing malloc libraries do not provide persistent memory support or enumeration support, which leads to many problems. In fact, existing malloc libraries are not designed to store heaps (i.e., heap memory) on persistent memory, but they dynamically allocate memory that is integrated with the operating system (OS) and subject to OS interaction. As a result, when data stored in memory is lost due to application restart, when the OS rewrites or otherwise interferes with the allocated data, or when other problems arise, memory and processing inaccuracies occur, as described below.
[0003] In addition, the existing malloc library does not provide enumeration support, which increases processing time and reduces computing efficiency. The lack of this enumeration support often leads to inefficient serial processing of packet data, uneven CPU distribution and memory locking. To elaborate, a server or virtual machine using the existing malloc library allocates multiple (e.g., four, eight, etc.) CPU cores (e.g., worker threads) for processing packet data. For example, in a four-CPU core system, when an application calls an existing malloc library, the existing malloc library horizontally divides the CPU core by allocating three of these cores to data processing, and the application uses the fourth core for enumeration operations. Therefore, in these cases, enumeration operations (or simply referred to as "enumeration") are strictly limited to a single CPU core, and are prevented from operating in parallel on other CPU cores, even when other CPU cores are not fully utilized. In addition, due to the use of a single CPU core, memory locking, lock contention and memory access errors often occur in the case of existing malloc libraries, because memory blocks are owned by other cores and / or heaps rather than by those cores that access them.
[0004] As an additional example of inefficiency, existing malloc libraries suffer from unequal CPU utilization and poor distribution caused by enumeration. As mentioned above, enumeration usually requires its own CPU core, which cannot be used for packet processing. Therefore, even if the enumeration core is idle or not fully engaged, the existing malloc library does not allow packet processing to occur on an idle CPU core assigned to the enumeration (and vice versa).
[0005] In addition, many existing malloc libraries are inflexibly limited to passive synchronization functions (e.g., synchronous programming models), which process packet data as it arrives, regardless of processing importance. Because service computations are done synchronously, existing malloc libraries require O(N) computations due to serializing and deserializing streams (e.g., in current services, all streams and objects are converted into new objects (e.g., at O(n))). For example, using existing malloc libraries, it takes an average of 40 seconds to process one million packet data streams. In fact, with existing malloc libraries, enumeration is performed serially on a single core and cannot be delayed, resulting in a significant reduction in processing throughput.
[0006] These and other problems result in significant inefficiency, inaccuracy, and inflexibility of existing malloc libraries. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] The detailed description provides one or more implementations with additional specificity and detail, as briefly described below, through the use of the accompanying drawings.
[0008] Figure 1A to Figure 1B A schematic diagram of a computing system environment is shown for implementing a memory allocation system in accordance with one or more implementations.
[0009] Figure 2 An example workflow for processing packet data utilizing a new memory allocation function library called PmemMalloc is shown in accordance with one or more implementations.
[0010] Figure 3 A schematic diagram illustrating processing packet data using the PmemMalloc library according to one or more implementations.
[0011] FIG. 4A to FIG. 4B An example architecture of the PmemMalloc functionality according to one or more implementations is shown.
[0012] Figure 5 A graph comparing CPU utilization between an existing malloc library and a PmemMalloc library according to one or more implementations is shown.
[0013] Figure 6A series of example actions are shown for servicing packet data using a new memory allocator library (ie, PmemMalloc) according to one or more implementations.
[0014] Figure 7 Some components that may be included in a computer system are shown. DETAILED DESCRIPTION
[0015] The implementation of the present disclosure provides the benefits of a system, a non-transient computer-readable medium and a method for a memory allocation system with a new memory allocation function library called PmemMalloc (or PmemMalloc library) and / or solves one or more of the aforementioned or other problems in the art problem. As described above, the PmemMalloc library promotes pre-allocation, partitioning and fixed shared memory blocks. By utilizing the PmemMalloc library, the memory allocation system can overcome the above-mentioned problems that hinder the persistence and enumeration of existing malloc libraries. In fact, the PmemMalloc library enables the memory allocation system to perform service calculations in parallel across multiple CPU cores / worker threads, equally distributes calculations between threads, and prioritizes / delays services in other improvements. It is noteworthy that the PmemMalloc library provides the main structures (e.g., persistence, enumeration and debuggability) that are unavailable to existing malloc libraries.
[0016] As described below, the memory allocation system implements persistence by implementing a heap on shared memory. In addition, the memory allocation system implements high-performance enumeration by partitioning memory to each data thread in a partitioned manner and attaching user metadata to each memory block. In addition, the memory allocation system performs enumeration based on the following structures: 1) parallelism, 2) equal allocation of CPUs, 3) latency, and 4) lock-free. In addition, the memory allocation system implements debuggability by reading shared memory and metadata from the outside.
[0017] As also described below, the memory allocation system provides an enumeration function that encourages applications to migrate some of their computations from packet processing to delayed enumeration, which improves the throughput of packet processing and effectively utilizes CPU cycles that would otherwise be idle. In addition, the memory allocation system also provides high priority enumeration to encourage enumeration to run at high speed with packet processing. As a result, as long as there is more available memory for deferring delayed operations, the memory allocation system indirectly improves throughput by consuming more idle CPU cycles.
[0018] To illustrate, in one or more implementations, a memory allocation system receives packet data at a network device including an application (e.g., a network application and / or a virtual application). In response to receiving the packet data, the memory allocation system can generate a memory heap within a pre-allocated, partitioned, and fixed shared memory block. In addition, the memory allocation system can attach allocation metadata to each block of the shared memory block (e.g., indicating an object type among other indicators), and perform memory operations across the shared memory blocks based on the allocation metadata.
[0019] As provided below, the memory allocation system disclosed herein provides multiple technical benefits in terms of computational efficiency, accuracy, and flexibility compared to conventional systems. In fact, the memory allocation system provides multiple practical applications that provide benefits and / or solve problems associated with memory allocation on computing devices (including computing devices in cloud computing systems).
[0020] In detail, in one or more implementations, the PmemMalloc library provides significant efficiency improvements for packet services. For example, in various implementations, a memory allocation system utilizes the PmemMalloc library in a data plane development kit (DPDK) server class environment within a cloud computing system, as provided below. For example, the memory allocation system may receive packet data via a network device including an application for processing packet data.
[0021] As part of processing (also referred to as "serving") input packet data, the packet library may enable a memory allocation system to generate memory heaps within shared memory blocks, where the shared memory blocks are pre-allocated, partitioned, and fixed. Furthermore, in various implementations, the memory allocation system equally distributes the amount of pre-allocated, fixed memory among DPDK worker threads (e.g., CPU cores).
[0022] By utilizing the memory heap of the shared memory block, the memory allocation system can overcome the persistence problems that hinder the current memory allocation library. For example, unlike the existing malloc library, the use of shared memory provides data persistence, even across application restarts (i.e., restarts). In addition, by utilizing shared memory, the memory allocation system avoids OS interaction because the shared memory is separated (e.g., partitioned) from the memory used by the OS (e.g., the memory heap of the shared memory block is pre-allocated without operating system integration). As a result, the OS will not rewrite or interfere with service operations.
[0023] Additionally, the memory allocation system provides significant efficiency gains for computing device services for packet data (e.g., packet streams). To illustrate, as described above, existing memory allocation libraries operate at O(n). In contrast, the memory allocation system operates at O(1) because shared memory eliminates the requirement for serialization and deserialization. In particular, although current service methods (e.g., existing malloc libraries) need to convert all streams and objects to new objects before accepting packets from a network interface card (NIC) (which requires O(n) operations), the memory allocation system delays these conversions by packets so that packets are immediately accepted from the NIC, resulting in operations at O(1). Therefore, although the existing malloc library takes 40 seconds to serve 1 million streams, the memory allocation system can serve 1 million streams in 2 seconds (i.e., 20X efficiency gain).
[0024] Additional advantages of the PmemMalloc library include a small code footprint, especially for memory allocation systems operating on servers like DPDK. Similarly, memory allocation systems using shared memory provide improved monitoring and debugging of object and data validation.
[0025] In various implementations, the memory allocation system can attach allocation metadata to each block of shared memory blocks, which also improves operational efficiency. For example, in one or more implementations, using the PmemMalloc library, the memory allocation system can store a small amount of user-defined metadata with each shared memory block. Part of the metadata can include an object type (e.g., the type of data stored), which allows the memory allocation system to select and efficiently service memory blocks with object types.
[0026] In addition, by creating and utilizing pre-allocated partitioned memory, the memory allocation system also overcomes the enumeration problem that inhibits existing malloc libraries. As mentioned above, existing malloc libraries do not directly support enumeration operations. On the contrary, the memory allocation system and PmemMalloc library provide enhanced and direct enumeration support. As described below, the memory allocation system provides parallel computing, distributed computing between worker threads, lock-free memory computing, delayed computing, and improved CPU usage.
[0027] To elaborate, the memory allocation system provides improved parallel computing by enabling enumeration to occur across multiple CPU cores rather than on a single core. In various implementations, the memory allocation system slices packet data across shared memory blocks and / or CPU cores (e.g., four, eight, etc.) for parallel processing. Specifically, the memory allocation system can use all CPU cores for both packet processing and enumeration, rather than allocating a portion of the CPU cores to packet processing and the remaining CPU cores to enumeration.
[0028] Similarly, the memory allocation system can provide improved CPU allocation across CPU cores. Because each of the CPU cores can be used for packet processing and enumeration, the memory allocation system can evenly distribute service tasks. Therefore, the memory allocation system unlocks previously unused CPU cores for packet processing or enumeration, which would otherwise be wasted. In addition, this allows calculations to be performed on CPU cores that own or create memory, which greatly improves efficiency.
[0029] In addition, the memory allocation system can reduce and improve CPU resource consumption by enabling lock-free memory functions. In fact, as a result of sharing memory blocks independently of OS interference, the memory allocation system reduces memory locking and improves throughput by better utilizing previously idle CPU resources.
[0030] In addition, the memory allocation system can improve service by delaying processing. For example, the memory allocation system introduces an active asynchronous programming model ("active asynchronous" or simply "asynchronous") on top of a traditional passive synchronous method (e.g., a synchronous programming model). In various implementations, the memory allocation system prioritizes enumeration operations and delays less important or time-sensitive tasks to be completed when the CPU core enters an idle cycle. Therefore, based on the priority of the enumeration operation, the memory allocation system processes high priority enumerations while delaying low priority enumerations until the CPU idle cycle begins. As a result, the CPU verifies and / or services the most important (e.g., higher priority) operations first, while also reducing idle CPU time by delaying lower priority enumeration operations.
[0031] As described above, by utilizing shared memory, the memory allocation system can solve the persistence problem of the above-mentioned existing malloc library. In fact, the memory allocation system does not need to perform serialization or deserialization, because the shared memory is accessible and the process can be easily attached thereto. In addition, by pre-allocating and partitioning the shared memory (e.g., proportional to the number of CPU cores / threads), the memory allocation system can solve the above-mentioned enumeration problem. For example, the memory allocation system retains the advantages of a server similar to DPDK, wherein the packet from the NIC can be shared to all CPU cores, due to no OS interaction and faster execution of operations, enumeration is run on all CPU cores in parallel with equal CPU distribution, operates in lock-free mode, and delays lower or optional enumeration tasks when the CPU core is idle.
[0032] As explained in the above discussion, the present disclosure utilizes a variety of terms to describe the features and advantages of one or more implementations described herein. For example, as used herein, a "cloud computing system" refers to a network of connected computing devices (e.g., network devices) that provide various services to client devices. For example, a cloud computing system can be a distributed computing system that includes a collection of physical server devices (e.g., server nodes) organized in a hierarchical structure that includes computing regions, clusters, virtual local area networks (VLANs), racks, load balancers, fault domains, etc.
[0033] In addition, the features and functions described in conjunction with a cloud computing system may similarly relate to racks, fault domains, or other hierarchical structures of physical server devices. A cloud computing system may refer to a private or public cloud computing system. In some implementations, a cloud computing system may include an orchestrator that manages servers, server clusters, server racks, server nodes, and / or other server groups of computing devices. In various implementations, a computing device including one or more virtual machines (e.g., a "host device") may include a hypervisor that facilitates hosting virtual machines on the host device.
[0034] As used herein, a "virtual machine" refers to a simulation of a computer system on a server node that provides the functionality of one or more applications on a cloud computing system. In various implementations, a host device allocates computing cores and / or memory to a virtual machine running on the host device. A virtual machine can provide the functionality required to execute one or more operating systems. In addition, a virtual machine can utilize a hypervisor on a processor of a server device that supports virtual replication of hardware. It should be understood that although the specific examples and implementation features and functions described herein with respect to virtual machines can be similarly applied to virtual machine groups and / or computing instance types.
[0035] As used herein, "application" refers to a virtual service provided by a cloud computing system. An application can be implemented on a virtual machine or another cloud computing device. For example, the backplane of a cloud computing system may include an application for processing and / or serving packet data. In some implementations, an application is a virtual application and / or a network application.
[0036] As used herein, "packet data", "data packet" or "packet" refers to data sent over a network. A series of packets may be referred to as a packet stream or simply a stream. Typically, a packet includes control information (e.g., header and trailer) and user data (e.g., payload).
[0037] As used herein, "shared memory" or "shared memory block" refers to memory on a computing device that can be accessed by multiple applications, CPU cores, worker threads, and / or programs. The size of the shared memory block can be fixed, pre-allocated, and / or partitioned for the operating system. Additionally, multiple CPU cores (e.g., virtual and / or real) can access the same block of shared memory, and / or seamlessly exchange stored data. Typically, the shared memory is random access memory (RAM).
[0038] Typically, a memory heap refers to memory on a computing device that resides in a free-floating area of memory used by applications, programs, and operating systems to store global variables. For example, when allocated via dynamic memory allocation, global variables are stored in heap memory space. However, as used herein, the term "memory heap" refers to memory that is pre-allocated, fixed, partitioned, and typically part of shared memory. Typically, the memory heap is random access memory (RAM).
[0039] As used herein, "memory allocation library" or simply "malloc" refers to a library of functions for allocating memory blocks on the heap. An application or program accesses a memory block via a pointer returned by a malloc function or call. When the memory is no longer needed, the pointer is passed to a "free" function to free the memory so that it can be used in other ways.
[0040] As used herein, "PmemMalloc library" refers to a library of improved functions for allocating memory blocks. In various implementations, the PmemMalloc library includes functions from malloc (or works in conjunction with malloc) and includes additional functions and features. In some implementations, the PmemMalloc library includes active asynchrony (e.g., an active asynchronous programming model) or operates in conjunction with active asynchrony (e.g., an active asynchronous programming model). Although the PmemMalloc library facilitates the operation of the PmemMalloc function, the term "PmemMalloc library" can be used to indicate the use of the PmemMalloc function.
[0041] Similarly, as used herein, "PmemMalloc" refers to a memory allocator that pre-allocates shared memory blocks using the PmemMalloc library. In some implementations, PmemMalloc allocates one or more fixed memory partitions to each DPDK worker thread (or simply referred to as "worker thread"). In addition, PmemMalloc can allocate partitioned and fixed-size shared memory blocks, and the allocated shared memory blocks can be equally allocated between CPU cores / worker threads. In various implementations, when implemented on shared memory, PmemMalloc allows objects to persist on application restarts (e.g., restarts). In an example implementation, PmemMalloc allocates calculations so that they will be executed on the CPU core where the memory is located, which results in a calculation without a lock state. In addition, PmemMalloc facilitates the allocation of service calculations so that the calculations will be executed on the CPU core where the memory is located. As described above, PmemMalloc has significant advantages and improvements compared to existing malloc libraries.
[0042] As used herein, "Data Plane Development Kit" (DPDK) refers to utilizing the pmemmalloc library in a server environment similar to the Data Plane Development Kit (DPDK) with a cloud computing system. For context, in some cases, a DPDK-like server includes software that provides a set of data plane libraries and NIC drivers for offloading TCP packet processing from the operating system kernel to a process running in another space (e.g., user space). In many implementations, a DPDK-like server has reserved resources (e.g., a fixed amount of memory and CPU / worker threads). In an example implementation, a DPDK-like server facilitates avoiding interaction with the operating system to improve throughput.
[0043] As used herein, "memory partitioning" refers to memory that is partitioned or partitioned between all DPDK worker threads / CPU cores (e.g., CPU cores on a DPDK-like server). In one or more implementations, memory is partitioned equally between DPDK worker threads / CPU cores. In various implementations, each DPDK worker thread / CPU core allocates and releases memory from its own partition without any locking contention.
[0044] As used herein, "memory operations" refer to operations performed on memory blocks. Memory operations may include basic memory operations (also known as "packet processing operations"), such as malloc and free (e.g., non-bulk operations) supported by the malloc library and other existing libraries. Memory operations may also include enumeration operations (or simply "enumeration"). The terms "bulk memory operations", "bulk operations", "enumeration", and "traversal memory" may be used interchangeably. Examples of enumeration / bulk memory operations include zero blocking, servicing of data structure changes, object expiration, port deletion, memory leak detection, etc.
[0045] As used herein, "allocation metadata" (or simply "metadata") refers to metadata generated and / or stored with a partitioned memory block. In various implementations, the allocation metadata includes a port number, an object type (e.g., the type of data stored), an object version, a timestamp, application-specific data, and / or user-specific data. Object types can include dozens of different types of objects, including various streams. In one or more implementations, the memory allocation system utilizes the allocation metadata to filter and select shared memory blocks that include the same object type (e.g., all packets with the same metadata object type tag) for combined processing.
[0046] Additional details will now be provided regarding a memory allocation system for improving memory allocation and data packet services within a cloud computing system. For example, Figure 1A to Figure 1B 1 shows a schematic diagram of an environment 100 (i.e., a digital media system environment) for implementing a memory allocation system 106. As shown in the figure, Figure 1A A cloud computing system 101 including a computing device 102 , a client device 130 , and a network 132 is included.
[0047] As shown, cloud computing system 101 includes computing device 102, such as a server device or other type of computing device. Computing device 102 may represent multiple computing devices within cloud computing system 101 (e.g., a node cluster having multiple server nodes). In addition, although not shown, in some implementations, cloud computing system 101 includes a server device having an orchestrator that manages various functions within cloud computing system 101, such as monitoring allocations, directing incoming network traffic to virtual machines, and / or otherwise managing operations within cloud computing system 101.
[0048] As shown, computing device 102 includes application 104, memory allocation system 106, and storage device 108. For example, memory allocation system 106 provides functionality to application 104 (e.g., a network or virtual application) to allocate memory from memory device 108 for servicing data packets. For example, memory allocation system 106 provides application 104 with a pmemmalloc library that allows the application (and / or memory allocation system 106) to create pre-allocated, partitioned, and fixed blocks of shared memory, as described above. Figure 1B Additional details regarding the memory allocation system 106 are provided.
[0049] As further shown, the environment 100 includes a plurality of client devices 130 in communication with the computing device 102 via a network 132. For example, one of the client devices 130 causes packet data to be sent to the computing device 102 for processing. In some implementations, the processed data is returned to one of the client devices 130. In some implementations, other data is returned to one of the client devices 130. In some implementations, the processed data is sent elsewhere and / or stored by the computing device 102.
[0050] In one or more implementations, client device 130 represents various types of computing devices, including, for example, mobile devices, desktop computers, or other types of computing devices. In addition, network 132 may include one or more networks that use one or more communication platforms or technologies to transmit data. For example, network 132 may include the Internet or other data links that enable electronic data to be transmitted between various client devices and devices of cloud computing system 101. Figure 7 Provides additional details about these computing devices and networks. In addition, although Figure 1A A particular arrangement of cloud computing system 101 , computing device 102 , client device 130 , and network 132 is shown, but environment 100 may include any number of systems, computing devices, and / or networks.
[0051] As mentioned above, in Figure 1B Additional details about the memory allocation system 106 are provided in . Specifically, Figure 1B The components of the memory allocation system 106 are shown. Figure 1A102 of the computing device 102. As shown, the memory allocation system 106 includes a packet manager 110, a shared memory manager 112, a metadata manager 114, a priority manager 116, an enumeration manager 118, and a storage manager 120. As also shown, the storage manager 120 includes packet data 122, memory pointers 124, metadata 126, and priorities 128. Although the components are shown on the memory allocation system 106, in some implementations, one or more components are located elsewhere, such as on the application 104, on the computing device 102 itself, or external to the computing device 102. In addition, one or more components may belong to the PmemMalloc library.
[0052] As shown, the memory allocation system 106 includes a packet manager 110. In various implementations, the packet manager 110 receives, accesses, provides, edits, modifies, identifies, creates, or otherwise manages packet data 122. For example, the packet manager 110 may communicate with a NIC (e.g., a real or virtual NIC) on the computing device 102 to accept incoming data packets for the memory allocation system 106 to service, forward, or otherwise process.
[0053] In one or more implementations, the packet manager 110 can perform packet processing. For example, the packet manager 110 performs basic memory allocation library functions and operations on the incoming packet data, as further described below. In some implementations, the packet manager 110 can also provide processed packet data to an external computing device.
[0054] As shown, the memory allocation system 106 includes a shared memory manager 112. In various implementations, the shared memory manager 112 facilitates creating, allocating, modifying, adding, removing, identifying, accessing, or otherwise managing shared memory blocks. For example, in one or more implementations, in conjunction with processing packet data, the shared memory manager 112 pre-allocates fixed blocks of shared memory from the memory device 108 on the computing device 102. In addition, the shared memory manager 112 can partition the shared memory blocks to be independent of the OS of the computing device 102 (e.g., not integrated with the OS). In addition, the shared memory manager 112 can create and / or provide memory pointers 124 (e.g., one or more memory pointers) that indicate one or more locations of the shared memory blocks within the memory device 108.
[0055] As shown, the memory allocation system 106 includes a metadata manager 114. In various implementations, the metadata manager 114 attaches, creates, identifies, edits, modifies, receives, accesses, provides, or otherwise manages metadata 126 (e.g., allocation metadata). For example, the metadata manager 114 attaches allocation metadata to some or all of the pre-allocated, fixed, and partitioned shared memory blocks. In various implementations, the metadata manager 114 attaches metadata (e.g., user-defined metadata) to objects stored in the shared memory blocks.
[0056] In some implementations, the metadata manager 114 includes information about objects corresponding to objects stored in the memory device 108. For example, the metadata manager 114 includes a port number, an object type (e.g., the type of data stored), an object version, a timestamp, application-specific data, and / or user-specific data within the metadata of the object and / or shared memory block. Furthermore, in various implementations, the metadata manager 114 utilizes the metadata to identify and / or select objects and / or shared memory blocks having a target metadata tag (e.g., to find all shared memory blocks having a target object type).
[0057] As shown, the memory allocation system 106 includes an enumeration manager 118. In various implementations, the memory allocation system 106 performs enumeration, bulk operations, and memory walks as part of processing the packet data 122. Examples of enumeration operations include servicing data structure changes, object expiration, port deletion, etc. In some implementations, the enumeration manager 118 utilizes functions from the PmemMalloc library to perform various enumeration operations.
[0058] In addition, the enumeration manager 118 can utilize the priority 128 to determine when to service the packet data. For example, the priority 128 of the shared memory block and / or object can indicate the priority level of the packet data (e.g., high priority bulk operation, low priority operation, and selectable priority operation) so that the enumeration manager 118 can determine when to optimally process it. Additional details about enumeration and priority are provided below.
[0059] Additionally, the memory allocation system 106 includes a storage manager 120. In various implementations, the storage manager 120 may include any data used by any component of the memory allocation system 106 in performing the features and functions described herein. For example, the storage manager 120 may include packet data 122, memory pointers 124, metadata 126, and priorities 128, each of which is included above. In various implementations, the storage manager 120 also includes code and / or instructions for the PmemMalloc library.
[0060] Combination Figure 2Additional details regarding an example implementation of the memory allocation system 106 are discussed. For example, Figure 2 An example workflow for processing packet data using a new memory allocation function library called PmemMalloc according to one or more implementations is shown. As shown in the figure, Figure 2 A series of actions 200 are shown for the memory allocation system 106 to process packet data utilizing the PmemMalloc library.
[0061] like Figure 2 As shown, a series of actions 200 includes an action 210 of detecting packet data arriving at a computing device having an application such as a network and / or a virtual application. For example, in various implementations, the memory allocation system 106 receives packets from a NIC (e.g., real or virtual) and / or a polling mode driver. For example, one or more client devices cause the packets to be provided to the memory allocation system 106 for processing.
[0062] also, Figure 2 A series of actions 200 are shown including an action 220 of processing packet data using a pre-allocated memory allocation function (PmemMalloc). For example, in one or more implementations, the memory allocation system 106 obtains a pre-allocated, fixed, and partitioned shared memory block for an application. For example, the memory allocation system 106 can help process packet data by providing a PmemMalloc library to the application, and the application executes a call and utilizes the function from the PmemMalloc library. In some implementations, the memory allocation system 106 can allocate shared memory equally with the working CPU cores / threads of the computing device.
[0063] In various implementations, the memory allocation system 106 performs packet processing of data packets by sharding (e.g., splitting and sharing) the packets to all CPU cores / threads. In addition, the memory allocation system 106 can perform various basic (e.g., non-batch) memory operations, such as passive operations, on the packet data (e.g., streams). In various implementations, the memory allocation system 106 can use all CPU cores / worker threads in an equal and distributed manner to perform packet processing operations. FIG. 4A to FIG. 4B Provides additional details about packet processing.
[0064] In some implementations, the memory allocation system 106 also additionally performs enumeration operations on the packet data. As described above, the memory allocation system 106 can perform batch operations on some or all CPU cores / threads in parallel and equally distributed on the CPU cores / threads based on the functions in the PmemMalloc library. In addition, the memory allocation system 106 can utilize the passive and active programming models from the PmemMalloc library to prioritize higher priority packet processing tasks while deferring lower priority packet processing tasks, which improves CPU usage and packet processing efficiency. Figure 3 (extended version of action 220) and FIG. 4A to FIG. 4B Provides additional details about enumeration and PmemMalloc.
[0065] As shown, the series of actions 200 includes an action 230 of providing the processed packets. For example, in some implementations, the memory allocation system 106 provides the processed data packets to the requesting client device. In some implementations, the memory allocation system 106 provides the processed data to other locations of the computing device, cluster nodes, or other places in the cloud computing system. In some implementations, the memory allocation system 106 provides the processed data to a computing device external to the cloud computing system.
[0066] As mentioned above, Figure 3 Provides additional details about enumeration and PmemMalloc. Specifically, Figure 3 A diagram of processing packet data using the PmemMalloc library according to one or more implementations is shown. As also described above, Figure 3 Is from Figure 2 In some implementations, an application (e.g., a network application) performs an extended version of action 220 of processing the received packet data using a pre-allocated memory allocation function (PmemMalloc) in conjunction with the memory allocation system 106. Figure 3 Some or all of the sub-actions shown.
[0067] As shown in the figure, Figure 3The action 220 in includes multiple sub-actions, which includes generating a first sub-action 321 of pre-allocated, partitioned and fixed shared memory blocks. For example, in various implementations, the memory allocation system 106 receives packet data for processing, as described above. As part of processing packet data, the memory allocation system 106 can create pre-allocated blocks of shared memory. In addition, the memory allocation system 106 can partition shared memory blocks and fix their sizes. In various implementations, the memory allocation system 106 applies the heap memory structure of the existing malloc library to the shared memory blocks. In this way, the memory allocation system 106 utilizes the known heap memory structure while increasing the benefits of the pre-allocated, fixed memory blocks from the OS partition.
[0068] In conjunction with the first sub-action 321 of generating a shared memory block, the memory allocation system 106 can create metadata and attach the metadata to each shared memory block in the shared memory block. In fact, as shown in the figure, the action series 200 includes a sub-action 322 that attaches the allocation metadata to the shared memory block. As described above, the metadata can indicate and / or mark the object type associated with each shared memory block (e.g., the object stored in the shared memory block). Metadata (e.g., allocation metadata) can also include other indicators, such as object version, timestamp, port number for processing objects, application-specific data and / or user-specific data.
[0069] In some implementations, the metadata includes a processing priority tag associated with the object. In an alternative implementation, each object belongs to a priority level (eg, objects are pre-assigned to high, low, or selectable priorities). Priority processing of the memory allocation system 106 is further described below.
[0070] As shown, the action series 200 includes a sub-action 323 that equally distributes the shared memory blocks on a fixed number of worker threads / CPU cores. For example, in various implementations, the memory allocation system 106 pairs the shared memory blocks with the CPU cores to achieve efficient processing of the received packet data. In addition, due to the benefits provided by the shared memory blocks, the memory allocation system 106 does not have to horizontally split the CPU cores between packet processing and enumeration, but can equally distribute the shared memory blocks across all CPU cores for both packet processing and enumeration.
[0071] In various implementations, the memory allocation system 106 shards (e.g., copies and shares) the packet data into each of the shared memory blocks and / or CPU cores. In this way, the CPU cores can maximize the processing of the data packets because each of the CPU cores can access the packet data. Depending on the implementation, sharding can occur before or after sub-action 323.
[0072] As shown, the action series 200 includes a sub-action 324 of processing the packet data and performing a high priority enumeration operation across the CPU cores according to the metadata. For example, in various implementations, the memory allocation system 106 identifies the packet data and / or objects that may require immediate and / or priority processing. In various implementations, the memory allocation system 106 identifies objects that require synchronous programming model functions (e.g., packet processing and passive synchronous operations) and objects that require high priority asynchronous functions (e.g., high priority enumeration operations).
[0073] In various implementations, the memory allocation system 106 utilizes metadata attached to objects and / or shared memory blocks to filter (e.g., identify) objects with asynchronous high priority. In this way, the memory allocation system 106 can first process and / or enumerate objects that require immediate attention, while deferring lower priority or selectable priority objects when CPU resources are available.
[0074] In various implementations, the memory allocation system 106 facilitates parallel processing across each of the CPU cores. Thus, passive synchronous packet processing and / or high priority enumeration operations can utilize all possible CPU resources without any CPU idle waiting because it is allocated to different types of processing operations. In fact, the memory allocation system 106 facilitates equal CPU allocation without memory locks.
[0075] In multiple implementations, the memory allocation system 106 utilizes metadata (e.g., allocating metadata) to allocate objects between operation types. As described above, the metadata may include the stored data type (e.g., UF stream, LF stream, etc.), timestamp, port number, etc. Therefore, in some implementations, when the application and / or memory allocation system 106 creates a shared memory block (e.g., first sub-action 321), it may create the metadata as a memory object and / or tag the object with an object type tag (e.g., the stored data type) (e.g., sub-action 322). Then, during enumeration, the application 104 and / or the memory allocation system 106 may identify and traverse (e.g., enumerate or batch process) all objects and / or shared memory blocks of a specific object type (e.g., having the same metadata tag). In practice, the memory allocation system 106 enables PmemMalloc to filter the desired memory blocks based on the metadata and enumerate all packets with the target metadata object type tag (e.g., process all packets with UF stream, LF stream, etc.).
[0076] Additionally, in some implementations, the memory allocation system 106 identifies objects that require high priority enumeration operations based on metadata. For example, the memory allocation system 106 classifies a first subset of object types as high priority enumeration operations, a second subset of object types as low priority enumeration operations, and / or a third subset of object types as optional priority enumeration operations. In this way, by identifying metadata of objects and / or shared memory blocks, the memory allocation system 106 can know their processing priority.
[0077] As shown, the action series 200 includes a sub-action 325 that postpones the execution of low priority enumeration operations across CPU cores until the CPU cores are idle. For example, the memory allocation system 106 first processes passive synchronization and high priority asynchronous enumeration operations (e.g., high priority batch operations), and then the memory allocation system 106 processes low priority asynchronous enumeration operations (low priority batch operations). In some implementations, the memory allocation system 106 waits until the CPU idle time increases to exceed or exceed the CPU utilization threshold level. In other words, when the CPU utilization level drops below the CPU utilization threshold level, the memory allocation system 106 can start processing priority asynchronous enumeration operations. Additional disclosures about this subject matter are provided below.
[0078] In various implementations, delayed low priority enumeration includes two types. For example, delayed low priority enumeration can include low priority enumeration with infinite delay. In these cases, even if the memory allocation system 106 does not utilize CPU cycles in a very long period of time, it also postpones enumeration. In addition, the memory allocation system 106 processes the nodes and / or memory blocks in the enumeration list by packet processing over the time period, takes them out of the enumeration and reduces the size of the enumeration over the time period (for example, this can occur with data structure change enumeration and address mapping enumeration). In the example implementation, the low priority enumeration with infinite delay is the only enumeration type, and the nodes in the list shrink over the time period because the nodes are processed by packet processing.
[0079] In addition, delayed low priority enumerations may include low priority enumerations with a determined degree of delay. For example, when the packet engine omits processing of various nodes and / or memory blocks (e.g., object expiration enumerations) for a period of time, memory pressure begins to increase. Therefore, the memory allocation system 106 may temporarily promote these enumerations (e.g., low priority enumerations with a clear delay) to high priority for a short period of time until the memory pressure decreases.
[0080] As shown, the action series 200 includes a sub-action 326 for processing optional priority enumeration. For example, in various implementations, the memory allocation system 106 identifies grouped data and / or objects that are useful for processing given additional idle CPU cycles, but are not necessary for processing. In practice, optional enumeration refers to operations that are not guaranteed to be processed by the memory allocation system 106 and can be stopped due to insufficient resources.
[0081] As mentioned above, FIG. 4A to FIG. 4B Provides additional details on packet processing, enumeration, and utilizing the PmemMalloc library. Specifically, FIG. 4A to FIG. 4B An example architecture of the PmemMalloc functionality according to one or more implementations is shown. As shown, FIG4 includes an application 104 and a PmemMalloc architecture 400. In some implementations, the application 104 is part of a backplane and / or a virtual filtering platform (e.g., a virtual switching platform for host software-defined networking in a cloud computing system).
[0082] Specifically, PmemMalloc framework 400 includes basic operations 430, which may include synchronous programming model functions 432 (e.g., basic memory allocation operations). In fact, memory allocation system 106 may perform basic operations when receiving a packet, such as creating a stream, updating a stream, or terminating a stream. Basic operations may be passive synchronization operations, and are similar to existing Malloc libraries. However, the difference of memory allocation system 106 is that PmemMalloc stores a small amount of metadata for each memory block, and memory allocation system 106 utilizes these metadata to improve processing priority and other functions, as described below. In various implementations, basic operations 430 represent a first memory operation module.
[0083] In addition, the PmemMalloc architecture 400 includes a PmemMalloc asynchronous programming function 410, which includes an enumeration operation 420, a cache 426, and a metric and log 428 (e.g., a cross log). In addition, the enumeration operation 420 includes various operations, including asynchronous high priority 422, asynchronous low priority 424, and asynchronous optional priority operations. In one or more implementations, the enumeration operation 420 represents an enumeration (e.g., housekeeping) operation, such as registering and deregistering shared memory blocks.
[0084] In some implementations, the PmemMalloc asynchronous programming function 410 represents a second memory operation module. In addition, as shown, the PmemMalloc architecture 400 includes a second memory operation module built on top of the first memory operation module. In fact, in one or more implementations, the active asynchronous module is located on top of the traditional passive synchronous method for memory management (but using a heap in pre-allocated shared memory instead of releasing a heap in floating memory).
[0085] like Figure 4A As shown, the PmemMalloc architecture 400 facilitates various API and / or operation calls from the application 104, as shown by arrows 440 to 444. For example, the first arrow 440 can represent a Malloc, free, open, or close call of the application 104 (e.g., supported by the Malloc library). As described above, unlike the existing Malloc library that relies on unpartitioned and unpreallocated heap memory, the memory allocation system 106 enables these calls to be made to preallocated shared memory blocks. In fact, the basic operation 430 and / or the synchronization programming model function 432 can utilize the heap memory created in the shared memory block, which is preallocated, fixed, partitioned, attached with metadata, etc.
[0086] In addition, as shown in the figure, second arrow 442 can represent application 104 registration, batch cancellation, batch operation or other batch (e.g., enumeration) call to PmemMalloc asynchronous programming function 410. In some implementations, the call of second arrow 442 corresponds to main loop. In additional implementations, application 104 can perform a call in the call during CPU idle state, such as batch operation, which will be described below. In fact, because memory allocation system 106 performs calculations on the CPU core that owns or creates memory, memory allocation system 106 can perform these operations more effectively than existing Malloc library.
[0087] In addition, as shown, the third arrow 444 can represent the enumeration operation 420 and / or the PmemMalloc asynchronous programming function 410 providing a callback to the application 104. For example, the memory allocation system 106 can provide an indication of the data processed and / or the data within the serviced shared memory block. In some implementations, because the PmemMalloc architecture 400 includes an asynchronous model, the memory allocation system 106 can perform enumeration (e.g., business / housekeeping (function) functions) and / or non-base code as part of the callback.
[0088] The following is a sample algorithm for utilizing the PmemMalloc library on the Virtual Filtering Platform (VFP).
[0089] 1. If there is any change in the data structure (e.g., uniform stream / layered stream data structure), VFP starts as follows:
[0090] a. Set the flag service_inprogress = 1 (this flag is set only when the data structure changes.).
[0091] b. Register in PmemMalloc and start enumeration.
[0092] 2. If a packet is received and service_inprogress == 1, do the following:
[0093] a. Get the stream pointer from the hash table.
[0094] b. Check PmemMalloc to see if the stream is patched or non-patched.
[0095] (PmemMalloc knows if it is patched based on metadata containing the version of the UF / LF data structure)
[0096] c. If it is non-patched, convert it to new using the transform function.
[0097] d. Transformation function - A computationally rich function that converts a stream from any version to any other version. This function is called in two contexts:
[0098] i. Synchronize during the processing of a packet, or
[0099] ii. From an asynchronous enumeration.
[0100] When all streams are switched, service_inprogress is set to 0 and the switch function will not be called yet.
[0101] e. Process the remainder of the computation in the current manner.
[0102] 3. If a packet is received and service_inprogress == 0, do the following:
[0103] a. Process the packet in the current manner (eg, step 2e).
[0104] 4. Repeat steps 2 and 3.
[0105] 5. During object expiration, check service_inprogress == 1 and perform similar actions as step 2.
[0106] 6. Active asynchronous execution during CPU idle time – callback:
[0107] a. When the enumeration completes the full scan, it uses a callback to indicate that the traversal is complete (eg, inside the callback, set service_inprogress = 0).
[0108] b. When the enumeration sees an unpatched object (when an object is converted from an old version to a new version, the metadata is updated accordingly). Then, it calls the callback. The callback converts the unpatched object to the new object, which is similar to step 2d.
[0109] VFP Service - Data Structure Algorithm
[0110] Similarly, following is a sample algorithm for the enumeration algorithm.
[0111] 1. Register an enumeration for an object type: (e.g., a unified stream of ports).
[0112] a. Parameter: object type.
[0113] 2. Traverse all memory blocks in order:
[0114] b. If the object's type matches, the callback is called.
[0115] 3. Repeat step -2 until all blocks are traversed.
[0116] 4. Callback to indicate traversal completion, at which point service_inprogress will become 0.
[0117] Enumeration Algorithm
[0118] While current servicing methods (e.g., existing malloc libraries) require that all flows and objects be converted to new objects before packets from the NIC will be accepted, requiring O(n) operations, the memory allocation system 106 defers packet conversion and immediately accepts packets from the NIC, resulting in operations at O(1) (e.g., the memory allocation system 106 receives packet data corresponding to the application from the NIC before the flows are converted to new objects). Therefore, while it takes 40 seconds to service one million flows using the existing malloc library, the memory allocation system can service one million flows in 2 seconds.
[0119] As described above, in an example implementation, the memory allocation system 106 can improve efficiency by deferring less critical and / or time-sensitive tasks based on CPU usage. For example, unlike existing malloc libraries, which are general-purpose and not designed for memory-intensive applications, the memory allocation system 106 efficiently supports bulk memory operations (i.e., enumeration). In fact, in many implementations, the memory allocation system 106 can perform memory calculations without locks, distribute calculations equally among threads, and traverse memory blocks to perform calculations during CPU idle cycles. To further illustrate, Figure 4B A representation of CPU usage for synchronous programming model functions and asynchronous programming model functions is shown.
[0120] In one or more implementations, the memory allocation system 106 will utilize an active asynchronous model to perform batch memory operations for servers similar to DPDK. These batch memory operations may include memory-intensive tasks, and for various operations such as virtual filtering platform (VFP) services, port deletion, object expiration, etc., it may be necessary to periodically traverse memory objects. Therefore, although some of these operations are considered passive synchronization in existing malloc libraries and must be processed urgently, the memory allocation system 106 can postpone one or more of these operations under the PmemMalloc library and the active asynchronous model. In fact, as described below, the memory allocation system 106 can utilize an active asynchronous model to improve the efficiency of a computing device (e.g., a server) by dividing the calculation into two processing sets, and the two processing sets occur at different times based on CPU utilization.
[0121] To illustrate, Figure 4B Includes CPU non-idle cycles 450 and CPU idle cycles 460. During CPU non-idle cycles 450, the memory allocation system 106 performs calculations that cannot be delayed or postponed. For example, when certain events occur like receiving packets, these calculations need to be performed. As shown, CPU non-idle cycles 450 include synchronous programming model functions 432 and asynchronous high priority 422 (e.g., high priority active asynchronous programming model functions).
[0122] In one or more implementations, the synchronous programming model function 432 (e.g., passive synchronization) refers to a computation that the memory allocation system 106 performs synchronously when an external event such as receiving a packet occurs. This is often referred to as passive synchronization because it does not require any threads and is only performed when an external event occurs. In addition, passive can mean that the computation inside the passive component is only triggered and executed when the function is called.
[0123] In some implementations, asynchronous high priority 422 refers to calculations that the memory allocation system 106 performs asynchronously when an external or internal event occurs. For example, external events may include receiving a control command, and internal events may include timer expiration, memory usage exceeding a limit, etc. In fact, active asynchronous operations use the term "active" to indicate that the active component is triggered and executed when the function is called. In addition, the memory allocation system 106 also periodically performs calculations as part of enumeration (e.g., via an additional housekeeping / enumeration thread). According to some implementations, the priority between passive synchronization and asynchronous high priority is the same. In some implementations, the entire CPU cycle is shared between passive synchronization and asynchronous high priority in an m:n ratio.
[0124] As shown in the figure, Figure 4B A CPU idle cycle 460 is included with asynchronous high priority 422, asynchronous low priority 424 (e.g., active asynchronous programming model functions), and asynchronous optional priority 425 operations. In other words, asynchronous low priority 424 can be shared with asynchronous high priority 422 and executed when the CPU is idle, while asynchronous optional priority 425 can be executed when additional CPU resources are available. In various implementations, after completing passive synchronization and asynchronous high priority functions, the CPU / thread core can begin to enter CPU idle cycle 460. Therefore, the memory allocation system 106 can utilize available CPU resources by performing delayed service operations. In other words, after the asynchronous high priority operation is completed, the memory allocation system 106 can use the remaining CPU cycles for asynchronous low priority operations.
[0125] In various implementations, asynchronous low priority 424 refers to computations that are delayed by the memory allocation system 106 for later execution. For example, when these delayed computations reach CPU idle cycles 460, the memory allocation system 106 can switch or execute them. Therefore, under the PmemMalloc framework, when the thread / CPU core enters the idle state, each worker thread has a dedicated CPU core at the end of the synchronous task. But before the idle state, the memory allocation system 106 performs low priority asynchronous computations.
[0126] In fact, by performing calculations in a delayed manner (e.g., using an asynchronous model), the memory allocation system 106 can better consume idle CPU cycles. In addition, in some implementations, asynchronous low priority 424 refers to calculations that can postpone the memory allocation system 106, such as zeroing of memory, expiration of objects, etc., which can be performed when the CPU is idle and at a lower priority.
[0127] In some implementations, asynchronous selectable priority 425 refers to a computation that the memory allocation system 106 defers execution upon completion of an asynchronous low priority 424 operation. Alternatively, the memory allocation system 106 may disable asynchronous selectable priority 425 when computing resources are otherwise needed. Examples of enumerations associated with asynchronous selectable priority 425 include header / tail validation and object content validation.
[0128] As described above, the memory allocation system 106 provides various enumeration types and attributes, including high priority enumerations, delay-determined enumerations, delay-uncertain enumerations, and delay-optional enumerations. In fact, as previously provided, high priority enumerations are not delayed and can be similar to packet processing and other time-sensitive operations; delay-uncertain enumerations include operations that may be exhausted, and therefore, the memory allocation system 106 promotes them to high priority to relieve memory pressure, delay-uncertain enumerations include operations in which the size is gradually reduced even if the enumeration is inactive; and delay-optional enumerations include operations that are not guaranteed and can be stopped if computing resources are insufficient. In addition, the flags or attributes of the enumerations are rearranged, which include recursive, user-defined / system-defined, multiple / single, memory release (e.g., these enumerations release memory), and priority between delay enumerations.
[0129] As described above, the memory allocation system 106 provides a number of improvements to the existing malloc library. To further illustrate, Figure 5 A diagram 500 is shown comparing CPU utilization between an existing malloc library and a PmemMalloc library according to one or more implementations. As shown, Figure 5 The first graph 510 includes CPU utilization of the current system (eg, existing malloc library) and the second graph 520 of CPU utilization of the PmemMalloc library utilized by the memory allocation system 106. In addition, the first graph 510 and the second graph 520 represent processing of packet data using four CPU cores.
[0130] As shown, the group processing (e.g., black area) in the first graph 510 peaks at 75% of full CPU utilization because the existing malloc libraries split the CPU cores horizontally and they cannot use the fourth CPU core enumerated by the application allocation. In contrast, as shown in the second graph 520, the memory allocation system 106 can perform group processing across all CPU cores.
[0131] In addition, the first figure 510 shows that when the packet processing is easy, the existing malloc library cannot use all CPU resources. As a result, the first figure 510 includes a large number of idle CPU cycles (e.g., the brightest area at the top). On the contrary, because the memory allocation system 106 can allocate operations across each CPU core in the CPU core and perform all object operation types on any CPU core, the second figure 520 shows a very small amount of idle CPU cycles. In addition, because the memory allocation system 106 can postpone certain enumeration tasks, when the packet processing is low, the memory allocation system 106 can use more CPU cores to postpone the enumeration tasks.
[0132] The following disclosure demonstrates additional details about one or more of the implementations of the above-described memory allocation system 106. As described above, the memory allocation system 106 adopts an active asynchronous model within the framework of the PmemMalloc library. As an advantage of active asynchronous, all worker threads (e.g., CPU cores) can operate in their own memory partitions, which are pre-allocated to minimize lock contention to near zero. By doing so, the throughput of a computing device (e.g., a server) will increase.
[0133] Specifically, the computing device distributes CPU cycles equally among the CPU cores (e.g., distributes computation equally to all cores assigned to the DPDK worker threads). In addition, computation is performed in parallel using a lock-free or near lock-free state, which enables processing throughput to rise to its highest level (e.g., each CPU core / thread uses a pre-allocated separate memory partition, and asynchronous computation will traverse or scan the memory partition at different priority levels with a lock-free state, and contention between threads or CPU cores is close to zero).
[0134] Regarding lock-free, in various implementations, the memory allocation system 106 provides various structures for lock-free. For lock-free, there may be three interrelated substructures associated with this structure: a) lazy release, b) no lock, and c) zero release or memory leak detection. For lazy release, all objects in PmemMalloc can be lazy released. Lazy release can be achieved by marking the object as free, thereby delaying most calculations related to free. In addition, lazy release is lightweight and garbage can be collected in a delayed manner. The advantage of the lazy release configuration is that it gives more CPU cycles to process packets. According to test data, malloc and free use equal calculations, which means that lazy release will reduce the calculation of malloc-free by 40-50%, and these CPUs can be used for packet processing. In fact, as described above, the PmemMalloc library supports various release changes, including normal release, zero release, and lazy release. In addition, the PmemMalloc library enables Free to be transparently changed to lazy release without affecting the current application code (for example, from the point of view of the application code, no change is detected).
[0135] For lock-free, in order to make PmemMalloc run in lock-free mode, the memory allocation system 106 may need cross-malloc, and cross-free needs to be disabled. Cross-malloc means that memory is allocated from one core, but owned by another core. If the data thread does not have free memory in its own memory partition, PmemMalloc will obtain memory from the adjacent partition. This will use the memory optimally, but at the expense of locks. If the packet is evenly sharded to all CPUs, cross-malloc can be disabled because it saves less memory. "Cross-free" means that memory is released from one core, but owned by another core. For cross-free, any thread can release any object, so cross-free will require locking. However, if PmemMalloc is configured with lazy release, cross-free will be automatically disabled.
[0136] For zero release or memory leak detection, no release API from the application may be required, and objects may be garbage collected in some cases. In PmemMalloc, each object includes a timestamp as part of its metadata. Therefore, if an object is not released or updated, PmemMalloc can detect it as part of the system enumeration and notify the application that the specific object suffers from a leak. The memory allocation system 106 and / or the application can then release it. If PmemMalloc is configured for zero release, cross-release will be automatically disabled. But the application can still explicitly call release, which is internally interpreted as lazy release.
[0137] In addition, the PmemMalloc library can include PmemMalloc primitives such as non-OOM and memory leak detection. For non-OOM, OOM will never occur in the PmemMalloc library due to pre-allocated memory. For memory leak detection, PmemMalloc has no memory leaks because timestamps are attached to objects. In fact, the memory allocation system will detect leaks and notify the application and / or fix the memory leaks.
[0138] Additionally, with respect to the application impact of zero deallocation on applications, during packet processing in memory-intensive applications, applications will extensively request either a) create / update or b) delete processing types, where the computations done for each type are roughly the same). There is also an internal operation called object expiration, typically implemented as one or more timers. The final result of the delete packet processing and object expiration will result in the deletion of the object.
[0139] In addition, one method of increasing the speed of packet processing is to reduce deletion operations as much as possible. Therefore, in various implementations, the memory allocation system 106 does this in the following manner: when a delete request arrives, the memory allocation system 106 marks the main object as similar to expiration. In addition, the memory allocation system 106 links the remaining objects to the main object that will be deleted by the object expiration enumeration. Therefore, instead of calling release, the memory allocation system 106 detects expired objects and generates callbacks in a delayed manner.
[0140] In various implementations, the zero release performed by the memory allocation system 106 is similar to how Java collects garbage objects. Therefore, in pmemMalloc, the memory allocation system 106 can only collect objects of a specific type and then notify the application so that the application issues a lazy release or a normal release for deletion.
[0141] As an additional advantage of active asynchrony, CPU idle cycles will be used effectively because idle CPU cycles are used for delayed calculations by traversing all memory objects in parallel. In some cases, if the CPU core is busy with requests, active asynchronous calculations can be automatically converted to passive synchronization. In some implementations, the memory allocation system 106 determines that the computing device (e.g., server) is busy when the percentage of CPU idle cycles is less than a first threshold or the number of packets received per second is higher than a second threshold. Through the above operations, calculations performed outside the DPDK worker thread or in the control thread will be minimized. In addition, asynchronous active will facilitate incrementally offloading calculations from the control thread to the worker thread, and when these calculations are completed, the number of worker threads can be increased.
[0142] In addition, the memory allocation system 106 provides several advantages by offloading computation from packet processing. In fact, the memory allocation system 106 migrates computation from packet processing to delayed enumeration, which significantly improves the throughput of packet processing and effectively utilizes otherwise idle CPU cycles. For example, the memory allocation system 106 can offload Calloc, lazy release, zero release, and mutable object expiration from packet processing.
[0143] In detail, the Calloc operation may include asynchronously filling a release block with zeros. During packet processing, the memory allocation system 106 saves the application of performing calloc calculations with no changes / minimal changes in the application code. With the lazy release operation, the memory allocation system 106 can convert all release calls to lazy releases. In lazy release, the object will be released during the delayed enumeration. Here, there is no need to change the application code. In addition, for the lazy release operation, the memory allocation system 106 offloads the packet processing and helps make the object lock-free (otherwise, under the existing Malloc library, locks will be required for cross release).
[0144] With zero release operations, "delete requests" in packet processing become lightweight operations by offloading the memory allocation system 106 to delayed enumeration. Therefore, instead of fully processing an object, partial object processing is done by marking the primary object as a zero release object and using callbacks to delete the remaining objects as part of delayed enumeration. As described above, the memory allocation system 106 converts the delete request to be similar to object expiration. Object expiration is triggered due to the expiration time, and the delete request from the NIC will generate and / or mark objects with zero marks, and these objects will be deleted in a delayed manner similar to object expiration.
[0145] With variable object expiration operations, the existing Malloc library can offload the task, but it is not efficient because the existing enumeration is fixed object expiration and it is not efficient because the application enumeration is not efficient. In contrast, the memory allocation system 106 uses variable expiration idle cycles to efficiently offload and release memory, rather than fixed expiration variable object expiration operations.
[0146] As a further advantage of active asynchrony, the memory allocation system 106 can increase DPDK worker threads by migrating batches of computations from control threads. Thus, memory-intensive computations can be migrated from control threads to parallel DPDK worker threads. By doing so, computations can be performed in parallel with virtually no locks. Furthermore, by distributing equal computations between worker threads with virtually no locks, the memory allocation system 106 can achieve high throughput with the same DPDK worker threads, which can result in high throughput compared to unequal allocation of locks with existing malloc libraries.
[0147] Above, enumeration (e.g., housekeeping operations) is mentioned as part of active asynchrony. In various implementations, enumeration is a function called from the backplane just before the CPU is about to become idle and / or once every n packets. Enumeration consumes a small amount of CPU computation. In one or more implementations, enumeration includes two types of tasks. The first task is a low priority enumeration task. Here, when the CPU is idle, the low priority enumeration is called. The second task is a high priority task. Here, after processing every n packets, an enumeration is called.
[0148] In various implementations, active asynchronous enumeration provides at least two structures to address the problem of improving throughput and latency. Structure 1 includes converting synchronous computations into asynchronous computations. Here, the computations performed during the creation, update, and termination processes will be synchronous. Such computations should be very efficient and should be as small as possible to achieve better latency and throughput. One way to achieve this is to move / convert some existing synchronous computations to asynchronous, as described in detail above. In fact, asynchronous computations can be pre-pended or delayed to make synchronization smaller.
[0149] For illustration, taking memory block zeroing as an example, the calculation will make synchronous calculations smaller, reduce delays and improve throughput. In various implementations, when there are idle CPU cycles, the memory allocation system 106 performs asynchronous zeroing, otherwise during high loads, zeroing will occur synchronously in the current manner. By providing an example of a service due to a data structure change, the stream conversion from an older version to a newer version can be postponed or delayed until the memory allocation system 106 needs it. Currently, for existing malloc libraries, the entire service requires large synchronous calculations, and this causes interrupts / voltage drop cycles that interfere with the processing of new or existing streams.
[0150] In various implementations, Structure 2 includes distributing CPU compute instructions / memory equally among all DPDK cores. Here, distributing CPU and memory resources equally among all DPDK cores (e.g., 4, 8, etc. cores) will improve overall throughput (CPS) and latency, such as in the case of object expiration and live migration.
[0151] In some implementations, regarding the design details of the enumeration, the memory allocation system 106 utilizes an asynchronous paradigm (e.g., Pmem_Bulk) having two equal parts. For example, the asynchronous paradigm includes a top half with a callback in the application that provides business details and is registered before the enumeration begins. In particular, when the enumeration is completed in all CPU core partitions, the callback can occur in each node / block. In addition, the asynchronous paradigm can include a bottom half that includes traversing all nodes of the enumeration (e.g., Pmem_Bulk). Here, the memory allocation system 106 can operate in parallel for all partitions.
[0152] Additionally, with respect to the lifecycle of an asynchronous paradigm (e.g., Pmem_Bulk), the lifecycle can register the enumeration with flags and filters, receive callbacks for each chunk on multiple cores in parallel, and receive a single callback when the enumeration is complete.
[0153] As described above, the memory allocation system 106 may include various enumeration types for the asynchronous paradigm (e.g., Pmem_Bulk). For example, the enumeration types may include a high priority type that runs with packet processing by performing both packet processing and high priority enumeration operations (where the CPU core is shared between passive synchronous and asynchronous high priority in an m:n ratio), and a low priority type that runs only during CPU idle time. However, in some implementations, the memory allocation system 106 may provide the low priority type as a high priority type if necessary when memory is low.
[0154] In addition, the enumeration type may include an optional priority type. The memory allocation system 106 may run the optional priority type during CPU idle time. However, in various implementations, when memory is low, the memory allocation system 106 may disable the enumeration with the optional priority type.
[0155] To further illustrate, the following is an example algorithm for an enumeration that implements an asynchronous paradigm (e.g., Pmem_Bulk).
[0156] While While(1){ / *1. Synchronous 2. High 3. Low 4. Optional* /
[0157] / * In non-idle state, split CPU m:n between sync and enumeration high * /
[0158] Npkt = Poll_Recv_pkt (m); / * synchronization * /
[0159] Npkt = Npkt + pmem_run_enumerate_high (n) / * Run enumeration - high * /
[0160] If (Npkt==0) {
[0161] Npkt = Pmem_run_enumerate_low_optional() / * Run enumeration low and optional * / If (Npkt == 0) {
[0162] Lcore_idle(); / *Core sleep, interrupt switching* /
[0163] }}}
[0164] Pmem_Bulk: Enumeration implementation algorithm
[0165] Additionally, below is an example algorithm for enumeration (bulk operations) in an asynchronous paradigm (e.g., pmem_bulk).
[0166] Pmem_Bulk: Perform computation on cores that own memory with the help of metadata.
[0167] 1. Register the task: After starting the registration traversal, the task will register with Pmem_Bulk.
[0168] 2. Parallel and equal CPU contribution: All data threads traverse all partitions independently. When all traversals are completed from all partitions, the "complete_callback" will be called. After that, the task will be deleted.
[0169] New software structure 1: parallel : Data is evenly sharded across all partitions, and bulk operations will be more scalable if parallelized, so the computation is evenly distributed among the parallel cores.
[0170] New software structure 2: Equal CPU distribution : Due to sharding, batch computation data is sharded across all partitions, and if it is parallelized, batch operations will be more scalable.
[0171] 3. Delay: Low priority traversal during idle cycles, high priority traversal steals cycles from synchronization.
[0172] In Pmem_Bulk, tasks will be delayed as long as possible until memory resources are adequate for this idle cycle. This idle cycle will be maximized and unused memory will be converted into additional CPU power.
[0173] Structure 3: Delay: Delay memory calculation: use as many idle cycles as possible.
[0174] a) Additional memory: Waiting for longer idle periods,
[0175] b) Extra CPU: Uses idle cycles.
[0176] For example, object expiration, object deletion, port deletion, verification, cleaning up memory blocks, etc.
[0177] 4. Lock-free: It is lock-free as the traversal is done by the main CPU.
[0178] Structure 4: No lock : Since there is one data thread per core, no locks are required.
[0179] 5. Piggyback: All repeating tasks will use the same traversal as any single non-repeating task, and it can be piggybacked on top of any non-repeating task or dummy task (saving CPU computation).
[0180] Structure 5: Piggyback: Traverse multiple tasks at once by executing callbacks one after another on the same object. This greatly reduces the cost. (For example, object expiration, port deletion, object deletion, object validation, etc., all use the same traversal and they can start from any point in the traversal.)
[0181] 6. Non-repeating: A non-repeating task is a separate traversal for each. It strictly starts from the first block and completes when it reaches the end.
[0182] Pmem_Bulk: Batch operation algorithm
[0183] As mentioned above, existing malloc libraries do not support enumeration and must rely on the application to perform enumeration. The following table provides a comparison between enumeration in the application and enumeration using the PmemMalloc library.
[0184]
[0185]
[0186]
[0187] Table 1
[0188] As shown in Table 1, regarding the traversal of objects, in the application, the data structure used is not unified, and there are different methods for different enumerations. In PmemMalloc, it is unified and does two ways: a) using a linked list b) sequential scanning. Sequential scanning is used when multiple enumerations are piggybacked and the filters are mostly common.
[0189] Regarding the additional memory for each enumeration, the application requires additional data structures such as a hash table with a linked list to do the enumeration. This memory is proportional to the size of the enumeration. PmemMalloc does not require any additional memory. Instead, it requires a small amount of memory to maintain metadata for the enumeration. This small memory is created during enumeration registration.
[0190] Regarding the additional code for each enum, the application needs a separate function to iterate over the enum list. This is hardly reusable between different enums. In PmemMalloc, a very common / similar code is used for all enums.
[0191] Regarding the asynchronous framework for enumeration, in the application, enumeration performs traversal and functions on each node access, and the code is tightly coupled between traversal and execution nodes. In PmemMalloc, traversal and execution / consumption nodes can be completely decoupled. Therefore, the enumeration traversal is completed in a general way by the memory allocation system 106, and consumption is completed by callbacks in the application.
[0192] Regarding zero free, this is not available in applications. The memory allocation system 106 can use callbacks to the application to free blocks due to object expiration. Regarding memory leak detection, this is not available in applications. In PmemMalloc, this functionality is very similar to zero free. However, zero free is intentional, while memory leaks are unintentional.
[0193] Regarding CPU cache / memory cache, in the application, since the enumeration is global, the memory cache will have more noise. In PmemMalloc, the enumeration is local, so the CPU will access the memory it owns and will not cause noise to the neighboring CPUs. In addition, since it is a batch operation, it disturbs the CPU cache due to the large scan, but this happens during the non-packet processing time of all cores, and this is for non-high priority enumeration. In addition, assuming that such high priority enumeration is less, if all other cores are also busy with packet processing, the memory locality construction during the packet processing stage will be complete. During delayed enumeration, there will be a lot of disturbances in the memory in all cores.
[0194] About debuggability, in PmemMalloc, since enumeration metadata exists in shared memory, it can be easily monitored from the outside without connecting to the application. As another advantage, the memory allocation system 106 can monitor all enumerations, including determining at what stage the traversal is and how much work is left.
[0195] Further with respect to debuggability, in various implementations, the memory allocation system 106 may provide a standalone application (e.g., called PmemAnalyser) that reads the PmemMalloc shared memory in read-only mode. Then, using the metadata and the heap, the memory allocation system 106 may extract information from the outside. For example, the memory allocation system 106 may extract information about object trees (e.g., which object is linked to which other object in the system), memory leaks (e.g., based on timestamps and orphan objects, memory leaks will be detected (orphan objects are objects that are not linked to any other objects)), detailed CPU computation segmentation (e.g., how CPU cycles are spent on group processing, all types of enumeration and idle cycles, when a thread switches from one type of computation to another, it records timestamps in a partitioned area of shared memory), and heap-related metrics.
[0196]
[0197] Table 2
[0198] Regarding latency, additional details of the memory allocation system 106 regarding delayed enumeration operations are provided below. In various implementations, the degree of differentiation involves improving throughput by deferring low priority computations to the throughput of high priority computations. In some implementations, the computations include passive synchronization, high, low, and optional priority computations. As described above, the memory allocation system 106 can perform passive synchronization and high priority enumeration as quickly as possible. In addition, the memory allocation system 106 can delay low and optional enumerations as long as free memory and / or idle CPUs are available.
[0199] In various implementations, latency provides a jitter buffer mechanism. For example, latency can improve throughput and latency by consuming idle cycles and releasing memory (e.g., it gives its own cycles to synchronize / high computation). In addition, latency allows the memory allocation system 106 to determine the throughput of low priority and optional priority enumerations. In addition, latency provides a mechanism to consume as many idle cycles as possible until there is no free memory, and when the free memory exceeds the enumeration, the memory allocation system 106 can wait for idle cycles for a long time.
[0200] With respect to memory, latency requires less memory because processing can wait for CPU idle cycles for a short time, thus consuming a smaller amount of idle cycles. In fact, through latency, fewer enumerations are run at high priority compared to existing malloc libraries. In addition, with more free memory, the memory allocation system 106 can wait for idle CPU cycles for a long time, so it consumes a larger amount of idle CPU cycles. In addition, because the input packets and high priority are not uniform, latency allows the memory allocation system 106 to better manage and process bursts of input packet data faster. (For example, the latency acts as a jitter buffer).
[0201] In some implementations, the memory allocation system 106 controls latency by increasing / continuing low latency if available freed memory > 15%, decreasing low latency by moving from low to high if freed memory < 15%, stopping low latency by moving all low to high if freed memory < 5%, and / or disabling optional enumeration if freed memory < 30%.
[0202] In one or more implementations, the memory allocation system 106 measures the performance improvement of latency by identifying the percentage of CPU cycles consumed by low enumeration (e.g., when the CPU is in an idle state). For example, when 70 units are idle and 30 units are not idle, the memory allocation system 106 determines that latency contributes more than 70 CPU units to passive synchronization / high priority enumeration by deferring low priority and optional priority enumeration until the CPU is idle.
[0203] Now go to Figure 6 , which illustrates an example flow diagram including a series of actions 600 for facilitating packet data processing using a memory allocator library and / or managing memory according to one or more implementations. Figure 6 Actions according to one or more implementations are shown, but alternative implementations may omit, add, reorder, and / or modify Figure 6 Any action shown. Figure 6 The actions of may be performed as part of a method. Alternatively, the non-transitory computer readable medium may include a method that, when executed by one or more processors, causes a computing device to perform Figure 6 In a further implementation, the system may execute Figure 6 action.
[0204] For example, Figure 6A series of actions 600 are shown for servicing packet data using a new memory allocator library (i.e., PmemMalloc). The series of actions 600 include an action 610 of receiving packet data at an application. For example, action 610 may involve receiving packet data on a network device that includes an application that processes packet data. In some implementations, action 610 includes receiving packet data corresponding to the application from a network interface card before the stream is converted into a new object. In some implementations, action 610 includes receiving packet data corresponding to the application from a network interface card before the stream is converted into a new object.
[0205] As further shown, action series 600 includes action 620 of generating pre-allocated, partitioned and fixed shared memory blocks. For example, action 620 may involve generating a memory heap in response to receiving packet data and in pre-allocated, partitioned and fixed shared memory blocks. In some implementations, the memory heap of the shared memory block includes a memory that is continuous across restart or restart application. In various implementations, the memory heap of the shared memory block is pre-allocated without operating system integration. In one or more implementations, action 620 includes pre-allocated and partitioned shared memory blocks across a fixed number of data threads, a fixed number of CPUs and a fixed number of memories. In an example implementation, CPU calculations and shared memory blocks occur on a data plane development server device.
[0206] As further shown, a series of actions 600 includes an action 630 of attaching metadata to a shared memory block. For example, action 630 may include attaching allocation metadata to each block of the shared memory block. In various implementations, action 630 includes generating allocation metadata for each block of the shared memory block before attaching the allocation metadata to the corresponding block of the shared memory block. In some implementations, the allocation metadata for each block of the shared memory block includes an object type, a timestamp, and a port number.
[0207] As further shown, a series of actions 600 include an action 640 of performing memory operations across shared memory blocks based on metadata. For example, action 640 may involve performing one or more memory operations across shared memory blocks based on allocation metadata. In some implementations, one or more memory operations include packet processing operations or enumeration operations. In some implementations, action 640 includes performing one or more memory operations on a data thread on which a corresponding memory block is created (or owned).
[0208] In one or more implementations, action 640 includes performing one or more memory operations across the shared memory block based on priorities of enumerated operations including high priority bulk operations, low priority operations, and / or selectable priority operations. In some implementations, action 640 includes performing one or more memory operations across the shared memory block and further includes processing high priority bulk operations while delaying low priority bulk operations until the CPU idle time increases above a threshold level.
[0209] In various implementations, act 640 includes filtering memory blocks from the shared memory blocks based on object type, and performing an enumeration operation on the filtered memory blocks having the same object type. In some implementations, act 640 includes slicing the packet data across the shared memory blocks, performing one or more memory operations in parallel across the shared memory blocks, and performing both the packet processing operation and the enumeration operation across multiple CPU data threads using the shared memory blocks.
[0210] A "network" is defined as one or more data links capable of transmitting electronic data between computer systems and / or modules and / or other electronic devices. When information is transmitted or provided to a computer via a network or another communication connection (hardwired, wireless, or a combination of hardwired or wireless), the computer properly views the connection as a transmission medium. Transmission media may include networks and / or data links that may be used to carry the required program code means in the form of computer executable instructions or data structures and may be accessed by general or special purpose computers. Combinations of the above should also be included within the scope of computer readable media.
[0211] In addition, the network described herein may represent a network or a collection of networks (such as the Internet, an intranet, a virtual private network (VPN), a local area network (LAN), a wireless local area network (WLAN), a cellular network, a wide area network (WAN), a metropolitan area network (MAN), or a combination of two or more such networks) through which one or more computing devices can access the storage allocation system 106. In fact, the network described herein may include one or more networks that use one or more communication platforms or technologies to transmit data. For example, the network may include the Internet or other data links that enable the transmission of electronic data between respective client devices and components (e.g., server devices and / or virtual machines thereon) of a cloud computing system.
[0212] In addition, upon reaching various computer system components, program code means in the form of computer executable instructions or data structures may be automatically transferred from a transmission medium to a non-transitory computer readable storage medium (device) (or vice versa). For example, a computer executable instruction or data structure received via a network or data link may be cached in RAM within a network interface module (NIC) and then ultimately transferred to the computer system RAM and / or a less volatile computer storage medium (device) at the computer system. Therefore, it should be understood that a non-transitory computer readable storage medium (device) may be included in a computer system component that also (or even primarily) utilizes a transmission medium.
[0213] Computer executable instructions include, for example, when instructions and data are executed by a processor, making a general-purpose computer, a special-purpose computer, or a special-purpose processing device perform a specific function or function group. In some implementations, computer executable instructions are executed by a general-purpose computer to convert a general-purpose computer into a special-purpose computer that implements the elements of the present disclosure. Computer executable instructions can be, for example, binary code, intermediate format instructions such as assembly language, or even source code. Although the subject matter has been described in a language dedicated to structural features and / or method actions, it should be understood that the subject matter defined in the attached claims is not necessarily limited to the features or actions described above. On the contrary, the described features and actions are disclosed as an example form of implementing the claims.
[0214] Figure 7 Illustrated are certain components that may be included within computer system 700. Computer system 700 may be used to implement the various computing devices, components, and systems described herein.
[0215] In various implementations, computer system 700 may represent one or more client devices, server devices, or other computing devices described above. For example, computer system 700 may refer to various types of client devices that can access data on a cloud computing system. For example, a client device may refer to a mobile device, such as a mobile phone, a smart phone, a personal digital assistant (PDA), a tablet computer, a laptop computer, or a wearable computing device (e.g., a headset or a smart watch). A client device may also refer to a non-mobile device, such as a desktop computer, a server node (e.g., from another cloud computing system), or another non-portable device.
[0216] Computer system 700 includes processor 701. Processor 701 may be a general purpose single-chip or multi-chip microprocessor (e.g., Advanced RISC (Reduced Instruction Set Computer) Machine (ARM)), a dedicated microprocessor (e.g., a digital signal processor (DSP)), a microcontroller, a programmable gate array, etc. Processor 701 may be referred to as a central processing unit (CPU). Although processor 701 is shown as Figure 7 A single processor is used in the computer system 700, but in an alternative configuration, a combination of processors (e.g., an ARM and DSP) could be used.
[0217] The computer system 700 also includes a memory 703 in electronic communication with the processor 701. The memory 703 may be any electronic component capable of storing electronic information. For example, the memory 703 may be implemented as a random access memory (RAM), a read-only memory (ROM), a magnetic disk storage medium, an optical storage medium, a flash memory device in a RAM, an onboard memory included with the processor, an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) memory, a register, etc., including combinations thereof.
[0218] Instructions 705 and data 707 may be stored in the memory 703. The instructions 705 may be executed by the processor 701 to implement some or all of the functions disclosed herein. Executing the instructions 705 may include using the data 707 stored in the memory 703. Any of the various examples of modules and components described herein may be implemented in part or in whole as instructions 705 stored in the memory 703 and executed by the processor 701. Any of the various examples of data described herein may be data 707 stored in the memory 703 and used during the execution of the instructions 705 by the processor 701.
[0219] The computer system 700 may also include one or more communication interfaces 709 for communicating with other electronic devices. The one or more communication interfaces 709 may be based on wired communication technology, wireless communication technology, or both. Some examples of the one or more communication interfaces 709 include a universal serial bus (USB), an Ethernet adapter, a wireless adapter operating according to the Institute of Electrical and Electronics Engineers (IEEE) 702.11 wireless communication protocol, Wireless communication adapter and infrared (IR) communication port.
[0220] The computer system 700 may also include one or more input devices 711 and one or more output devices 713. Some examples of the one or more input devices 711 include a keyboard, a mouse, a microphone, a remote control device, a button, a joystick, a trackball, a touchpad, and a light pen. Some examples of the one or more output devices 713 include speakers and a printer. A specific type of output device that is often included in the computer system 700 is a display device 715. The display device 715 used with the implementations disclosed herein may utilize any suitable image projection technology, such as a liquid crystal display (LCD), a light emitting diode (LED), a gas plasma, electroluminescence, etc. A display controller 717 may also be provided for converting data 707 stored in the memory 703 into text, graphics, and / or moving images (as appropriate) displayed on the display device 715.
[0221] The various components of the computer system 700 may be coupled together via one or more buses, which may include a power bus, a control signal bus, a status signal bus, a data bus, etc. For clarity, Figure 7 The individual buses are shown as bus system 719.
[0222] Those skilled in the art will appreciate that the present invention can be practiced in a network computing environment with many types of computer system configurations, including personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, tablet computers, pagers, routers, switches, etc. The present disclosure can also be implemented in a distributed system environment, where local and remote computer systems linked by a network (by hardwired data links, wireless data links, or by a combination of hardwired and wireless data links) all perform tasks. In a distributed system environment, program modules can be located in local and remote memory storage devices.
[0223] The techniques described herein may be implemented in hardware, software, firmware, or any combination thereof, unless explicitly described as being implemented in a particular manner. Any features described as modules, components, etc. may also be implemented together in a logic device, or implemented separately as discrete but interoperable logic devices. If implemented in software, the techniques may be implemented at least in part by a non-transitory processor-readable storage medium containing instructions, which, when executed by at least one processor, perform one or more of the methods described herein. Instructions may be organized into routines, programs, objects, components, data structures, etc., which may perform specific tasks and / or implement specific data types, and which may be combined or distributed as needed in various implementations.
[0224] Computer-readable media can be any available media that can be accessed by a general or special purpose computer system. Computer-readable media that store computer-executable instructions are non-transitory computer-readable storage media (devices). Computer-readable media that carry computer-executable instructions are transmission media. Therefore, by way of example and not limitation, implementations of the present disclosure may include at least two distinct types of computer-readable media: non-transitory computer-readable storage media (devices) and transmission media.
[0225] As used herein, non-transitory computer-readable storage media (devices) may include RAM, ROM, EEPROM, CD-ROM, solid-state drives ("SSD") (e.g., RAM-based), flash memory, phase change memory ("PCM"), other types of memory, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other media that can be used to store desired program code means in the form of computer-executable instructions or data structures and that can be accessed by a general or special purpose computer.
[0226] The steps and / or actions of the methods described herein may be interchangeable with each other without departing from the scope of the claims. In other words, unless the proper operation of the described method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims.
[0227] The term "determining" encompasses a variety of actions, and thus, "determining" may include calculating, computing, processing, deriving, investigating, searching (e.g., searching in a table, a database, or another data structure), ascertaining, etc. Furthermore, "determining" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), etc. Furthermore, "determining" may include resolving, selecting, choosing, establishing, etc.
[0228] The terms "comprising," "including," and "having" are intended to be inclusive, and mean that there may be additional elements in addition to the listed elements. Furthermore, it should be understood that references to "an implementation" or "multiple implementations" of the present disclosure are not intended to be interpreted as excluding the existence of additional implementations that also include the described features. For example, any element or feature described with respect to an implementation herein may be combined with any element or feature of any other implementation described herein where compatible.
[0229] The present disclosure may be embodied in other specific forms without departing from its spirit or features. The described implementation is considered to be illustrative rather than restrictive. Therefore, the scope of the present disclosure is indicated by the appended claims rather than by the preceding description. Changes within the meaning and scope of the equivalents of the claims will be included within their scope.
Claims
1. A computer-implemented method for managing memory, comprising: receiving packet data via a network device, the network device including an application for processing the packet data; In response to receiving the packet data, generating a memory heap within a pre-allocated, partitioned and fixed shared memory block; appending allocation metadata to each block of the shared memory block; as well as One or more memory operations are performed across the shared memory block based on the allocation metadata.
2. The computer-implemented method of claim 1 , further comprising: The allocation metadata is generated for each block of the shared memory block before appending the allocation metadata to the corresponding block of the shared memory block. 3 . The computer-implemented method of claim 2 , wherein the allocation metadata for each block of the shared memory block comprises an object type, a timestamp, and a port number. 4 . The computer-implemented method of claim 3 , wherein the one or more memory operations comprise a packet processing operation or an enumeration operation.
5. The computer-implemented method of claim 4, wherein executing the one or more memory operations across the shared memory block is further based on a priority level of the enumerated operations, the enumerated operations comprising high priority bulk operations, low priority operations, and selectable priority operations.
6. The computer-implemented method of claim 5, wherein performing the one or more memory operations across the shared memory block further comprises: Process high priority batch operations while deferring lower priority batch operations until the CPU idle time increases above a threshold level.
7. The computer-implemented method of claim 4, wherein performing the one or more memory operations across the shared memory block comprises: filtering memory blocks from the shared memory blocks based on object type; as well as An enumeration operation is performed on the filtered memory blocks having the same object type.
8. The computer-implemented method of any one of claims 1-7, wherein performing the one or more memory operations across the shared memory block comprises: slicing the packet data across the shared memory blocks; executing the one or more memory operations in parallel across the shared memory block; as well as Both packet processing operations and enumeration operations are performed across multiple CPU data threads using the shared memory block.
9. The computer-implemented method of any of claims 1-8, wherein the memory heap of the shared memory block comprises memory that persists across restarts of the application.
10. The computer implemented method of claim 1, wherein the memory heap of the shared memory block is pre-allocated without operating system integration.
11. The computer-implemented method of claim 1 , further comprising: The packet data corresponding to the application is received from a network interface card before the stream is converted into a new object.
12. The computer-implemented method of claim 1 , wherein generating the memory heap of the shared memory block comprises: The shared memory blocks are pre-allocated and partitioned across a fixed number of data threads, a fixed number of CPUs, and a fixed amount of memory.
13. The computer implemented method of claim 1, wherein CPU computation and the shared memory block occur on a data plane development server device.
14. The computer-implemented method of claim 1 , wherein performing the one or more memory operations across the shared memory block comprises: The one or more memory operations are performed on a data thread that created a corresponding memory block.
15. A system comprising at least one processor and a storage medium storing instructions, wherein when the instructions are executed by the at least one processor, the system executes the method of any one of claims 1 to 14.
