Memory management method and apparatus

Abstract

The application discloses a memory management method and device, relates to the technical field of heterogeneous computing, and comprises the following steps: obtaining a memory allocation request of a device, and searching for a page-locked memory block corresponding to the memory allocation request to allocate the memory block to the corresponding device; receiving a request for releasing the memory block, identifying a device computing flow associated with the memory block, and recording a synchronization event for each computing flow; storing the recorded synchronization event and metadata of the memory block in a to-be-processed event queue; querying whether an event in the to-be-processed event queue is completed; and when it is queried that a specific event is completed, recycling the corresponding memory block to an idle memory pool, so that the technical problem that in the related art, high-cost memory management function calls are too frequent, performance is low, and it is difficult to cooperate with device asynchronous operation, and data competition and errors are easily caused is solved, and the technical effect that the efficiency of memory allocation and the overall throughput of a system are greatly improved under the premise of ensuring memory safety is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of heterogeneous computing, and particularly relates to a memory management method and device. BACKGROUND

[0002] In heterogeneous computing, frequent data transfer between the host and the device is one of the main performance bottlenecks, although the use of pinned memory (PM) can significantly optimize the data transfer effect, but the cost of its allocation and release operation is high.

[0003] At present, the related technology can perform memory allocation operation through a direct management mode and a simple cache mode. The direct management mode can allocate memory every time it is needed, and release it immediately after use. The simple cache mode can place the released memory block in the cache pool for subsequent allocation operation,

[0004] However, in the related technology, the high-cost memory management function is called too frequently, which makes the performance low and difficult to coordinate with the device asynchronous operation, and is easy to cause the memory block in the cache to still be used by the device, thereby causing data competition and errors, which needs to be solved urgently. SUMMARY

[0005] The present application provides a memory management method and device to at least solve the technical problem that the high-cost memory management function is called too frequently in the related technology, which makes the performance low and difficult to coordinate with the device asynchronous operation, and is easy to cause data competition and errors.

[0006] The present application provides a memory management method, comprising the following steps: obtaining a memory allocation request of a target device, and detecting whether a background thread of a target host is started to obtain a corresponding thread detection result, and determining a corresponding memory block searching strategy based on the thread detection result, and searching a target memory block corresponding to the memory allocation request according to the memory block searching strategy; allocating the target memory block to the target device, and obtaining a to-be-released memory pointer of the target device, and searching corresponding memory block metadata according to the to-be-released memory pointer, and detecting whether the memory block metadata exists associated device computing flow, wherein, in the case that it is detected that the associated device computing flow exists, a synchronization event corresponding to the associated device computing flow is inserted into a queue head position of a to-be-processed event queue; determining a total number of event-memory block pairs in the to-be-processed event queue, and taking out a corresponding current event-memory block pair from the end of the to-be-processed event queue, and determining an incoming parameter corresponding to the current event-memory block pair, so as to determine a corresponding memory block recycling mechanism according to the incoming parameter and the total number, and recycling the memory block corresponding to the current event-memory block pair based on the memory block recycling mechanism.

[0007] The application further provides a memory management device, comprising: a memory allocation module, configured to acquire a memory allocation request of a target device, detect whether a target host starts a background thread to obtain a thread detection result, determine a corresponding memory block searching strategy based on the thread detection result, and search for a target memory block corresponding to the memory allocation request according to the memory block searching strategy; a computing flow detection module, configured to allocate the target memory block to the target device, acquire a to-be-released memory pointer of the target device, search for memory block metadata corresponding to the to-be-released memory pointer, and detect whether the memory block metadata exists an associated device computing flow, wherein, in a case where it is detected that the associated device computing flow exists, a synchronization event corresponding to the associated device computing flow is inserted into a queue head position of a to-be-processed event queue; and a memory recycling module, configured to determine a total number of event-memory block pairs in the to-be-processed event queue, take out a corresponding current event-memory block pair from an end of the to-be-processed event queue, determine an incoming parameter corresponding to the current event-memory block pair, determine a memory block recycling mechanism according to the incoming parameter and the total number, and recycle a memory block corresponding to the current event-memory block pair based on the memory block recycling mechanism.

[0008] The application further provides an electronic device, comprising: a memory configured to store a computer program; and a processor configured to execute the computer program to implement the steps of any of the memory management methods.

[0009] The application further provides a non-volatile computer readable storage medium, wherein the non-volatile computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of any of the memory management methods.

[0010] The application further provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement the steps of any of the memory management methods.

[0011] By the present application, the memory allocation request of the target device can be obtained, and it is detected whether the target host starts a background thread to obtain a corresponding thread detection result. Based on the thread detection result, a corresponding memory block search strategy is determined, and the target memory block corresponding to the memory allocation request is searched according to the memory block search strategy. The target memory block is allocated to the target device, and the to-be-released memory pointer of the target device is obtained, and the corresponding memory block metadata is searched according to the to-be-released memory pointer, and it is detected whether the memory block metadata exists associated device computing flow. In the case where it is detected that there is an associated device computing flow, the synchronization event corresponding to the associated device computing flow is inserted into the queue head position of the to-be-processed event queue. The total number of event-memory block pairs in the to-be-processed event queue is determined, and the corresponding current event-memory block pair is taken out from the end of the to-be-processed event queue, and the incoming parameter corresponding to the current event-memory block pair is determined, so as to determine the corresponding memory block recycling mechanism according to the incoming parameter and the total number, and recycle the memory block corresponding to the current event-memory block pair based on the memory block recycling mechanism. Therefore, the technical problem that the high-cost memory management function call is too frequent in the related art, which causes low performance and is difficult to coordinate with device asynchronous operation, and easily causes data competition and error can be solved. The technical effect that the cache management of the page-locked memory is combined with the query of the device end event, so that the memory safety is guaranteed, and the efficiency of memory allocation and the overall throughput of the system are greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creating laborious work.

[0013] Figure 1 The flow chart of the memory management method provided by the embodiments of the present application;

[0014] Figure 2 The architecture schematic diagram of the page-locked memory management system provided by an embodiment of the present application;

[0015] Figure 3 The execution logic schematic diagram of the memory allocation provided by an embodiment of the present application;

[0016] Figure 4 The example diagram of the memory management device according to the embodiments of the present application.

[0017] Among them, 10-memory management device, 100-memory allocation module, 200-computing flow detection module, 300-memory recycling module. DETAILED DESCRIPTION

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0019] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0020] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] The specific application environment architecture or specific hardware architecture on which the execution of the memory management method depends is described here.

[0022] The embodiments of this application provide a memory management method.

[0023] like Figure 1 The diagram shown is a flowchart of a memory management method according to an embodiment of this application. The memory management method includes the following steps:

[0024] In step S101, the memory allocation request of the target device is obtained, and it is detected whether the target host has started a background thread to obtain the corresponding thread detection result. Based on the thread detection result, the corresponding memory block search strategy is determined, and the target memory block corresponding to the memory allocation request is searched according to the memory block search strategy.

[0025] Those skilled in the art should understand that heterogeneous computing refers to the process in which different types of processing units in a computing system work together to complete a computing task. This process typically involves a host, device, or accelerator.

[0026] The host can include a general processor responsible for logical control, task scheduling, etc., and the device or accelerator can include a special processor such as a GPU (Graphics Processing Unit), an NPU (Neural network Processing Unit), an FPGA (Field-Programmable Gate Array), etc., responsible for large-scale parallel computing. The page-locked host memory managed by the embodiments of the present application needs to perform efficient and secure data interaction with the GPU / NPU and other devices.

[0027] In heterogeneous computing, the system memory directly managed and accessed by the central processor is the host memory, which is different from the device memory in that the device memory (such as the GPU memory) is the memory on the accelerator and cannot be directly accessed by the CPU (Central Processing Unit). In the embodiments of the present application, data must be transmitted between the host memory and the device memory.

[0028] Therefore, in order to realize the cooperative management of the cache-type host memory and events for heterogeneous computing, the embodiments of the present application can first acquire the memory allocation requirements initiated by each device, detect whether the host has a background thread running, and generate a thread detection result; secondly, the embodiments of the present application can match a corresponding memory block searching strategy according to the thread detection result to search for a target memory block corresponding to the memory allocation requirement and allocate it to the corresponding device.

[0029] Thus, the embodiments of the present application can match the best memory block searching strategy through thread detection, thereby improving the memory block positioning accuracy and allocation efficiency, quickly meeting the memory requirements of different devices, and optimizing the running performance of the devices.

[0030] Optionally, in an embodiment of the present application, before acquiring the memory allocation request of the target device, the method further includes: constructing a page-locked memory allocator instance corresponding to the target device and initializing the page-locked memory allocator instance; and performing memory allocation and recycling operations on the page-locked memory using the initialized page-locked memory allocator instance, wherein the page-locked memory allocator instance includes a free list, a pending event queue, a memory block meta-information hash table, and a configuration file.

[0031] It should be noted that the architecture of the page-locked memory management system in the embodiments of the present application is as shown in Figure 2 .

[0032] The page-locked memory management system mainly includes a page-locked memory allocator, which sets up a page-locked memory allocator for each device. In the embodiment of the present application, it is assumed that there are N devices in total, and each device has a page-locked memory allocator, i.e., from page-locked memory allocator 0 to page-locked memory allocator N-1, each page-locked memory allocator is responsible for handling the page-locked memory application and release related to the device.

[0033] It can be understood that the page-locked memory is the default pageable memory of the operating system. The page-locked memory is a memory page locked in the physical RAM (Random Access Memory) by a specific API (Application Programming Interface) (such as cudaHostAlloc), so as to allow the device to exchange data with the host memory through direct memory access without the participation of the CPU for copying, the transmission rate is extremely high, and the physical address is fixed and will not be swapped out by the operating system during transmission.

[0034] Further, the embodiment of the present application can initialize the page-locked memory management system. Specifically, the embodiment of the present application can initialize N page-locked memory allocator instances (N is the number of devices), each instance includes a configuration file, a hash table for storing all allocated memory block meta-information (data_to_block_), a multi-level free list (free_list_), the free list at each level can be organized according to the memory block size (2 power), an event-memory block pair queue to be handled (events_to_handle_) (i.e., a pending event queue, which stores a double-ended queue (std::deque) of (event, memory block) pairs, all memory blocks released by Free but still have incomplete events will enter this queue and wait to be processed).

[0035] It should be noted that the multi-level free list in the embodiment of the present application, i.e., the free memory pool is a multi-level structure, each level corresponds to a specific size of memory block range, wherein the memory block size is aligned and classified according to the power of 2; for example, the embodiment of the present application can use a two-level size classification free list structure, and use a vector (std::vector <freelist>free_list_)manages free memory blocks, and each slot of the vector corresponds to a specific size range (e.g., 2 0 ,2 1 ,..., 2 63 bytes).

[0036] In addition, the embodiment of the present application can determine whether the background thread needs to be enabled to process the event according to the configuration file, and start the background thread in the initialization process when the background thread needs to be started.

[0037] Therefore, the embodiment of the present application can configure a dedicated allocator for each device, and can support the on-demand starting of the background thread, thereby improving the pertinence and flexibility of memory management, and optimizing the application and release processing efficiency of the page-locked memory. In addition, the embodiment of the present application can combine the cache management of the page-locked memory with the query of the device-side event through an event-driven memory recycling algorithm, thereby greatly improving the efficiency of memory allocation and the overall throughput of the page-locked memory management system under the premise of ensuring memory safety.

[0038] Optionally, in an embodiment of the present application, the memory allocation request of the target device is obtained, and it is detected whether the background thread of the target host is started to obtain a corresponding thread detection result, and based on the thread detection result, a memory block searching strategy is determined, and the target memory block corresponding to the memory allocation request is searched according to the memory block searching strategy, including: judging whether the background thread of the target host is started according to the thread detection result; if the background thread of the target host is not started, processing the events in the to-be-processed event queue, and after the processing of the events in the to-be-processed event queue is completed, performing a preset rounding-up operation on the request memory block size corresponding to the memory allocation request to obtain the corresponding target memory block; if the background thread of the target host is started, performing a preset rounding-up operation on the request memory block size to obtain the target memory block; searching for the target memory block in the free list, and judging whether the target memory block whose event processing is completed is found in the free list; when the target memory block is found in the free list, allocating the target memory block to the target device; when the target memory block is not found in the free list, re-judging whether the background thread of the target host is started, wherein, when the background thread of the target host is not started, performing a corresponding backup allocation operation on the target memory block to allocate the target memory block to the target device.

[0039] In actual execution, the embodiment of the application can configure a low-priority work thread (i.e., a background thread) independent of the main computing thread, continuously poll the pending event queue, and asynchronously complete event query and memory block recycling, so that the high-latency event query operation can be separated from the critical main thread, and the main thread can directly obtain a prepared memory block from the free list with a high probability when performing memory allocation, thereby significantly reducing memory allocation latency.

[0040] As an implementable manner, the embodiment of the application can determine whether to enable the background thread according to actual demand of the host.

[0041] When the background thread is not enabled, the main thread can synchronously process events in the pending event queue for all memory blocks.

[0042] When the background thread is enabled, a memory allocation request cannot immediately obtain a corresponding memory block from the free list, and the main thread synchronously processes events in the pending event queue according to the size of the required memory block. At this time, the embodiment of the application can traverse the pending event queue to find a memory block that matches the size of the target memory block and whose event has been completed, and immediately return after finding, thereby avoiding unnecessary query of all events and maintaining the efficiency of query in the synchronous mode.

[0043] In the specific execution process, the embodiment of the application can first receive a memory allocation request of a target device, the request size is size_request, and determine whether the host starts the background thread. If the background thread is not started, all events in the pending event queue are synchronously processed, the parameter size passed into the event processing function is -1 at this time, and size_request is rounded up to the nearest power of 2, i.e., roundup_size.

[0044] Secondly, the embodiment of the application can attempt to obtain a memory block of the request memory block size corresponding to the memory allocation request (i.e., a target memory block) from the free list, calculate the corresponding index according to roundup_size to lock the corresponding free list, and attempt to obtain a free memory block from the free list. If the corresponding free memory block is successfully obtained, it is marked as "allocated memory block" and immediately returned.

[0045] If the background thread is started, the embodiment of the application can attempt to process events in the pending event queue, the parameter size passed into the event processing function is equal to roundup_size at this time, and after the corresponding event processing is completed, the target memory block is again attempted to be found from the free list, and after the target memory block is found in the free list, the target memory block is allocated to the target device.

[0046] In addition, it should be noted that if the target memory block is not found in the free list, the embodiment of the present application needs to re-judge whether the host starts the background thread, wherein, when the host does not start the background thread, the target memory block is subjected to corresponding backup allocation operation, so as to allocate the target memory block to the target device.

[0047] It can be understood that the request memory block size corresponding to the memory allocation request of the embodiment of the present application is rounded up to the nearest power of 2, and the idle block in the corresponding free list slot is searched, so as to greatly reduce the overhead of searching for a suitable memory block (i.e. the target memory block), realize the allocation and release with approximate O(1) time complexity, and effectively reduce the memory fragmentation.

[0048] Therefore, the embodiment of the present application can adapt the memory block searching strategy according to the thread state, so as to first process the event when the background thread is not started, and directly search for the corresponding memory block when the background thread is started, and combine the backup allocation strategy, so as to effectively improve the pertinence and efficiency of memory allocation, so that the device memory demand can be quickly responded.

[0049] Optionally, in an embodiment of the present application, the corresponding backup allocation operation is performed on the target memory block to allocate the target memory block to the target device, comprising: calling a preset underlying system to allocate new page-locked memory, creating new memory block metadata corresponding to the new page-locked memory, and adding the new memory block metadata to a preset allocated block list, so as to allocate the target memory block to the target device through the allocated block list.

[0050] It should be noted that when the target host does not start the background thread, the embodiment of the present application performs backup allocation operation on the target memory block and allocates it to the target device as follows:

[0051] 1. When the regular allocation path cannot meet the demand, the system will trigger the preset backup mechanism, first call the special interface (such as cudaHostAlloc) in the underlying system for allocating new page-locked memory, and apply new page-locked memory resources for the target device;

[0052] 2. After successfully obtaining the memory, immediately create metadata containing memory address, size, state identifier and other key information for the new memory block, and these metadata will be structuredly stored to ensure traceability;

[0053] 3. The newly generated memory block metadata is included in the allocated block list according to the preset rule for unified management, and the list index mechanism is realized for fast retrieval and state updating, and finally the allocation of the target memory block to the target device is completed.

[0054] In addition, if the underlying system call fails, causing the new memory block allocation to fail, the system can trigger a multi-level error reporting mechanism. In addition to returning the error information immediately, the embodiment of the present application can also record the failure details to the system log, thereby providing a basis for subsequent troubleshooting. If it is a fatal error, the program safety exit process is started to avoid abnormal resource occupation.

[0055] Thus, the embodiment of the present application guarantees the reliability of memory allocation and improves the system stability in extreme scenarios through the structured fallback allocation process and error handling mechanism.

[0056] In step S102, the target memory block is allocated to the target device, and the to-be-released memory pointer of the target device is obtained. The corresponding memory block metadata is searched according to the to-be-released memory pointer, and it is detected whether the memory block metadata is associated with a device computing flow. If it is detected that the memory block metadata is associated with a device computing flow, the synchronization event corresponding to the associated device computing flow is inserted into the head position of the to-be-processed event queue.

[0057] Further, after the target memory block is allocated to the target device, the embodiment of the present application can obtain the to-be-released memory pointer of the device, and search the corresponding memory block metadata according to the to-be-released memory pointer. Since the memory block can be used on multiple flows, the embodiment of the present application can detect whether the memory block metadata is associated with a device computing flow. If there is an associated device computing flow, the synchronization event of the corresponding computing flow is inserted into the head position of the to-be-processed event queue.

[0058] It should be noted that the device computing flow represents a sequence of a series of ordered operations (such as kernel function execution and memory copy) on a device (such as a GPU). The operations in different flows can be executed asynchronously and concurrently.

[0059] Thus, the embodiment of the present application improves the cooperation efficiency of memory release and device computing flow by preferentially processing the synchronization event of the associated computing flow, and safely recovers the corresponding memory block after the operations on all related computing flows are completed, thereby optimizing the memory recycling speed.

[0060] Optionally, in one embodiment of the present application, a to-be-released memory pointer of a target device is acquired, and corresponding memory block metadata is searched according to the to-be-released memory pointer, and whether the memory block metadata has an associated device computing stream is detected, wherein in a case where it is detected that the associated device computing stream exists, a synchronization event corresponding to the associated device computing stream is inserted into a queue head position of a to-be-processed event queue, including: detecting whether the memory block metadata has an associated device computing stream; if the associated device computing stream exists, recording a synchronization event corresponding to the associated device computing stream, and data pairing the synchronization event and the memory block metadata to obtain a corresponding event-memory block pair, and inserting the event-memory block pair into the queue head position of the to-be-processed event queue, and performing a preset self-increment operation on an event quantity parameter in the memory block metadata; and if the associated device computing stream does not exist, determining a memory size of to-be-released memory corresponding to the to-be-released memory pointer, and inserting the to-be-released memory into a free list corresponding to the memory size.

[0061] In actual execution, when a memory block is released, the system does not immediately recycle it to the free cache pool, but first checks the device computing stream associated with it. Specifically, an embodiment of the present application can first receive a pointer data pointing to to-be-released memory, and search for corresponding memory block metadata block through the pointer data, and detect whether the block has an associated device computing stream (streams_to_handle_).

[0062] If the associated device computing stream does not exist (i.e. no associated stream), it indicates that the memory block of the to-be-released memory pointer corresponding to the to-be-released memory can be immediately reused, and an embodiment of the present application can determine the memory size of the to-be-released memory corresponding to the to-be-released memory pointer, and can directly insert the memory block of the to-be-released memory into the free list corresponding to the memory size.

[0063] If the associated device computing stream exists (i.e. associated stream exists), an embodiment of the present application can record a synchronization event (Event) for each associated stream (i.e. computing stream using the memory block), and place these events in the head position of the to-be-processed event queue (events_to_handle_) after pairing them with the block (i.e. after associating them with the memory block), and the events_num_ of the block is incremented by 1. It should be noted that at this time, the block has not entered the free list, and is in a "to-be-recycled" state.

[0064] It should be noted that the embodiments of the present application can periodically query these events (i.e. device computing flow) through a background thread or the main thread itself, and only when a certain event is queried as "completed" (i.e. the corresponding device computing is completed), the memory block associated with it is safely moved to the free list for subsequent allocation, thereby avoiding reallocation of the memory still being used by the device.

[0065] It should be noted that the synchronization event represents a point for marking and synchronizing the execution status of the device flow, and the occurrence time of an event in a certain flow can be recorded, and whether the event is "completed" (i.e. all operations prior to the event in the flow have been completed) is queried, thereby realizing the key mechanism for synchronizing the asynchronous operations between the host and the device.

[0066] Therefore, the embodiments of the present application can classify the memory release process according to whether the memory block metadata is associated with the device computing flow, so as to preferentially process the associated computing flow event, optimize the memory recycling efficiency and idle memory management, and effectively improve the resource reuse rate.

[0067] Optionally, in an embodiment of the present application, detecting whether the memory block metadata is associated with the device computing flow comprises: obtaining the identification information corresponding to the computing task of the target device, and recording the attribute information of the device computing flow to which the computing task belongs according to the identification information, the attribute information including the type, priority and data interaction range of the computing flow; establishing a mapping relationship between the memory block metadata and the attribute information, and storing the mapping relationship in a preset association information table, and when receiving the memory block release request, querying the mapping record corresponding to the to-be-released memory in the association information table; extracting the device computing flow attribute information associated with the to-be-released memory from the mapping record, determining the target device computing flow based on the device computing flow attribute information, and sending a state verification request to the target device computing flow to obtain the association degree information between the current processing data corresponding to the target device computing flow and the to-be-released memory according to the state verification request; determining whether the target device computing flow is still using the memory data of the to-be-released memory according to the association degree information; if the target device computing flow is still using the memory data of the to-be-released memory, it is determined that the memory block metadata is associated with the device computing flow; if the target device computing flow stops using the memory data of the to-be-released memory, it is determined that the memory block metadata is not associated with the device computing flow, and the association relationship corresponding to the target device computing flow is deleted from the mapping record.

[0068] As a way that can be implemented, the process of detecting whether the memory block metadata is associated with the device computing flow by the embodiments of the present application is as follows:

[0069] 1. Capture the unique identification of the target device computing task, synchronize the recording of the type of device computing flow (such as special-purpose / general-purpose), priority level, and device range of data interaction, and other attribute information to form a structured attribute file.

[0070] 2. Build a dynamic mapping relationship between the memory block metadata and the attribute file, and use a time sequence label to mark the association establishment time, store it in the association information table and establish a fast index; after receiving the memory block release request, the embodiment of the application can retrieve the association information table through the to-be-released memory address, and extract the attribute information of the corresponding device computing flow.

[0071] 3. Based on the attribute information, send a timestamped state verification request to the target device computing flow, obtain the real-time association degree of its current processing data and the to-be-released memory, including data dependency weight, access frequency decay coefficient, and other quantitative indicators; if the association degree is higher than the preset threshold, it is determined that the computing flow is still using the memory data, which is an effective association; otherwise, it is an invalid association, and the mapping relationship is immediately deleted from the association information table, and the association release log is recorded.

[0072] Thus, the embodiment of the application can quantize the association degree to accurately determine the association state between the computing flow and the memory, dynamically maintain the mapping relationship, thereby effectively improving the accuracy of the memory release opportunity and reducing resource occupation and conflicts.

[0073] In step S103, the total number of event-memory block pairs in the to-be-processed event queue is determined, and the corresponding current event-memory block pair is taken out from the end of the to-be-processed event queue, and the incoming parameter corresponding to the current event-memory block pair is determined, to determine the corresponding memory block recycling mechanism according to the incoming parameter and the total number, and recycle the memory block corresponding to the current event-memory block pair based on the memory block recycling mechanism.

[0074] After that, the embodiment of the application can determine the total number of event-memory block pairs in the to-be-processed event queue, and take out the current event-memory block pair from the end (i.e. the last position) of the to-be-processed event queue, and obtain its incoming parameter; secondly, the embodiment of the application can determine the corresponding memory block recycling mechanism in combination with the incoming parameter and the total number, to recycle the memory block corresponding to the current event-memory block pair.

[0075] In summary, the embodiment of the application first responds to the received request to release the page-locked memory block, identifies at least one device computing flow associated with the memory block, and records a synchronization event for each device computing flow in the at least one device computing flow; secondly, the embodiment of the application can associate the recorded synchronization event with the metadata of the memory block and store it in the to-be-processed event queue, and asynchronously or synchronously query whether the event in the to-be-processed event queue is completed; after the specific event is found to be completed, the embodiment of the application can recycle the memory block corresponding to the specific event to the idle memory pool for subsequent allocation request.

[0076] It should be noted that the asynchronous operation refers to that the thread initiating the operation can continue to execute the subsequent code without waiting for the operation to be completed, and the operation is executed in the background by other hardware (such as a GPU). In the embodiments of the present application, the execution of the device computing stream, the data transmission between the host and the device, and the event query of the background thread are all asynchronous operations.

[0077] Therefore, the embodiments of the present application can support two event processing modes (i.e., synchronous or asynchronous), and can be configured according to the application scenarios (such as delay-sensitive or throughput-sensitive), so as to orderly perform the memory block recycling operation, improve the memory resource recycling efficiency, and optimize the memory management efficiency.

[0078] Optionally, in an embodiment of the present application, the total number of event-memory block pairs in the to-be-processed event queue is determined, and the corresponding current event-memory block pair is taken out from the end of the to-be-processed event queue, and the incoming parameter corresponding to the current event-memory block pair is determined, so as to determine the corresponding memory block recycling mechanism according to the incoming parameter and the total number, and recycle the memory block corresponding to the current event-memory block pair based on the memory block recycling mechanism, including: judging whether the incoming parameter corresponding to the current event-memory block pair is a preset parameter; if the incoming parameter is the preset parameter, processing the event in the to-be-processed event queue, and querying whether the event corresponding to the current event-memory block pair is completed, if it is queried that the event corresponding to the current event-memory block pair is not completed, storing the current event-memory block pair at the end position of the to-be-processed event queue, otherwise, placing the event object of the current event-memory block pair in the preset event pool, and performing a preset self-decrement operation on the event number parameter in the memory block metadata of the current event-memory block pair; if the incoming parameter is not the preset parameter, determining the target memory block size corresponding to the incoming parameter, and processing the event corresponding to the memory block of the target memory block size in the to-be-processed event queue, and judging whether the number of processed events is greater than the total number; if the number of processed events is greater than the total number, placing the current event-memory block pair at the head position of the to-be-processed event queue; if the number of processed events is less than or equal to the total number, performing a preset self-increment operation on the number of processed events, and judging whether the size of the memory block corresponding to the current event-memory block pair is the target memory block size, wherein, when the size of the memory block corresponding to the current event-memory block pair is not equal to the target memory block size, a new event-memory block pair is extracted from the end of the to-be-processed event queue, and a preset incoming parameter analysis operation is re-executed on the new event-memory block pair to obtain the corresponding incoming parameter analysis result, and the corresponding event in the to-be-processed event queue is processed according to the incoming parameter analysis result.

[0079] Specifically, the process of the event processing and the memory block recycling in the embodiments of the present application is as follows:

[0080] Step 1, obtaining the total number max of event-memory block pairs in the current event queue to handle (events_to_handle_);

[0081] Step 2, taking out an event-memory block pair (i.e. the current event-memory block pair) pair from the tail (i.e. the end position) of events_to_handle_;

[0082] Step 3, the embodiment of the application can judge whether the incoming parameter size is -1 (i.e. the preset parameter), if size is -1, it means that the embodiment of the application needs to process all memory size memory block corresponding events, and execute step 5; otherwise, only process the size (i.e. the target memory block size corresponding to the incoming parameter) memory block corresponding events, execute step 4;

[0083] Step 4, judge whether the number of events processed is greater than max, if yes, put pair back to the head position (i.e. the front end position) of events_to_handle_, then end processing; otherwise, add 1 to the number of events processed (i.e. execute the self-add operation), judge whether the size of the block corresponding to pair is not equal to size (i.e. judge whether the size of the memory block corresponding to the current event-memory block pair is the target memory block size), if yes, the embodiment of the application can also put pair back to the head of events_to_handle_, and execute step 2;

[0084] Step 5, query event status: call the interface of device runtime to query whether the event (i.e. the event corresponding to the current event-memory block pair) has been completed;

[0085] Step 6, if the event is not completed: if size is -1, put pair back to the tail of events_to_handle_, and end processing; otherwise, put pair back to the head of events_to_handle_, then execute step 2;

[0086] Step 7, if the event is completed: return the event object to the event pool, and reduce events_num_ of the corresponding memory block block by 1 (i.e. execute the corresponding self-decrement operation).

[0087] Therefore, the embodiment of the application can flexibly filter the events to be processed through the total number of event-memory block pairs and the incoming parameter, accurately process the event status according to different scenes, and orderly return the uncompleted events, recycle the memory resources of the completed events and update the count, thereby improving the accuracy and efficiency of event processing and memory recycling.

[0088] Optionally, in an embodiment of the present application, after performing the preset decrement operation on the event number parameter in the memory block metadata of the current event-memory block pair, it further comprises: judging whether the event number parameter after decrement is zero; if the event number parameter after decrement is not zero, processing the event in the current event-memory block pair and re-performing the preset decrement operation on the event number parameter, and detecting whether the event number parameter after decrement is zero; if the event number parameter after decrement is zero, recycling the memory block corresponding to the current event-memory block pair and inserting the memory block corresponding to the current event-memory block pair into the free list.

[0089] It should be noted that after completing the event processing and performing the decrement operation on the event number parameter in the memory block metadata, the embodiment of the present application can judge whether the events_num of the block (i.e. the event number parameter in the memory block metadata of the current event-memory block pair) is zero in real time. If the parameter is not zero after decrement, it indicates that there are still associated events to be processed, and the embodiment of the present application can continue to perform iterative processing on the remaining events in the current event-memory block pair and synchronously perform the decrement operation on the event number parameter until the parameter is zero to stop the loop processing.

[0090] If the event number parameter is zero after decrement, it indicates that all events associated with the memory block have been completed, at which time the embodiment of the present application can start the memory block recycling process: the embodiment of the present application can first perform state verification on the memory block corresponding to the current event-memory block pair, remove it from the allocated list after confirming that there is no residual associated resource, then locate the corresponding free list according to the memory block size, and insert it into the free list according to the memory block size sorting rule to ensure that it can be quickly searched during subsequent allocation. At the same time, the embodiment of the present application can update the statistical information of the free list to provide data support for the memory allocation strategy.

[0091] Thus, the embodiment of the present application can achieve accurate recycling by dynamically tracking the number of events, process the unfinished events in stages, guarantee the safe reuse of memory blocks, and improve the recycling efficiency and management accuracy of memory resources.

[0092] Optionally, in an embodiment of the present application, after recycling the memory block corresponding to the current event-memory block pair based on the memory block recycling mechanism, it further comprises: determining the event type of the remaining events in the event queue to be processed, and determining the corresponding event processing mechanism according to the event type to process the corresponding remaining events based on the event processing mechanism, and recycling the memory block corresponding to the remaining events to the free list after the processing of the remaining events is completed; traversing the free list and calling the memory block in the free list to empty the memory block metadata hash table corresponding to the memory block.

[0093] As an implementable manner, after the memory recycling of the event-memory block pair is completed based on the memory block recycling mechanism, the cache emptying process can be started by the embodiment of the application, and the specific process is as follows:

[0094] 1. The remaining events in the current to-be-processed event queue are processed in batches, and the state of the memory block associated with each remaining event is checked one by one to determine whether it meets the recycling condition. For the memory block that meets the recycling condition, it is accurately classified and recycled to the idle list according to the corresponding memory block size, so as to ensure that all reusable memory resources are included in the unified management pool.

[0095] 2. After the remaining events are processed and all memory blocks are recycled to the idle list, the embodiment of the application can further perform a deep cleaning operation on the idle list to extract the memory blocks in the list one by one by traversing all idle lists of different size levels, and call the preset bottom-layer system special interface (such as cudaFreeHost) to perform complete release at the physical layer and return the memory resources to the host system.

[0096] 3. The hash table storing these memory block meta information is synchronized to empty, and the key-value pairs corresponding to the released memory blocks in the hash table are deleted to avoid the occupation of system resources by residual metadata, so as to ensure that the memory management data is completely synchronized with the actual memory state.

[0097] It should be noted that the embodiment of the application can process the remaining events in the to-be-processed event queue through the following four stages, which are as follows:

[0098] 1. Event analysis stage:

[0099] The events in the to-be-processed event queue are structurally analyzed, and an "event metadata frame" containing an event type label, an associated memory block unique identifier, and an operation parameter set is output. The frame needs to be embedded with a timestamp and a priority weight to provide a basis for subsequent processing sorting.

[0100] 2. Mechanism matching stage:

[0101] The preset processing mechanism is called based on the event type label, and the mechanism is built-in "component interaction topology graph" (explicit hash table index rule, idle list merging strategy, etc.), and an execution plan containing an operation instruction sequence and component access permission is output. The plan is associated with specific resources with the memory block identifier in the metadata frame as an anchor point.

[0102] 3. Execution linkage stage:

[0103] According to the execution plan driving the memory allocator instance, the meta information update (such as hash table key value pair refreshing), memory block operation (such as page table mapping adjustment) are completed, and a processing reply containing operation result code and resource state snapshot is output. The resource state in the reply is used as a check reference for the next stage of cleaning.

[0104] 4. Cleaning and transferring stage:

[0105] According to the processing reply, the event processing integrity is checked, and the incremental cleaning algorithm is used to release the queue node resources occupied by the event. The queue pointer update instruction is output, and the next event address pointed by the queue pointer is directly used as the analysis input source of the event analysis stage.

[0106] It can be understood that the embodiments of the present application can process the memory block pairing queue event through the above-mentioned event analysis stage, mechanism matching stage, execution linkage stage and cleaning and transferring stage, thereby improving the pertinence of event processing and effectively guaranteeing the order and accuracy of memory operation.

[0107] Therefore, the embodiments of the present application can complete the remaining event processing and memory release in stages, and combine with the underlying calling system to completely empty the resources, avoid memory and metadata residues, improve the cleanliness of memory management, and guarantee the efficient circulation of system resources.

[0108] In summary, the embodiments of the present application can avoid frequent calling of high-cost page locking memory allocation functions through memory caching, and guarantee the safety of cache reuse through event query, and greatly improve the efficiency of memory allocation through multi-level free list. In addition, the event object pool (that is, through the design pattern of pre-creating and reusing a group of objects, the used event objects can be returned to the pool for direct acquisition when needed, without the need for re-creation) and the background thread mechanism can balance the load of CPU and device, improve the overall resource utilization, and can also count the allocation of different size memory blocks, cache hit rate, host allocation time, etc., providing reliable data and technical support for performance analysis and optimization.

[0109] The memory allocation process of the present application is described below by combining the accompanying drawings.

[0110] Figure 3 The execution logic diagram for memory allocation is shown in FIG. 1. As shown in FIG. 1, the memory allocation process of the present application is as follows: Figure 3

[0111] S301: receiving a memory allocation request;

[0112] S302: determining whether the background thread is started, if not, going to S303, otherwise going to S304;

[0113] ​S303: Process all events in the to-be-processed event queue, and go to S304;

[0114] S304: Perform up-rounding operation on the request memory block size corresponding to the memory allocation request;

[0115] S305: Try to obtain the memory block corresponding to the memory allocation request from the free list;

[0116] S306: Determine whether the corresponding memory block is successfully obtained, if yes, go to S3012, otherwise go to S307;

[0117] S307: Re-determine whether the background thread is started, if yes, go to S308, otherwise go to S3010;

[0118] S308: Try to process the events in the to-be-processed event queue, and obtain the memory block corresponding to the memory allocation request again after the event processing;

[0119] S309: Re-determine whether the corresponding memory block is successfully obtained, if yes, go to S3012, otherwise go to S3010;

[0120] S3010: Call the underlying system to perform backup allocation operation;

[0121] S3011: Determine whether the corresponding memory block is successfully obtained after the backup allocation operation, if yes, go to S3012, otherwise go to S3013;

[0122] S3012: End the memory allocation operation;

[0123] S3013: Program error and exit the memory allocation process.

[0124] Further, the embodiments of the present application can further optimize the memory block allocation and recovery process from three dimensions of dynamic adjustment of thread strategy and parameter adaptation, introduction of intelligent prediction and pre-allocation mechanism, and fine memory block state management and recovery scheduling.

[0125] 1. The process of dynamic adjustment of thread strategy and parameter adaptation is as follows:

[0126] (1) Collect system real-time load data, output a load index set containing CPU utilization, memory usage, input / output waiting time, etc., and take the index set as the input of the next step;

[0127] (2) Divide the load level according to the load index set, output the load level result, and take the result as the input of the next step;

[0128] (3) Adjust the background thread priority according to the load level result, output high priority configuration for high load level and low priority configuration for low load level, and apply the priority configuration to the background thread, and the output thread running state is taken as the input of the next step;

[0129] (4) Collect historical memory allocation records, output the request frequency data of each memory block size, and take the data as the input of the next step;

[0130] (5) Set a processing threshold for the high-frequency memory block size based on the request frequency data, output the threshold parameter, and take the parameter as the input of the next step;

[0131] (6) Calculate the proportion of the high-frequency memory block size in the to-be-processed events, output the proportion result, and take the proportion result and the threshold parameter as the input of the next step;

[0132] (7) If the proportion result exceeds the threshold parameter, trigger the directional processing mode, and output the directional processing instruction, which is used to guide the event processing module to process only the events corresponding to the high-frequency memory block size.

[0133] Therefore, the embodiment of the application can collect load data in real time through a system monitoring tool, determine high load when the CPU utilization is greater than or equal to 70% and the memory usage is greater than or equal to 80%, adjust the background thread priority from "medium" to "high", determine low load when the CPU utilization is less than or equal to 30% and the memory usage is less than or equal to 50%, and adjust the priority to "low", analyze allocation records in the past 30 days, set the processing threshold of a memory block size to 30% if the request frequency proportion of the memory block size is more than 20%, and process only the events corresponding to the size when the proportion of the size in the to-be-processed events is more than 30%.

[0134] Thus, the embodiment of the application can adjust the thread priority as needed, avoid resource waste and event accumulation, reduce invalid traversal through directional processing, improve event processing efficiency, shorten the memory block recycling time, and optimize the response speed of the system.

[0135] 2. The memory allocation process based on the intelligent prediction and pre-allocation mechanism is as follows:

[0136] (1) Collect the time sequence data of historical memory allocation requests, including request time, memory block size, and device identifier, output the time sequence data set, and take the data set as the input of the next step;

[0137] (2) Input the time sequence data set into a preset prediction model, output the predicted request amount of each memory block size in a future preset period, and take the request amount as the input of the next step;

[0138] (3) Calculate the pre-allocation quantity according to the predicted request amount, output the pre-allocation instruction, and take the instruction as the input of the next step;

[0139] (4) calling the backup allocation interface according to the pre-allocation instruction, allocating a corresponding number of memory blocks, outputting a pre-allocation memory block set, and storing the set in a preset cache area;

[0140] (5) collecting historical reuse interval data of the released memory blocks, outputting an interval data set, and taking the data set as an input of a next step;

[0141] (6) judging whether the memory blocks meet short-term reuse conditions based on the interval data set, outputting a judgment result, and taking the result as an input of a next step;

[0142] (7) if the judgment result is that the memory blocks meet the conditions, storing the memory blocks in a high-speed cache area and outputting a cache instruction, the instruction being used to inform a memory management module to preferentially call the memory blocks from the high-speed cache area.

[0143] For example, the embodiment of the application can use an LSTM (Long-Short Term Memory, long-short term memory artificial neural network) model to process time series data of nearly 90 days, predict the request quantity of each memory block size in the next 7 days, determine the pre-allocation quantity according to 80% of the request quantity, such as predicting that the request quantity of a certain size is 100 times, and pre-allocating 80 blocks; recording the reuse interval of the released memory blocks, and if the reuse interval of nearly 5 times is all less than or equal to 24 hours, it is determined that the short-term reuse condition is met, and the memory blocks are stored in the high-speed cache area and preferentially called from the area during allocation.

[0144] Therefore, the embodiment of the application can pre-allocate high-frequency demand memory blocks, reduce expensive system calls at the bottom, and further shorten the reuse memory search path of the high-speed cache area of the embodiment of the application, improve the allocation efficiency, and reduce the system resource consumption.

[0145] 3. The process of fine memory block state management and recycling scheduling is as follows:

[0146] (1) collecting allocation records of the memory blocks, outputting allocation times data of each memory block, and taking the data as an input of a next step;

[0147] (2) setting a heat value for the memory blocks based on the allocation times data, increasing the heat value by 1 for each allocation, decreasing the heat value by 2 if no allocation is performed for a preset period, outputting a heat value parameter, and taking the parameter as an input of a next step;

[0148] (3) sorting the memory blocks in the idle list according to the heat value parameter, outputting a sorting instruction, and using the instruction to guide the idle list to arrange the memory blocks in descending order of the heat value;

[0149] (4) collecting priority configuration information of the device computing flow, outputting a priority data set, and taking the data set as an input of a next step;

[0150] (5) In the case of associating multiple computing flows with the memory block, the priority level is marked for each computing flow according to the priority data set, and a priority list is output, which is used as the input of the next step;

[0151] (6) The processing order of the event is determined according to the priority list, and a processing order instruction is output, which is used to guide the event processing module to process the associated events from high to low priority.

[0152] As an implementable manner, in the embodiments of the present application, the hotness value of the memory block is increased by 1 each time the memory block is allocated, and is decreased by 2 each time the memory block is not allocated for 7 days. The free list is sorted from high to low according to the hotness value, and the memory block with a hotness value greater than or equal to 10 is placed at the top. In addition, the device computing flow configuration can be read in the embodiments of the present application, the real-time computing flow priority is set to "high", the batch computing flow priority is set to "medium", and the background computing flow priority is set to "low". When the memory block is associated with multiple flows, the corresponding events are processed in the order of "high-medium-low", and the events_num_ of the memory block is updated after the events are completed.

[0153] Therefore, according to the hotness sorting strategy, the embodiments of the present application can effectively improve the search efficiency of high-frequency memory blocks, process events according to priority, speed up the memory block recycling efficiency, and ensure the stable operation of the system core function.

[0154] Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and a general hardware platform as required, of course, it can also be realized by hardware, but in many cases the former is a better implementation.

[0155] The embodiments of the present application also provide a memory management device.

[0156] As shown in Figure 4 , the memory management device 10 comprises a memory allocation module 100, a computing flow detection module 200, and a memory recycling module 300.

[0157] The memory allocation module 100 is configured to obtain a memory allocation request of a target device, detect whether a background thread of the target host is started to obtain a corresponding thread detection result, determine a corresponding memory block search strategy based on the thread detection result, and search for a target memory block corresponding to the memory allocation request according to the memory block search strategy.

[0158] The computing flow detection module 200 is configured to allocate the target memory block to the target device, obtain a to-be-released memory pointer of the target device, search for corresponding memory block metadata according to the to-be-released memory pointer, and detect whether there is an associated device computing flow in the memory block metadata. In the case where it is detected that there is an associated device computing flow, a synchronization event corresponding to the associated device computing flow is inserted into a queue head position of a to-be-processed event queue.

[0159] The memory recycling module 300 is configured to determine a total number of event-memory block pairs in the to-be-processed event queue, take a current event-memory block pair corresponding to an end of the to-be-processed event queue, determine an incoming parameter corresponding to the current event-memory block pair, determine a memory block recycling mechanism corresponding to the current event-memory block pair according to the incoming parameter and the total number, and recycle a memory block corresponding to the current event-memory block pair based on the memory block recycling mechanism.

[0160] Optionally, in an embodiment of the present application, the memory management apparatus 10 of the embodiment of the present application further comprises a construction module and an initialization module.

[0161] The construction module is configured to construct a page-locked memory allocator instance corresponding to the target device before obtaining a memory allocation request of the target device, and initialize the page-locked memory allocator instance.

[0162] The initialization module is configured to perform memory allocation and recycling operations on the page-locked memory by using the initialized page-locked memory allocator instance, wherein the page-locked memory allocator instance comprises a free list, a to-be-processed event queue, a memory block meta-information hash table, and a configuration file.

[0163] Optionally, in an embodiment of the present application, the memory management apparatus 10 of the embodiment of the present application further comprises an event processing module and a list traversal module.

[0164] The event processing module is configured to determine a type of a remaining event in the to-be-processed event queue after recycling the memory block corresponding to the current event-memory block pair based on the memory block recycling mechanism, determine an event processing mechanism corresponding to the remaining event according to the type of the remaining event, process the remaining event based on the event processing mechanism, and recycle a memory block corresponding to the remaining event to the free list after the processing of the remaining event is completed.

[0165] The list traversal module is configured to traverse the free list, and call a memory block in the free list to clear a memory block meta-information hash table corresponding to the memory block.

[0166] Optionally, in an embodiment of the present application, the memory allocation module 100 comprises a first judgment unit, a first rounding unit, a second rounding unit, a second judgment unit, a searching unit, and a backup allocation unit.

[0167] The first judgment unit is configured to determine whether the target host starts a background thread according to the thread detection result.

[0168] The first rounding unit is configured to process the event in the pending event queue if the target host does not start the background thread, and perform a preset upward rounding operation on the request memory block size corresponding to the memory allocation request to obtain the target memory block.

[0169] The second rounding unit is configured to perform a preset upward rounding operation on the request memory block size to obtain the target memory block if the target host has started the background thread.

[0170] The second judging unit is configured to search for the target memory block in the free list, and determine whether the target memory block in which the event has been completed is found in the free list.

[0171] The searching unit is configured to allocate the target memory block to the target device when the target memory block is found in the free list.

[0172] The backup allocation unit is configured to re-determine whether the target host starts the background thread when the target memory block is not found in the free list, wherein the target memory block is allocated to the target device by performing a corresponding backup allocation operation on the target memory block when the target host does not start the background thread.

[0173] Optionally, in an embodiment of the present application, the computing flow detection module 200 comprises a detection unit, an insertion unit and a determination unit.

[0174] The detection unit is configured to detect whether the memory block metadata has an associated device computing flow.

[0175] The insertion unit is configured to record a synchronization event corresponding to the associated device computing flow if the associated device computing flow exists, and perform data pairing between the synchronization event and the memory block metadata to obtain a corresponding event-memory block pair, insert the event-memory block pair into a head position of the pending event queue, and perform a preset self-increment operation on an event quantity parameter in the memory block metadata.

[0176] The determination unit is configured to determine a memory size of the to-be-released memory corresponding to the to-be-released memory pointer if the associated device computing flow does not exist, and insert the to-be-released memory into a free list corresponding to the memory size.

[0177] Optionally, in an embodiment of the present application, the memory recycling module 300 comprises a third judging unit, a self-decrement unit, a fourth judging unit, a head placement unit and a tail extraction unit.

[0178] The third judging unit is configured to determine whether a transmission parameter corresponding to the current event-memory block pair is a preset parameter.

[0179] The decrement unit is configured to, if the incoming parameter is the preset parameter, process an event in the to-be-processed event queue, and query whether the event corresponding to the current event-memory block pair is completed, and if the event corresponding to the current event-memory block pair is not completed, store the current event-memory block pair at an end position of the to-be-processed event queue, or if the event corresponding to the current event-memory block pair is completed, place an event object of the current event-memory block pair in a preset event pool, and perform a preset decrement operation on an event quantity parameter in metadata of the memory block of the current event-memory block pair.

[0180] The fourth determining unit is configured to, if the incoming parameter is not the preset parameter, determine a target memory block size corresponding to the incoming parameter, process an event corresponding to a memory block with the target memory block size in the to-be-processed event queue, and determine whether the processed event quantity is greater than the total quantity.

[0181] The head placing unit is configured to, if the processed event quantity is greater than the total quantity, place the current event-memory block pair at a head position of the to-be-processed event queue.

[0182] The end extracting unit is configured to, if the processed event quantity is less than or equal to the total quantity, perform a preset increment operation on the processed event quantity, and determine whether a size of the memory block corresponding to the current event-memory block pair is the target memory block size, wherein, if the size of the memory block corresponding to the current event-memory block pair is not equal to the target memory block size, extract a new event-memory block pair from an end position of the to-be-processed event queue, re-perform the preset incoming parameter analysis operation on the new event-memory block pair to obtain a corresponding incoming parameter analysis result, and process a corresponding event in the to-be-processed event queue according to the incoming parameter analysis result.

[0183] Optionally, in an embodiment of the present application, the memory management device 10 of the embodiment of the present application further comprises a zero judgment module, a first analysis module and a second analysis module.

[0184] The zero judgment module is configured to determine whether the event quantity parameter after the preset decrement operation is zero.

[0185] The first analysis module is configured to, if the event quantity parameter after the decrement operation is not zero, process the event in the current event-memory block pair, re-perform the preset decrement operation on the event quantity parameter, and detect whether the event quantity parameter after the decrement operation is zero.

[0186] The second analysis module is configured to, if the event quantity parameter after the decrement operation is zero, recycle the memory block corresponding to the current event-memory block pair, and insert the memory block corresponding to the current event-memory block pair into the free list.

[0187] Optionally, in an embodiment of the present application, the backup allocation unit comprises a creating subunit configured to invoke a preset underlying system to allocate new page-locked memory, create new memory block metadata corresponding to the new page-locked memory, and add the new memory block metadata to a preset allocated block list, so as to allocate the target memory block to the target device through the allocated block list.

[0188] Optionally, in an embodiment of the present application, the computing flow detection module 200 further comprises an obtaining unit, a establishing unit, a verifying unit, a fifth judging unit, a first determining unit, and a second determining unit.

[0189] The obtaining unit is configured to obtain identification information corresponding to a computing task of a target device, and record attribute information of a device computing flow to which the computing task belongs according to the identification information, the attribute information comprising a type, a priority, and a data interaction range of the device computing flow.

[0190] The establishing unit is configured to establish a mapping relationship between memory block metadata and attribute information, and store the mapping relationship in a preset association information table, and when a memory block release request is received, query a mapping record corresponding to the memory to be released in the association information table.

[0191] The verifying unit is configured to extract device computing flow attribute information associated with the memory to be released from the mapping record, determine a target device computing flow based on the device computing flow attribute information, and send a state verification request to the target device computing flow, so as to obtain association degree information between current processing data corresponding to the target device computing flow and the memory to be released according to the state verification request.

[0192] The fifth judging unit is configured to determine whether the target device computing flow is still using memory data of the memory to be released according to the association degree information.

[0193] The first determining unit is configured to determine that the memory block metadata exist an associated device computing flow if the target device computing flow is still using the memory data of the memory to be released.

[0194] The second determining unit is configured to determine that the memory block metadata do not exist an associated device computing flow if the target device computing flow stops using the memory data of the memory to be released, and delete an association relationship corresponding to the target device computing flow from the mapping record.

[0195] The features of the embodiments of the memory management device can be referred to the related descriptions of the embodiments of the memory management method, which will not be repeated here.

[0196] Embodiments of the present application also provide an electronic device comprising a memory and a processor, the memory storing a computer program, and the processor being configured to run the computer program to perform the steps in any of the above memory management method embodiments.

[0197] The embodiment of the present application further provides a non-volatile computer readable storage medium, which stores a computer program, wherein the computer program is arranged to execute the steps in any of the memory management method embodiments when running.

[0198] In an example embodiment, the non-volatile computer readable storage medium can include, but is not limited to, a U disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0199] The embodiment of the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps in any of the memory management method embodiments.

[0200] The embodiment of the present application further provides another computer program product, which comprises a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in any of the memory management method embodiments.

[0201] The skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0202] The above describes in detail the memory management method, device, equipment and medium provided by the present application. The principles and implementation modes of the present application are described by applying specific examples in this paper, and the above example description is only used to help understand the method and core idea of the present application. It should be pointed out that, for the ordinary skilled in the art, without departing from the principles of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.< / freelist>

Claims

1. A memory management method, characterized in that, Includes the following steps: The system obtains the memory allocation request of the target device and detects whether the target host has started a background thread to obtain the corresponding thread detection result. Based on the thread detection result, the system determines the corresponding memory block search strategy and searches for the target memory block corresponding to the memory allocation request according to the memory block search strategy. The target memory block is allocated to the target device, and the memory pointer to be released of the target device is obtained. The corresponding memory block metadata is found according to the memory pointer to be released, and the memory block metadata is checked to see if there is an associated device computing stream. If the associated device computing stream is detected, the synchronization event corresponding to the associated device computing stream is inserted into the head position of the queue of pending events. Determine the total number of event-memory block pairs in the event queue to be processed, and take out the corresponding current event-memory block pair from the end of the event queue to be processed. Determine the input parameters corresponding to the current event-memory block pair, so as to determine the corresponding memory block reclamation mechanism according to the input parameters and the total number, and reclaim the memory block corresponding to the current event-memory block pair based on the memory block reclamation mechanism. The steps include obtaining the memory pointer to be released from the target device, finding the corresponding memory block metadata based on the memory pointer, and detecting whether the memory block metadata has an associated device computing stream. If the associated device computing stream is detected, the synchronization event corresponding to the associated device computing stream is inserted at the head of the queue of events to be processed. Detect whether the metadata of the memory block has an associated device compute stream; If the associated device computing stream exists, the synchronization event corresponding to the associated device computing stream is recorded, and the synchronization event is paired with the memory block metadata to obtain the corresponding event-memory block pair. The event-memory block pair is inserted into the head position of the queue of events to be processed, and a preset increment operation is performed on the event count parameter in the memory block metadata. If the associated device computation stream does not exist, the memory size of the memory to be released corresponding to the memory pointer to be released is determined, and the memory to be released is inserted into the free list corresponding to the memory size.

2. The memory management method according to claim 1, characterized in that, Before obtaining the memory allocation request from the target device, the process also includes: Construct a page-locked memory allocator instance corresponding to the target device, and initialize the page-locked memory allocator instance; The initialized page-locked memory allocator instance is used to perform memory allocation and reclamation operations on page-locked memory. The page-locked memory allocator instance includes a free list, a pending event queue, a memory block metadata hash table, and a configuration file.

3. The memory management method according to claim 2, characterized in that, After reclaiming the memory block corresponding to the current event-memory block pair based on the memory block reclamation mechanism, the process further includes: The event types of the remaining events in the event queue to be processed are determined, and the corresponding event processing mechanism is determined according to the event types. The remaining events are processed based on the event processing mechanism, and the memory blocks corresponding to the remaining events are reclaimed to the free list after the remaining events are processed. The free list is traversed, and the memory blocks in the free list are called to clear the memory block metadata hash table corresponding to the memory blocks.

4. The memory management method according to claim 2, characterized in that, The process of obtaining the memory allocation request of the target device, detecting whether the target host has started a background thread to obtain the corresponding thread detection result, determining the corresponding memory block search strategy based on the thread detection result, and searching for the target memory block corresponding to the memory allocation request according to the memory block search strategy includes: Based on the thread detection results, determine whether the target host has started the background thread; If the target host has not started the background thread, the events in the pending event queue are processed, and after the events in the pending event queue are processed, a preset rounding up operation is performed on the size of the requested memory block corresponding to the memory allocation request to obtain the corresponding target memory block; If the target host has started the background thread, then the requested memory block size is rounded up to obtain the target memory block. The target memory block is searched in the free list, and it is determined whether a target memory block whose memory block event has been completed is found in the free list. When the target memory block is found in the free list, the target memory block is allocated to the target device; If the target memory block is not found in the free list, it is re-evaluated whether the target host has started the background thread. If the target host has not started the background thread, a corresponding backup allocation operation is performed on the target memory block to allocate the target memory block to the target device.

5. The memory management method according to claim 1, characterized in that, The steps include determining the total number of event-memory block pairs in the event queue to be processed, retrieving the corresponding current event-memory block pair from the end of the event queue, determining the input parameters corresponding to the current event-memory block pair, determining the corresponding memory block reclamation mechanism based on the input parameters and the total number, and reclamating the memory block corresponding to the current event-memory block pair based on the memory block reclamation mechanism. Determine whether the input parameter corresponding to the current event-memory block pair is a preset parameter; If the input parameter is a preset parameter, then the events in the event queue to be processed are processed, and the event corresponding to the current event-memory block pair is queried as to whether it is completed. If the event corresponding to the current event-memory block pair is not completed, then the current event-memory block pair is stored at the end of the event queue to be processed. Otherwise, the event object of the current event-memory block pair is placed in the preset event pool, and the event count parameter in the memory block metadata of the current event-memory block pair is decremented by a preset operation. If the input parameter is not the preset parameter, then the target memory block size corresponding to the input parameter is determined, and the events corresponding to the memory block of the target memory block size in the event queue to be processed are processed, and it is determined whether the number of processed events is greater than the total number. If the number of events processed is greater than the total number, then the current event-memory block pair is placed at the head of the queue of events to be processed; If the number of events processed is less than or equal to the total number, a preset increment operation is performed on the number of events processed, and it is determined whether the size of the memory block corresponding to the current event-memory block pair is the target memory block size. If the size of the memory block corresponding to the current event-memory block pair is not equal to the target memory block size, a new event-memory block pair is extracted from the end of the event queue to be processed, and a preset input parameter analysis operation is re-executed on the new event-memory block pair to obtain the corresponding input parameter analysis result. The corresponding event in the event queue to be processed is then processed according to the input parameter analysis result.

6. The memory management method according to claim 5, characterized in that, After performing a preset decrement operation on the event count parameter in the memory block metadata of the current event-memory block pair, the method further includes: Determine if the event count parameter after decrementing is zero; If the event count parameter after decrement is not zero, then the events in the current event-memory block pair are processed, and the preset decrement operation is performed again on the event count parameter, and it is checked whether the event count parameter after decrement is zero. If the event count parameter after decrementing is zero, then the memory block corresponding to the current event-memory block pair is reclaimed, and the memory block corresponding to the current event-memory block pair is inserted into the free list.

7. The memory management method according to claim 4, characterized in that, The step of performing a corresponding backup allocation operation on the target memory block to allocate the target memory block to the target device includes: The system calls a preset underlying system to allocate new page-locked memory, creates new memory block metadata corresponding to the new page-locked memory, and adds the new memory block metadata to a preset allocated block list, so as to allocate the target memory block to the target device through the allocated block list.

8. The memory management method according to claim 1, characterized in that, The detection of whether the memory block metadata exists as an associated device computation stream includes: Obtain the identification information corresponding to the computing task of the target device, and record the attribute information of the computing flow of the device to which the computing task belongs based on the identification information. The attribute information includes the type, priority, and data interaction range of the computing flow. Establish a mapping relationship between the memory block metadata and the attribute information, and store the mapping relationship in a preset association information table. When a memory block release request is received, query the association information table for the mapping record corresponding to the memory to be released. Extract device computing flow attribute information associated with the memory to be released from the mapping record, determine the target device computing flow based on the device computing flow attribute information, and send a status verification request to the target device computing flow to obtain the correlation information between the current processing data corresponding to the target device computing flow and the memory to be released according to the status verification request; Based on the correlation information, it is determined whether the target device's computing stream is still using the memory data in the memory to be released; If the target device computing stream is still using the memory data of the memory to be released, it is determined that the memory block metadata has an associated device computing stream; If the target device computing stream stops using the memory data of the memory to be released, it is determined that there is no associated device computing stream in the memory block metadata, and the association relationship corresponding to the target device computing stream is deleted from the mapping record.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for implementing the steps of the memory management method as described in any one of claims 1 to 8 when executing the computer program.

Images