An anti-explosion terminal artificial intelligence memory pool management method

Abstract

The application discloses an anti-explosion terminal artificial intelligence memory pool management method, belongs to the technical field of embedded terminal memory resource management, and aims at solving the problems of memory fragment accumulation, insufficient task resource isolation and difficult recycling of abnormal retained memory when anti-explosion terminal artificial intelligence reasoning tasks run concurrently. The application generates a memory demand list by constructing a weight memory pool, an input / output memory pool and an intermediate feature map memory pool, configures task priority, memory quota, independent virtual memory address space and address access permission boundary, pre-allocates memory blocks before task starting and generates a memory allocation record table, controls fragment arrangement during task running, and releases occupied memory blocks and recycles abnormal retained memory blocks after task ending. The application can reduce memory fragment accumulation, reduce the influence of cross-task access and abnormal occupation, and is suitable for fields such as anti-explosion terminal on-site inspection, equipment state identification and data collection.

Description

Technical Field

[0001] This invention belongs to the field of embedded terminal memory resource management technology, specifically relating to a method for managing and controlling an artificial intelligence memory pool in an explosion-proof terminal. Background Technology

[0002] With the increasing use of explosion-proof terminals in scenarios such as on-site inspection, equipment status identification, and data collection, the AI ​​inference tasks on the terminal side are gradually shifting from single-model operation to multi-task concurrent operation. These terminals are typically limited by size, power consumption, and heat dissipation conditions, resulting in relatively limited available memory resources. Furthermore, the AI ​​model generates different types of data during loading and inference, including model weights, input / output buffers, and intermediate feature maps from various network layers. These data vary in terms of duration of use, access frequency, and size, placing higher demands on terminal memory management.

[0003] Existing embedded terminals typically employ general dynamic memory allocation mechanisms. Some solutions also set up memory pools for AI inference processes, reuse memory space based on tensor lifecycles, or statically allocate some buffers before model execution. Other solutions restrict resource access ranges for different tasks through task priorities, resource quotas, or address space isolation. These techniques can, to some extent, reduce the overhead of repeated memory allocation or minimize resource interference between different tasks.

[0004] However, existing solutions are mostly designed for memory reuse of single AI models, isolation of general process resources, or allocation of ordinary memory pools. They fail to address the characteristics of concurrent operation of AI inference tasks and business tasks in explosion-proof terminals by incorporating model weights, input / output data, and intermediate feature maps into separate memory pools for unified management. If data from different lifecycles are still mixed in the same memory space for allocation and release, discrete free memory blocks can easily form after long-term operation, leading to insufficient contiguous memory. If only general task isolation is performed, it is difficult to establish a direct correlation between task priority, memory quota, and AI inference memory allocation process, and critical tasks may still be affected by the memory consumption of low-priority tasks.

[0005] Furthermore, when an AI inference task exits abnormally or its release process is incomplete, allocated memory blocks may remain occupied. Existing memory management systems, lacking memory allocation records corresponding to the task's lifecycle, often struggle to promptly locate abnormally lingering memory blocks belonging to completed tasks, and also find it difficult to create a closed loop between pre-allocation, runtime defragmentation, and post-task recycling. Therefore, a memory pool management method suitable for AI inference scenarios in explosion-proof terminals is needed to continuously manage memory resources during model loading, task execution, and task termination. Summary of the Invention

[0006] To address the problems of memory fragmentation accumulation, insufficient task resource isolation, and difficulty in timely reclamation of abnormally retained memory during concurrent execution of AI inference tasks in explosion-proof terminals in existing technologies, this invention proposes the following solutions: A method for managing an AI-powered memory pool in an explosion-proof terminal includes: S1. Construct a weighted memory pool, an input / output memory pool, and an intermediate feature map memory pool with independent address boundaries in the physical memory space of the artificial intelligence chip in the explosion-proof terminal; S2. Receive the target artificial intelligence model loading request, parse the target artificial intelligence model, and generate memory requirement lists for the weight memory pool, input / output memory pool, and intermediate feature map memory pool, respectively. S3. Determine the task priority of the target task, configure memory quota for the target task, and allocate an independent virtual memory address space and address access permission boundary for the target task; S4. Before the target task starts, pre-allocate memory blocks from the weight memory pool, input / output memory pool and intermediate feature map memory pool according to the memory requirement list, map the pre-allocated memory blocks to independent virtual memory address spaces, and generate a memory allocation record table. S5. During the execution of the target task, monitor the memory operation status of the weight memory pool, input / output memory pool and intermediate feature map memory pool, obtain the system load and the operation status of the emergency task, and control the start or pause of defragmentation based on the memory operation status, system load and the operation status of the emergency task. S6. After the target task is completed, release the memory blocks occupied by the target task according to the memory allocation record table, and reclaim the abnormally lingering memory blocks that are still occupied beyond the task's lifecycle.

[0007] Furthermore, the construction of the weighted memory pool, input / output memory pool, and intermediate feature map memory pool with mutually independent address boundaries as described in S1 includes: Based on the lifecycle, size characteristics, and access frequency of AI inference data, the memory space corresponding to the model weights is divided into a weight memory pool, the memory space corresponding to the input data and output results is divided into an input-output memory pool, and the memory space corresponding to the intermediate feature maps of each network layer is divided into an intermediate feature map memory pool.

[0008] Furthermore, the construction of the weighted memory pool, input / output memory pool, and intermediate feature map memory pool with mutually independent address boundaries, as described in S1, also includes: Configure multiple fixed-size memory blocks for the weight memory pool, input / output memory pool, and intermediate feature map memory pool respectively; When a memory request is received, the target memory pool is determined based on the data type corresponding to the memory request, and the smallest memory block that meets the capacity requirement is selected from the fixed-size memory blocks in the target memory pool.

[0009] Furthermore, the memory requirement list for generating the weight memory pool, input / output memory pool, and intermediate feature map memory pool, respectively, as described in S2, includes: Analyze the network structure, number of parameters, and input / output specifications of the target artificial intelligence model; The memory requirements of the model weights are determined based on the number of parameters, and the memory requirements of the model weights are associated with the weight memory pool. Determine the memory requirements of the input / output buffers based on the input / output specifications, and associate the memory requirements of the input / output buffers with the input / output memory pool; The memory requirements of intermediate feature maps in each network layer are determined based on the network structure, and the memory requirements of intermediate feature maps in each network layer are associated with the intermediate feature map memory pool.

[0010] Furthermore, S3, determining the task priority of the target task and configuring memory quotas for the target task, includes: The task priorities of the target tasks are divided into urgent, important, and general levels; Configure differentiated memory quota limits based on task priority; When memory resources are insufficient, priority should be given to fulfilling the memory requests of urgent tasks; When the memory request of an urgent task cannot be satisfied, pause the important or general task and release the corresponding memory resources.

[0011] Furthermore, S3's allocation of an independent virtual memory address space and address access permission boundaries for the target task includes: An independent virtual memory address space is allocated for the target task based on the hardware memory management unit; The allowed address range for the target task is determined based on the task identifier and memory quota of the target task. Configure the allowed access address range for the target task as the allowed access zone, and configure the memory address ranges corresponding to other tasks as the prohibited access zone; When the target task accesses a restricted area, the access is blocked.

[0012] Furthermore, the generation of the memory allocation record table described in S4 includes: Record the memory pool to which the pre-allocated memory block belongs, the address of the pre-allocated memory block, the status of the pre-allocated memory block, the target task information to which the pre-allocated memory block belongs, and the correspondence between the pre-allocated memory block and the independent virtual memory address space.

[0013] Furthermore, the step S5, which controls the start or pause of defragmentation based on memory operating status and task priority, includes: Periodically obtain the free block distribution, fragmentation rate, and system load of the weight memory pool, input / output memory pool, and intermediate feature map memory pool; When the fragmentation rate of any memory pool exceeds the preset fragmentation rate threshold, the system load is lower than the preset load threshold, and there are no urgent tasks running, a background defragmentation thread is started to merge the discrete free memory blocks in the memory pools whose fragmentation rate exceeds the preset fragmentation threshold and update the memory address mapping table. If an urgent task is detected during defragmentation, defragmentation will be paused.

[0014] Furthermore, the reclaiming of abnormally persistent memory blocks that remain occupied beyond the task's lifecycle, as described in S6, includes: Periodically compare task status and memory block usage status; When the task to which the memory block belongs has been completed but the memory block is still occupied, the memory block is identified as an abnormally inactive memory block. Change the abnormally lingering memory block to an idle state and update the memory management table.

[0015] Based on the same inventive concept, the present invention also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above methods.

[0016] Compared with the prior art, the present invention has the following beneficial effects: By constructing weight memory pools, input / output memory pools, and intermediate feature map memory pools with independent address boundaries in the physical memory space of the AI ​​chip in the explosion-proof terminal, and making the memory requirement list correspond to different memory pools respectively, it is possible to reduce the situation where model weights, input / output data, and intermediate feature maps are mixed in the same memory space for allocation and release, thereby reducing the accumulation of memory fragments caused by the cross-occupancy of data at different lifecycles.

[0017] By pre-allocating memory blocks according to the memory requirement list before the target task starts, mapping the pre-allocated memory blocks to an independent virtual memory address space, and generating a memory allocation record table, the number of times the target task dynamically requests memory during its operation can be reduced. This makes the memory allocation process correspond to the task lifecycle, thereby reducing the problems of memory request failure and difficulty in tracking memory resources during operation.

[0018] By determining the task priority of the target task, configuring memory quotas for the target task, and allocating an independent virtual memory address space and address access permission boundary for the target task, it is possible to limit the memory access range between different tasks and associate memory resource allocation with task priority, thereby reducing the impact of low-priority tasks occupying the memory resources of critical tasks and abnormal cross-task access.

[0019] By monitoring the memory operation status of each memory pool during the execution of the target task, and controlling the start or pause of defragmentation based on the memory operation status and task priority, resource conflicts between the defragmentation process and the execution of critical tasks can be avoided, thereby reducing the impact on the execution of critical tasks while defragmenting discrete free memory blocks.

[0020] By releasing the memory blocks occupied by the target task according to the memory allocation record table after the target task is completed, and reclaiming abnormally retained memory blocks that are still occupied beyond the task's lifecycle, the residual memory resources corresponding to the completed task can be located, thereby reducing the decrease in memory availability caused by the continuous occupation of abnormally retained memory.

[0021] This invention features memory pool classification and management, task resource isolation, and task lifecycle recycling capabilities. It enables continuous management of memory resources during the execution of AI inference tasks in explosion-proof terminals and is applicable to fields such as on-site inspection of explosion-proof terminals, equipment status identification, and data acquisition. Attached Figure Description

[0022] Figure 1 This is a flowchart of the explosion-proof terminal artificial intelligence memory pool management method described in the implementation method. Detailed Implementation

[0023] The technical solutions in the embodiments of the present invention will now be clearly and completely described in conjunction with the accompanying drawings.

[0024] Implementation Method 1 A method for managing an AI-powered memory pool in an explosion-proof terminal includes: S1. Construct a weighted memory pool, an input / output memory pool, and an intermediate feature map memory pool with independent address boundaries in the physical memory space of the artificial intelligence chip in the explosion-proof terminal; S2. Receive the target artificial intelligence model loading request, parse the target artificial intelligence model, and generate memory requirement lists for the weight memory pool, input / output memory pool, and intermediate feature map memory pool, respectively. S3. Determine the task priority of the target task, configure memory quota for the target task, and allocate an independent virtual memory address space and address access permission boundary for the target task; S4. Before the target task starts, pre-allocate memory blocks from the weight memory pool, input / output memory pool and intermediate feature map memory pool according to the memory requirement list, map the pre-allocated memory blocks to independent virtual memory address spaces, and generate a memory allocation record table. S5. During the execution of the target task, monitor the memory operation status of the weight memory pool, input / output memory pool and intermediate feature map memory pool, obtain the system load and the operation status of the emergency task, and control the start or pause of defragmentation based on the memory operation status, system load and the operation status of the emergency task. S6. After the target task is completed, release the memory blocks occupied by the target task according to the memory allocation record table, and reclaim the abnormally lingering memory blocks that are still occupied beyond the task's lifecycle.

[0025] By treating the weight memory pool, input / output memory pool, and intermediate feature map memory pool as continuous management objects for loading the target AI model, running the task, and recycling the memory at the end of the task, the memory usage generated at different stages can be recorded and processed along the same management chain, thereby improving the continuity of the memory resource management process.

[0026] Furthermore, the construction of the weighted memory pool, input / output memory pool, and intermediate feature map memory pool with mutually independent address boundaries as described in S1 includes: Based on the lifecycle, size characteristics, and access frequency of AI inference data, the memory space corresponding to the model weights is divided into a weight memory pool, the memory space corresponding to the input data and output results is divided into an input-output memory pool, and the memory space corresponding to the intermediate feature maps of each network layer is divided into an intermediate feature map memory pool.

[0027] By determining the ownership of memory pools according to the lifecycle, size characteristics, and access frequency of AI inference data, model weights, input data, output results, and intermediate feature maps of each network layer can enter the corresponding memory space, thus facilitating subsequent allocation and reclamation according to data type.

[0028] Furthermore, the construction of the weighted memory pool, input / output memory pool, and intermediate feature map memory pool with mutually independent address boundaries, as described in S1, also includes: Configure multiple fixed-size memory blocks for the weight memory pool, input / output memory pool, and intermediate feature map memory pool respectively; When a memory request is received, the target memory pool is determined based on the data type corresponding to the memory request, and the smallest memory block that meets the capacity requirement is selected from the fixed-size memory blocks in the target memory pool.

[0029] By configuring multiple fixed-size memory blocks within each memory pool and selecting the smallest size memory block according to data type and capacity requirements, memory requests can be matched within a limited range, thereby reducing unnecessary memory usage.

[0030] Furthermore, the memory requirement list for generating the weight memory pool, input / output memory pool, and intermediate feature map memory pool, respectively, as described in S2, includes: Analyze the network structure, number of parameters, and input / output specifications of the target artificial intelligence model; The memory requirements of the model weights are determined based on the number of parameters, and the memory requirements of the model weights are associated with the weight memory pool. Determine the memory requirements of the input / output buffers based on the input / output specifications, and associate the memory requirements of the input / output buffers with the input / output memory pool; The memory requirements of intermediate feature maps in each network layer are determined based on the network structure, and the memory requirements of intermediate feature maps in each network layer are associated with the intermediate feature map memory pool.

[0031] By converting the network structure, parameter count, and input / output specifications of the target AI model into a list of memory requirements corresponding to three types of memory pools, memory allocation before task startup has a clear basis, thereby reducing the uncertainty caused by temporary calculations and temporary requests during the runtime phase.

[0032] Furthermore, S3, determining the task priority of the target task and configuring memory quotas for the target task, includes: The task priorities of the target tasks are divided into urgent, important, and general levels; Configure differentiated memory quota limits based on task priority; When memory resources are insufficient, priority should be given to fulfilling the memory requests of urgent tasks; When the memory request of an urgent task cannot be satisfied, pause the important or general task and release the corresponding memory resources.

[0033] By configuring differentiated memory quota limits according to task priority and prioritizing memory requests from urgent tasks when memory resources are insufficient, memory resources can be scheduled according to task level, thereby reducing resource conflicts during task concurrency.

[0034] Furthermore, S3's allocation of an independent virtual memory address space and address access permission boundaries for the target task includes: An independent virtual memory address space is allocated for the target task based on the hardware memory management unit; The allowed address range for the target task is determined based on the task identifier and memory quota of the target task. Configure the allowed access address range for the target task as the allowed access zone, and configure the memory address ranges corresponding to other tasks as the prohibited access zone; When the target task accesses a restricted area, the access is blocked.

[0035] By dividing the virtual memory address space into independent virtual memory spaces through a hardware memory management unit and configuring memory blocks corresponding to different tasks as allowed or prohibited access areas, the access range of the target task is constrained by address permissions, thereby reducing cross-task memory access.

[0036] Furthermore, the generation of the memory allocation record table described in S4 includes: Record the memory pool to which the pre-allocated memory block belongs, the address of the pre-allocated memory block, the status of the pre-allocated memory block, the target task information to which the pre-allocated memory block belongs, and the correspondence between the pre-allocated memory block and the independent virtual memory address space.

[0037] By recording the memory pool, address, status, task information, and virtual address space correspondence of the pre-allocated memory blocks in the memory allocation record table, the pre-allocation results can be jointly invoked by subsequent task execution monitoring and task end-of-task reclamation, thereby maintaining consistency in the allocation and reclamation basis.

[0038] Furthermore, the step S5, which controls the start or pause of defragmentation based on memory operating status and task priority, includes: Periodically obtain the free block distribution, fragmentation rate, and system load of the weight memory pool, input / output memory pool, and intermediate feature map memory pool; When the fragmentation rate of any memory pool exceeds the preset fragmentation rate threshold, the system load is lower than the preset load threshold, and there are no urgent tasks running, a background defragmentation thread is started to merge the discrete free memory blocks in the memory pools whose fragmentation rate exceeds the preset fragmentation threshold and update the memory address mapping table. If an urgent task is detected during defragmentation, defragmentation will be paused.

[0039] By using fragmentation rate, system load, and the running status of urgent tasks as control conditions for defragmentation, the background defragmentation thread can avoid the running period of urgent tasks, thus balancing the merging of discrete free memory blocks and the continuity of task execution.

[0040] Furthermore, the reclaiming of abnormally persistent memory blocks that remain occupied beyond the task's lifecycle, as described in S6, includes: Periodically compare task status and memory block usage status; When the task to which the memory block belongs has been completed but the memory block is still occupied, the memory block is identified as an abnormally inactive memory block. Change the abnormally lingering memory block to an idle state and update the memory management table.

[0041] Preferably, after modifying the abnormally lingering memory block to an idle state, the corresponding exception log is recorded.

[0042] By comparing the task status and memory block occupancy status, memory blocks whose tasks have ended but are still occupied are identified as abnormally lingering memory blocks and changed to an idle status. This allows abnormally occupied memory blocks to re-enter an allocatable state, thereby improving the consistency between the memory management table and the actual task status.

[0043] The method described in this embodiment can be executed by a processor calling a computer program, which can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the above-described method for managing the artificial intelligence memory pool of explosion-proof terminals can be implemented.

[0044] Implementation Method 2 This embodiment uses the explosion-proof terminal artificial intelligence memory pool management method as a specific example, combined with... Figure 1 The flowchart shown illustrates the process of memory pool construction, task resource isolation, pre-allocation loading, defragmentation, and full lifecycle recycling during the concurrent execution of AI inference tasks and business tasks in explosion-proof terminals.

[0045] like Figure 1 As shown, this implementation constructs a closed-loop memory management system consisting of "layered pooling control - address space isolation - pre-allocation loading - automated defragmentation - full lifecycle recycling". In the physical memory space of the AI ​​chip in the explosion-proof terminal, based on the lifecycle, size characteristics, and access frequency of various types of data during AI inference, three memory pools with independent address boundaries are defined: a weight memory pool, an input / output memory pool, and an intermediate feature map memory pool. The weight memory pool stores data corresponding to model weights, the input / output memory pool stores data corresponding to input data and output results, and the intermediate feature map memory pool stores data corresponding to intermediate feature maps of each network layer. These three memory pools are isolated and managed independently, ensuring that the memory space for different types of data is controlled separately.

[0046] Each memory pool employs a fixed-size memory block management model, with multiple standardized memory block specifications set according to the typical size of the corresponding data. Upon receiving a memory request, the target memory pool is determined based on the data type corresponding to the memory request, and the smallest memory block that meets the capacity requirement is selected from the fixed-size memory blocks of the target memory pool.

[0047] The memory fragmentation rate is defined by the following formula:

[0048] In the formula, For memory fragmentation rate, This represents the size of the largest contiguous free memory block in the current memory pool. This represents the total free memory size of the memory pool. A higher fragmentation rate indicates that the free memory blocks in the current memory pool are more scattered, and there is less free space available for contiguous allocation.

[0049] Memory block adaptation selection follows the minimum fit rule, for blocks of size... The memory request is selected from the corresponding memory pool specification set. Select the smallest memory block that meets the capacity requirement, i.e.:

[0050] In the formula, For the selected memory block size, For specifications Any memory block specification in the, This rule specifies the required memory capacity. It ensures that memory requests are matched to the required capacity within the corresponding memory pool.

[0051] During the stage of loading the target AI model into the explosion-proof terminal, the system receives the target AI model loading request, parses the network structure, parameter quantity and input / output specifications of the target AI model, pre-calculates the memory capacity required for the entire process of a single inference of the target AI model, including the memory usage of model weights, input / output buffers and intermediate feature maps of each network layer, and generates memory requirement lists for the weight memory pool, input / output memory pool and intermediate feature map memory pool respectively.

[0052] The formula for calculating the total memory requirement for single-model inference is:

[0053] In the formula, This represents the total memory requirement for single-model inference. The memory usage of the model weight parameters. The buffer capacity for input data and output results. This represents the total memory usage of intermediate feature maps across all network layers during the inference process. Based on the aforementioned memory requirement list, the memory requirements of the model weights are associated with the weight memory pool, the memory requirements of the input / output buffers are associated with the input / output memory pool, and the memory requirements of the intermediate feature maps of each network layer are associated with the intermediate feature map memory pool.

[0054] Before a target task is initiated, its priority is determined based on its task type. Memory quotas are configured for the target task, and an independent virtual memory address space and address access permission boundaries are allocated for it based on the hardware memory management unit. The target task's priority is divided into three levels: urgent, important, and general. Urgent tasks include oil and gas leak detection and fire alarm tasks; important tasks include equipment fault diagnosis and weld inspection tasks; and general tasks include data uploading and report generation tasks.

[0055] The formula for calculating the maximum allocatable memory capacity of a priority task at level k is:

[0056] In the formula, This represents the maximum allocatable memory capacity for a priority task of level k, where k is the task priority number. This corresponds to the total capacity of the tiered memory pool. This refers to the priority quota coefficient. Specifically, the quota coefficient for urgent tasks is 1.0, for important tasks it is 0.5, and for general tasks it is 0.2. When memory resources are insufficient, memory requests from urgent tasks are prioritized; if the memory requests of urgent tasks cannot be fulfilled, important or general tasks are paused and their corresponding memory resources are released.

[0057] During address access control, the pre-allocated memory block corresponding to the target task is configured as an accessible region, while the memory blocks corresponding to other tasks are configured as prohibited regions. When the target task accesses a prohibited region, the hardware memory management unit intercepts the access, ensuring that each task can only access memory resources within its own address range.

[0058] Before the target task officially starts, all required memory blocks are pre-allocated from the weighted memory pool, input / output memory pool, and intermediate feature map memory pool according to the memory requirement list, and their states are locked. The pre-allocated memory blocks are mapped to independent virtual memory address spaces, and a memory allocation record table is generated. This record table records the memory pool to which the pre-allocated memory block belongs, the address of the pre-allocated memory block, the status of the pre-allocated memory block, the target task information to which the pre-allocated memory block belongs, and the correspondence between the pre-allocated memory block and the independent virtual memory address space. During the execution of the target task, the pre-allocated memory blocks are accessed within the independent virtual memory address space; dynamic memory allocation and deallocation operations are no longer performed.

[0059] During the execution of the target task, a memory operation status monitoring process is established. This process periodically scans the free block distribution, memory occupancy rate, and fragmentation rate parameters of the weighted memory pool, input / output memory pool, and intermediate feature map memory pool, and also acquires system load and the running status of urgent tasks. When the fragmentation rate of any memory pool exceeds a preset fragmentation rate threshold, the system load is below a preset load threshold, and no urgent tasks are running, a background defragmentation thread is initiated. This thread merges the discrete free memory blocks in memory pools with fragmentation rates exceeding the preset threshold and updates the memory address mapping table to ensure consistency between the defragmented free memory block status and address mapping.

[0060] Defragmentation is performed in a low-priority background thread. If an urgent task is detected during defragmentation, defragmentation is paused and the corresponding runtime resources are released, allowing the urgent task to execute first. Once the urgent task's running status is cleared, defragmentation can be resumed based on the memory status and task priority.

[0061] After the target task is completed, all memory blocks occupied by the target task are located according to the memory allocation record table. The located memory blocks are released in batches and returned to the free block queue of the corresponding weight memory pool, input / output memory pool, or intermediate feature map memory pool. During release, the status of the memory block corresponding to the target task in the memory allocation record table is changed from occupied to free, and the memory management table is updated synchronously so that the released memory block can participate in the memory allocation of subsequent tasks.

[0062] For free memory blocks in the memory pool, a Least Recently Used (LRU) strategy is used for management. The idleness of memory blocks is evaluated by an idle weight, and the formula for calculating the idle weight is as follows:

[0063] In the formula, For idle weight, The duration of continuous idle time for a memory block. This is the memory reclamation detection cycle. The higher the idle weight, the longer the memory block has been unused. Small blocks of free memory with an idle weight exceeding a set threshold are preferentially merged into contiguous large blocks, allowing free memory blocks to participate in reorganization and reuse according to their idle level.

[0064] Simultaneously, task status and memory block occupancy status are periodically compared. When the task to which a memory block belongs has ended but the memory block is still in use, it is identified as an abnormally suspended memory block. After identifying an abnormally suspended memory block, its status is changed to idle, and the memory management table is updated. Furthermore, after changing the abnormally suspended memory block to idle, the corresponding exception log is recorded.

[0065] In this implementation, the weighted memory pool, input / output memory pool, and intermediate feature map memory pool are used to hold different types of AI inference data; the memory requirement list is used to determine the pre-allocation requirements of each memory pool before the target task starts; task priority and memory quota are used to determine the memory resource allocation order and allocation limit of the target task; the independent virtual memory address space and address access permission boundary are used to restrict the memory access range of the target task; the memory allocation record table is used to record the correspondence between pre-allocated memory blocks, target tasks, and independent virtual memory address spaces; memory running status and task priority are used to control the start or pause of defragmentation; task status and memory block occupancy status are used to identify abnormally lingering memory blocks and perform reclamation. The above processes are executed sequentially along the loading, startup, running, and termination of the target task, forming a continuous memory pool management process.

[0066] The method described in this embodiment can be executed by a processor calling a computer program, which can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the above-described method for managing the artificial intelligence memory pool of explosion-proof terminals can be implemented.

[0067] The embodiments of the present invention have been described above. These embodiments are used to help understand the technical solutions of the present invention and should not be construed as limiting the scope of the technical solutions of the present invention. Equivalent substitutions or conventional modifications made by those skilled in the art to the embodiments without departing from the technical concept of the present invention should all fall within the scope of the technical solutions of the present invention.

Claims

1. A method for managing an artificial intelligence memory pool in an explosion-proof terminal, characterized in that, include: S1. Construct a weighted memory pool, an input / output memory pool, and an intermediate feature map memory pool with independent address boundaries in the physical memory space of the artificial intelligence chip in the explosion-proof terminal; S2. Receive the target artificial intelligence model loading request, parse the target artificial intelligence model, and generate memory requirement lists for the weight memory pool, input / output memory pool, and intermediate feature map memory pool, respectively. S3. Determine the task priority of the target task, configure memory quota for the target task, and allocate an independent virtual memory address space and address access permission boundary for the target task; S4. Before the target task starts, pre-allocate memory blocks from the weight memory pool, input / output memory pool and intermediate feature map memory pool according to the memory requirement list, map the pre-allocated memory blocks to independent virtual memory address spaces, and generate a memory allocation record table. S5. During the execution of the target task, monitor the memory operation status of the weight memory pool, input / output memory pool and intermediate feature map memory pool, obtain the system load and the operation status of the emergency task, and control the start or pause of defragmentation based on the memory operation status, system load and the operation status of the emergency task. S6. After the target task is completed, release the memory blocks occupied by the target task according to the memory allocation record table, and reclaim the abnormally lingering memory blocks that are still occupied beyond the task's lifecycle.

2. The method according to claim 1, characterized in that, S1 describes the construction of a weighted memory pool, an input / output memory pool, and an intermediate feature map memory pool with mutually independent address boundaries, including: Based on the lifecycle, size characteristics, and access frequency of AI inference data, the memory space corresponding to the model weights is divided into a weight memory pool, the memory space corresponding to the input data and output results is divided into an input-output memory pool, and the memory space corresponding to the intermediate feature maps of each network layer is divided into an intermediate feature map memory pool.

3. The method according to claim 2, characterized in that, S1 further includes constructing a weighted memory pool, an input-output memory pool, and an intermediate feature map memory pool with mutually independent address boundaries: Configure multiple fixed-size memory blocks for the weight memory pool, input / output memory pool, and intermediate feature map memory pool respectively; When a memory request is received, the target memory pool is determined based on the data type corresponding to the memory request, and the smallest memory block that meets the capacity requirement is selected from the fixed-size memory blocks in the target memory pool.

4. The method according to claim 1, characterized in that, The memory requirement list for generating the weight memory pool, input / output memory pool, and intermediate feature map memory pool, as described in S2, includes: Analyze the network structure, number of parameters, and input / output specifications of the target artificial intelligence model; The memory requirements of the model weights are determined based on the number of parameters, and the memory requirements of the model weights are associated with the weight memory pool. Determine the memory requirements of the input / output buffers based on the input / output specifications, and associate the memory requirements of the input / output buffers with the input / output memory pool; The memory requirements of intermediate feature maps in each network layer are determined based on the network structure, and the memory requirements of intermediate feature maps in each network layer are associated with the intermediate feature map memory pool.

5. The method according to claim 1, characterized in that, S3, which describes determining the task priority of the target task and configuring memory quotas for the target task, includes: The task priorities of the target tasks are divided into urgent, important, and general levels; Configure differentiated memory quota limits based on task priority; When memory resources are insufficient, priority should be given to fulfilling the memory requests of urgent tasks; When the memory request of an urgent task cannot be satisfied, pause the important or general task and release the corresponding memory resources.

6. The method according to claim 1, characterized in that, S3 describes allocating an independent virtual memory address space and address access permission boundaries for the target task, including: An independent virtual memory address space is allocated for the target task based on the hardware memory management unit; The allowed address range for the target task is determined based on the task identifier and memory quota of the target task. Configure the allowed access address range for the target task as the allowed access zone, and configure the memory address ranges corresponding to other tasks as the prohibited access zone; When the target task accesses a restricted area, the access is blocked.

7. The method according to claim 1, characterized in that, S4 describes the generation of a memory allocation record table, which includes: Record the memory pool to which the pre-allocated memory block belongs, the address of the pre-allocated memory block, the status of the pre-allocated memory block, the target task information to which the pre-allocated memory block belongs, and the correspondence between the pre-allocated memory block and the independent virtual memory address space.

8. The method according to claim 1, characterized in that, S5 describes controlling the start or pause of defragmentation based on memory running status and task priority, including: Periodically obtain the free block distribution, fragmentation rate, and system load of the weight memory pool, input / output memory pool, and intermediate feature map memory pool; When the fragmentation rate of any memory pool exceeds the preset fragmentation rate threshold, the system load is lower than the preset load threshold, and there are no urgent tasks running, a background defragmentation thread is started to merge the discrete free memory blocks in the memory pools whose fragmentation rate exceeds the preset fragmentation threshold and update the memory address mapping table. If an urgent task is detected during defragmentation, defragmentation will be paused.

9. The method according to claim 1, characterized in that, S6 describes the process of reclaiming abnormally persistent memory blocks that remain occupied beyond the task's lifecycle, including: Periodically compare task status and memory block usage status; When the task to which the memory block belongs has been completed but the memory block is still occupied, the memory block is identified as an abnormally inactive memory block. Change the abnormally lingering memory block to an idle state and update the memory management table.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images