Heterogeneous computing scheduling method, system, device and storage medium

Abstract

The disclosure provides a heterogeneous computing scheduling method, system, device and storage medium, and relates to the technical field of artificial intelligence. In some embodiments of the disclosure, according to the task descriptor, the page table mapping relationship of the input-output memory management unit is configured without the participation of the system core, and address translation preprocessing is completed; the virtual address interval accessed subsequently by the neural network processing unit is predicted; before the neural network processing unit initiates a memory access request, according to the virtual address interval, the page table hardware traversal of the input-output memory management unit is triggered, the page table entry is obtained, and the page table entry is loaded into the input-output conversion buffer; a start signal is sent to the neural network processing unit, so that the neural network processing unit accesses the shared main memory through the input-output memory management unit based on the page table mapping relationship. Through the dual-core division of the system core and the control core, the system overhead caused by the participation of the operating system in the underlying scheduling is reduced, and the additional power consumption overhead is reduced.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a heterogeneous computing scheduling method, system, device and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology, edge computing scenarios are facing increasingly stringent requirements for real-time performance, energy efficiency, and data processing throughput. In typical edge AI systems, heterogeneous computing architectures consisting of central processing units (CPUs) and neural network processing units (NPUs) have become the mainstream solution. To maximize system performance and reduce data transfer overhead, achieving efficient memory sharing between the CPU and NPU, especially "zero-copy" data interaction, has become a key technological breakthrough.

[0003] In existing technologies, to achieve data interaction between the CPU and NPU, several solutions have evolved, including data copying, coarse-grained mapping of shared physical memory, and virtual memory sharing based on the IOMMU. Among these, the zero-copy solution based on static IOMMU mapping typically involves the operating system or device driver pre-configuring the IOMMU page table before the NPU task starts, establishing a mapping relationship between CPU virtual addresses and NPU virtual addresses, and maintaining this mapping largely unchanged during task execution. While these existing technologies achieve zero-copy data sharing to some extent, they cannot adapt to the bursty and discontinuous data access characteristics of the NPU in highly dynamic, low-latency edge AI application scenarios. This leads to frequent address translation errors and expensive page table traversal operations, increasing access latency and power consumption.

[0004] Currently, heterogeneous computing scheduling methods suffer from long access latency and significant additional power consumption. Summary of the Invention

[0005] This disclosure provides a heterogeneous computing scheduling method, system, device, and storage medium to at least solve the problems of long access latency and large additional power consumption in existing systems.

[0006] The technical solution disclosed herein is as follows:

[0007] This disclosure provides a heterogeneous computing scheduling method, applied to the control core of a heterogeneous computing scheduling system, including:

[0008] Receive a task descriptor issued by the system core; wherein, the task descriptor is a descriptor generated by the system core based on the task request corresponding to the received AI task;

[0009] Without relying on system kernel intervention, configure the page table mapping relationship of the input / output memory management unit according to the task descriptor;

[0010] Based on the access behavior characteristics and historical execution information of the AI ​​task, predict the virtual address range to be accessed by the neural network processing unit in the future;

[0011] Before the neural network processing unit initiates a memory access request, the page table hardware traversal of the input / output memory management unit is triggered according to the virtual address range to obtain page table entries, and the page table entries are loaded into the input / output conversion backup buffer;

[0012] A start signal is sent to the neural network processing unit so that the neural network processing unit can access the shared main memory through the input / output memory management unit according to the start signal and the page table mapping relationship.

[0013] Optionally, configuring the page table mapping relationship of the input / output memory management unit according to the task descriptor includes:

[0014] The task descriptor is parsed to obtain virtual address information;

[0015] Based on the virtual address information, page table mapping relationships are configured in the input / output memory management unit via a dedicated interface.

[0016] Optionally, predicting the virtual address range subsequently accessed by the neural network processing unit based on the access behavior characteristics and historical execution information of the AI ​​task includes:

[0017] The access behavior characteristics of the AI ​​task are input into the address prediction model along with historical execution information to obtain a virtual address range;

[0018] The input features of the address prediction model include at least: static prior features, dynamic observation features, and statistical correlation features;

[0019] The static prior features include: memory access mode parameters parsed from the task descriptor, wherein the memory access mode parameters include: data step size, tensor dimension, data block size and starting virtual address;

[0020] The dynamic observation features include: the virtual address access history or statistical representation generated by the neural network processing unit in the current task execution phase;

[0021] The statistical correlation features include: historical task memory access feature statistics or feature fingerprints corresponding to the current operator type.

[0022] Optionally, based on the input features, the virtual address range is inferred using at least one of the following methods:

[0023] When the memory access mode exhibits linear or quasi-linear characteristics, inferences are made based on the relationship between data step size and address offset;

[0024] By maintaining an address access record window with configurable length, periodic memory access patterns are identified and inferred based on the address difference distribution within the window;

[0025] Inference is made based on the operator memory access state transition model and combined with the task progress information of the neural network processing unit.

[0026] Optionally, after predicting the virtual address range to be subsequently accessed by the neural network processing unit based on the access behavior characteristics and historical execution information of the AI ​​task, the method further includes:

[0027] Determine whether the address translation entry corresponding to the target virtual address already exists in the input / output translation backup buffer; if it does, skip the prefetch operation for the target virtual address.

[0028] The number of page table entries prefetched in a single operation is adjusted based on memory bandwidth utilization, the number of remaining cache entries in the input / output conversion backup buffer, and the throughput requirements of the neural network processing unit.

[0029] For a missed virtual address, the input / output memory management unit is triggered to perform a page table traversal through a dedicated interface, and the obtained page table entries are loaded into the input / output translation backup buffer.

[0030] Optionally, after predicting the virtual address range to be subsequently accessed by the neural network processing unit based on the access behavior characteristics and historical execution information of the AI ​​task, the method further includes:

[0031] If a virtual address access initiated by a neural network processing unit misses a prefetched page table entry, or if the input / output conversion backstop miss rate exceeds a preset hit rate threshold or miss threshold within a preset window time, the prediction depth is reduced, and the highest priority address translation channel is activated; and

[0032] The address prediction model is calibrated based on the actual memory access behavior sequence;

[0033] If the AI ​​task is detected to have entered a new operator phase, the prediction state is reset.

[0034] This disclosure also provides a heterogeneous computing scheduling system, including:

[0035] The system core is used to generate a task descriptor based on the task request corresponding to the AI ​​task, and to send the task descriptor to the control core;

[0036] The control core is used to configure the page table mapping relationship of the input / output memory management unit according to the task descriptor, predict the virtual address range to be accessed by the neural network processing unit in the future according to the access behavior characteristics and historical execution information of the AI ​​task, and trigger page table traversal and load page table entries before the neural network processing unit initiates a memory access request.

[0037] The neural network processing unit is used to execute the AI ​​task and accesses the shared memory through the input / output memory management unit;

[0038] The input / output memory management unit is used for page table hardware traversal to obtain page table entries and load the page table entries into the input / output conversion backup buffer;

[0039] A shared main memory is used to store the data buffer and the task descriptor.

[0040] This disclosure also provides an electronic device, including:

[0041] processor;

[0042] Memory used to store processor-executable instructions;

[0043] The processor is configured to execute instructions to implement the steps in the above method.

[0044] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0045] This disclosure also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the method described above.

[0046] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0047] In some embodiments of this disclosure, a task descriptor issued by the system core is received; based on the task descriptor, the page table mapping relationship of the input / output memory management unit is configured without the participation of the system core, and address translation preprocessing is completed; based on the access behavior characteristics and historical execution information of the AI ​​task, the virtual address range to be subsequently accessed by the neural network processing unit is predicted; before the neural network processing unit initiates a memory access request, the page table hardware traversal of the input / output memory management unit is triggered according to the virtual address range to obtain page table entries, and the page table entries are loaded into the input / output translation backup buffer; a start signal is sent to the neural network processing unit, enabling it to access the shared main memory through the input / output memory management unit based on the page table mapping relationship. Through the dual-core division of labor between the system core and the control core, the system overhead caused by the operating system's involvement in low-level scheduling is reduced, and additional power consumption is lowered.

[0048] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0049] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0050] Figure 1 A schematic diagram of the structure of a heterogeneous computing scheduling system 10 provided for an exemplary embodiment of this disclosure;

[0051] Figure 2 A flowchart illustrating a heterogeneous computing scheduling method provided for an exemplary embodiment of this disclosure;

[0052] Figure 3 A flowchart illustrating another heterogeneous computing scheduling method provided for an exemplary embodiment of this disclosure;

[0053] Figure 4 A schematic diagram of the structure of a heterogeneous computing scheduling device provided for an exemplary embodiment of this disclosure;

[0054] Figure 5 A schematic diagram of another heterogeneous computing scheduling device provided as an exemplary embodiment of this disclosure;

[0055] Figure 6 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this disclosure;

[0056] Figure 7 A schematic diagram of the structure of another electronic device provided as an exemplary embodiment of this disclosure. Detailed Implementation

[0057] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0058] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure.

[0059] It should be noted that the user information involved in this disclosure includes, but is not limited to, user device information and user personal information; the collection, storage, use, processing, transmission, provision and disclosure of user information in this disclosure all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0060] To address the aforementioned technical issues, in some embodiments of this disclosure, a task descriptor issued by the system core is received; wherein, the task descriptor is a descriptor generated by the system core based on the task request corresponding to the received AI task; the page table mapping relationship of the input / output memory management unit is configured according to the task descriptor; the virtual address range to be subsequently accessed by the neural network processing unit is predicted based on the access behavior characteristics and historical execution information of the AI ​​task; the page table hardware traversal of the input / output memory management unit is triggered according to the virtual address range to obtain page table entries, and the page table entries are loaded into the input / output conversion backup buffer; a start signal is sent to the neural network processing unit, and the neural network processing unit accesses the shared main memory through the input / output memory management unit according to the start signal and the page table mapping relationship, thereby achieving virtual address space sharing while ensuring the secure isolation of the input / output memory management unit, reducing bandwidth consumption caused by data movement, and reducing access latency; through the dual-core division of labor between the system core and the control core, the system overhead caused by the operating system's participation in low-level scheduling is eliminated, reducing additional power consumption overhead.

[0061] The technical solutions provided by the embodiments of this disclosure are described in detail below with reference to the accompanying drawings.

[0062] Figure 1 This is a schematic diagram of the structure of a heterogeneous computing scheduling system 10 provided for an exemplary embodiment of this disclosure. (See diagram below.) Figure 1 As shown, the heterogeneous computing scheduling system 10 includes a system core 11, a control core 12, a neural network processing unit 13, an input / output memory management unit 14, and a shared main memory 15.

[0063] In some embodiments of this disclosure, system core 11 is used to generate a task descriptor based on the task request corresponding to the AI ​​task and send the task descriptor to the control core. Control core 12 is used to configure the page table mapping relationship of the input / output memory management unit based on the task descriptor, and predict the virtual address range subsequently accessed by the neural network processing unit based on the access behavior characteristics and historical execution information of the AI ​​task. Neural network processing unit 13 is used to execute the AI ​​task and access the shared memory through the input / output memory management unit. Input / output memory management unit 14 is used for page table hardware traversal to obtain page table entries and load the page table entries into the input / output conversion backup buffer. Shared main memory 15 is used to store data buffers and task descriptors.

[0064] To provide a more complete and clear explanation of this disclosure, the following explanations are provided for the technical terms used in this disclosure.

[0065] An NPU (Neural Processing Unit) is a dedicated hardware processor designed specifically for artificial intelligence and machine learning tasks. It mimics the structure of neurons and synapses in the human brain and is designed to efficiently perform large-scale parallel computations (primarily matrix multiplication and convolution operations) in deep learning algorithms.

[0066] The MMU (Memory Management Unit) is a hardware component integrated inside (or adjacent to) the CPU. Its primary function is processor-side virtual address translation.

[0067] The IOMMU (Input-Output Memory Management Unit) is a bridge connecting high-speed peripherals and system memory. It is mainly used for virtual address translation on the NPU side.

[0068] IOTLB (I / O Translation Lookaside Buffer) is a key hardware component inside the IOMMU.

[0069] NoC (Network-on-Chip) is a communication architecture used to connect various processing cores, storage modules, input / output interfaces, and other components within a single integrated circuit chip (such as a multi-core processor or a system-on-a-chip, SoC).

[0070] It should be noted that the aforementioned shared main memory and on-chip interconnect network together form a shared memory subsystem. The aforementioned IOMMU and MMU together form an address translation subsystem. The system core, control core, and NPU access the shared main memory through the on-chip interconnect network. The IOMMU is located between the NPU and the shared main memory and is used to perform address translation and access control on virtual address access requests initiated by the NPU.

[0071] In this embodiment, the system kernel is used to run a general-purpose operating system and manage high-level applications. As the global management hub of the system, the system kernel is mainly configured to perform the following functions:

[0072] Environment maintenance: It carries the execution of user-mode AI applications and kernel-mode operating systems, and is responsible for general OS functions such as process scheduling and memory space planning.

[0073] Task abstraction and descriptor generation: In response to AI task requests initiated by upper-layer applications, the system kernel creates a corresponding AI task context. Specifically, the system kernel generates a task descriptor.

[0074] Resource authorization and asynchronous delivery: The system core allocates the necessary virtual address space for the AI ​​task and writes the generated task descriptor to a designated area within the shared storage subsystem or a hardware communication mailbox. Subsequently, the system core delivers the task request to the control core asynchronously.

[0075] It is worth noting that the system core in this embodiment does not directly participate in the underlying micro-scheduling of the NPU, nor does it directly maintain the runtime state of the IOMMU page table (such as the creation, updating or page fault handling of page table entries), thereby avoiding latency jitter caused by operating system context switching.

[0076] In this embodiment, the control core is used to perform real-time scheduling control for the NPU and collaborative management related to address translation without participating in the operating system scheduling path. As the real-time scheduling unit of the NPU and the dynamic page table management entity of the IOMMU, the control core is mainly configured to perform the following functions:

[0077] Task takeover and resolution: Listen for task notifications from the system kernel, read and parse task descriptors from the shared storage subsystem, and obtain the data virtual address range and access attributes associated with the task.

[0078] Dynamic mapping management: Based on the resolved virtual address range, the control kernel directly manipulates the IOMMU's page table base address register and page table memory structure to dynamically establish, update, or reclaim the mapping relationship between virtual addresses and physical addresses. This process is completed locally by the control kernel without needing to enter the host operating system kernel.

[0079] Proactive predictive prefetching mechanism: Based on the memory access behavior characteristics of the current task and historical execution information, predict the virtual address range that may be accessed in the future, and trigger IOMMU page table traversal and IOTLB update in advance to reduce access latency caused by missing address translation.

[0080] Zero-copy triggering: After confirming that the critical page table entries required by the current task have been successfully loaded into the IOTLB and have the correct access permissions, the control core directly sends a start signal to the NPU. This allows the NPU to directly access task data in shared main memory through the IOMMU, completely avoiding redundant data copying between CPU memory and NPU video memory, reducing bus bandwidth usage and power consumption.

[0081] Fine-grained operation management: During NPU task execution, the control verifies the data buffer status, handles possible page faults or access violations, and only reports interrupts to the system kernel when a major error that cannot be handled locally occurs, thus minimizing interference to the system kernel.

[0082] In this embodiment, the NPU is a dedicated AI acceleration hardware engine, comprising multiple computing arrays, on-chip cache, and a data flow controller. The NPU is configured to: receive a start signal and a task context pointer from the control core; execute neural network operators such as convolution, matrix multiplication, and activation functions according to the logic indicated by the task descriptor; during execution, all memory access requests initiated by the NPU carry virtual addresses and are translated by the IOMMU to directly read and write data buffers in the shared main memory without software intervention in address translation.

[0083] In addition to the heterogeneous computing scheduling system 10 provided above, some embodiments of this disclosure also provide a heterogeneous computing scheduling method. The heterogeneous computing scheduling method provided in this disclosure may rely on the heterogeneous computing scheduling system 10 provided above, but is not limited to the heterogeneous computing scheduling system 10 provided in the above embodiments.

[0084] From the perspective of nuclear control, Figure 2 This is a flowchart illustrating a heterogeneous computing scheduling method provided as an exemplary embodiment of this disclosure. Figure 2 As shown, the method includes:

[0085] S201: Receive the task descriptor issued by the system core; wherein, the task descriptor is a descriptor generated by the system core according to the task request corresponding to the received AI task;

[0086] S202: Configure the page table mapping relationship of the input / output memory management unit according to the task descriptor without relying on the intervention of the system kernel;

[0087] S203: Based on the access behavior characteristics and historical execution information of the AI ​​task, predict the virtual address range to be accessed by the neural network processing unit in the future;

[0088] S204: Before the neural network processing unit initiates a memory access request, the page table hardware traversal of the input / output memory management unit is triggered according to the virtual address range to obtain the page table entries and load the page table entries into the input / output conversion backup buffer;

[0089] S205: Send a start signal to the neural network processing unit so that the neural network processing unit can access the shared main memory through the input / output memory management unit according to the start signal and the page table mapping relationship.

[0090] From the perspective of the system kernel, Figure 3 A flowchart illustrating another heterogeneous computing scheduling method provided as an exemplary embodiment of this disclosure. (See attached diagram.) Figure 3 As shown, the method includes:

[0091] S301: Receives task requests corresponding to AI tasks from the upper-layer framework;

[0092] S302: Based on the task request, allocate the data buffer corresponding to the AI ​​task in the shared main memory and establish the virtual address space corresponding to the AI ​​task;

[0093] S303: Generate a task descriptor based on the virtual address space;

[0094] S304: Send the task descriptor to the control kernel.

[0095] In this embodiment, the execution entity of the above method can be a server. The implementation form of the server is not limited. For example, the server can be a conventional server, a cloud server, a cloud host, a virtual center, or other server equipment. The server's components mainly include a processor, hard disk, memory, system bus, and other common computer architecture types.

[0096] In this embodiment, the system core receives task requests corresponding to AI tasks from the upper-layer framework; based on the task requests, it allocates a data buffer corresponding to the AI ​​task in the shared main memory and establishes a virtual address space corresponding to the AI ​​task; based on the virtual address space, it generates a task descriptor; and it sends the task descriptor to the control core. This embodiment achieves decoupling of task scheduling and execution control by having the system core uniformly manage upper-layer AI task requests, centrally complete data buffer allocation and virtual address space construction in the shared main memory, and generate standardized task descriptors that are then sent to the control core. This mechanism not only effectively isolates multi-task data and avoids memory conflicts using the virtual address space, but also reduces the underlying resource management burden on the control core, allowing it to focus on task execution. This significantly reduces task startup latency and improves the system's response speed, resource utilization, and overall throughput for high-concurrency AI tasks.

[0097] In this embodiment, the control core receives a task descriptor from the system core; the task descriptor is generated by the system core based on the task request corresponding to the received AI task; based on the task descriptor, the page table mapping relationship of the input / output memory management unit is configured; based on the access behavior characteristics and historical execution information of the AI ​​task, the virtual address range to be subsequently accessed by the neural network processing unit is predicted; based on the virtual address range, the page table hardware traversal of the input / output memory management unit is triggered to obtain page table entries, and the page table entries are loaded into the input / output conversion backup buffer; a start signal is sent to the neural network processing unit, and the neural network processing unit accesses the shared main memory through the input / output memory management unit according to the start signal and the page table mapping relationship. Specifically, without relying on system core access, the page table mapping relationship of the input / output memory management unit is configured based on the task descriptor. Virtual address space sharing is achieved while ensuring the secure isolation of the input / output memory management units, reducing bandwidth consumption caused by data movement and reducing access latency; through the dual-core division of labor between the system core and the control core, the system overhead caused by the operating system's involvement in low-level scheduling is eliminated, reducing additional power consumption.

[0098] In some embodiments of this disclosure, a task descriptor is generated based on the virtual address space. One possible approach is to generate a task descriptor containing memory access metadata and control metadata based on the virtual address space. The memory access metadata includes, but is not limited to, the virtual address range and access permission attributes of the data to be processed, while the control metadata includes, but is not limited to, the AI ​​operator type, memory access mode parameters, task execution priority, and control information. The system core receives a task request from the upper-layer AI inference framework, allocates a physical data buffer for the task in shared main memory, and constructs an independent virtual address space mapping the buffer. Based on this virtual address space, the system core generates a standardized task descriptor and writes it to a designated area of ​​shared main memory or a hardware communication mailbox. This disclosure reduces the complexity and latency of the control core parsing task configuration by deeply integrating memory access metadata and control metadata into a unified task descriptor; simultaneously, it improves the flexibility of memory management and enhances the overall throughput efficiency of AI tasks by utilizing the virtual address space pre-established by the system core and explicit access permission attributes.

[0099] In some embodiments of this disclosure, the control core sends task descriptors to the control core asynchronously.

[0100] In some embodiments of this disclosure, the page table mapping relationship of the Input / Output Memory Management Unit is configured based on the task descriptor. One possible approach is to parse the task descriptor to obtain virtual address information; based on the virtual address information, the page table mapping relationship is configured in the Input / Output Memory Management Unit through a dedicated interface. Specifically, the control core parses the task descriptor to obtain the virtual address information required for the operation of the neural network processing unit. The system core does not participate in the maintenance of the underlying page table, thereby solving the problem of excessively long scheduling links caused by system calls. Based on the parsed virtual address information, the control core dynamically establishes or updates the corresponding page table mapping relationship in the IOMMU through a dedicated interface, realizing the mapping configuration from the NPU virtual address to the shared main memory physical address. The implemented mapping has fine-grained characteristics and is adjusted during task execution based on the operator execution progress or phased memory access changes, rather than being statically configured all at once before startup, thereby improving the security and flexibility of memory access. This disclosure employs a task context-based dynamic page table mapping management mechanism. Compared to static mapping, this mechanism can establish or update page table mapping relationships on demand based on the memory access characteristics of different AI tasks, thereby improving memory resource utilization and enhancing the system's adaptability to complex memory access patterns. By dynamically managing the page table mapping of the input / output memory management unit through the control core, in at least one implementation, shadow page tables or equivalent intermediate mapping structures can be combined to avoid or reduce the additional overhead caused by system core interrupts or context switching triggered by page table updates in traditional architectures, thereby improving the real-time performance and determinism of heterogeneous task scheduling.

[0101] In some embodiments of this disclosure, the virtual address range to be subsequently accessed by the neural network processing unit is predicted based on the access behavior characteristics and historical execution information of the AI ​​task. One possible approach is to input the access behavior characteristics and historical execution information of the AI ​​task into an address prediction model to obtain the virtual address range. The input features of the address prediction model include at least: static prior features, dynamic observation features, and statistical correlation features. The static prior features include: memory access pattern parameters parsed from the task descriptor, including: data step size, tensor dimension, data block size, and starting virtual address. The dynamic observation features include: the virtual address access history or statistical representation generated by the neural network processing unit in the current task execution stage. The statistical correlation features include: historical task memory access feature statistics or feature fingerprints corresponding to the current operator type. The control kernel combines the access behavior characteristics and historical execution information to calculate the virtual address range that the NPU may access in the next stage. This disclosure analyzes the memory access behavior characteristics of AI tasks to predict the virtual address range subsequently accessed by the neural network processing unit, and pre-triggers page table traversal and loads the corresponding page table entries into the address translation cache before the memory access request is initiated, thereby reducing the access waiting time caused by address translation misses and improving the continuity and throughput performance of the neural network processing unit execution process.

[0102] In some embodiments of this disclosure, based on input features, at least one of the following methods is used to infer the virtual address range subsequently accessed by the neural network processing unit. These inference methods include, but are not limited to, the following:

[0103] Inference Method 1: When the memory access mode exhibits linear or quasi-linear characteristics, inference is made based on the relationship between data step size and address offset;

[0104] Inference Method 2: By maintaining an address access record window with configurable length, inference is made based on the periodic memory access patterns identified by the address difference distribution within the window;

[0105] Inference Method 3: Inference is made based on the operator memory access state transition model and combined with the task progress information of the neural network processing unit. Specifically, the range of data pages that may be accessed in the next stage is inferred based on the operator memory access state transition model and combined with the task progress information of the neural network processing unit, thereby inferring the virtual address range.

[0106] In some embodiments of this disclosure, after predicting the virtual address range to be subsequently accessed by the neural network processing unit based on the access behavior characteristics and historical execution information of the AI ​​task, a page table warm-up operation is performed. One possible approach is to determine whether the address translation entry corresponding to the target virtual address already exists in the input / output translation lookup buffer; if it does, the prefetching operation for the target virtual address is skipped; the number of page table entries prefetched in a single operation is adjusted according to the memory bandwidth utilization, the number of remaining cache entries in the input / output translation lookup buffer, and the throughput requirements of the neural network processing unit; for virtual addresses that are not hit, the input / output memory management unit is triggered to perform page table traversal through a dedicated interface, and the obtained page table entries are loaded into the input / output translation lookup buffer. For example, conflict checking: First, the IOTLB is searched. If the translation entry corresponding to the predicted address already exists, the corresponding prefetch operation is skipped. Prefetch depth control: The number of page table entries prefetched in a single operation is dynamically adjusted according to the current memory bandwidth utilization of the system, the number of remaining cache entries in the IOTLB, and the current real-time throughput requirements of the NPU to avoid over-prefetching. Active loading: For a missed predicted address, the control core triggers the page table traversal of the IOMMU through a dedicated interface and actively and asynchronously loads the obtained page table entries into the address translation cache such as the IOTLB. This process is transparent to the NPU and does not block its normal computation in at least one implementation.

[0107] In some embodiments of this disclosure, after predicting the virtual address range subsequently accessed by the neural network processing unit based on the access behavior characteristics and historical execution information of the AI ​​task, a prediction deviation processing and adaptive adjustment mechanism is implemented. One possible approach is to reduce the prediction depth and activate the highest priority address translation channel when it is detected that a virtual address access initiated by the neural network processing unit misses a prefetched page table entry, or the IOTLB miss rate exceeds a preset hit rate threshold or miss threshold within a preset window time; and to perform feedback calibration on the address prediction model based on the actual memory access behavior sequence; and to automatically reset the prediction state machine when it is detected that the AI ​​task enters a new operator cycle or execution phase. Specifically, while performing page table prediction prefetching, the control core evaluates the effectiveness of the prediction results in real time through an internal monitoring unit. When it is detected that a virtual address access actually initiated by the NPU misses a prefetched page table entry, or the IOTLB miss rate exceeds a threshold within a preset window time, the control core executes at least one of the following processing strategies when a prediction deviation occurs:

[0108] Degradation and rapid recovery strategy: Temporarily reduce prediction depth and simultaneously enable high-priority address translation channels to prioritize filling page table entries required by the NPU for current access.

[0109] Adaptive parameter calibration: Recalculate the step size or offset parameters of the current task based on the actual memory access behavior sequence, and perform online feedback calibration on the prediction model.

[0110] Task boundary adaptation: When a task is detected to be entering a new operator cycle or execution phase, the prediction state machine is automatically reset to avoid the impact of outdated prediction models on new memory access patterns.

[0111] Through the above mechanism, even when the prediction is inaccurate, the system can still maintain the correctness of address translation and task execution, while providing feedback for the adaptive optimization of subsequent prediction strategies.

[0112] This disclosure embodiment monitors address translation cache hits in real time during task execution and executes mechanisms such as prediction depth adjustment, model parameter calibration, or prediction strategy switching when prediction deviations are detected. This enables the system to maintain stable address translation performance under complex and variable AI operator memory access modes, improving the overall system robustness. Through prediction prefetching and dynamic scheduling mechanisms, memory access efficiency and execution performance are improved while reducing invalid memory accesses and unnecessary scheduling overhead, thereby reducing system power consumption and improving the system's energy efficiency ratio to a certain extent.

[0113] In some embodiments of this disclosure, after confirming that the address translation path (including page table entry readiness status and permission verification) required for the current task is ready, the control core directly sends a start signal to the NPU. The NPU utilizes the prepared translation entries to directly access the shared main memory through the IOMMU, achieving zero-copy data interaction during computation. After confirming that the relevant prefetch entries are ready by querying the IOMMU's status information, the control core sends a start command to the NPU through a dedicated hardware trigger path. Embodiments of this disclosure achieve virtual address space sharing between the neural network processing unit and the system main memory through an input / output memory management unit, enabling the neural network processing unit to directly access data in the shared main memory, reducing or avoiding traditional data copying processes, thereby reducing memory bandwidth usage, reducing latency overhead caused by data movement, and improving overall memory access efficiency.

[0114] In some embodiments of this disclosure, during NPU execution, the control core tracks the actual memory access progress and IOTLB hit status of the NPU through a real-time monitoring module. If a prediction deviation or memory access mode change (such as an AI task switching operator) is detected, the control core dynamically adjusts the IOMMU page table mapping relationship and prediction prefetching strategy to ensure the real-time performance and determinism of the scheduling process. During the dynamic execution of tasks, the control core monitors the hit status and prediction effectiveness of the address translation cache in real time through the IOMMU-related address translation performance monitoring mechanism, thereby constructing an adaptive closed-loop control process at runtime. Real-time deviation determination: When abnormal fluctuations occur in the address translation cache hit behavior due to factors such as data alignment offset, operator execution path changes, or dynamic task switching, for example, consecutive misses or an overall hit level below a preset safety threshold, the control core immediately captures the abnormal state. The above monitoring and determination processes are all completed locally by the control core without reporting interruptions to the system core or operating system, thereby ensuring the continuity and real-time performance of the main system execution. Online policy calibration: After determining the prediction deviation, the control core triggers an online calibration mechanism, adaptively adjusting the prediction parameters by re-analyzing recent address access trajectories. The control core can dynamically switch between multiple address access prediction strategies or update parameters such as the prediction window and step size estimation method online. Simultaneously, the control core can dynamically adjust the prefetch depth or prefetch rhythm based on the current system bus or storage subsystem load status to achieve a balance between prefetch benefits and bandwidth consumption.

[0115] In some embodiments of this disclosure, after the NPU completes its computation task, the control core receives a completion event from the NPU and reports the task execution result to the system core. Subsequently, the control core reclaims the IOMMU page table entries associated with the task, releases the corresponding translation cache space, and completes the closed loop of the entire scheduling cycle.

[0116] Figure 4 This is a schematic diagram of the structure of a heterogeneous computing scheduling device 40 provided for an exemplary embodiment of this disclosure. Figure 4 As shown, the heterogeneous computing scheduling device 40 includes: a receiving module 41, a configuration module 42, a prediction module 43, a loading module 44, and a sending module 45.

[0117] The receiving module 41 is used to receive the task descriptor issued by the system core; wherein the task descriptor is a descriptor generated by the system core according to the task request corresponding to the received AI task;

[0118] Configuration module 42 is used to configure the page table mapping relationship of the input / output memory management unit according to the task descriptor without relying on the intervention of the system kernel;

[0119] The prediction module 43 is used to predict the virtual address range to be accessed by the neural network processing unit in the future based on the access behavior characteristics and historical execution information of the AI ​​task.

[0120] Before the neural network processing unit initiates a memory access request, the loading module 44 is used to trigger the page table hardware traversal of the input / output memory management unit according to the virtual address range, obtain the page table entries, and load the page table entries into the input / output conversion backup buffer.

[0121] The sending module 45 is used to send a start signal to the neural network processing unit, so that the neural network processing unit can access the shared main memory through the input / output memory management unit according to the start signal and the page table mapping relationship.

[0122] Optionally, when configuring the page table mapping relationship of the input / output memory management unit according to the task descriptor, the configuration module 42 is used to:

[0123] The task descriptor is parsed to obtain virtual address information;

[0124] Based on the virtual address information, page table mapping relationships are configured in the input / output memory management unit through a dedicated interface.

[0125] Optionally, when predicting the virtual address range subsequently accessed by the neural network processing unit based on the access behavior characteristics and historical execution information of the AI ​​task, the prediction module 43 is used to:

[0126] By inputting the access behavior characteristics of AI tasks and historical execution information into the address prediction model, a virtual address range is obtained.

[0127] The input features of the address prediction model include: static prior features, dynamic observation features, and statistical correlation features. Static prior features include: memory access pattern parameters parsed from the task descriptor, which include: data step size, tensor dimension, data block size, and starting virtual address. Dynamic observation features include: the virtual address access history or statistical representation generated by the neural network processing unit in the current task execution stage. Statistical correlation features include: historical task memory access feature statistics or feature fingerprints corresponding to the current operator type.

[0128] Optionally, the prediction module 43 infers the virtual address range subsequently accessed by the neural network processing unit based on the input features using at least one of the following methods:

[0129] When the memory access mode exhibits linear or quasi-linear characteristics, the virtual address range is inferred based on the relationship between the data step size and the address offset.

[0130] Maintain a configurable address access record window. By analyzing the address difference distribution pattern within the address access record window, identify non-continuous and periodic memory access patterns to infer virtual address ranges.

[0131] Based on the operator memory access state transition model and combined with the task progress information of the neural network processing unit, the virtual address range is inferred.

[0132] Optionally, after predicting the virtual address range subsequently accessed by the neural network processing unit based on the access behavior characteristics and historical execution information of the AI ​​task, the prediction module 43 can also be used for:

[0133] Determine if the address translation entry corresponding to the target virtual address already exists in the input / output translation backup buffer; if it does, skip the prefetch operation for the target virtual address.

[0134] Adjust the number of page table entries for a single prefetch based on memory bandwidth utilization, the number of remaining cache entries in the input / output conversion backup buffer, and the throughput requirements of the neural network processing unit.

[0135] For a missed virtual address, the input / output memory management unit is triggered to perform a page table traversal through a dedicated interface, and the obtained page table entries are loaded into the input / output translation backup buffer.

[0136] Optionally, after predicting the virtual address range subsequently accessed by the neural network processing unit based on the access behavior characteristics and historical execution information of the AI ​​task, the prediction module 43 can also be used for:

[0137] If a virtual address access initiated by a neural network processing unit misses a prefetched page table entry, or if the miss rate of the input / output conversion backup buffer exceeds a preset hit rate threshold or miss threshold within a preset window time, the prediction depth is reduced, and the highest priority address translation channel is activated; and

[0138] The step size or offset parameters of the AI ​​task are recalculated based on the actual memory access behavior sequence, and the address prediction model is calibrated by feedback.

[0139] The predictive state machine is automatically reset when the AI ​​task is detected to be entering a new operator cycle or execution phase.

[0140] Figure 5 A schematic diagram of another heterogeneous computing scheduling device 50 provided for an exemplary embodiment of this disclosure. (See diagram below.) Figure 5 As shown, the heterogeneous computing scheduling device 50 includes: a receiving module 51, an allocation module 52, a generation module 53, and a distribution module 54.

[0141] Among them, the receiving module 51 is used to receive the task request corresponding to the AI ​​task of the upper-layer framework;

[0142] The allocation module 52 is used to allocate a data buffer corresponding to the AI ​​task in the shared main memory according to the task request, and to establish a virtual address space corresponding to the AI ​​task.

[0143] The generation module 53 is used to generate task descriptors based on the virtual address space;

[0144] The distribution module 54 is used to distribute task descriptors to the control core.

[0145] Optionally, when generating a task descriptor based on the virtual address space, the generation module 53 is used for:

[0146] Based on the virtual address space, a task descriptor containing memory access metadata and control metadata is generated; wherein, the memory access metadata includes: the virtual address range and access permission attributes of the data to be processed, and the control metadata includes: AI operator type, memory access mode parameters, task execution priority and control information.

[0147] Optionally, the generation module 53 can also be used for:

[0148] Write the task descriptor to a designated area of ​​shared main memory or a hardware communication mailbox.

[0149] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0150] Figure 6 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present disclosure. For example... Figure 6 As shown, the electronic device includes a memory 61 and a processor 62. Additionally, the electronic device also includes a power supply component 63 and a communication component 64.

[0151] Memory 61 is used to store computer programs and can be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device.

[0152] The memory 61 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0153] Communication component 64 is used for data transmission with other devices.

[0154] The processor 62 executes computer instructions stored in the memory 61 to: receive a task descriptor issued by the system core; wherein the task descriptor is a descriptor generated by the system core according to the task request corresponding to the received AI task; configure the page table mapping relationship of the input / output memory management unit according to the task descriptor without relying on the intervention of the system core; predict the virtual address range to be subsequently accessed by the neural network processing unit according to the access behavior characteristics and historical execution information of the AI ​​task; before the neural network processing unit initiates a memory access request, trigger the page table hardware traversal of the input / output memory management unit according to the virtual address range, obtain the page table entries, and load the page table entries into the input / output translation backup buffer; and send a start signal to the neural network processing unit so that the neural network processing unit can access the shared main memory through the input / output memory management unit according to the start signal and the page table mapping relationship.

[0155] Accordingly, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program. When the computer-readable storage medium stores a computer program, and the computer program is executed by one or more processors, it causes one or more processors to perform... Figure 2 Each step in the method embodiment.

[0156] Accordingly, embodiments of this disclosure also provide a computer program product, which includes a computer program / instructions that are executed by a processor. Figure 2 Each step in the method embodiment.

[0157] Figure 7 A schematic diagram of the structure of another electronic device provided as an exemplary embodiment of this disclosure. For example... Figure 7 As shown, the electronic device includes a memory 71 and a processor 72. Additionally, the electronic device also includes a power supply component 73 and a communication component 74.

[0158] Memory 71 is used to store computer programs and can be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device.

[0159] The memory 71 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0160] Communication component 74 is used for data transmission with other devices.

[0161] The processor 72 can execute computer instructions stored in the memory 71 to: receive task requests corresponding to AI tasks in the upper-layer framework; allocate a data buffer corresponding to the AI ​​task in the shared main memory according to the task request, and establish a virtual address space corresponding to the AI ​​task; generate a task descriptor according to the virtual address space; and send the task descriptor to the control core.

[0162] Accordingly, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program. When the computer-readable storage medium stores a computer program, and the computer program is executed by one or more processors, it causes one or more processors to perform... Figure 3 Each step in the method embodiment.

[0163] Accordingly, embodiments of this disclosure also provide a computer program product, which includes a computer program / instructions that are executed by a processor. Figure 3 Each step in the method embodiment.

[0164] The above Figure 6 and Figure 7 The communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0165] The above Figure 6 and Figure 7 The power supply component provides power to the various components of the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which it resides.

[0166] The aforementioned electronic devices also include a display screen and audio components.

[0167] The display includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions, but also the duration and pressure associated with the touch or swipe operation.

[0168] An audio component may be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals may be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0169] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0171] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0173] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0174] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0175] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0176] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0177] The above are merely specific embodiments of this disclosure, enabling those skilled in the art to understand or implement this disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to these embodiments, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A heterogeneous computing scheduling method, characterized in that, Control cores used in heterogeneous computing scheduling systems include: Receive a task descriptor issued by the system core; wherein, the task descriptor is a descriptor generated by the system core based on the task request corresponding to the received AI task; Without relying on system kernel intervention, configure the page table mapping relationship of the input / output memory management unit according to the task descriptor; Based on the access behavior characteristics and historical execution information of the AI ​​task, the virtual address range to be subsequently accessed by the neural network processing unit is predicted, including: The access behavior characteristics of the AI ​​task are input into the address prediction model along with historical execution information to obtain a virtual address range; The input features of the address prediction model include at least: static prior features, dynamic observation features, and statistical correlation features; The static prior features include: memory access mode parameters parsed from the task descriptor, wherein the memory access mode parameters include: data step size, tensor dimension, data block size and starting virtual address; The dynamic observation features include: the virtual address access history or statistical representation generated by the neural network processing unit in the current task execution phase; The statistical correlation features include: historical task memory access feature statistics or feature fingerprints corresponding to the current operator type; Before the neural network processing unit initiates a memory access request, the page table hardware traversal of the input / output memory management unit is triggered according to the virtual address range to obtain page table entries, and the page table entries are loaded into the input / output conversion backup buffer; A start signal is sent to the neural network processing unit so that the neural network processing unit can access the shared main memory through the input / output memory management unit according to the start signal and the page table mapping relationship.

2. The method according to claim 1, characterized in that, The step of configuring the page table mapping relationship of the input / output memory management unit according to the task descriptor includes: The task descriptor is parsed to obtain virtual address information; Based on the virtual address information, page table mapping relationships are configured in the input / output memory management unit via a dedicated interface.

3. The method according to claim 1, characterized in that, Based on the input features, the virtual address range is inferred using at least one of the following methods: When the memory access mode exhibits linear or quasi-linear characteristics, inferences are made based on the relationship between data step size and address offset; By maintaining an address access record window with configurable length, periodic memory access patterns are identified and inferred based on the address difference distribution within the window; Inference is made based on the operator memory access state transition model and combined with the task progress information of the neural network processing unit.

4. The method according to claim 1, characterized in that, After predicting the virtual address range to be subsequently accessed by the neural network processing unit based on the access behavior characteristics and historical execution information of the AI ​​task, the method further includes: Determine whether the address translation entry corresponding to the target virtual address already exists in the input / output translation backup buffer; if it does, skip the prefetch operation for the target virtual address. The number of page table entries prefetched in a single operation is adjusted based on memory bandwidth utilization, the number of remaining cache entries in the input / output conversion backup buffer, and the throughput requirements of the neural network processing unit. For a missed virtual address, the input / output memory management unit is triggered to perform a page table traversal through a dedicated interface, and the obtained page table entries are loaded into the input / output translation backup buffer.

5. The method according to claim 1, characterized in that, After predicting the virtual address range to be subsequently accessed by the neural network processing unit based on the access behavior characteristics and historical execution information of the AI ​​task, the method further includes: If a virtual address access initiated by a neural network processing unit misses a prefetched page table entry, or if the input / output conversion backstop miss rate exceeds a preset hit rate threshold or miss threshold within a preset window time, the prediction depth is reduced, and the highest priority address translation channel is activated; and The address prediction model is calibrated based on the actual memory access behavior sequence; If the AI ​​task is detected to have entered a new operator phase, the prediction state is reset.

6. A heterogeneous computing scheduling system, characterized in that, include: The system core is used to generate a task descriptor based on the task request corresponding to the AI ​​task, and to send the task descriptor to the control core; The control core is used to configure the page table mapping relationship of the input / output memory management unit according to the task descriptor, predict the virtual address range to be accessed by the neural network processing unit in the future according to the access behavior characteristics and historical execution information of the AI ​​task, and trigger page table traversal and load page table entries before the neural network processing unit initiates a memory access request. The neural network processing unit is used to execute the AI ​​task and accesses the shared memory through the input / output memory management unit; The input / output memory management unit is used for page table hardware traversal to obtain page table entries and load the page table entries into the input / output conversion backup buffer; A shared main memory is used to store the data buffer and the task descriptor; The control core is used to input the access behavior characteristics of the AI ​​task and historical execution information into the address prediction model to obtain a virtual address range; The input features of the address prediction model include at least: static prior features, dynamic observation features, and statistical correlation features; The static prior features include: memory access mode parameters parsed from the task descriptor, wherein the memory access mode parameters include: data step size, tensor dimension, data block size and starting virtual address; The dynamic observation features include: the virtual address access history or statistical representation generated by the neural network processing unit in the current task execution phase; The statistical correlation features include: historical task memory access feature statistics or feature fingerprints corresponding to the current operator type.

7. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute instructions to implement the steps of the method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-5.

9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-5.

Images