AI data handling optimization method based on multi-core system

Abstract

The invention discloses an AI data handling optimization method based on a multi-core system. The data handling optimization method comprises the steps that S1, initialization is carried out; s2, asynchronous request; s3, asynchronous processing; and S4, parallel execution. The processor A for executing the AI operation is always in the user state and does not need to fall into the kernel state, so that the system calling overhead is avoided; in the process that the processor A executes the AI operation, the processor B schedules DMA hardware to complete transmission processing of next data required by the AI reasoning operation, the DMA hardware and the AI reasoning operation are completely parallel, and when DMA transmission time is shorter than AI reasoning operation time, time overhead of DMA data transmission is completely masked; based on the software and hardware collaboration framework, the utilization rate of the computing part of the processor A can be close to 100%.

Description

Technical Field

[0001] This invention belongs to the field of AI inference technology, specifically relating to an AI data handling optimization method based on a multi-core system. Background Technology

[0002] AI inference engines need to access large amounts of data and perform computations based on this data when executing inference tasks, and this data is often reused repeatedly within a short period of time. To improve the computational efficiency of AI inference, the AI ​​computing unit of a chip typically incorporates TCM (Tightly Coupled Memory, which has higher access efficiency than DRAM main memory). When performing a local inference operation, the AI ​​inference engine needs to perform two steps: first, move the data from DRAM main memory to TCM; second, use the data in TCM to perform the inference operation. The higher the efficiency of these two steps, the better the performance of the AI ​​inference engine. This invention is a multi-processor collaborative framework applied to SMP multi-core systems. Through hardware and software collaboration, it improves the overall efficiency of TCM data movement and AI inference operations, thereby achieving higher AI computing utilization and inference performance.

[0003] AI inference performance is limited by the "memory wall," meaning that the computing power of computing units increases much faster than the growth of memory bandwidth, resulting in data supply speed not keeping up with computing demands. Data transfer performance has become the core bottleneck of AI inference. Current AI inference system optimization mainly focuses on optimizing the hardware transmission itself (such as adding conversion functions when accessing data via DMA to reduce the data conversion load of AI computing components).

[0004] The prior art disclosed on March 19, 2019, is an invention patent entitled "A Multi-channel DMA Controller and Convolutional Neural Network Acceleration Method for Convolutional Neural Network Acceleration" (application number: CN201811425474.5). This patent improves the computational efficiency and memory usage efficiency of convolutional neural network accelerators by specifying specific channels for different AI computing tasks.

[0005] The prior art disclosed on November 1, 2024, is an invention patent entitled "A DMA memory access optimization method based on a multi-core AI chip" (application number: CN202411354313.7). This patent reduces the number of DRAM accesses by merging DMA requests, thereby improving access efficiency.

[0006] Existing technologies primarily focus on optimizing DMA hardware transfers. However, this optimization suffers from waiting issues in single-core architectures or synchronous models. AI inference is typically built on Linux operating systems. In homogeneous AI computing systems (where AI computing power comes from CPU extensions), the AI ​​inference engine runs in user space, while the DMA driver usually runs in kernel space. Calls to the DMA driver by the AI ​​inference service introduce significant time overhead: In the DMA data transfer model, the AI ​​inference engine needs to interrupt its user-space computation tasks and enter Linux kernel space via system calls. The trapping and recovery of these system calls require additional processing time, which can become substantial when system calls are frequent. Furthermore, the kernel driver initiates DMA transfers and waits for the DMA task to complete. During this time, the AI ​​computing core cannot perform AI inference operations, wasting AI computing resources. Summary of the Invention

[0007] The purpose of this invention is to solve the aforementioned technical problems in the prior art and provide an AI data transfer optimization method based on a multi-core system. Through a software and hardware collaborative framework, the AI ​​inference engine only needs to fill in the data block information to be transferred in user space, without needing system calls to trap into the kernel, to complete the data block transfer request. Furthermore, data transfer and AI inference computation are asynchronous and parallel, and AI computation is basically not interrupted during AI inference, with AI computing power resource utilization approaching 100%.

[0008] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: An AI data handling optimization method based on a multi-core system includes the following steps: S1, Initialization Configure the DMA service in the system kernel. The DMA service requests an MSI interrupt and obtains the interrupt ID. At the same time, it establishes a shared memory region mapped to the user space. The shared memory region includes a DMA Message space for storing the DMA request queue and an interrupt Message space for triggering the MSI interrupt.

[0009] S2, Asynchronous Request The AI ​​inference engine running in user mode on processor A submits a data transfer request containing transfer parameters to the DMA Message space and triggers an MSI interrupt by writing the interrupt ID to the interrupt Message space.

[0010] S3, Asynchronous Processing In response to the MSI interrupt in step S2, processor B reads the data transfer request from the DMA Message space, configures and starts the DMA controller to execute the corresponding data transfer operation.

[0011] S4, Parallel Execution After submitting the first data transfer request, processor A waits for the first data transfer to complete, and then performs AI inference calculations based on the transferred data. Furthermore, before executing the calculation of the current batch of data, processor A submits the data transfer request for the next batch of data in advance, thereby realizing asynchronous parallelism between data transfer and AI inference calculations.

[0012] Furthermore, the DMA Message space consists of an array containing several task structures, each including data transfer information and task status. Preferably, the DMA Message space contains 32 task structures.

[0013] Furthermore, the data transport information includes at least one of the following: data length, data address, and transport method.

[0014] Furthermore, the task status includes at least one of the following: idle, submitted, in transit, and transit completed.

[0015] Furthermore, after step S3, an interrupt handling step is also included: after the DMA controller completes the data transfer, it sends a completion interrupt to the processor B; the processor B responds to the completion interrupt, updates the status flag of the corresponding request in the DMA Message space to transfer completed, and checks whether there is a subsequent transfer request with a status of "committed" in the DMA request queue. If so, the next data transfer operation is started immediately.

[0016] Furthermore, in step S1, the specific process of the MSI interrupt is as follows: an MSI interrupt is generated by writing a specific interrupt ID to the MSI controller space; during the initialization phase of processor B, an MSI interrupt ID is requested, and the MSI space is mapped to the application layer space visible to processor A through mmap. Processor A needs to obtain the aforementioned MSI interrupt ID, submit the task to the DMA Message, and then write the interrupt ID into the mapped space to trigger the MSI interrupt.

[0017] Furthermore, after completing the AI ​​inference operation on the current batch of data, processor A checks the status flag of the next batch of data in the DMA Message space by calling the query interface; if the status flag indicates that the transfer is complete, then the next batch of data is used to continue executing the AI ​​inference operation.

[0018] Furthermore, the query interface allows setting a timeout period; if the status flag changes to "transfer completed" is not found within the timeout period, the process will wait until the set timeout period is reached or until the task is completed.

[0019] Furthermore, during the system startup phase, constraints are dedicated to processor B in response to MSI interrupts and DMA controller transfer completion interrupts.

[0020] Furthermore, in step S4, after processor A submits the first data transfer request, it synchronously waits for the DMA Message space state to change to "transfer completed" before starting the first AI inference operation and submitting the second data transfer request.

[0021] A multi-core system optimized for AI data transfer includes: The DMA controller is used to perform data transfer. Several processors A are used to run the user-mode AI inference engine; Several processors B are used to respond to MSI interrupts and DMA controller transfer completion interrupts; The shared memory region is mapped to DMA Message space and interrupt Message space; Processor A is configured to submit a data transfer request in user mode and trigger an MSI interrupt by performing write operations to the DMA Message space and the interrupt Message space; Processor B is configured to: in response to an MSI interrupt, retrieve a request from the DMA Message space and drive the DMA controller.

[0022] Furthermore, processor A uses a processor core that accelerates the operation of the AI ​​inference engine, while processor B uses a processor core that focuses on general-purpose computing.

[0023] The present invention, by adopting the above-described technical solution, has the following beneficial effects: This invention avoids system call overhead by keeping processor A, which performs AI calculations, in user mode without needing to enter kernel mode. While processor A is executing AI calculations, processor B schedules DMA hardware to handle the transfer of the next data required for AI inference. The two processes run in parallel, and when the DMA transfer time is less than the AI ​​inference time, the DMA data transfer overhead is completely masked. Based on this software and hardware collaborative framework, the utilization rate of processor A's computing components can reach nearly 100%.

[0024] In step S1 of this invention, an efficient communication mechanism between user-mode programs and the DMA controller is achieved by configuring DMA services in the kernel and establishing a user-mode accessible shared memory region. This design avoids frequent kernel-mode and user-mode switching, significantly improving data transfer efficiency and response speed, while providing a reliable hardware support foundation for asynchronous data transfer.

[0025] In step S2 of this invention, by directly submitting a transmission request in user mode and triggering an interrupt, the computation task and data transmission are decoupled. This asynchronous request mechanism allows the AI ​​inference engine to continue executing other computation tasks while waiting for data transmission, effectively reducing the processor's idle waiting time and improving the overall resource utilization and task processing efficiency of the system.

[0026] In step S3 of this invention, a dedicated processing core is responsible for responding to and processing DMA requests, achieving physical isolation and specialized division of labor between computational tasks and data transfer tasks. This collaborative model allows the two processing cores to fully leverage their respective strengths, with processor B focusing on efficiently managing and scheduling DMA transfer tasks to ensure the timeliness and accuracy of data transmission. Simultaneously, this design avoids idle states caused by computational cores waiting for data transmission, enabling more efficient and rational utilization of system resources, improving the overall system throughput, and achieving parallel execution of computation and transmission.

[0027] In step S4 of this invention, the time overlap between data transfer operations and AI inference computation is successfully achieved through a cleverly designed parallel execution mechanism. This parallel processing mode effectively masks the latency caused by data transmission, making the computing core almost always in an active working state. The pre-submission mechanism ensures the continuity of data supply and avoids idle periods caused by the computing core waiting for data. This highly parallel execution mode significantly improves the overall execution efficiency of AI inference tasks, enabling the system to run continuously at a level close to the theoretical peak performance. Attached Figure Description

[0028] The present invention will be further described below with reference to the accompanying drawings: Figure 1 This is a system architecture diagram of an AI data handling optimization method based on a multi-core system according to the present invention; Figure 2 This is an asynchronous timing diagram of processor A and processor B in this invention; Figure 3 This is a flowchart of the present invention. Detailed Implementation

[0029] A multi-core system optimized for AI data transfer includes: a DMA controller for performing data transfer; several processors A for running a user-mode AI inference engine; several processors B for responding to MSI interrupts and transfer completion interrupts of the DMA controller; a shared memory region mapped to a DMA message space and an interrupt message space; processors A are configured to submit data transfer requests in user mode and trigger MSI interrupts by writing to the DMA message space and the interrupt message space; processors B are configured to retrieve requests from the DMA message space and drive the DMA controller in response to MSI interrupts. Processors A employ processor cores designed to accelerate the operation of the AI ​​inference engine, while processors B employ processor cores focused on general-purpose computing.

[0030] This invention constructs a highly efficient data transfer optimization system through the rational allocation of hardware resources and the meticulous design of software logic; it achieves perfect separation of computation and transmission, fully leverages the parallel processing capabilities of multi-core systems, and provides strong performance support for AI inference applications; it adopts targeted core type selection to achieve optimal configuration of system functions; dedicated computing cores ensure the efficient execution of AI inference tasks, while general-purpose computing cores provide flexible interrupt handling and system management capabilities, and the two work together to improve the overall performance of the system.

[0031] like Figures 1 to 3 As shown, this invention provides an AI data handling optimization method based on a multi-core system, comprising the following steps: S1, Initialization Configure the DMA service in the system kernel. The DMA service requests an MSI interrupt and obtains the interrupt ID. At the same time, it establishes a shared memory region mapped to the user space. The shared memory region includes a DMA Message space for storing the DMA request queue and an interrupt Message space for triggering the MSI interrupt.

[0032] S2, Asynchronous Request The AI ​​inference engine running in user mode on processor A submits a data transfer request containing transfer parameters to the DMA Message space and triggers an MSI interrupt by writing the interrupt ID to the interrupt Message space.

[0033] S3, Asynchronous processing: In response to the MSI interrupt in step S2, processor B reads the data transfer request from the DMA Message space, configures and starts the DMA controller to execute the corresponding data transfer operation.

[0034] S4, Parallel Execution After submitting the first data transfer request, processor A waits for the first data transfer to complete, and then performs AI inference calculations based on the transferred data. Furthermore, before executing the calculation of the current batch of data, processor A submits the data transfer request for the next batch of data in advance, thereby realizing asynchronous parallelism between data transfer and AI inference calculations.

[0035] The DMA Message space consists of an array of task structures, each containing data transfer information and a task status. Data transfer information includes at least one of data length, data address, and transfer method. Task status includes at least one of idle, committed, transferring, and transferred. The DMAMessage space contains 32 task structures. By designing the DMA Message space as a queue containing multiple task structures, a standardized task management framework is established. Each task structure fully encapsulates the key information required for data transfer and its current execution status, providing a clear context for multi-task parallel processing. This structured design allows multiple transfer tasks to be queued and managed independently, avoiding mutual interference between tasks. It also lays the foundation for task priority scheduling and resource allocation, significantly improving the standardization and systematic nature of DMA task management. The data transfer information includes key parameters such as data length, address, and transfer method, ensuring that the DMA controller can accurately execute each transfer task. The complete parameter set enables the DMA hardware to autonomously complete the entire data transfer process from source to destination without continuous intervention from the processing core. This design improves the accuracy and reliability of data transmission, providing a reliable data supply guarantee for complex AI workloads. The task status employs a multi-level identification mechanism, providing precise state tracking for task coordination in asynchronous environments. Clear state divisions enable the processing core to understand the execution progress of each task in real time, thereby making appropriate scheduling decisions. The state machine mechanism effectively prevents task duplication or omission, ensuring the integrity and consistency of task processing.

[0036] Following step S3, an interrupt post-processing step is also included: after the DMA controller completes data transfer, it sends a completion interrupt to processor B; processor B responds to the completion interrupt, updates the status flag of the corresponding request in the DMA Message space to "transfer completed," and checks whether there is a subsequent transfer request with a status of "committed" in the DMA request queue. If so, it immediately starts the next data transfer operation. Through an intelligent interrupt handling mechanism, continuous automated execution of DMA transfer tasks is achieved. This design ensures that after processing the current transfer task, subsequent pending tasks can be checked and started immediately, minimizing the idle time of the DMA controller. The pipelined task processing mode significantly improves the utilization efficiency of DMA hardware, allowing the data transfer bandwidth to be fully utilized. At the same time, timely status updates ensure the accurate transmission of task completion information, providing a reliable basis for the use of data in the computing core, thereby constructing an efficient and reliable data supply chain.

[0037] In step S1, the specific process of the MSI interrupt is as follows: An MSI interrupt is generated by writing a specific interrupt ID to the MSI controller space; during the initialization phase, processor B requests an MSI interrupt ID and maps the MSI space to the application layer space visible to processor A via mmap. Processor A needs to obtain the aforementioned MSI interrupt ID, submit the task to the DMA Message, and then write the interrupt ID into the mapped space to trigger the MSI interrupt. This memory-based interrupt triggering mechanism achieves efficient and low-latency inter-core communication. Compared to traditional interrupts, MSI interrupts avoid the additional overhead of the interrupt controller and provide more accurate and faster interrupt responses. By directly exposing the interrupt triggering capability to user-mode programs through memory mapping, the computing core can directly wake up the processing core without kernel intervention, greatly shortening the interrupt response path latency. This design provides crucial low-latency guarantees for AI computing scenarios with high real-time requirements, ensuring that the system can respond to data processing needs in a timely manner.

[0038] After completing the AI ​​inference operation on the current batch of data, processor A checks the status flag of the next batch of data in the DMA Message space by calling the query interface. If the status flag indicates "transfer complete," the AI ​​inference operation continues using the next batch of data. The query interface allows setting a timeout period; if the status flag is not changed to "transfer complete" within the timeout period, the process waits until the set timeout period or until the task is completed. This proactive query mechanism achieves precise synchronization between computation and data transmission, ensuring that the AI ​​inference engine can always begin processing immediately after data is ready. This avoids unnecessary waiting time and prevents wasted computational resources due to premature queries. The query mechanism provides the computational core with a flexible means of checking data availability, enabling task scheduling to be dynamically adjusted based on actual data preparation, maintaining the continuity and stability of the computational pipeline, and providing a reliable data supply guarantee for continuous and efficient AI inference operations. The introduction of a configurable timeout mechanism enhances the system's handling capability and robustness under abnormal conditions. The timeout setting prevents indefinite waiting due to DMA transfer failures, ensuring that the system maintains a certain level of responsiveness even when some components malfunction; this improves the system's adaptability and stability in the face of various abnormal situations.

[0039] During system startup, constraints are dedicated to processor B for responding to MSI interrupts and DMA controller transfer completion interrupts. This specialized allocation of interrupt responses ensures that critical interrupts are handled promptly and reliably, guaranteeing the continuity and stability of AI inference task execution. Simultaneously, centralized interrupt management simplifies system complexity, improves maintainability, and provides more reliable infrastructure support for high-performance computing environments.

[0040] In step S4, after processor A submits the first data transfer request, it synchronously waits for the DMA Message space state to change to "transfer complete" before starting the first AI inference operation and submitting the second data transfer request. This initial synchronous waiting mechanism ensures data integrity and computational correctness during system startup. This design lays a reliable foundation for subsequent asynchronous parallel execution, avoiding computational errors caused by incomplete data processing.

[0041] like Figure 1As shown, processor A, represented by A100, is a processor core that accelerates the AI ​​inference engine and runs multiple AI applications. During AI computation, it generates data transfer requirements. This method pre-sends these requirements to the DMA control processor (X100) via MSI and a task queue (transfer-req-array in the diagram). Processor B, represented by X100, is a processor core focused on general-purpose computing. Upon receiving the request and reading the relevant parameter configuration, X100 uses high-speed DMA (HS-AIDMA in the diagram) to complete the data transfer. X100 is also responsible for task distribution and status updates after task completion. During this process, A100 can continue AI computation after submitting the task. When data needs to be transferred, it checks the task status; if the transfer is complete, AI computation can begin.

[0042] Specific implementation process: 1. The system enters the running state from the start of operation; 2. Processor A runs an AI application, configures the parameters required for this transfer task in the DMA Message space, and triggers an MSI interrupt by writing to the interrupt Message space in the encapsulated interface to notify processor B to start the DMA transfer. After the notification is completed, the original application continues to run. 3. In the MSI interrupt or the interrupt where the previous DMA transfer is completed, processor B obtains the DMA parameters of the new task from the DMA Message space and completes the configuration and startup of DMA. 4. After responding to the interrupt indicating that the DMA transfer is complete, processor B updates the status of the current DMA transfer task in the DMA Message space and checks whether a new transfer task has been submitted. If so, the previous step is repeated. 5. Before executing an inference operation, processor A submits the next DMA transfer request. After completing an inference operation, it actively queries the status of the next DMA transfer task and waits for it to be completed before preparing to execute a new inference operation.

[0043] like Figure 2 As shown in the diagram, the APIs include: API1: Used to initialize shared memory resources and MSI interrupt resources, allocate a shared memory region in user mode and kernel mode to create the task queue corresponding to this application, and obtain the resources required to trigger an MSI interrupt, including the address and interrupt number.

[0044] API2: Used to provide AI applications with the ability to submit tasks. Specifically, the data structure corresponding to the task is submitted to the task queue, the task status is marked as updated and DMA has not been moved, and then an MSI interrupt is triggered.

[0045] API 3: Provides AI applications with a way to query task status and allows callers to set timeouts. It checks the task status in the task queue to see if the DMA transfer is complete; if not, it waits at most for the set timeout or until the task is completed.

[0046] Specific implementation process: P1. Start the hardware device and start the relevant software. In this method, the DMA controller driver is loaded and the hardware is initialized during the kernel startup phase of the software startup. The constraint is that the processor B responds to the DMA interrupt. P2. During the preparation phase, the AI ​​application calls API1 provided by the DMA framework that serves AI to prepare resources.

[0047] After the AI ​​application on processor A (P3) starts running, it will generate a data transfer request. This request is broken down into DMA parameters according to regulations and submitted as a task via API2. Upon receiving the task, processor B drives the DMA transfer, while processor A continues running the AI ​​application asynchronously. Further details will be provided below. When the P4 AI application needs the data corresponding to this task, it calls API3 to check the status. This status will be updated to shared memory in the interrupt when the DMA transfer is completed.

[0048] Step P31 is derived from step P3. Its specific function is to update the task status on the shared memory to "updated" and "DMA not moved" by calling API2, and trigger an MSI interrupt. After the call is completed, processor A continues to run the AI ​​application. Step P32 is derived from step P3. Its specific function is as follows: If processor B is not in the DMA interrupt handler or the previous MSI interrupt handler at this time, processor B responds to the current MSI interrupt and checks whether there are any updated but unprocessed tasks in the task queue. If so, DMA is started for transfer and the status is updated to DMA in progress. If processor B is in the DMA interrupt handler or the previous MSI interrupt handler when the MSI is triggered, the latest submitted task will be started and its status updated in the current interrupt.

[0049] Step P41 is derived from step P4. Its specific function is as follows: after the DMA hardware completes the current transfer task, the status of the transfer task is updated to DMA completed in the DMA interrupt handling function, and the task queue is checked for new tasks. If there are new tasks, the transfer is restarted.

[0050] Step P42 is derived from step P4. Its specific function is: the AI ​​application calls API3 to check the task status on shared memory, and continues to run the AI ​​application after completion.

[0051] This invention can modify the logic of processor B described above, changing it from initiating DMA transfer to other resource requirements, such as operations on PCIe bus resources, and sending notifications to other processors. When application-layer tasks have high real-time requirements, the time spent outside of the effective task can be asynchronously implemented using the logic of this invention.

[0052] This invention avoids system call overhead by keeping processor A, which performs AI calculations, in user mode without needing to enter kernel mode. While processor A is executing AI calculations, processor B schedules DMA hardware to handle the transfer of the next data required for AI inference. The two processes run in parallel, and when the DMA transfer time is less than the AI ​​inference time, the DMA data transfer overhead is completely masked. Based on this software and hardware collaborative framework, the utilization rate of processor A's computing components can reach nearly 100%.

[0053] The above are merely specific embodiments of the present invention, but the technical features of the present invention are not limited thereto. Any simple changes, equivalent substitutions, or modifications made based on the present invention to solve essentially the same technical problems and achieve essentially the same technical effects are all covered within the protection scope of the present invention.

Claims

1. A method for AI data transfer optimization based on a multi-core system, characterized in that The method comprises the following steps: S1, initialization In the system kernel, a DMA service is configured, which applies for an MSI interrupt and obtains an interrupt ID, and establishes a shared memory region mapped to a user space, which comprises a DMA Message space for storing a DMA request queue and an interrupt Message space for triggering an MSI interrupt; S2, asynchronous request An AI inference engine in a user space of a processor A is run, a data transmission request containing transmission parameters is submitted to the DMA Message space, and an MSI interrupt is triggered by writing the interrupt ID into the interrupt Message space; S3, asynchronous processing A processor B responds to the MSI interrupt in step S2, reads the data transmission request from the DMA Message space, configures and starts a DMA controller to perform a corresponding data transfer operation; S4, parallel execution The processor A waits for the completion of the first data transfer after submitting the first data transmission request, and then performs AI inference calculation based on the transferred data; and the processor A submits the transmission request of the next batch of data in advance before performing the calculation of the current batch of data, so as to realize the asynchronous parallel of data transfer and AI inference calculation.

2. The AI data transfer optimization method based on a multi-core system according to claim 1, characterized in that: The DMA Message space is composed of an array containing a plurality of task structures, and the task structure comprises data transfer information and a task state.

3. The AI data movement optimization method based on a multi-core system according to claim 2, characterized in that: The data transfer information comprises at least one of data length, data address and transfer mode.

4. The AI data movement optimization method based on a multi-core system according to claim 2, characterized in that: The task state comprises at least one of idle, submitted, transferring and transfer completed.

5. The AI data movement optimization method based on a multi-core system according to claim 1, characterized in that: After step S3, there is also a post-interrupt processing step: when the DMA controller completes the data transfer, a completion interrupt is sent to the processor B; the processor B updates the state flag of the corresponding request in the DMA Message space to "transfer completed" in response to the completion interrupt, and checks whether there is a subsequent transmission request with a state of "submitted" in the DMA request queue, and if so, the next data transfer operation is started immediately.

6. The AI data movement optimization method based on a multi-core system according to claim 1, characterized in that: In step S1, the specific process of the MSI interrupt is: writing a specific interrupt id into the MSI controller space to generate an MSI interrupt; the processor B applies for an MSI interrupt id in the initialization stage, and maps the MSI space to the application layer space visible to the processor A through mmap, and the processor A needs to obtain the above MSI interrupt id, write the interrupt id into the mapped space after submitting the task to the DMA Message, and trigger the MSI interrupt.

7. The AI data movement optimization method based on a multi-core system according to claim 1, characterized in that: After the processor A completes the AI inference operation of the current batch of data, the state flag of the next batch of data in the DMA Message space is checked by calling a query interface; if the state flag is "transfer completed", the next batch of data is used to continue the AI inference operation.

8. The AI data movement optimization method based on a multi-core system according to claim 7, characterized in that: The query interface allows setting a timeout time; if the state flag is not changed to "transfer completed" within the timeout time, waiting until the timeout time is set or the task is completed.

9. The AI data movement optimization method based on a multi-core system according to claim 1, characterized in that: In the system startup phase, the constraint is that the processor B is dedicated to responding to the MSI interrupt and the DMA controller's transfer completion interrupt.

10. The AI data movement optimization method based on a multi-core system according to claim 1, characterized in that: In the step S4, after the processor A submits the first data transfer request, it synchronously waits for the DMA Message space state to become the carrying completion, and then starts the first AI inference operation and submits the second transfer request.

11. An AI data movement optimized multi-core system for implementing the method of any of claims 1-10, characterized in that, Comprise: a DMA controller for performing data carrying; a plurality of processors A for running user-mode AI inference engines; a plurality of processors B for responding to MSI interrupts and DMA controller transfer completion interrupts; a shared memory area mapped with a DMA Message space and an interrupt Message space; The processor A is configured to submit a data carrying request in the user mode and trigger an MSI interrupt by writing to the DMA Message space and the interrupt Message space; The processor B is configured to respond to the MSI interrupt, obtain the request from the DMA Message space, and drive the DMA controller.

12. The AI data movement optimized multi-kernel system of claim 11, wherein: The processor A uses a processor core that accelerates the running of the AI inference engine, and the processor B uses a processor core that focuses on general computing.

Images