Data processing method and device for AI application, equipment, medium and program product

Abstract

The invention provides a data processing method and device for an AI application, equipment, a medium and a program product, and relates to the technical field of data processing.The method comprises the steps that a training data set is read from storage equipment according to a first data loading instruction sent by a CPU; performing data enhancement on data in the training data set, generating to-be-trained data corresponding to the batch identifier, and writing the to-be-trained data into a first data buffer area in the GPU; obtaining model intermediate state information written by the GPU from a second data buffer area of the DPU; and obtaining model data obtained by final training and written by the GPU from the second data buffer area. By introducing the DPU into the artificial intelligence server, necessary operations in artificial intelligence model training can be unloaded from the CPU of the artificial intelligence server to the DPU to be executed, so that processing resources of the CPU are released, processing time delay is reduced, execution efficiency of the CPU is improved, and data processing efficiency of an AI application is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to an AI application data processing method and device, equipment, medium and program product. BACKGROUND

[0002] At present, in the training application of an artificial intelligence (AI) model, the capacity of a training data set is usually large (for example, the data set for training a GPT-4 large model reaches 1 PetaBytes), and in the training preparation stage, the AI server (cluster) needs to read the training data set from an external network storage device. At the same time, the training data set needs to be preprocessed, and the above operations are all processed by the CPU (Central Processing Unit) of the AI server host in series. Compared with the actual training operation, the above operations are time-consuming, which reduces the efficiency of the model training stage. Therefore, the low data processing efficiency of AI applications has become a problem to be solved. SUMMARY

[0003] The present application provides an AI application data processing method, device, equipment, medium and program product, which can solve the problem of low data processing efficiency of AI applications in the prior art and improve the data processing efficiency of AI applications.

[0004] The present application provides an AI application data processing method applied to a DPU, which includes the following steps: reading a training data set from a storage device according to a first data loading instruction sent by a central processing unit (CPU); the first data loading instruction contains a batch identifier for identifying a training data batch; performing data enhancement on the data in the training data set to generate training data corresponding to the batch identifier, and writing the training data into a first data buffer in a graphics processing unit (GPU); the training data is used for the GPU to perform a model training operation; obtaining model intermediate state information written by the GPU from a second data buffer of the DPU; the model intermediate state information is generated in a model training process and is used for training interruption recovery or model performance analysis; obtaining model data obtained by final training from the second data buffer.

[0005] According to a data processing method for an AI application provided in this application, the GPU memory pre-allocates a first storage space, the first storage space including a first data structure for point-to-point direct memory access operations; the first data structure includes a first data buffer and a first notification ring; the first notification ring is used to receive notifications of data writing from the peer and notifications of data processing completion from the peer. After writing the training data into the first data buffer in the graphics processing unit (GPU), the method further includes: According to the point-to-point direct memory access write operation, a first notification message of the training data is written to the address pointed to by the tail pointer of the first notification ring; the first notification message is used to notify the GPU that data has arrived in the first data buffer.

[0006] According to the data processing method for an AI application provided in this application, the DPU has a second storage space pre-allocated in its memory. The second storage space includes a second data structure for point-to-point direct memory access operations. The second data structure includes a second data buffer and a second notification ring. The second notification ring is used to receive notifications of data writing from the peer and notifications of data processing completion from the peer. The step of obtaining the intermediate state information of the model written by the GPU from the second data buffer of the DPU includes: The second notification ring receives a second notification message containing the intermediate state information of the model sent by the GPU. According to the second notification message, the intermediate state information of the model written by the GPU is obtained from the second data buffer.

[0007] According to the data processing method for an AI application provided in this application, the step of obtaining the final trained model data written by the GPU from the second data buffer includes: The third notification message of the model data sent by the GPU is received through the second notification ring; According to the third notification message, the final trained model data written by the GPU is obtained from the second data buffer.

[0008] According to a data processing method for an AI application provided in this application, the DPU and the GPU are coupled to the same peripheral component interconnect standard PCI bridge; the DPU and the GPU communicate through the PCI bridge; or, the DPU and the GPU communicate through memory read and write operations.

[0009] According to the data processing method for an AI application provided in this application, the data processing method for the AI ​​application further includes: Receive the second data loading instruction sent by the CPU; According to the second data loading instruction, the verification dataset is read from the storage device; Perform at least one of the following operations on the data in the verification dataset: data decoding, data augmentation, and data format conversion, to generate data to be verified. A point-to-point direct memory access write operation is performed to write the data to be verified into the first data buffer in the GPU; the data to be verified is used by the GPU to perform model verification operations.

[0010] According to the data processing method for an AI application provided in this application, the data processing method for the AI ​​application further includes: Receive the first read instruction for AI inference model data sent by the CPU; According to the first read instruction, the AI ​​inference model data is read from external storage; The AI ​​inference model data is sent to the CPU; the AI ​​inference model data is used by the CPU to perform model preprocessing operations.

[0011] According to the data processing method for an AI application provided in this application, after sending the AI ​​inference model data to the CPU, the method further includes: The system receives a second read instruction for pre-computed features sent by the CPU; the pre-computed features are feature data that are pre-computed and stored for direct use by the AI ​​inference model. According to the second read instruction, the pre-computed features are read from the external storage; Perform a point-to-point direct memory access write operation to write the pre-computed features into the first data buffer in the GPU.

[0012] According to the data processing method for an AI application provided in this application, the data processing method for the AI ​​application further includes: Receive the data to be inferred sent by the front-end application; Feature extraction is performed on the data to be inferred to obtain feature data; A point-to-point direct memory access write operation is performed to write the feature data into the first data buffer in the GPU; the feature data is used by the GPU to perform inference operations.

[0013] This application also provides a data processing device for AI applications, including the following modules: The data reading module is used to read the training dataset from the storage device according to the first data loading instruction sent by the central processing unit (CPU); the first data loading instruction includes a batch identifier for identifying the batch of training data. The data augmentation module is used to augment the data in the training dataset, generate training data corresponding to the batch identifier, and write the training data into the first data buffer in the graphics processing unit (GPU); the training data is used by the GPU to perform model training operations. The first writing module is used to obtain the intermediate state information of the model written by the GPU from the second data buffer of the DPU; the intermediate state information of the model is generated during the model training process and is used for training interruption recovery or model performance analysis. The second write module is used to obtain the final trained model data written by the GPU from the second data buffer.

[0014] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a data processing method for any of the AI ​​applications described above.

[0015] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a data processing method for any of the AI ​​applications described above.

[0016] This application also provides a computer program product, including a computer program that, when executed by a processor, implements a data processing method for any of the AI ​​applications described above.

[0017] The AI ​​application data processing method, apparatus, device, medium, and program product provided in this application read a training dataset from a storage device according to a first data loading instruction sent by the CPU; the first data loading instruction includes a batch identifier for identifying the batch of training data; data augmentation is performed on the data in the training dataset to generate training data corresponding to the batch identifier, and the training data is written to a first data buffer in the GPU; the training data is used for the GPU to perform model training operations; intermediate model state information written by the GPU is obtained from a second data buffer of the GPU; the intermediate model state information is generated during model training and is used for training interruption recovery or model performance analysis; the final trained model data written by the GPU is obtained from the second data buffer. This application, by introducing a DPU into an artificial intelligence server, can offload necessary operations in artificial intelligence model training from the CPU of the artificial intelligence server to the DPU for execution, thereby freeing up CPU processing resources, reducing processing latency, and improving CPU execution efficiency, thus improving the data processing efficiency of AI applications. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the data processing method for the AI ​​application provided in this application.

[0020] Figure 2 This is a hardware co-architecture diagram for AI acceleration provided in this application.

[0021] Figure 3 This is a schematic diagram of the Peer-to-Peer DMA implementation between the video memory of the AI ​​accelerator card and the onboard memory of the DPU provided in this application.

[0022] Figure 4 This is a schematic diagram illustrating the communication method between the video memory of the AI ​​accelerator card and the onboard memory of the DPU provided in this application.

[0023] Figure 5 This is a schematic diagram of the artificial intelligence model training and application process provided in this application.

[0024] Figure 6 This is a schematic diagram of the processing instruction sequence interaction inside the artificial intelligence server for the artificial intelligence model training application provided in this application.

[0025] Figure 7This is a schematic diagram of the artificial intelligence model inference application process provided in this application.

[0026] Figure 8 This is a schematic diagram of the processing instruction sequence interaction inside the artificial intelligence server of the artificial intelligence model inference application provided in this application.

[0027] Figure 9 This is a schematic diagram of the data processing device for the AI ​​application provided in this application.

[0028] Figure 10 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] In related technologies, artificial intelligence involves two application scenarios: model training and model inference. The complete training process of an AI model consists of several epochs, each epoch involving training on a full, large-scale training dataset. Each epoch includes stages such as data augmentation, forward propagation, backpropagation, and weight updates. After several epochs of iteration, a model validation / testing step is performed to finally obtain a model (model structure and weight parameter set) that meets the accuracy requirements. The large-scale training dataset and the final output model are typically stored in a dedicated network distributed storage device. During the model training preparation phase, through a data loading operation, the AI ​​server (cluster) reads the large-scale training dataset from the external, dedicated network distributed storage into its local memory. Then, the CPU performs data decompression, data decoding, data augmentation, shuffling, and batching operations on the data entries in the dataset. Finally, through H2D (host-to-device, i.e., from server host memory to GPU / accelerator card memory) operations, the data batches are sent to the GPU (i.e., AI accelerator card) for corresponding computations. After each epoch is completed, a checkpoint mechanism (a technique for persistent data storage and state recovery) can be executed as needed to export the intermediate state data of the trained model from the GPU to the server's storage space, preparing for subsequent breakpoint training and other fine-tuning. Finally, after all epochs are completed, the final trained model data is output from the GPU to the server's storage space.

[0031] In addition, in AI model inference applications, for common real-time inference scenarios, the data to be inferred is typically submitted online by the front-end application to the back-end AI server. The AI ​​server reads the necessary pre-computed features from external network storage devices. First, the AI ​​server performs pre-processing operations to complete the feature extraction and transformation steps of the data to be inferred. Second, the AI ​​server sends the transformed data features to be inferred to the GPU / AI accelerator card for corresponding calculations through H2D operations (the inference model has been pre-loaded into the GPU / AI accelerator card). After the calculation is completed, the calculation result is returned to the AI ​​server host memory by the GPU / AI accelerator card through D2H (device to host, i.e., from GPU memory to server host memory). After the CPU performs post-processing steps, the result is returned to the front-end application.

[0032] In the process of training and inference applications of artificial intelligence models, the main problems are inefficient hardware resource scheduling and data transmission becoming a performance bottleneck, as detailed below: (1) Problems in the model training scenario: a) Low efficiency in the training preparation stage: Operations such as reading the training dataset (from external network storage to the server), decompression, decoding, shuffling, and data augmentation are all processed serially by the CPU. These operations are time-consuming and do not match the subsequent GPU-dominated training operation rhythm, slowing down the progress of the entire training preparation stage, making the pre-training preparation work an efficiency bottleneck. b) The checkpoint mechanism is time-consuming: When checking the intermediate state of the model, the GPU needs to export the intermediate state data through D2H operation and then write it from the host memory to the external network storage. When the amount of intermediate state data is very large, the D2H transmission and host writing to storage process will consume a lot of time, seriously affecting the overall progress and efficiency of the training process.

[0033] (2) Problems in the model inference scenario: a. Time-consuming pre-inference processing, increasing latency: The submission of data to be inferred, the reading of pre-computed features (from the feature repository stored on the external network), decompression, decoding, and feature extraction and transformation before inference all rely on CPU processing. These operations are time-consuming, making the pre-inference preparation steps an efficiency bottleneck and increasing the overall latency of online inference. b. Data transmission links drag down efficiency: During inference, the CPU needs to send the transformed data features to be inferred into the video memory through H2D operations, which takes time. After the GPU completes the calculation, the result needs to be returned to the host memory through D2H operations, and then the CPU needs to perform post-processing before returning it to the front-end application. Multiple data transmission links (H2D, D2H) consume time, further reducing inference efficiency and increasing the latency of online inference.

[0034] In short, during training and inference, the CPU faces significant pressure in serially processing non-computationally intensive tasks (such as data preprocessing and data transfer coordination), and the frequent and time-consuming data transfer between different hardware (CPU, GPU / accelerator card, storage device) becomes a key factor restricting overall performance.

[0035] It should be understood that the Data Processing Unit (DPU) has independent processing capabilities such as a processor (low-power multi-core ARM processor), onboard memory, and storage space. By introducing the DPU into the artificial intelligence server, some necessary operations in artificial intelligence model training and model inference can be offloaded from the CPU of the artificial intelligence server to the DPU for execution, thereby freeing up CPU processing resources, reducing processing latency, and improving CPU execution efficiency.

[0036] To address the aforementioned issues, this application proposes a data processing method for AI applications, specifically an accelerated processing method for artificial intelligence applications, encompassing accelerated processing in both AI model training and inference scenarios. This application utilizes a Data Processing Unit (DPU) to rapidly load and preprocess large-scale training and validation data for AI model training from external network storage devices, accelerating the model's checkpoint implementation mechanism and thus speeding up AI model training. It also proposes a method for rapidly loading and preprocessing real-time online request data in AI model inference applications, thereby accelerating AI model inference.

[0037] It should be noted that the data processing method for the AI ​​application provided in this application is executed by a DPU.

[0038] The following is combined with Figures 1-10 This application describes the data processing methods, apparatus, devices, media, and program products for AI applications.

[0039] Figure 1 This is a flowchart illustrating the data processing method for the AI ​​application provided in this application, such as... Figure 1 As shown, the method includes the following: Step 101: Read the training dataset from the storage device according to the first data loading instruction sent by the central processing unit (CPU).

[0040] The first data loading instruction contains a batch identifier to identify the batch of training data. It should be understood that in AI training, data is typically processed in batches (e.g., training the model with 32 images and 64 texts at a time), rather than loading all data at once (too much data would be too much for memory / GPU memory). The batch identifier is a unique identifier (which can be a number, sequence number, index, etc., such as batch_001, batch 5) used to precisely locate a specific batch of training data.

[0041] The functions of batch identifiers can include: 1) Data location: Storage devices may store massive amounts of training data in batches (e.g., batch 1, batch 2... batch N). Batch identifiers enable the DPU to directly locate the target batch, avoiding the reading of irrelevant data. 2) Matching training rhythm: During model training, after each batch of data computation is completed (e.g., forward propagation, backward propagation), the next batch of data needs to be loaded. Batch identifiers ensure that the batch loaded by the DPU is perfectly matched to the current training progress (e.g., when training reaches the 10th round, batch 3, the corresponding batch is loaded). 3) Supporting parallel / distributed training: In multi-GPU distributed training, different GPUs may need to process different sub-batches of data. Batch identifiers can be further subdivided (e.g., batch_001_sub0, batch_001_sub1), enabling the DPU to accurately distribute data.

[0042] The CPU can generate a first data load instruction based on the training progress (such as the current round, completed batches), specifying the batch identifier needed (e.g., loading the 8th batch of training data). After receiving the first data load instruction, the DPU parses the batch identifier and, based on it, locates and reads the training data (which may include the original data and corresponding labels) from an independent network distributed storage device. For example, the DPU can read training data from an independent network distributed storage device through file interfaces, block interfaces, or SPDK (Storage Performance Development Kit). File interface: Reads data by file / directory level (e.g., image datasets are organized by category folders), adapting to common data organization formats; Block interface: Skips file system overhead and directly accesses storage block-level data, pursuing ultimate IO performance; SPDK: Through user-space driving, polling mechanisms, etc., it further reduces storage I / O latency and CPU intervention, enabling the DPU to pull data more efficiently.

[0043] The DPU leverages efficient interfaces to directly read training datasets from independent distributed storage, offloading the CPU's storage I / O burden. This allows the throughput and latency of data reading to better match the demands of AI training (especially large-scale training) for high-throughput, low-latency data supply.

[0044] Step 102: Perform data augmentation on the data in the training dataset to generate training data corresponding to the batch identifier, and write the training data into the first data buffer in the graphics processing unit (GPU).

[0045] In artificial intelligence model training, data augmentation is a technique that generates new training samples by reasonably and diversely modifying the original training data without changing the core semantics / labels of the data. Its core goal is to expand the scale of training data, enrich the diversity of data distribution, and ultimately help the model improve its generalization ability (reduce overfitting and allow the model to perform better on unseen real data).

[0046] It should be understood that different types of data (such as images, text, and speech) have different enhancement methods. For example, image data enhancement can include geometric transformations, pixel-level transformations, and advanced enhancements; text data enhancement can include synonym replacement, random insertion / deletion, sentence reordering, and context expansion; and speech data enhancement can include additive noise, volume adjustment, speech rate adjustment, pitch shifting, time cropping, and frequency masking.

[0047] The DPU augments the data in the training dataset, generating training data corresponding to batch identifiers. This training data is used by the GPU to perform model training operations. Simultaneously, the training data is written to the first data buffer in the GPU. It should be understood that the DPU takes over the data augmentation work from the CPU, uses hardware parallelism to accelerate the generation of training data for the corresponding batches, and then directly writes it to the GPU buffer, allowing the GPU to start training immediately without waiting for data.

[0048] In one embodiment, a peer-to-peer direct memory access (P2P DMA) write operation is performed to write the training data to the first data buffer in the GPU. It should be understood that in P2P DMA, data transmission between devices (such as DPUs and GPUs) and memory requires CPU scheduling (e.g., DPU → CPU memory → GPU), consuming significant CPU resources. DMA technology allows peripherals to transfer data directly with memory or other peripherals without CPU intervention, essentially freeing up CPU resources. Peer-to-peer is a further optimization of DMA; data is transferred directly between two peripherals (DPU and GPU) without passing through host memory (i.e., system memory managed by the CPU). This is equivalent to the DPU and GPU communicating directly without going through CPU memory. The core characteristics of peer-to-peer are no CPU intervention, bypassing host memory, and the shortest transmission path.

[0049] After the DPU completes data augmentation, it identifies the batch of training data that needs to be transferred to the GPU according to preset rules (or instructions issued by the CPU in advance) and initiates a point-to-point DMA write request. At this time, the DPU directly establishes a communication link with the GPU (usually based on the PCIe bus P2P protocol, which requires hardware support from both the DPU and GPU), without notifying the CPU to participate in scheduling. The data path is: DPU's local cache → PCIe direct connection channel → GPU's first data buffer. Through point-to-point direct memory access, the DPU skips the CPU and writes the training data directly to the GPU buffer, reducing CPU resource consumption and avoiding the data transfer bottleneck caused by the CPU, thereby improving data delivery efficiency and the overall progress of subsequent GPU model training.

[0050] In one embodiment, the DPU can also perform operations such as data decompression, data decoding, data augmentation, and data shuffling on the training dataset to generate training data corresponding to batch identifiers. It should be understood that raw data (such as image and text datasets) is often stored in compressed formats (such as JPEG images or .gz compressed files) to save storage / transmission costs. Decompression restores the compressed data to its original, processable form (e.g., decompressing JPEG into an RGB pixel matrix, or decompressing a compressed file into its original file), providing uncompressed raw data for subsequent steps. It should be understood that data decoding is a fine-grained conversion of the encoding format, translating the data from its stored encoding into a structure that the program can directly understand. It should be understood that randomly shuffling the data order is crucial. If the data is stored in an ordered manner according to category / pattern (e.g., first all images of "cats," then all images of "dogs"), not shuffling it will cause the model to remember the data order, resulting in poor generalization ability. Shuffling makes the data categories in each batch more random, allowing the model to learn general features more efficiently.

[0051] Step 103: Obtain the intermediate state information of the model written by the GPU from the second data buffer of the DPU.

[0052] Intermediate state information is generated during model training and is used for training interruption recovery or model performance analysis. Specifically, intermediate state information is key data dynamically generated during AI model training, not the final result. It is a snapshot of a certain stage of training and can include: model weights after the current batch training (such as weight matrices and bias terms for each layer), optimizer status (such as momentum parameters and learning rate adjustment records); completed training epochs, current batch index, loss function value, accuracy, and other metrics; current training hyperparameters (such as batch size and learning rate), and hardware resource usage (such as GPU memory usage). It should be understood that if training is interrupted due to power outages, hardware failures, etc., it does not need to start from scratch; the progress before the interruption can be restored based on the intermediate state information. By analyzing the intermediate states at different stages (such as loss change trends and weight distribution), training strategies can be optimized (such as adjusting the learning rate) or model problems can be diagnosed (such as early warning of overfitting).

[0053] When the model is trained to a preset node (e.g., after each epoch or every 100 batches), the GPU will directly transfer the generated intermediate model state information to a pre-allocated second data buffer in the DPU memory via a P2P DMA write operation. The DPU, by monitoring its own second notification ring, detects that the GPU has written the intermediate state information and then reads this data from the second data buffer.

[0054] Optionally, after acquiring the data, the DPU will perform further operations (such as compression and verification) and then write it to external storage (such as a distributed storage system) for long-term storage, or send some information back to the CPU for real-time analysis according to CPU instructions.

[0055] Step 104: Obtain the final trained model data written by the GPU from the second data buffer.

[0056] The final model data obtained from training is the final output data generated after the AI ​​model has undergone a complete training process (such as multiple iterations and parameter optimization). This data may include: the weight matrices of each layer (such as convolutional kernel parameters and fully connected layer weights), optimized core parameters such as bias terms; network topology definitions (such as the connection relationships between layers and activation function types), input and output dimensions, quantization information (such as whether FP16 / INT8 precision is used); and hyperparameters at the end of training (such as the final learning rate) and performance metrics (such as the final accuracy and loss value), used to verify the model's effectiveness. This data is the final output of training and will subsequently be used for model deployment (such as inference services), secondary training (such as fine-tuning), or long-term archiving.

[0057] Once model training is complete (e.g., reaching the preset number of epochs or achieving the required accuracy), the GPU organizes the final model data into a standardized format (such as PyTorch's .pth or TensorFlow's .pb structure) and writes it to a pre-allocated second data buffer in the DPU. After the GPU completes the write, it sends a notification that the data has been written via a second notification ring. Upon receiving the notification, the DPU directly retrieves the model data from the second data buffer without needing to go through CPU memory.

[0058] Optionally, after acquiring the data, the DPU can perform data verification (such as CRC check to ensure that the transmission is error-free); format conversion (such as converting to the common ONNX format for easy cross-framework deployment); and writing to external storage (such as distributed file system or object storage) to complete the persistent storage of the model.

[0059] The data processing method for AI applications provided in this application involves reading a training dataset from a storage device according to a first data loading instruction sent by a central processing unit (CPU). The first data loading instruction includes a batch identifier for identifying batches of training data. Data augmentation is performed on the data in the training dataset to generate training data corresponding to the batch identifier. This training data is then written to a first data buffer in a graphics processing unit (GPU). The training data is used by the GPU to perform model training operations. Intermediate model state information written by the GPU is obtained from a second data buffer of the GPU. This intermediate model state information is generated during model training and is used for training interruption recovery or model performance analysis. Finally, the final trained model data written by the GPU is obtained from the second data buffer. This application, by introducing a GPU into an AI server, can offload necessary operations in AI model training from the CPU of the AI ​​server to the GPU for execution, thereby freeing up CPU processing resources, reducing processing latency, and improving CPU execution efficiency, thus improving the data processing efficiency of AI applications.

[0060] In one embodiment, the GPU's memory pre-allocated a first storage space, the first storage space including a first data structure for point-to-point direct memory access operations; the first data structure including a first data buffer and a first notification ring; the first notification ring is used to receive notifications of data writing from the peer and notifications of data processing completion from the peer. After writing the training data into the first data buffer in the graphics processing unit (GPU), the method further includes: According to the point-to-point direct memory access write operation, a first notification message of the training data is written to the address pointed to by the tail pointer of the first notification ring; the first notification message is used to notify the GPU that data has arrived in the first data buffer.

[0061] Inside an AI server, the GPU (AI accelerator card) and DPU reside on the same PCI bridge. (Reference) Figure 2 , Figure 2 The architecture showcases a collaborative approach to CPU scheduling, DPU data processing, and GPU computing hardware for AI acceleration, enabling efficient memory access and device communication between components through a memory management unit (MMU / IOMMU) and a PCI bridge.

[0062] (1) Core components and functions: CPU Core + MMU: The CPU is responsible for overall scheduling, while the MMU (Memory Management Unit) serves the CPU, translating the CPU's virtual address into a physical address through the process page table to enable reading and writing to main memory.

[0063] Main Memory: A shared memory resource of the system that stores programs and data. It is a common data pool for the CPU, DPU, and GPU.

[0064] DPU+Onboard Memory+IOMMU: The DPU focuses on data preprocessing (decompression, enhancement, etc.) and has its own onboard memory for local caching; the DPU's IOMMU (Input / Output Memory Management Unit) is responsible for address translation between the DPU and main memory, and between the DPU and the GPU, and supports direct memory access (DMA) and direct device-to-device (P2P) connections.

[0065] GPU / AI accelerator card + HBM + IOMMU: The GPU is responsible for computationally intensive AI training / inference, while HBM (High Bandwidth Memory) provides high-bandwidth, low-latency local storage. The GPU's IOMMU is also responsible for address translation between the GPU and main memory, and between the GPU and DPU, which improves efficiency in conjunction with P2P communication.

[0066] PCI bridge: A communication bridge connecting the DPU and GPU, providing hardware routing for device-to-device (P2P) direct connection.

[0067] (2) Data flow and collaboration logic: The core of the architecture is to reduce the main memory transfer and realize direct communication between devices, thereby accelerating the AI ​​workload: The CPU's routine interaction with main memory: The CPU uses the MMU and process page tables to translate virtual addresses into physical addresses and read and write programs and data in main memory (this is the basic process of general-purpose computing).

[0068] Direct peer-to-peer (P2P) communication between the DPU and GPU: The DPU and GPU communicate via a PCI bridge and their respective IOMMUs (as shown by the dotted arrows in the diagram). Data preprocessed by the DPU can be addressed via the IOMMU and directly written to the GPU's HBM through the PCI bridge (without needing to store it in main memory first). Results calculated by the GPU can also be directly transmitted back to the DPU's onboard memory or written to main memory via P2P. This design bypasses main memory intermediaries, significantly reducing data transmission latency and improving the overall efficiency of AI training / inference.

[0069] Efficient access between peripherals and main memory (DMA): When the DPU or GPU needs to access main memory, it uses its own IOMMU to translate addresses and read / write main memory in DMA mode (without full CPU intervention), freeing the CPU to do more core scheduling work.

[0070] refer to Figure 3 , Figure 3This demonstrates the implementation of Peer-to-Peer DMA between the AI ​​accelerator card's video memory and the DPU's onboard memory. Both the DPU and GPU's memory reserve some space for data structures used in Peer-to-Peer DMA operations, including a data buffer (DataBuffer) and a notification ring (Notification Ring), with the following functions: The DataBuffer is used to receive DMA Write data from the peer. For example, when the DPU transmits training data to the AI ​​accelerator card, the data is written directly to the AI ​​accelerator card's DataBuffer; conversely, when the AI ​​accelerator card returns results to the DPU, the data is written to the DPU's DataBuffer. It should be understood that the DataBuffer is a contiguous memory area with a defined starting address and capacity, ensuring that data is not written out of bounds and can be accurately read by the receiver.

[0071] The Notification Ring is used to receive notifications from the peer that data has been written and to receive notifications that data processing is complete. For example, when the DPU finishes writing data to the AI ​​accelerator card, it will write a "data written" notification (including data location, length, etc.) to the AI ​​accelerator card's Notification Ring; after the AI ​​accelerator card finishes processing the data, it will write a "processing complete" notification to the DPU's Notification Ring.

[0072] Both the DPU and GPU use their respective IOMMUs (Input / Output Memory Management Units) to translate logical memory addresses into actual physical memory addresses. Data written by the DPU to logical addresses can be correctly mapped to physical addresses in GPU memory (and vice versa), avoiding data errors caused by address confusion. The DPU and GPU drivers undergo necessary customization. The DPU driver initializes the DPU device through a series of configuration registers, informing it of various parameters, including: the GPU's DataBuffer start address, capacity, Notification Ring start address, Head pointer, and Tail pointer. The GPU driver initializes the GPU device through a series of configuration registers, informing it of various parameters, including: the DPU's DataBuffer start address, capacity, Notification Ring start address, Head pointer, and Tail pointer. It should be understood that the Head / Tail pointers are used to manage the read / write order of notifications (Head points to the latest pending notification, Tail points to the end of the processed notifications), preventing notification confusion.

[0073] refer to Figure 3When the DPU needs to write data to the AI ​​accelerator card, it first writes the data in its memory directly to the AI ​​accelerator card's DataBuffer via a Peer-to-Peer DMA Write operation. Then, it writes a notification message (containing the starting address, size, and unique sequence number of the data) to the address pointed to by the Tail Pointer of the AI ​​accelerator card's Notification Ring via a Peer-to-Peer DMA Write operation, informing the AI ​​accelerator card that data has arrived in the buffer and needs processing. Upon receiving the notification from the Notification Ring, the AI ​​accelerator card retrieves the relevant data from the DataBuffer and processes it. Then, the AI ​​accelerator card uses a Peer-to-Peer DMA Write operation to write a notification message to the address pointed to by the Tail Pointer of the DPU's Notification Ring, informing the AI ​​accelerator card that the data (uniquely identified by its sequence number) has been processed. The operation steps are similar when the AI ​​accelerator card needs to write data to the DPU.

[0074] In this embodiment, a first storage space is pre-allocated in the GPU's memory, specifically for P2P DMA interaction between the DPU and the GPU. This space contains two core data structures: a first data buffer, a dedicated memory area for the GPU to receive training data; and a first notification ring, a ring-shaped memory area used to transmit notification messages about the data transmission status. Its function can be to receive a notification from the DPU that "data has been written to the first data buffer" or to send a notification to the DPU that "data processing is complete" (completing bidirectional synchronization).

[0075] After the DPU writes the training data to the GPU's first data buffer via P2P DMA, it writes the first notification message of the training data to the address pointed to by the tail pointer of the first notification ring. The specific process is as follows: 1) Determining the notification write location: The first notification ring has two key pointers, the head pointer and the tail pointer. The tail pointer points to the address where the next new notification can be written; the head pointer points to the address of the earliest notification that the GPU has not yet processed. The DPU will find the address currently pointed to by the tail pointer of the first notification ring as the location to write the notification.

[0076] 2) Write the first notification message: The DPU writes the first notification message to this address. The message content may include: the starting address and length of the data in the first data buffer, the batch identifier corresponding to the data (to ensure that the GPU matches the training progress), and verification information (to ensure that the notification itself has not been tampered with).

[0077] It should be understood that the purpose of the first notification message is to proactively inform the GPU that new training data has arrived in the first data buffer. Because the GPU does not check the buffer for new data in real time when it is focused on computation, the notification ring allows the GPU to passively receive reminders without waiting idly or frequently querying, thus improving efficiency.

[0078] In this embodiment, after the DPU writes the training data to the GPU's first data buffer via P2P DMA, it sends a notification that the data has arrived (to the address pointed to by the tail pointer) to the GPU's first notification ring, enabling the GPU to efficiently detect new data and process it in a timely manner, thus solving the efficiency problem of data synchronization between devices.

[0079] Based on the above embodiments, the DPU has a second storage space pre-allocated in its memory. The second storage space includes a second data structure for point-to-point direct memory access operations. The second data structure includes a second data buffer and a second notification ring. The second notification ring is used to receive notifications of data writing from the peer and notifications of data processing completion from the peer. The step of obtaining the intermediate state information of the model written by the GPU from the second data buffer of the DPU includes: The second notification ring receives a second notification message containing the intermediate state information of the model sent by the GPU. According to the second notification message, the intermediate state information of the model written by the GPU is obtained from the second data buffer.

[0080] The DPU has a pre-planned second storage space in its memory, specifically for receiving data transferred by the GPU via P2P DMA. This second storage space contains two core data structures that jointly support data transmission and synchronization: The second data buffer is a contiguous memory area used to store intermediate model state information (such as weights and loss values ​​for a particular batch) written by the GPU via point-to-point direct memory access (P2P DMA). The second data buffer has a defined starting address and capacity (configured in advance) to avoid address confusion.

[0081] The second notification ring is a ring-shaped memory area used to transmit notification messages about data transfer status. It acts as a communication semaphore between the DPU and GPU. Its function is to receive notifications from the GPU that data has been written to the second data buffer, and to send notifications to the GPU that data processing is complete.

[0082] It should be understood that the second notification ring is key to achieving efficient collaboration between the DPU and GPU. Its core principle is to replace frequent queries with message passing, thus avoiding resource waste. Specifically, it involves two key roles: Notification message: Generated by the sender (GPU), it contains key information about the data, such as the starting address of the model's intermediate state information in the second data buffer, the data length, the corresponding training batch identifier, checksum, etc. (so that the DPU can accurately locate and verify the data).

[0083] Pointer Management: The notification ring maintains message order using a head pointer and a tail pointer. The tail pointer points to the next location where a new notification can be written (when the GPU sends a notification, it writes the message to the address pointed to by the tail pointer and updates the tail pointer); the head pointer points to the location of the earliest notification that the DPU has not yet processed (the DPU reads the notifications to be processed from here and updates the head pointer after processing). This design ensures that notifications are processed in a first-in, first-out order, preventing message corruption.

[0084] The process of retrieving the intermediate model state information written by the GPU from the DPU's second data buffer consists of two key steps, forming a complete synchronization link: First, the second notification message is received via the second notification ring. After the GPU writes the intermediate model state information to the DPU's second data buffer via P2PDMA, it immediately generates a second notification message (containing information such as data location and length). The GPU writes this notification message to the address pointed to by the tail pointer in the DPU's second notification ring via P2P DMA and updates the tail pointer (telling the DPU that a new notification has arrived). The DPU detects the arrival of a new notification message by monitoring the pointer changes in the second notification ring (without CPU intervention), thus triggering subsequent processing. Second, data is retrieved from the second data buffer based on the notification message. The DPU reads the second notification message pointed to by the head pointer in the second notification ring and parses out the specific location (starting address) and length of the intermediate model state information in the second data buffer. Based on the parsing result, the DPU directly reads the corresponding intermediate model state information (such as the currently trained weights, loss values, etc.) from its own second data buffer. After the read is complete, the DPU updates the head pointer of the second notification ring (marking that the notification has been processed) and can send feedback to the GPU through the notification ring to indicate that the processing is complete (ensuring that the GPU knows that the data has been received and can continue to send the next batch).

[0085] In this embodiment, the DPU first receives a notification from the GPU that data has been written, using a pre-allocated second storage space, and then accurately retrieves intermediate model state information from the buffer based on the notification. Through hardware-level data storage and notification loop synchronization, efficient and reliable bidirectional communication between the DPU and GPU is achieved, solving the problems of resource waste and inefficient synchronization in data interaction. This is a key mechanism to ensure reliable state transfer during model training.

[0086] Based on the above embodiments, obtaining the final trained model data written by the GPU from the second data buffer includes: The third notification message of the model data sent by the GPU is received through the second notification ring; According to the third notification message, the final trained model data written by the GPU is obtained from the second data buffer.

[0087] The process of the DPU acquiring the final model data consists of two steps, forming a strict send-confirmation mechanism to ensure that the data is not lost or corrupted: (1) Receiving the third notification message via the second notification ring: After the GPU completes all training steps and generates the final model data, it will directly transfer the data to the DPU's second data buffer via P2P DMA write operation (without going through the CPU, which is fast and reliable). After the data is written, the GPU immediately generates a third notification message, which may include: the starting address and total length of the final model data in the second data buffer; the data checksum (such as the CRC value, used by the DPU to verify whether the data is complete and has not been tampered with); and model-related metadata (such as the model name, training end time, etc.). The GPU writes the third notification message to the DPU's second notification ring via P2P DMA (the specific position is specified by the tail pointer of the notification ring to ensure that the messages are arranged in order) and updates the tail pointer. The DPU detects new notification messages by monitoring the pointer changes of the second notification ring (hardware-level awareness, no CPU intervention required) and prepares to read the data.

[0088] (2) Obtaining data from the second data buffer based on the third notification message: The DPU reads the third notification message from the second notification ring (reading from the position pointed to by the head pointer to ensure sequential processing), and parses out the specific position and checksum of the final model data in the second data buffer. Based on the parsing result, the DPU reads the complete final model data from its own second data buffer and verifies the data integrity through the checksum (if the verification fails, it can request the GPU to resend through the notification ring). After reading and verification, the DPU updates the head pointer of the second notification ring (marking that the notification has been processed) and can send an acknowledgment message to the GPU through the second notification ring indicating that the data has been successfully received (letting the GPU know that the final data has been safely delivered and related resources can be released). The DPU will perform subsequent processing on the obtained final model data (such as converting the format to ONNX and writing it to the distributed storage system for persistent storage).

[0089] In this embodiment, after generating the final training model data, the GPU writes it to the DPU's second data buffer via P2P DMA and sends a third notification message to the second notification ring. The DPU retrieves and verifies the data from the buffer based on the notification message, completing the reliable delivery of the final model. This mechanism, through a combination of hardware direct connection, message synchronization, and data verification, ensures both the integrity of the final model data and maximizes transmission efficiency.

[0090] Based on the above embodiments, the DPU and GPU are coupled to the same peripheral component interconnect standard PCI bridge; the DPU and GPU communicate through the PCI bridge; or, the DPU and GPU communicate through memory read and write operations.

[0091] It should be understood that a PCIe bridge is a hardware component on a computer motherboard that connects multiple PCIe devices (such as GPUs, DPUs, and network cards) and coordinates data transmission between them, acting as a transportation hub between devices. When two devices (DPU and GPU) are connected to the same PCIe bridge, their communication does not require multiple bridging stages or CPU intervention, resulting in shorter physical links and lower latency. This provides the hardware foundation for high-speed communication between the DPU and GPU, avoiding the latency and bandwidth loss caused by crossing multiple bridges.

[0092] The communication methods between the DPU and the GPU are divided into two categories, each adapted to different data transmission scenarios: 1) Communication via PCI Bridge (Direct Link Communication): Utilizing the native communication capabilities of the PCIe bus, the DPU and GPU exchange data directly through a PCI bridge, without relying on memory as an intermediary. The technical implementation is based on the PCIe peer-to-peer (P2P) communication protocol, allowing the DPU and GPU to directly access each other's PCIe address spaces. For example, when the DPU wants to send data to the GPU, it directly accesses the GPU's PCIe device address through the PCI bridge and writes the data to the GPU's video memory; similarly, when the GPU wants to send data to the DPU, it directly accesses the DPU's PCIe device address through the PCI bridge to complete the data transfer. This communication method is suitable for scenarios with high real-time requirements and moderate data volumes, such as the DPU transmitting preprocessed batch data to the GPU during training, or the GPU feeding back intermediate calculation results to the DPU. Its advantages include a short transmission path (only through the PCI bridge), low latency (no memory intermediary required), and no CPU resource consumption (completed directly by hardware).

[0093] 2) Communication via memory read / write operations (shared memory communication): Using shared memory as an intermediary, the DPU and GPU exchange data by reading and writing to the same memory region. The technical implementation is as follows: The system pre-allocates a shared memory region in memory and maps its address to the address spaces of the DPU and GPU (address translation is performed through the IOMMU to ensure both sides can access it); the DPU writes the data to be transmitted into the shared memory and then notifies the GPU that the data has been written; the GPU reads the data from the shared memory, processes it, writes the result back to the shared memory, and then notifies the DPU that the data has been processed. This communication method is suitable for scenarios with large amounts of data and relatively low real-time requirements, such as when the GPU writes the final model data to the shared memory after training, which is then read and stored by the DPU on an external device; or when the DPU transmits a large amount of configuration parameters and metadata to the GPU. Its advantages are that it can utilize the large capacity of memory to transmit massive amounts of data, and the implementation logic is relatively simple (it can be done through a standard memory read / write interface), and it has better compatibility (no additional support for the PCIe P2P protocol is required from the device).

[0094] In one embodiment, reference Figure 4 , Figure 4 This demonstrates two communication methods between the GPU memory and the DPU's onboard memory. Figure 4In this system, Main Memory (MS) is the host's general-purpose memory, accessible to devices such as the CPU and GPU, but its bandwidth and latency are limited resources. GPU Memory (GPU VRAM) is high-speed memory dedicated to the GPU, used to store intermediate data during computation. It offers fast access speeds, but direct interaction with other devices requires special design. The DPU (including onboard MEM and ARM) is a data processing unit with its own onboard memory (MEM) and ARM core, responsible for tasks such as data preprocessing (e.g., decompression and enhancement). The MMU / IOMMU is a memory management unit responsible for translating virtual addresses to physical addresses: the MMU serves the CPU, managing the address mapping of main memory; the IOMMU serves peripherals (such as the DPU and GPU), supporting address translation when peripherals directly access main memory or communicate with each other. The PCIe Switch is the PCIe bus switch, responsible for routing communication between peripherals (such as between the DPU and GPU).

[0095] exist Figure 4 In the text, the paths and characteristics of the two communication methods are as follows: (1) Indirect communication (via system main memory): The path is DPU onboard memory → system main memory → GPU video memory. The DPU and GPU do not interact directly and need to use system main memory as a relay station to transfer data. For example, the DPU first copies the data from the onboard MEM to the main memory, and then the GPU reads the data from the main memory into its own video memory. The reverse (GPU → DPU) is similar: the GPU first writes the data to the main memory, and then the DPU reads it into the onboard MEM.

[0096] (2) Direct Communication (Peer-to-Peer, P2P, via PCIe direct connection): The path is DPU onboard memory → GPU video memory (via PCIe Switch + IOMMU). The DPU and GPU communicate directly via the PCIe bus, using the IOMMU to achieve address translation and DMA technology to bypass the system main memory. For example, the DPU can directly write data from the onboard MEM to the GPU video memory via DMA; or the GPU can directly read data from the DPU's onboard MEM. The IOMMU is responsible for translating the virtual address of the DPU / GPU into a physical address that the other party can recognize, ensuring the legitimacy of direct access.

[0097] In this embodiment, the DPU and GPU are connected at the hardware level through the same PCIe bridge, supporting two communication methods: direct communication via the PCIe P2P protocol (low latency, small data) and indirect communication via shared memory (large capacity, high compatibility). This design allows both to flexibly choose the optimal data transmission path according to actual needs, which is the foundation for AI acceleration hardware collaboration, and the ultimate goal is to improve the efficiency and reliability of the entire system.

[0098] Based on the above embodiments, the data processing method for the AI ​​application further includes: Receive the second data loading instruction sent by the CPU; According to the second data loading instruction, the verification dataset is read from the storage device; Perform at least one of the following operations on the data in the verification dataset: data decoding, data augmentation, and data format conversion, to generate data to be verified. A point-to-point direct memory access write operation is performed to write the data to be verified into the first data buffer in the GPU; the data to be verified is used by the GPU to perform model verification operations.

[0099] When the model training reaches a point requiring validation (e.g., after a training epoch), the CPU sends a second data load instruction to the DPU, explicitly stating that a validation dataset needs to be loaded to prepare for model validation. Upon receiving this instruction, the DPU reads the validation dataset from a storage device (e.g., distributed storage, high-speed SSD). This validation dataset is new data independent of the training set and is used to evaluate whether the model has truly learned general patterns. It should be understood that data I / O (from storage to memory) is a time-consuming operation. The DPU is dedicated to handling data I / O and preprocessing, freeing up the CPU to focus on training logic scheduling.

[0100] The validation data read by the DPU may be raw and unprocessed (such as compressed images or encoded text). Therefore, the DPU needs to perform at least one preprocessing operation on it: data decoding, data augmentation, and data format conversion. Finally, the validation data is generated, which is standardized data that meets the requirements of GPU computing and can be directly used for model validation.

[0101] The DPU writes the data to be verified directly to the GPU's first data buffer via point-to-point direct memory access (P2P DMA) operations. The GPU reads the data from the first data buffer and performs verification operations using the currently trained model (such as forward propagation to obtain metrics like loss and accuracy on the validation set). These metrics are used to determine if the model is overfitting (e.g., high accuracy on the training set but low accuracy on the validation set), thereby guiding subsequent training strategies (e.g., adjusting the learning rate or stopping training early).

[0102] This application embodiment accelerates model verification across the entire chain by using a dedicated DPU for data processing and P2P direct connection to the GPU for transmission, thereby freeing up the CPU and improving training iteration efficiency.

[0103] To further analyze and explain the application of artificial intelligence model training, refer to... Figures 5-6 and the following embodiments.

[0104] refer to Figure 5 , Figure 5 This demonstrates the entire process of training an artificial intelligence model. The process begins with training initiation, proceeds through training preparation, multiple epoch iterations (including data processing and model computation), saving intermediate states, training validation, and finally saving model information and ending training. The core logic is the collaborative division of hardware tasks (CPU scheduling, DPU data processing, and GPU / accelerator card computing power support). The division of labor and key steps of each hardware module are as follows: (1) CPU: Overall scheduling of the training process: Triggers the training process and completes initialization (such as model structure loading and hyperparameter configuration). Controls the Epoch loop: Executes to start a new round of Epoch: Starts a complete traversal of the training dataset; Executes to obtain the next batch of training data: Extracts a batch of data from the dataset for single iteration training; Determines whether the Epoch is complete: If not, iterates to obtain the next batch; If complete, proceeds to the next step; Determines whether the training is complete: If not, it returns to start a new round of Epoch to continue iteration; If complete, it executes to save model information and then ends training.

[0105] (2) DPU: Data processing and intermediate state management: Perform data augmentation operations (such as image rotation / flipping, text synonym replacement, etc.) on the batch training data acquired by the CPU to generate more diverse training samples and improve the model's generalization ability. After each epoch is completed, save the model's intermediate state (such as the current weights, optimizer state, etc.) to enable training interruption recovery (avoid restarting training from the beginning).

[0106] (3) GPU / AI accelerator card: The core of computing power for computationally intensive tasks. It uses training data enhanced by DPU to perform forward propagation in the model to obtain prediction results. The loss is calculated based on the prediction results and the true labels, and then the gradient of each parameter is solved by backpropagation. The model weights are updated using the gradients to optimize the model performance. At an appropriate time (such as after each epoch), the model in training is validated to evaluate the model's performance on the validation set (to determine whether it is overfitting and whether the strategy needs to be adjusted).

[0107] refer to Figure 6 , Figure 6 The document demonstrates the internal processing instruction sequence interaction flow of an AI server used for AI model training applications, including: (1) Loading training data and single-batch training: CPU initiates an epoch: The CPU sends a command to start a training epoch, thus initiating the current training cycle. DPU preprocesses and transmits batches: reads the first batch of the training dataset from distributed storage (via file interface / block interface / SPDK interface); performs data preprocessing, such as decompression, decoding, enhancement, shuffling, and batch composition (generating a format that the GPU can directly compute); and writes the batch directly to GPU memory via Peer-to-Peer DMA Write (skipping CPU memory and reducing latency). GPU performs computations by reading batches from video memory and sequentially executing forward propagation, gradient calculation, back propagation, and weight update. The process of processing subsequent batches in a loop continues: DPU loading, preprocessing, P2P transmission, and GPU computation are repeated until all batches of the current epoch have been processed.

[0108] (2) Model intermediate state checkpoint (interruption recovery / performance analysis): CPU triggers checkpoint: When an epoch ends, the CPU initiates an instruction to save the intermediate state of the model; GPU transfers intermediate state to DPU: The GPU directly transmits intermediate state (such as current weights and optimizer state) to the DPU via P2P DMAWrite to complete the persistence of intermediate state (without CPU intervention).

[0109] (3) Validation phase (evaluating the model's generalization ability): CPU initiates validation: After checkpointing is complete, the CPU sends out a model validation command; The DPU preprocesses and transmits the validation batch: It reads the validation dataset batch from distributed storage and performs preprocessing (same as the training data process). The validation batch is then written to GPU memory via P2P DMA. The GPU performs validation calculations: it reads the validation batch and completes the validation process.

[0110] (4) Training completion and final model saving: The CPU determines when training is complete: when all epochs have ended, the CPU sends an instruction to save the final model; GPU transfers final model to DPU: The GPU directly transfers the final trained model (complete weights, structure, etc.) to the DPU via P2P DMAWrite to complete model persistence.

[0111] Based on the above embodiments, the data processing method for the AI ​​application further includes: Receive the first read instruction for AI inference model data sent by the CPU; According to the first read instruction, the AI ​​inference model data is read from external storage; The AI ​​inference model data is sent to the CPU; the AI ​​inference model data is used by the CPU to perform model preprocessing operations.

[0112] The CPU sends the first read instruction to the DPU, explicitly stating that it needs to read AI inference model data from external storage. Upon receiving the first read instruction, the DPU reads the AI ​​inference model data from external storage. It should be understood that external storage I / O operations (such as disk reads and network storage fetches) are typically time-consuming and resource-intensive. Having the DPU handle these tasks allows the CPU to avoid being blocked while waiting for data reads and instead execute more critical computational logic. The DPU then sends the read AI inference model data to the CPU. After receiving the data, the CPU performs model preprocessing operations: Format conversion: Convert the model format during the training phase to the format optimized for inference; Data quantization: Quantize model parameters from FP32 (high precision but resource-intensive) to INT8 (low precision but less computational power, suitable for inference). Structural trimming: Simplify the model structure to accelerate reasoning.

[0113] In this embodiment, the DPU handles the IO-intensive task of reading AI inference model data from external storage, preventing the CPU from being blocked by time-consuming storage IO operations. This allows the CPU to focus on model preprocessing (such as format conversion, quantization, and structure pruning), significantly improving CPU computing power utilization. On the other hand, as dedicated hardware for data IO, the DPU's efficient storage access capabilities can accelerate model data loading and shorten the preparation cycle before inference. Simultaneously, the CPU preprocesses the model data transmitted by the DPU, optimizing the model to a format and accuracy more suitable for subsequent GPU / AI accelerator card inference. This reduces format adaptation and computational redundancy in the subsequent inference process, laying the foundation for low latency and high throughput in the overall AI inference process.

[0114] Based on the above embodiments, after sending the AI ​​inference model data to the CPU, the method further includes: The system receives a second read instruction for pre-computed features sent by the CPU; the pre-computed features are feature data that are pre-computed and stored for direct use by the AI ​​inference model. According to the second read instruction, the pre-computed features are read from the external storage; Perform a point-to-point direct memory access write operation to write the pre-computed features into the first data buffer in the GPU.

[0115] It should be understood that pre-computed features are feature data that is calculated and stored in advance before inference (rather than being extracted from the original data during inference). For example, in image inference, general features (such as edge and texture features) can be calculated in advance to avoid repeatedly performing convolution extraction on the original image each time inference occurs; in recommendation systems, statistical features of users' historical behavior (such as click frequency and preference tags) can be calculated in advance and directly called during inference. These features are the input raw materials for AI inference models, and the purpose of pre-computation is to reduce redundant calculations during inference and shorten the time consumed per inference.

[0116] The CPU sends a second read instruction to the DPU, explicitly specifying the need to load pre-computed features from external storage. For example, the CPU decides when to load pre-computed features based on the inference task requirements, ensuring that the features match the model and input data in a timely manner. After receiving the second read instruction, the DPU reads the pre-computed features from external storage. It should be understood that pre-computed features are usually large in volume (e.g., behavioral features of millions of users may reach GB levels). Reading from storage is an I / O-intensive task. As dedicated data processing hardware, the DPU excels at efficiently performing storage I / O operations (e.g., supporting multi-channel parallel reading and hardware-level cache optimization), which can prevent the CPU from being blocked while waiting for I / O, allowing the CPU to focus on inference logic scheduling.

[0117] The DPU writes pre-computed features directly to the GPU's first data buffer via point-to-point direct memory access (P2P DMA) operations, skipping the CPU memory transfer: the traditional path is DPU→CPU memory→GPU memory, which requires two data copies and has high latency; while P2P DMA allows the DPU and GPU to communicate directly, reducing data transfer latency, and is especially suitable for large-batch inference scenarios (such as processing 1024 user-requested features at once).

[0118] The pre-computed features in this application avoid repetitive feature extraction during inference, significantly reducing computational redundancy; the DPU undertakes the IO-intensive task of reading features from external storage, effectively offloading the CPU burden and allowing it to focus on inference logic scheduling; and the point-to-point direct memory access (P2P DMA) method of directly transferring to the GPU buffer skips the CPU memory transfer, reduces the number of data copies and transmission latency, and allows the pre-computed features to quickly reach the GPU computing core, ultimately achieving low latency and high throughput in the AI ​​inference process, significantly improving overall inference efficiency.

[0119] Based on the above embodiments, the data processing method for the AI ​​application further includes: Receive the data to be inferred sent by the front-end application; Feature extraction is performed on the data to be inferred to obtain feature data; A point-to-point direct memory access write operation is performed to write the feature data into the first data buffer in the GPU; the feature data is used by the GPU to perform inference operations.

[0120] The front-end application is the initiator of the inference request, such as mobile apps and web services. The data to be inferred is the raw input (such as images, text, voice, etc.).

[0121] The front-end application sends the data to be inferred to the DPU. It should be understood that the DPU receives the data directly, rather than the CPU. The reason is that the front-end data is usually unstructured / semi-structured (such as binary streams of images or strings of text) and may be accompanied by high-frequency, small-batch requests (such as thousands of image recognitions per second). The DPU has a dedicated network interface and data reception optimization (such as hardware-level protocol parsing and flow control), which can efficiently handle the front-end data input and avoid the CPU being occupied by the miscellaneous tasks of data reception.

[0122] The data to be inferred is in its raw form and cannot be directly used by the AI ​​model (for example, the raw image is a pixel matrix, while the model needs feature vectors after convolution and pooling). Therefore, the DPU needs to perform feature extraction operations. For example, for images, this may include resizing, normalization (mapping pixel values ​​to 0-1), edge detection, basic convolution, etc., to generate image features that the model can recognize; for text, this may include word segmentation, word vector conversion, context encoding, etc., to generate text feature sequences. These operations are pre-computation calculations for model inference. The advantage of having the DPU perform these operations is that the DPU has parallel data processing capabilities (such as a multi-core SIMD architecture), which is faster than the CPU's serial processing, and it avoids consuming GPU computing resources (the GPU should focus on core model inference).

[0123] The DPU writes extracted feature data directly to the GPU's first data buffer via point-to-point direct memory access (P2P DMA) operations. This step skips the CPU memory transfer: the traditional path is DPU → CPU memory → GPU memory, which requires two data copies and has high latency (especially for large features, such as batch transmission of 1024-dimensional image features). P2P DMA allows the DPU and GPU to communicate directly, reducing data transmission latency and ensuring that feature data arrives at the GPU quickly and is immediately used by the model for inference calculations.

[0124] In this embodiment, the DPU directly receives the front-end data to be inferred and completes feature extraction. This leverages its data processing expertise, reduces the non-core computing burden on the CPU and GPU, and avoids the raw data consuming GPU computing power. Furthermore, the point-to-point direct memory access and direct transmission to the GPU buffer bypasses the CPU-memory transfer, significantly reducing data transmission latency and copy overhead. This allows feature data to quickly reach the GPU for inference computation, ultimately significantly improving the end-to-end efficiency of AI inference.

[0125] To further analyze and explain the application of artificial intelligence model training, refer to... Figures 7-8 and the following embodiments.

[0126] refer to Figure 7 , Figure 7 It demonstrates the entire process of AI model inference, with the core being the efficient completion of the inference loop from front-end request to result return through hardware collaboration between general-purpose servers and AI servers (DPU preprocessing, CPU model management, and GPU computing power acceleration).

[0127] Specifically, the front-end application (general-purpose server) initiates the inference request (such as e-commerce recommendations or image recognition applications), sending batch inference requests containing a batch of data. The feature repository stores pre-computed feature data (such as user historical behavior features and general image features) to avoid redundant calculations during inference and improve efficiency. The model repository stores trained AI models (including structure and weight parameters), serving as the knowledge carrier for inference.

[0128] The DPU is responsible for data preparation before inference, reducing the burden on the CPU / GPU: it reads pre-generated features from the feature repository, combines them with the front-end batch requests, preprocesses the raw data (such as image normalization and text vectorization), and generates a format that the model can directly input.

[0129] The CPU is responsible for loading and optimizing the model, paving the way for GPU computing: it reads the trained model from the model repository, converts the model into a more efficient inference format (such as ONNX), and quantizes the parameters (such as FP32→INT8) to reduce the amount of computation and improve speed.

[0130] The GPU / accelerator card is the core of computing power, responsible for inference processing: it receives the optimized model processed by the CPU and the input data processed by the DPU, performs forward propagation of the model (inference only requires calculating the result using the model, without backpropagation), and quickly obtains intermediate results (such as classification probabilities and regression values). It then performs business-oriented processing on the GPU's intermediate results (such as converting classification probabilities to labels and filtering detection boxes), generates a final result that is understandable to humans / systems, and sends the result back to the requesting frontend to complete the inference response.

[0131] refer toFigure 8 , Figure 8 The document demonstrates the interaction flow of the processing instruction sequence within an AI server for AI model inference applications, including: (1) Inference Preparation (Model Loading, Format Optimization, Feature Preloading): The DPU reads the structure and weights of the AI ​​inference model from independent external storage and passes them to the CPU; the CPU reads the model again (ensuring consistency). The CPU performs model format conversion and data type quantization (e.g., converting the PyTorch model to ONNX format, or quantizing FP32 precision to INT8), the purpose of which is to make the model more suitable for efficient GPU inference (reducing computation and improving speed). After conversion, the CPU writes the model directly to GPU memory via Peer-to-Peer DMA Write (P2P DMA, direct memory access between devices, skipping CPU memory transfer) (avoiding redundant copying of traditional CPU memory → GPU memory, reducing latency). The DPU reads necessary pre-computed features (e.g., general image enhancement features, user behavior statistical features) from independent external storage, and also writes the pre-computed features directly to GPU memory via Peer-to-Peer DMA Write (coexisting with the model in GPU memory, and can be directly called during inference). After completion, the DPU reports to the CPU that inference preparation is complete (Ready).

[0132] (2) Inference Execution (Data Transfer and GPU Computation): The front-end application submits inference data (such as a batch of images to be classified or user requests to be recommended) to the DPU through a RESTful interface. The DPU preprocesses the inference data (such as image normalization, text vectorization, and batching to make the data meet the model input requirements). After preprocessing, the DPU writes the batch of data to be inferred directly to the GPU memory through Peer-to-Peer DMAWrite (skipping CPU memory again to ensure high-speed transmission). The GPU reads the optimized model pre-computed features to be inferred from the memory and performs forward propagation (the core computation of inference: using the model to predict the input data, no backpropagation is needed because inference only needs to calculate the result and does not update the model).

[0133] (3) Result Return (Post-processing and Front-end Response): After the GPU completes inference, it sends the original inference results (such as classification probabilities and regression values) back to the CPU through D2H (Device to Host) operations. The CPU performs post-processing on the results (such as converting classification probabilities into label names and filtering invalid detection boxes) to generate results that are understandable to the business. The DPU (or CPU coordination) returns the final processed results to the front-end application, completing the entire inference process.

[0134] The data processing apparatus for AI applications provided in this application is described below. The data processing apparatus for AI applications described below can be referred to in correspondence with the data processing method for AI applications described above.

[0135] refer to Figure 9 The data processing device for AI applications provided in this application includes a data reading module 901, a data enhancement module 902, a first writing module 903, and a second writing module 904.

[0136] The data reading module 901 is used to read the training dataset from the storage device according to the first data loading instruction sent by the central processing unit (CPU); the first data loading instruction includes a batch identifier for identifying the batch of training data. Data augmentation module 902 is used to augment the data in the training dataset, generate training data corresponding to the batch identifier, and write the training data into a first data buffer in the graphics processing unit (GPU); the training data is used by the GPU to perform model training operations. The first writing module 903 is used to obtain the model intermediate state information written by the GPU from the second data buffer of the DPU; the model intermediate state information is generated during the model training process and is used for training interruption recovery or model performance analysis. The second writing module 904 is used to obtain the final trained model data written by the GPU from the second data buffer.

[0137] The data processing apparatus for AI applications provided in this application reads a training dataset from a storage device according to a first data loading instruction sent by a central processing unit (CPU). The first data loading instruction includes a batch identifier for identifying batches of training data. Data augmentation is performed on the data in the training dataset to generate training data corresponding to the batch identifier. The training data is then written to a first data buffer in a graphics processing unit (GPU). The training data is used by the GPU to perform model training operations. Intermediate model state information written by the GPU is obtained from a second data buffer of the GPU. This intermediate model state information is generated during model training and is used for training interruption recovery or model performance analysis. Finally, the final trained model data written by the GPU is obtained from the second data buffer. By introducing a GPU into an AI server, this application can offload necessary operations in AI model training from the CPU of the AI ​​server to the GPU for execution, thereby freeing up CPU processing resources, reducing processing latency, and improving CPU execution efficiency, thus improving the data processing efficiency of AI applications.

[0138] In one embodiment, the GPU's memory pre-allocated a first storage space, the first storage space including a first data structure for point-to-point direct memory access operations; the first data structure including a first data buffer and a first notification ring; the first notification ring being used to receive notifications of data writing from the peer and notifications of data processing completion from the peer; the first write module 903 is further used for: According to the point-to-point direct memory access write operation, a first notification message of the training data is written to the address pointed to by the tail pointer of the first notification ring; the first notification message is used to notify the GPU that data has arrived in the first data buffer.

[0139] In one embodiment, the DPU has a second storage space pre-allocated in its memory. This second storage space includes a second data structure for point-to-point direct memory access operations. The second data structure includes a second data buffer and a second notification ring. The second notification ring is used to receive notifications of data writing from the peer and notifications of data processing completion from the peer. The first write module 903 is further configured to: The second notification ring receives a second notification message containing the intermediate state information of the model sent by the GPU. According to the second notification message, the intermediate state information of the model written by the GPU is obtained from the second data buffer.

[0140] In one embodiment, the second writing module 904 is further configured to: The third notification message of the model data sent by the GPU is received through the second notification ring; According to the third notification message, the final trained model data written by the GPU is obtained from the second data buffer.

[0141] In one embodiment, the DPU and the GPU are coupled to the same peripheral component interconnect standard PCI bridge; the DPU and the GPU communicate through the PCI bridge; or, the DPU and the GPU communicate through memory read and write operations.

[0142] In one embodiment, the data processing device for the AI ​​application further includes a model inference module, used for: Receive the second data loading instruction sent by the CPU; According to the second data loading instruction, the verification dataset is read from the storage device; Perform at least one of the following operations on the data in the verification dataset: data decoding, data augmentation, and data format conversion, to generate data to be verified. A point-to-point direct memory access write operation is performed to write the data to be verified into the first data buffer in the GPU; the data to be verified is used by the GPU to perform model verification operations.

[0143] In one embodiment, the model inference module is further configured to: Receive the first read instruction for AI inference model data sent by the CPU; According to the first read instruction, the AI ​​inference model data is read from external storage; The AI ​​inference model data is sent to the CPU; the AI ​​inference model data is used by the CPU to perform model preprocessing operations.

[0144] In one embodiment, the model inference module is further configured to: The system receives a second read instruction for pre-computed features sent by the CPU; the pre-computed features are feature data that are pre-computed and stored for direct use by the AI ​​inference model. According to the second read instruction, the pre-computed features are read from the external storage; Perform a point-to-point direct memory access write operation to write the pre-computed features into the first data buffer in the GPU.

[0145] In one embodiment, the model inference module is further configured to: Receive the data to be inferred sent by the front-end application; Feature extraction is performed on the data to be inferred to obtain feature data; A point-to-point direct memory access write operation is performed to write the feature data into the first data buffer in the GPU; the feature data is used by the GPU to perform inference operations.

[0146] Figure 10 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 10As shown, the electronic device may include a processor 1010, a communications interface 1020, a memory 1030, and a communication bus 1040, wherein the processor 1010, communications interface 1020, and memory 1030 communicate with each other via the communication bus 1040. The processor 1010 can call logical instructions in the memory 1030 to execute a data processing method for an AI application. This method includes: reading a training dataset from a storage device according to a first data loading instruction sent by the central processing unit (CPU); the first data loading instruction includes a batch identifier for identifying a batch of training data; performing data augmentation on the data in the training dataset to generate training data corresponding to the batch identifier; writing the training data into a first data buffer in the graphics processing unit (GPU); the training data is used by the GPU to perform model training operations; obtaining intermediate model state information written by the GPU from a second data buffer of the GPU; the intermediate model state information is generated during model training and is used for training interrupt recovery or model performance analysis; and obtaining the final trained model data written by the GPU from the second data buffer.

[0147] Furthermore, the logical instructions in the aforementioned memory 1030 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0148] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data processing method for AI applications provided by the above methods. The method includes: reading a training dataset from a storage device according to a first data loading instruction sent by a central processing unit (CPU); the first data loading instruction includes a batch identifier for identifying a batch of training data; performing data augmentation on the data in the training dataset to generate training data corresponding to the batch identifier; writing the training data into a first data buffer in a graphics processing unit (GPU); the training data is used for the GPU to perform model training operations; obtaining intermediate model state information written by the GPU from a second data buffer of the GPU; the intermediate model state information is generated during model training and is used for training interruption recovery or model performance analysis; and obtaining the final trained model data written by the GPU from the second data buffer.

[0149] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a data processing method for performing the AI ​​applications provided by the methods described above. The method includes: reading a training dataset from a storage device according to a first data loading instruction sent by a central processing unit (CPU); the first data loading instruction includes a batch identifier for identifying a batch of training data; performing data augmentation on the data in the training dataset to generate training data corresponding to the batch identifier; writing the training data into a first data buffer in a graphics processing unit (GPU); the training data is used for the GPU to perform model training operations; obtaining intermediate model state information written by the GPU from a second data buffer of the GPU; the intermediate model state information is generated during model training and is used for training interruption recovery or model performance analysis; and obtaining the final trained model data written by the GPU from the second data buffer.

[0150] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0151] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data processing method for an AI application, characterized in that, The data processing method for the AI ​​application, applied to the data processing unit (DPU), includes: According to the first data loading instruction sent by the central processing unit (CPU), the training dataset is read from the storage device; the first data loading instruction includes a batch identifier for identifying the batch of training data. Data augmentation is performed on the data in the training dataset to generate training data corresponding to the batch identifier, and the training data is written into the first data buffer in the graphics processing unit (GPU); the training data is used by the GPU to perform model training operations. The intermediate state information of the model written by the GPU is obtained from the second data buffer of the DPU; the intermediate state information of the model is generated during the model training process and is used for training interruption recovery or model performance analysis. Obtain the final trained model data written by the GPU from the second data buffer.

2. The data processing method for AI applications according to claim 1, characterized in that, The GPU has a first storage space pre-allocated in its memory. The first storage space includes a first data structure for point-to-point direct memory access operations. The first data structure includes a first data buffer and a first notification ring. The first notification ring is used to receive notifications of data writing from the peer and notifications of data processing completion from the peer. After writing the training data into the first data buffer in the graphics processing unit (GPU), the method further includes: According to the point-to-point direct memory access write operation, the first notification message of the data to be trained is written to the address pointed to by the tail pointer of the first notification ring; The first notification message is used to notify the GPU that data has arrived in the first data buffer.

3. The data processing method for AI applications according to claim 1, characterized in that, The DPU has a second storage space pre-allocated in its memory. The second storage space includes a second data structure for point-to-point direct memory access operations. The second data structure includes a second data buffer and a second notification ring. The second notification ring is used to receive notifications of data writing from the peer and notifications of data processing completion from the peer. The step of obtaining the intermediate state information of the model written by the GPU from the second data buffer of the DPU includes: The second notification ring receives a second notification message containing the intermediate state information of the model sent by the GPU. According to the second notification message, the intermediate state information of the model written by the GPU is obtained from the second data buffer.

4. The data processing method for AI applications according to claim 3, characterized in that, The step of retrieving the final trained model data written by the GPU from the second data buffer includes: The third notification message of the model data sent by the GPU is received through the second notification ring; According to the third notification message, the final trained model data written by the GPU is obtained from the second data buffer.

5. The data processing method for AI applications according to any one of claims 1 to 4, characterized in that, The DPU and the GPU are coupled to the same peripheral component interconnect standard PCI bridge; the DPU and the GPU communicate through the PCI bridge; or, the DPU and the GPU communicate through memory read and write operations.

6. The data processing method for AI applications according to claim 1, characterized in that, The data processing method for the AI ​​application also includes: Receive the second data loading instruction sent by the CPU; According to the second data loading instruction, the verification dataset is read from the storage device; Perform at least one of the following operations on the data in the verification dataset: data decoding, data augmentation, and data format conversion, to generate data to be verified. A point-to-point direct memory access write operation is performed to write the data to be verified into the first data buffer in the GPU; the data to be verified is used by the GPU to perform model verification operations.

7. The data processing method for AI applications according to claim 1, characterized in that, The data processing method for the AI ​​application also includes: Receive the first read instruction for AI inference model data sent by the CPU; According to the first read instruction, the AI ​​inference model data is read from external storage; The AI ​​inference model data is sent to the CPU; the AI ​​inference model data is used by the CPU to perform model preprocessing operations.

8. The data processing method for AI applications according to claim 7, characterized in that, After sending the AI ​​inference model data to the CPU, the method further includes: The system receives a second read instruction for pre-computed features sent by the CPU; the pre-computed features are feature data that are pre-computed and stored for direct use by the AI ​​inference model. According to the second read instruction, the pre-computed features are read from the external storage; Perform a point-to-point direct memory access write operation to write the pre-computed features into the first data buffer in the GPU.

9. The data processing method for AI applications according to claim 7 or 8, characterized in that, The data processing method for the AI ​​application also includes: Receive the data to be inferred sent by the front-end application; Feature extraction is performed on the data to be inferred to obtain feature data; A point-to-point direct memory access write operation is performed to write the feature data into the first data buffer in the GPU; the feature data is used by the GPU to perform inference operations.

10. A data processing device for AI applications, characterized in that, include: The data reading module is used to read the training dataset from the storage device according to the first data loading instruction sent by the central processing unit (CPU); the first data loading instruction includes a batch identifier for identifying the batch of training data. The data augmentation module is used to augment the data in the training dataset, generate training data corresponding to the batch identifier, and write the training data into the first data buffer in the graphics processor GPU. The data to be trained is used by the GPU to perform model training operations; The first writing module is used to obtain the intermediate state information of the model written by the GPU from the second data buffer of the DPU; the intermediate state information of the model is generated during the model training process and is used for training interruption recovery or model performance analysis. The second write module is used to obtain the final trained model data written by the GPU from the second data buffer.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the data processing method for the AI ​​application as described in any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data processing method of the AI ​​application as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the data processing method of the AI ​​application as described in any one of claims 1 to 9.

Images