A data processing method and related device

Abstract

A data processing method and related device, wherein, when executing an AI task, the data processor first performs preprocessing such as data decoding and random transformation on the data, and the image processor then performs the AI ​​task based on the preprocessing result, and performs reasoning or training on the neural network. The preprocessing of the data to be processed by the data processor and the execution of the AI ​​task by the image processor based on the preprocessing result can be performed in parallel to improve the efficiency of reasoning or training. In addition, the preprocessing calculation of the data by the data processor can reduce the computing load of the central processing unit and increase the stability of the computing system.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method and related devices. Background Art

[0002] Artificial Intelligence (AI) technology is a technology that uses digital computers to simulate and extend human perception of the environment and acquire knowledge and information. AI technology usually uses neural networks to process data, and infers based on the acquired data to obtain inference results. Usually, before using a neural network, a large amount of data is needed to train the neural network.

[0003] In order to improve the efficiency of reasoning or calculation, the data used can be preprocessed before using the neural network for reasoning or training. For example, taking the use of image data to train the neural network as an example, currently, when training the neural network, the central processing unit can perform preprocessing calculations such as image decoding and random transformation on the image data, and then transmit the preprocessed data to the image processor, which uses the preprocessed data to train the neural network.

[0004] In actual use, the pre-processing calculation of data by the central processing unit may easily cause the computing load of the central processing unit to be too high, resulting in instability of the computing system. Summary of the invention

[0005] The present application provides a data processing method and related devices. When executing AI tasks, the data processor performs pre-processing calculations on the data, which can reduce the computing load of the central processing unit and help improve the stability of the computing system.

[0006] In a first aspect, the present application provides a data processing method, which can be applied to a computing system, which may include a central processing unit, an image processor, and a data processor. The data processor can pre-process the data under the control of the central processing unit to obtain a pre-processing result, and transmit the pre-processing result to the image processor. The image processor can perform AI tasks based on the pre-processing result under the control of the central processing unit.

[0007] The present application adds a data processor to the computing system. When executing AI tasks, the data processor first performs data decoding, random transformation and other preprocessing on the data, and then the image processor performs the AI ​​task based on the preprocessing results, and performs reasoning or training on the neural network. The preprocessing of the data by the data processor and the process of the image processor performing the AI ​​task based on the preprocessing results can be performed in parallel to improve the efficiency of reasoning or training. For example, the data processor preprocesses the first batch of data, transmits the preprocessing results of the first batch of data to the image processor, and the image processor performs the AI ​​task based on the preprocessing results of the first batch of data. When the image processor performs the AI ​​task based on the preprocessing results of the first batch of data, the data processor can preprocess the second batch of data to achieve data preprocessing and AI tasks simultaneously. At the same time, compared with the preprocessing of data by the central processing unit, the preprocessing calculation of data by the data processor can reduce the computing load of the central processing unit and increase the stability of the computing system.

[0008] Furthermore, since the data processor has strong data processing capabilities, the data preprocessing rate can be increased, so that the data preprocessing rate can be higher than the inference efficiency or training efficiency when the image processor uses the preprocessing results to perform AI tasks. Therefore, before the image processor completes processing the preprocessing results of the current batch of data, the next batch of data available to the image processor is preprocessed, and the preprocessing results of the next batch of data are transmitted to the image processor in advance. This can reduce the idle waiting time of the image processor and maintain continuous data input to ensure that the image processor has a high time utilization rate.

[0009] In an optional implementation, the image processor and the data processor may be different chips on the same board. In another optional implementation, the image processor and the data processor may be chips on different boards.

[0010] In an optional implementation, preprocessing the data may include one or more of the following processes: decoding the data, randomly transforming the data, scaling the data, cropping the data, flipping the data, or converting the data into a color space.

[0011] In an optional implementation, the data processor may include multiple data preprocessing units, and the data may be multiple. The data processor may preprocess the multiple data in parallel through the multiple data preprocessing units to obtain preprocessing results corresponding to the multiple data.

[0012] By preprocessing multiple data in parallel using multiple data preprocessing units, the data processing efficiency can be further improved.

[0013] In an optional implementation, the data processor may include a plurality of preprocessing kernels connected in sequence, and different preprocessing kernels are used to perform different preprocessing operations on the data. Each preprocessing kernel may include one or more data preprocessing units. Exemplarily, the data processor may include a first preprocessing kernel, a second preprocessing kernel, and a third preprocessing kernel connected in sequence, that is, the data passes through the first preprocessing kernel, the second preprocessing kernel, and the third preprocessing kernel in sequence. Among them, the first preprocessing kernel is used to perform image scaling on the data, the second preprocessing kernel is used to perform image cropping on the data, and the third preprocessing kernel is used to perform image flipping on the data.

[0014] Compared with the preprocessing of data by the central processing unit, the central processing unit usually loads the data into the memory of the central processing unit during the preprocessing of data, and the central processing unit continuously reads data from the memory, calculates the read data, and then saves the intermediate calculation results to the memory, waits for the next calculation instruction, and then reads the intermediate calculation results of the previous calculation from the memory for the next calculation until the preprocessing process is completed. This process requires multiple interactions between the central processing unit and the memory, which makes the data preprocessing rate low, even lower than the reasoning efficiency or training efficiency when the image processor uses the preprocessing results to perform AI tasks, causing the image processor to wait for idle time, reducing the utilization of the image processor in the time dimension, thereby extending the execution time of the entire AI task. In this application, the data is preprocessed by the data processor. Since the data processor has good software and hardware programmability, multiple sequentially connected preprocessing kernels can be generated by programming, and different preprocessing kernels are used to perform different preprocessing operations on the data. For example, a data can sequentially pass through a first preprocessing kernel that performs image scaling on the data, a second preprocessing kernel that performs image cropping on the data, and a third preprocessing kernel that performs image flipping on the data, so that the data processor does not need to store intermediate calculation results in the DPU memory during the preprocessing process, and then repeatedly read the intermediate calculation results from the DPU memory for the next calculation. Therefore, there is no need for multiple interactions between the data processor and the DPU memory, which can improve the data preprocessing rate and make the data preprocessing rate higher than the inference efficiency or training efficiency when the image processor uses the preprocessing results to perform AI tasks. This can further reduce the waiting idleness of the image processor and maintain continuous data input to ensure that the image processor has a high time utilization rate.

[0015] In an optional implementation, after obtaining the preprocessing result, the data processor may save the preprocessing result to a first memory address in the data processor. The image processor may obtain the first memory address in the data processor for storing the preprocessing result, and then read the preprocessing result from the first memory address in the data processor by DMA.

[0016] In another optional implementation, after obtaining the preprocessing result, the data processor can obtain the target video memory address corresponding to the preprocessing result, and save the preprocessing result to the storage unit corresponding to the target video memory address through DMA. The target video memory address is the address of the storage unit in the video memory of the image processor for storing the preprocessing result.

[0017] In the above implementation, the preprocessing results can be transmitted between the data processor and the image processor in a direct manner, bypassing the central processor and the memory of the central processor, which can not only reduce the burden of the central processor, but also improve the data transmission rate. The data processor directly saves the preprocessing results to the video memory of the image processor through DMA, which is faster than the image processor reading the preprocessing results from the data processor, so the data transmission rate can be further improved.

[0018] In another optional implementation, after obtaining the preprocessing result, the data processor can obtain the second memory address corresponding to the preprocessing result, and save the preprocessing result to the storage unit corresponding to the second memory address by DMA. The second memory address is the address of the storage unit in the memory of the central processing unit for storing the preprocessing result. The central processing unit can save the preprocessing result stored in the storage unit corresponding to the second memory address to the storage unit corresponding to the target video memory address of the preprocessing result by DMA. The target video memory address is the address of the storage unit in the video memory of the image processor for storing the preprocessing result.

[0019] The above implementation can be applied to the scenario where the data processor and the image processor do not support direct communication, and the central processor can ensure that the preprocessing results are accurately transmitted to the image processor.

[0020] In a second aspect, the present application provides a data processing method, which can be executed by a data processor in a computing system, and the computing system can include a central processing unit, an image processor, and a data processor. The data processing method can include:

[0021] Under the control of the central processing unit, the data is preprocessed to obtain preprocessing results;

[0022] The preprocessing results are transmitted to the image processor; the image processor is used to perform AI tasks based on the preprocessing results.

[0023] In an optional implementation, the data processor includes multiple data preprocessing units, and there are multiple data. The data processor can preprocess the multiple data in parallel through the multiple data preprocessing units to obtain preprocessing results corresponding to the multiple data respectively.

[0024] In an optional implementation, after obtaining the preprocessing result, the data processor can obtain the target video memory address corresponding to the preprocessing result, and save the preprocessing result to the storage unit corresponding to the target video memory address through DMA. The target video memory address is the address of the storage unit in the video memory of the image processor for storing the preprocessing result.

[0025] In an optional implementation, after obtaining the preprocessing result, the data processor can obtain a second memory address corresponding to the preprocessing result; the second memory address is the address of a storage unit in the memory of the central processing unit for storing the preprocessing result. The data processor can save the preprocessing result to the storage unit corresponding to the second memory address through DMA, and save the preprocessing result to the storage unit corresponding to the target video memory address of the preprocessing result through the central processing unit; the target video memory address is the address of the storage unit in the video memory of the image processor for storing the preprocessing result.

[0026] In a third aspect, the present application provides a data processing unit, which can be applied to a data processor in a computing system, and the computing system may include a central processing unit, an image processor, and a data processor. The data processing unit may include:

[0027] The data preprocessing module is used to preprocess the data under the control of the central processing unit to obtain the preprocessing results;

[0028] The data transfer module is used to transmit the preprocessing results to the image processor; the image processor is used to perform AI tasks based on the preprocessing results.

[0029] In an optional implementation, the data processor includes a plurality of data preprocessing units; the data is a plurality of data; and the data preprocessing module can be specifically used for:

[0030] The multiple data are preprocessed in parallel by multiple data preprocessing units to obtain preprocessing results corresponding to the multiple data.

[0031] In an optional implementation, the data transfer module is specifically used to:

[0032] Obtaining a target video memory address corresponding to the preprocessing result; the target video memory address is an address of a storage unit in the video memory of the image processor for storing the preprocessing result;

[0033] The preprocessing result is saved to the storage unit corresponding to the target video memory address through DMA.

[0034] In another optional implementation, the data transfer module is specifically used to:

[0035] Acquire a second memory address corresponding to the preprocessing result; the second memory address is an address of a storage unit in the memory of the central processing unit for storing the preprocessing result;

[0036] The preprocessing result is saved to the storage unit corresponding to the second memory address through DMA, and the preprocessing result is saved to the storage unit corresponding to the target video memory address of the preprocessing result through the central processing unit; the target video memory address is the address of the storage unit in the video memory of the image processor for saving the preprocessing result.

[0037] In a fourth aspect, the present application provides a computing system, including a central processing unit, an image processor, and a data processor;

[0038] The central processing unit is used to control the data processor and image processor to perform artificial intelligence AI tasks;

[0039] The data processor is used to preprocess the data, obtain a preprocessing result, and transmit the preprocessing result to the image processor;

[0040] The image processor is used to perform AI tasks based on the preprocessing results.

[0041] In a fifth aspect, the present application provides a data processing chip, comprising a processor and a power supply circuit, wherein the power supply circuit is used to supply power to the processor, and the processor is used to execute a computer program to implement any one of the methods described in the second aspect above.

[0042] In a sixth aspect, the present application provides a data processing card, comprising an interface and a processor, wherein the interface is used for the processor to communicate with a central processing unit or an image processor; the processor is used to execute a computer program to implement any one of the methods described in the second aspect above.

[0043] In a seventh aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute any one of the methods provided in the second aspect above.

[0044] In an eighth aspect, the present application provides a computer program product, comprising computer executable instructions, which are used to enable a computer to execute any one of the methods provided in the second aspect.

[0045] The technical effects that can be achieved in any of the second to eighth aspects mentioned above can refer to the description of the beneficial effects in the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of the structure of a computing system provided in an embodiment of the present application;

[0047] Figure 2 An interactive flow chart of a data processing method provided in an embodiment of the present application;

[0048] Figure 3 A logical schematic diagram of a data transmission process provided in an embodiment of the present application;

[0049] Figure 4 An interactive flow chart of another data processing method provided in an embodiment of the present application;

[0050] Figure 5 A logical schematic diagram of another data transmission process provided in an embodiment of the present application;

[0051] Figure 6 An interactive flow chart of another data processing method provided in an embodiment of the present application;

[0052] Figure 7 A logical schematic diagram of another data transmission process provided in an embodiment of the present application;

[0053] Figure 8 A structural block diagram of a data processing unit provided in an embodiment of the present application;

[0054] Fig. 9 A schematic diagram of the structure of a data processing chip provided in an embodiment of the present application;

[0055] Fig.10 A schematic diagram of the structure of a data processing card provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0057] Before introducing the specific solutions provided by the embodiments of the present application, some terms in the present application are explained to facilitate understanding by those skilled in the art, and the terms in the present application are not limited.

[0058] (1) Data processor: A data-centric, input-output (IO) intensive chip that has the ability to improve the efficiency of computing systems and reduce the cost of computing systems. It is mainly aimed at computing for data center infrastructure applications and can flexibly offload virtualization, network, storage, security and other service loads to reduce the load on the central processing unit. The data processor is used to offload some functions of the central processing unit. For example, high-performance data processing functions in the network, storage or operating system that are not suitable for the central processing unit to be processed can be offloaded to the data processor to release the computing power of the central processing unit. The data processor can be a data processing unit (DPU), an infrastructure processing unit (IPU), a system on chip (SoC), an iNIC or a smartNIC, etc., which have offloading functions. Among them, iNIC or smartNIC can be understood as an intelligent network card. The data processor can refer to a data processing chip or a board with an integrated data processing chip. The board is plugged into the motherboard and communicates with components such as the CPU and image processor through a bus.

[0059] (2) Image processor: used to process computer graphics and image data, such as graphics rendering, image processing, etc. The image processor can be a graphic processing unit (GPU), which is used in graphics cards. The GPU has a large number of processing units and can quickly process large amounts of graphics and image data, greatly improving computing speed and efficiency. In the field of AI, the GPU can be used to train neural networks or perform inference based on neural networks. The image processor can refer to an image processing chip or a board with an integrated image processing chip. The board is plugged into the motherboard and communicates with components such as the CPU and image processor through a bus.

[0060] In the embodiments of the present application, "multiple" refers to two or more than two. In view of this, in the embodiments of the present application, "multiple" can also be understood as "at least two". "At least one" can be understood as one or more, for example, one, two or more. For example, including at least one means including one, two or more, and there is no restriction on which ones are included. For example, including at least one of A, B and C, then A, B, C, A and B, A and C, B and C, or A and B and C may be included. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / ", unless otherwise specified, generally indicates that the previously associated objects are in an "or" relationship.

[0061] Unless otherwise specified, the ordinal numbers such as "first" and "second" mentioned in the embodiments of the present application are used to distinguish multiple objects and are not used to limit the order, timing, priority or importance of the multiple objects.

[0062] The image data processing method provided in the embodiment of the present application can be applied to a computing system. Figure 1 The schematic diagram of the structure of a computing system is shown as an example. Figure 1 As shown, the computing system may include a central processing unit 100, a data processor 200, and an image processor 300. The central processing unit 100, the data processor 200, and the image processor 300 may be connected via a bus, for example, the central processing unit 100, the data processor 200, and the image processor 300 may be connected via a peripheral component interconnect Express (PCIe) bus or an extended industry standard architecture (EISA) bus. In some embodiments, the central processing unit 100, the data processor 200, and the image processor 300 may also be connected via a PCIe switch.

[0063] The CPU 100 may be a central processing unit (CPU), which is the control core of the computing system and the final execution unit for information processing and program execution, and is used to interpret computer instructions, and control and coordinate various hardware in the computing system. For example, the image processor 300 and the data processor 200 may use the acquired data to perform AI tasks, train a neural network, or perform reasoning based on a neural network under the control of the CPU 100.

[0064] The above-mentioned computing system can be a single computing device or a computing device cluster. The computing device cluster can include multiple computing devices. A computing device can be understood as a physical device such as a computer or a server. When the computing system is a computing device cluster composed of multiple computing devices, the image processor and the data processor can be located in the same computing device or in different computing devices (for example, the data processor of computing device A is used to pre-process data, and the image processor of computing device B performs AI tasks). When the image processor and the data processor are located in the same computing device, the image processor and the data processor can be different chips on the same board or chips on different boards.

[0065] Compared with the traditional computing system which only includes a central processing unit and an image processor, the computing system provided by the embodiment of the present application adds a data processor. In the process of executing AI tasks, the data processor can be used to unload the functions of pre-processing calculations such as image decoding and random transformation of image data originally performed by the central processing unit, so as to reduce the load of the central processing unit and improve the stability of the computing system.

[0066] Figure 2 A flowchart of the interaction between the central processing unit, the data processor and the image processor during the execution of an AI task is shown.

[0067] like Figure 2 As shown, in some embodiments, the process may include the following steps:

[0068] S201, the central processing unit starts the AI ​​task and performs initialization configuration operations for the AI ​​task.

[0069] In some optional embodiments, the computing system may further include a display, which may be connected to the central processing unit via a bus. The display may display a human-computer interaction interface to the user under the control of the central processing unit, and receive operation instructions for executing AI tasks input by the user through the human-computer interaction interface. In this embodiment, the AI ​​task is to train a neural network as an example. Assuming that the neural network is an image recognition network, after the user inputs an operation instruction for training the image recognition network through the human-computer interaction interface, the central processing unit may start the AI ​​task for training the image recognition network, and initialize and configure the image processor and the data processor respectively.

[0070] For example, the data processor may include a DPU memory, and the image processor may include a video memory. The central processing unit may access the DPU memory of the data processor and the video memory of the image processor through direct memory access (DMA). The central processing unit initializes and configures the data processor, which may include the central processing unit reserving storage space for AI tasks in the DPU memory of the data processor to save the data to be processed received by the data processor and the preprocessing results after preprocessing the data to be processed. The central processing unit initializes and configures the image processor, which may include the central processing unit reserving storage space for AI tasks in the video memory of the image processor to save the preprocessing results sent by the data processor and the task execution results after the image processor executes the AI ​​task.

[0071] After the CPU initializes and configures the image processor and the data processor, it can also perform related process control. For example, the CPU can send AI task start instructions to the image processor and the data processor respectively, so that both the image processor and the data processor start and execute the AI ​​task.

[0072] S202: The data processor preprocesses the data to obtain a preprocessing result.

[0073] Under the control of the central processing unit, the data processor can obtain the data to be processed of the AI ​​task and pre-process the data. Pre-processing the data may include one or more of the following multiple processes: decoding the data, performing random transformation on the data, performing image scaling on the data, performing image cropping on the data, performing image flipping on the data, or performing color space conversion on the data. Exemplarily, the central processing unit may notify the data processor of the amount of data to be processed, the method of obtaining the data to be processed, and the information such as what pre-processing operation to perform on the data to be processed. In the AI ​​task, the data processor loads data in batches. For example, assuming that the data to be processed is image data, the central processing unit notifies the data processor that batchsize=64, then the data processor can load 64 pictures at one time and pre-process the 64 pictures. The method of obtaining the data to be processed may include: obtaining the data to be processed from the network or obtaining the data to be processed locally.

[0074] For example, in an optional embodiment, the data processor may include a network interface, and the data processor may communicate with other devices in the network through a high-performance network interface. For example, the data processor has a high-performance network interface, and the network interface can be used for high-speed data processing and parsing. If the central processor notifies the data processor to obtain the data to be processed from the network, the data processor can obtain the data to be processed from other devices through the network interface.

[0075] In another optional embodiment, the computing system may further include a storage device, which may be a hard disk drive (HDD) or a solid-state drive (SSD). For example, assuming that the storage device is an SSD, if the central processor notifies the data processor to obtain the data to be processed locally, the data processor may obtain the data to be processed from the storage device through a peer-to-peer (P2P) direct transmission.

[0076] After the data processor obtains the data to be processed, it can directly transfer the data to be processed to the preprocessing core in the data processor for preprocessing, or temporarily cache the data to be processed in a large-capacity, high-bandwidth storage, and then the preprocessing core in the data processor preprocesses the data to be processed.

[0077] Exemplarily, the data processor can be a data processor based on a field programmable gate array (FPGA), or a data processor based on an application-specific integrated circuit (ASIC), or a data processor based on a system-on-chip (SoC). The data processor includes a programmable area, and the programmable area on the data processor generates a preprocessing kernel. The preprocessing kernel can be understood as an acceleration engine generated in the programmable area on the DPU. The acceleration engine refers to a programmable unit that can perform accelerated calculations. The flexible and programmable acceleration engine can realize the unloading of many basic application facilities to reduce the computing pressure of the central processing unit. For example, after the network interface of the data processor receives the data to be processed, the data to be processed can be transmitted to the preprocessing kernel, or the preprocessing kernel can obtain the data to be processed from the storage device and preprocess the data to be processed.

[0078] Among them, the preprocessing of the data to be processed may include but is not limited to random transformation processing such as decoding and normalization of the data. If the data to be processed is image data, it may also include random transformation processing such as image scaling (resize), color space conversion (cvtcolor), image cropping, image flipping, etc. In one embodiment, the decoding and random transformation processing of the above data can be performed by a data processor. In another embodiment, according to actual needs, a part of the random transformation processing such as image scaling, image cropping, image flipping, etc. can be performed by a data processor, and another part of the random transformation processing such as color space conversion, normalization, etc. can be performed by an image processor.

[0079] In some embodiments, considering the number of programmable resources on the data processor and the user's requirements for performance, the data processor may include a data preprocessing unit, which preprocesses the data to be processed to obtain a preprocessing result.

[0080] In other embodiments, the data processor includes multiple data preprocessing units, or the preprocessing core of the data processor may include multiple data preprocessing units, the data to be processed includes multiple data, and the data processor or the preprocessing core of the data processor may preprocess the multiple data in parallel through the multiple data preprocessing units to obtain preprocessing results corresponding to the multiple data. For example, the data to be processed includes 64 pictures, and the data processor or the preprocessing core of the data processor may preprocess the 64 pictures in parallel through the multiple data preprocessing units to obtain preprocessing results corresponding to the 64 pictures.

[0081] In actual application scenarios, the programmable area of ​​the data processor can generate multiple preprocessing kernels, each preprocessing kernel performs different preprocessing operations on the data. For example, it is assumed that the data processor includes three preprocessing kernels, namely the first preprocessing kernel, the second preprocessing kernel and the third preprocessing kernel. Among them, the first preprocessing kernel can be used to perform image scaling on the data, the second preprocessing kernel can be used to perform image cropping on the data, and the third preprocessing kernel can be used to perform image flipping on the data. The data to be processed can first pass through the first preprocessing kernel, and the first preprocessing kernel performs image scaling on the data to be processed. Then, the data to be processed after the image scaling passes through the second preprocessing kernel, and the second preprocessing kernel performs image cropping on the data to be processed. The data to be processed after the image cropping passes through the third preprocessing kernel, and the third preprocessing kernel performs image flipping on the data to be processed. Among them, the first preprocessing kernel, the second preprocessing kernel and the third preprocessing kernel can all include multiple data preprocessing units, the multiple data preprocessing units in the first preprocessing kernel can perform image scaling on multiple data in parallel, the multiple data preprocessing units in the second preprocessing kernel can perform image cropping on multiple data after image scaling in parallel, and the multiple data preprocessing units in the third preprocessing kernel can perform image flipping on multiple data after image cropping in parallel.

[0082] By preprocessing multiple data in parallel using multiple data preprocessing units, data processing efficiency can be improved.

[0083] S203, the data processor sends a preprocessing completion message to the central processor.

[0084] S204, the central processor sends a storage instruction message to the data processor.

[0085] The central processing unit receives the preprocessing completion message sent by the data processor, and can determine the address of the storage unit in the video memory of the image processor for storing the preprocessing results according to the address of the storage space reserved for the AI ​​task in the video memory of the image processor, that is, the target video memory address corresponding to the preprocessing result, generate a storage indication message containing the target video memory address, and send the storage indication message to the data processor. Among them, the storage unit for storing the preprocessing results can be a part of the storage space reserved for the AI ​​task in the video memory of the image processor. Since the storage space reserved for AI tasks is relatively large and reusable, the central processing unit can determine which storage units in the reserved storage space are used to store the preprocessing results to improve the utilization of the storage space.

[0086] S205: The data processor transmits the preprocessing result to the image processor.

[0087] The data processor can transmit the preprocessing result to the image processor according to the storage indication message. The data processor can save the preprocessing result directly to the video memory of the image processor through the P2P direct pass-through method, bypassing the central processor and the memory of the central processor. Figure 3 As shown, the data processor receives the storage indication message sent by the central processor, and the data processor can obtain the target video memory address corresponding to the preprocessing result from the storage indication message, and save the preprocessing result to the storage unit corresponding to the target video memory address through DMA. The target video memory address can be represented by the bus address of the video memory of the image processor.

[0088] To improve data processing efficiency, in some embodiments, a computing system may include multiple image processors, and multiple image processors jointly perform AI tasks. The bus addresses of the video memories of each image processor are different. After the data processor obtains the target video memory address, the image processor used to save the preprocessing result can be determined according to the target video memory address. For example, assume that the computing system includes two image processors, a first image processor and a second image processor. Among them, the bus address of the video memory of the first image processor is 0~16T, and the bus address of the video memory of the second image processor is 16~32T. If the target video memory address obtained by the data processor belongs to 0~16T, the data processor can transfer the preprocessing result to the video memory of the first image processor, and the storage unit corresponding to the target video memory address.

[0089] The data processor directly saves the preprocessing result to the video memory of the image processor through the P2P direct pass-through method, which is faster than the image processor reading the preprocessing result from the data processor, so the data transmission rate can be further improved.

[0090] S206: The image processor performs an AI task based on the preprocessing result.

[0091] Under the control of the central processing unit, the image processor can perform AI tasks based on the preprocessing results, obtain task execution results, and transmit the task execution results to the central processing unit and save them in the memory of the central processing unit. For example, the image processor can input the preprocessing results into the image recognition network, train the image recognition network, and obtain training parameters. In some embodiments, when the image processor performs AI tasks based on the preprocessing results, it can also perform random transformation processing such as color space conversion and normalization on the preprocessing results that the data processor did not perform when performing the preprocessing operation, and then input the input data into the image recognition network to train the image recognition network.

[0092] In the above embodiment, after the data processor obtains the preprocessing result of the data to be processed, the data processor acts as the master control and saves the preprocessing result to the video memory of the image processor. In another embodiment, after the data processor obtains the preprocessing result of the data to be processed, it can also be saved in the DPU memory of the data processor, and the image processor acts as the master control and reads the preprocessing result from the memory of the data processor. For example, Figure 4 Another flow chart of the interaction between the CPU, data processor and image processor during the execution of an AI task is shown. Figure 4 As shown, in some embodiments, the process may include the following steps:

[0093] S401, the central processing unit starts the AI ​​task and performs initialization configuration operations for the AI ​​task.

[0094] In this embodiment, the AI ​​task of training a neural network is still used as an example for explanation. Assuming that the neural network is an image recognition network, after the user inputs an operation instruction for training the image recognition network through the human-computer interaction interface, the central processing unit can start the AI ​​task of training the image recognition network and initialize and configure the image processor and the data processor respectively.

[0095] The central processor performs initialization configuration on the data processor, which may include the central processor reserving storage space for AI tasks in the DPU memory of the data processor, for storing the data to be processed received by the data processor and the preprocessing results after preprocessing the data to be processed. The central processor performs initialization configuration on the image processor, which may include the central processor reserving storage space for AI tasks in the video memory of the image processor, for storing the preprocessing results sent by the data processor and the task execution results after the image processor executes the AI ​​task.

[0096] After the CPU initializes and configures the image processor and the data processor, it can also perform related process control. For example, the CPU can send AI task start instructions to the image processor and the data processor respectively, so that both the image processor and the data processor start and execute the AI ​​task.

[0097] S402: The data processor preprocesses the data to obtain a preprocessing result.

[0098] The data processor can obtain the data to be processed of the AI ​​task under the control of the central processor, and pre-process the data to be processed. The process of the data processor pre-processing the data to be processed can be performed with reference to the above step S202, which will not be repeated here.

[0099] S403, the data processor sends a preprocessing completion message to the central processor.

[0100] S404: The central processing unit sends a storage instruction message to the image processor.

[0101] The central processor receives the preprocessing completion message sent by the data processor, and can determine the address of the storage unit in the DPU memory of the data processor for storing the preprocessing results, that is, the first memory address, based on the address of the storage space reserved for the AI ​​task in the DPU memory of the data processor, generate a storage indication message containing the first memory address, and send the storage indication message to the data processor, so that the data processor saves the preprocessing results to the storage unit corresponding to the first memory address. Among them, the storage unit for storing the preprocessing results can be a part of the storage space reserved for the AI ​​task in the DPU memory of the data processor. Since the storage space reserved for AI tasks is relatively large and reusable, the central processor can determine which storage units in the reserved storage space are used to store the preprocessing results to improve the utilization of the storage space.

[0102] After generating the storage indication message including the first memory address, the central processor may also send the storage indication message to the image processor, so that the image processor obtains the first memory address for storing the preprocessing result.

[0103] S405: The image processor reads the preprocessing result from the data processor.

[0104] The image processor reads the preprocessing result from the first memory address in the data processor according to the storage instruction message. Figure 5As shown, the image processor receives a storage indication message sent by the central processing unit, and can obtain a first memory address for storing the preprocessing result from the storage indication message. The image processor can read the preprocessing result from the first memory address in the data processor by DMA, and save the preprocessing result to a storage unit corresponding to the target video memory address in the video memory. Among them, the first memory address can be represented by the bus address of the DPU memory of the data processor. The target video memory address can be determined by the central processing unit after receiving the preprocessing completion message sent by the data processor, and can be carried in the storage indication message sent by the central processing unit to the image processor and sent to the image processor.

[0105] S406: The image processor performs an AI task based on the preprocessing result.

[0106] Under the control of the central processing unit, the image processor can execute AI tasks based on the preprocessing results, obtain task execution results, and transmit the task execution results to the central processing unit and save them in the memory of the central processing unit.

[0107] In the above embodiments, the flexible and programmable acceleration engine on the data processor is used to efficiently complete the data preprocessing tasks in the AI ​​tasks, which can reduce the computing load of the central processor for data preprocessing, reduce the pressure on the central processor, and improve the stability of the computing system. The preprocessing results are directly transmitted between the data processor and the image processor in a P2P manner, without relying on the central processor to transfer the preprocessing results, which can greatly improve the data transmission efficiency and further reduce the impact of the transfer of preprocessing results on the entire AI training process.

[0108] In some embodiments, when the AI ​​task is completed by multiple image processors together, it may be necessary to average the multiple calculation results obtained by the multiple image processors. The averaging process can also be performed by the data processor. In an optional embodiment, the data processor can obtain the video memory address for storing the calculation results in each image processor from the central processor, read each calculation result from the video memory address for storing each calculation result through DMA, determine the average value of the multiple calculation results, and then save the average value of the multiple calculation results to the corresponding video memory address in each image processor through DMA. In another optional embodiment, each image processor can obtain the DPU memory address for storing the calculation results in the data processor from the central processor, save each calculation result to the DPU memory address corresponding to each calculation result through DMA, the data processor can read each calculation result from each DPU memory address, determine the average value of the multiple calculation results, save the average value of the multiple calculation results to the DPU memory address corresponding to each calculation result, and each image processor reads the average value of the multiple calculation results from the corresponding DPU memory address and saves it to its own video memory. In another optional embodiment, considering that the speed of writing data is usually faster than the speed of reading data, in order to further improve the data processing rate, each image processor can obtain the DPU memory address used to store calculation results in the data processor from the central processing unit, and save each calculation result to the DPU memory address corresponding to each calculation result through DMA. The data processor can read each calculation result from each DPU memory address, determine the average value of multiple calculation results, and then obtain the video memory address of each image processor that stores the average value of the calculation results from the central processing unit, and save the average value of the multiple calculation results to the corresponding video memory address in each image processor through DMA.

[0109] In other embodiments, after obtaining the preprocessing result of the data to be processed, the data processor may also transmit the preprocessing result to the central processor, and the central processor transmits the preprocessing result to the image processor. Figure 6 Another flow chart of the interaction between the CPU, data processor and image processor during the execution of an AI task is shown. Figure 6 As shown, in some embodiments, the process may include the following steps:

[0110] S601, the central processing unit starts the AI ​​task and performs initialization configuration operations for the AI ​​task.

[0111] In this embodiment, the AI ​​task of training a neural network is still used as an example for explanation. Assuming that the neural network is an image recognition network, after the user inputs an operation instruction for training the image recognition network through the human-computer interaction interface, the central processing unit can start the AI ​​task of training the image recognition network and initialize and configure the image processor and the data processor respectively.

[0112] After the CPU initializes and configures the image processor and the data processor, it can also perform related process control. For example, the CPU can send AI task start instructions to the image processor and the data processor respectively, so that both the image processor and the data processor start and execute the AI ​​task.

[0113] S602: The data processor preprocesses the data to obtain a preprocessing result.

[0114] The data processor can obtain the data to be processed of the AI ​​task under the control of the central processor, and pre-process the data to be processed. The process of the data processor pre-processing the data to be processed can be performed with reference to the above step S202, which will not be repeated here.

[0115] S603, the data processor sends a preprocessing completion message to the central processor.

[0116] S604: The central processor sends a storage instruction message to the data processor.

[0117] When the central processing unit receives the preprocessing completion message sent by the data processor, it can determine the address of the storage unit in the memory of the central processing unit for storing the preprocessing results, i.e., the second memory address, based on the address of the storage space reserved for the AI ​​task in the DPU memory of the data processor, generate a storage indication message including the second memory address, and send the storage indication message to the data processor.

[0118] S605, the data processor transmits the preprocessing result to the central processor.

[0119] The data processor transmits the preprocessing result to the central processor according to the storage instruction message.

[0120] S606, the central processing unit transmits the preprocessing result to the image processor.

[0121] like Figure 7 As shown, the data processor receives the storage indication message sent by the central processor, obtains the second memory address corresponding to the preprocessing result from the storage indication message, and saves the preprocessing result to the storage unit corresponding to the second memory address through DMA.

[0122] After receiving the preprocessing completion message sent by the data processor, the central processor can also determine the target video memory address in the video memory of the image processor for storing the preprocessing result. The central processor can save the preprocessing result stored in the storage unit corresponding to the second memory address to the storage unit corresponding to the target video memory address of the preprocessing result in the image processor through DMA.

[0123] S607, the image processor performs an AI task based on the preprocessing result.

[0124] Under the control of the central processing unit, the image processor can execute AI tasks based on the preprocessing results, obtain task execution results, and transmit the task execution results to the central processing unit and save them in the memory of the central processing unit.

[0125] The above process can also be understood as the CPU maintaining a queue in the memory space corresponding to the second memory address, which is used to save the preprocessing results sent by the data processor in batches. Whenever the image processor is idle and the queue is not empty, the preprocessing results in the queue are directly transmitted to the image processor for subsequent inference training.

[0126] In an embodiment of the present application, when executing an AI task, before reasoning or training the neural network through a neural network, the data processor can first perform data decoding, random transformation and other preprocessing on the data to be processed, and then the image processor performs the AI ​​task based on the preprocessing result, and reasoning or training the neural network through the neural network. The preprocessing of the data to be processed by the data processor and the process of the image processor performing the AI ​​task based on the preprocessing result can be performed in parallel to improve the efficiency of reasoning or training. For example, the data processor preprocesses the first batch of data to be processed, transmits the preprocessing results of the first batch of data to be processed to the image processor, and the image processor performs the AI ​​task based on the preprocessing results of the first batch of data to be processed. When the image processor performs the AI ​​task based on the preprocessing results of the first batch of data to be processed, the data processor can preprocess the second batch of data to be processed, so as to realize data preprocessing and AI task synchronization. At the same time, compared with the preprocessing of data by the central processing unit, the preprocessing calculation of data by the data processor can reduce the computing load of the central processing unit and increase the stability of the computing system.

[0127] In addition, during the data preprocessing process, the CPU usually loads the data to be processed into the memory of the CPU first, and the CPU continuously reads the data from the memory, calculates the read data, and then saves the intermediate calculation results to the memory, waiting for the next calculation instruction, and then reads the intermediate calculation results of the previous calculation from the memory for the next calculation, until the preprocessing process is completed. This process requires multiple interactions between the CPU and the memory, resulting in a low data preprocessing rate, even lower than the inference efficiency or training efficiency when the image processor uses the preprocessing results to perform AI tasks, causing the image processor to be idle and waiting, reducing the utilization of the image processor in the time dimension, thereby extending the execution time of the entire AI task. In an embodiment of the present application, data is preprocessed by a data processor. Since the data processor has good software and hardware programmability, or in other words, the data processor has a programmable area, the data preprocessing process can be implemented by programming. For example, a data can sequentially pass through a first preprocessing kernel for image scaling, a second preprocessing kernel for image cropping, and a third preprocessing kernel for image flipping, without storing the intermediate calculation results in the DPU memory in the preprocessing process, and then repeatedly reading the intermediate calculation results from the DPU memory for the next calculation. Therefore, there is no need for multiple interactions between the data processor and the DPU memory, which can improve the data preprocessing rate, so that the data preprocessing rate can be higher than the inference efficiency or training efficiency when the image processor uses the preprocessing results to perform AI tasks, so that before the image processor completes the processing of the preprocessing results of the current batch of data, the next batch of data that can be used by the image processor is preprocessed, and the preprocessing results of the next batch of data are transmitted to the image processor in advance, which can reduce the waiting idleness of the image processor and maintain continuous data input to ensure that the image processor has a high time utilization rate.

[0128] In summary, the embodiments of the present application provide a data processing method that can effectively offload the data preprocessing computing load of the central processing unit, thereby improving the efficiency and stability of data preprocessing, so that the speed of data preprocessing matches the neural network inference speed or training speed of the image processor.

[0129] The above embodiments are described by taking the task of training a neural network as an example. In other embodiments, the AI ​​task may also be a task of performing data reasoning based on the data to be processed through a neural network. The above embodiments are all described by taking the neural network as an image recognition network and the data to be processed as image data. In other embodiments, the neural network may also be a speech recognition network, and the data to be processed may also be audio data or text data, etc. This application does not limit the data type of the data to be processed.

[0130] The data processing method provided in the embodiments of the present application can be applied to intelligent driving, artificial intelligence, cloud computing, finance and other fields. The data processor is used to unload and accelerate the preprocessing operations before AI reasoning training. The flexible and programmable acceleration engine in the data processor can generate a vectorized computing unit for data preprocessing. The data processor serves as the data entry of the computing system. After the data processor receives data from the network or storage, it immediately unloads and accelerates the preprocessing tasks in the AI ​​reasoning training, which can effectively reduce the computing load of the central processing unit, improve the computing efficiency of the preprocessing, and reduce the power consumption of the central processing unit during the data preprocessing calculation.

[0131] Based on the same technical concept as the above method embodiment, a data processing unit is also provided in the embodiment of the present application. The data processing unit can be applied to a data processor of a computing system, and the data processor can be connected to a central processing unit and an image processor via a bus. In some embodiments, Figure 8 As shown, the data processing unit 800 may include a data preprocessing module 801 and a data transfer module 802. The data processing unit 800 may be used to implement the functions of the above method embodiment, and thus may achieve the beneficial effects of the above method embodiment.

[0132] Among them, the data preprocessing module 801 can be used to preprocess the data to be processed of the AI ​​task under the control of the central processing unit to obtain the preprocessing results of the data to be processed; the data transfer module 802 can be used to transmit the preprocessing results to the image processor; the image processor is used to execute the AI ​​task based on the preprocessing results.

[0133] It should be noted that, in other embodiments, the data preprocessing module 801 can be used to execute any step in the data processing method, and the data transfer module 802 can be used to execute any step in the data processing method. The steps that the data preprocessing module 801 and the data transfer module 802 are responsible for implementing can be specified as needed, and the data preprocessing module 801 and the data transfer module 802 respectively implement different steps in the data processing method to achieve all the functions of the data reading device. In other embodiments, the data processing device may also include a data receiving module, which may receive data from the network or storage through the high-performance network interface of the data processor, and the received data may be temporarily cached in a large-capacity, high-bandwidth storage on the data processor, and then or directly transmitted to the data preprocessing module on the data processor.

[0134] Each functional module in the embodiment of the present application can be integrated into one processor, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional units.

[0135] Based on the same technical concept as the above method embodiment, a data processing chip is also provided in the embodiment of the present application. The data processing chip can be a computing chip. The data processing chip can be used as the above data processor, or applied to the data processor of the computing system. The data processor can be connected to the central processing unit and the image processor through a bus. The data processing chip can be used to implement the functions of the above method embodiment, so the beneficial effects of the above method embodiment can be achieved.

[0136] In some embodiments, the structure of the data processing chip 900 can be as follows: Fig. 9 As shown, it includes a processor 901 and a power supply circuit 902 connected to the processor 901. The processor 901 and the power supply circuit 902 can be connected to each other through a bus. The processor 901 can be a digital signal processor (digital signal processor, DSP), ASIC, field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components or other specific integrated circuits. The bus can be a peripheral component interconnect (peripheral component interconnect, PCI) bus or an extended industry standard architecture (extendedindustry standard architecture, EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. The power supply circuit 902 is used to power the processor 901 through the bus.

[0137] The processor 901 can be connected to a memory disposed outside the chip, or to a memory disposed inside the chip, to run software programs and modules stored in the memory, thereby executing various functional applications and data processing of the data processing chip 900, such as the data processing method provided in an embodiment of the present application.

[0138] In some embodiments, the processor 901 may include one or more processing units, and different processing units may be independent devices or integrated into one or more processors. The processor 901 may also include a controller, which may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0139] Based on the same technical concept as the above embodiment, a data processing card is also provided in the embodiment of the present application. The data processing card can be used to implement the functions of the above method embodiment, thereby achieving the beneficial effects of the above method embodiment.

[0140] In some embodiments, the structure of the data processing card 1000 can be as follows: Fig.10 As shown, it includes a processor 1001 and an interface 1002 connected to the processor 1001. The processor 1001 can be understood as the above-mentioned data processor, and the processor 1001 can adopt a general processor, such as a microprocessor, or other conventional processors. The processor 1001 can communicate with the central processing unit and the image processor in the computing system through the interface 1002.

[0141] In an optional embodiment, the data processing card 1000 may also include a memory, which may be the DPU memory in the above embodiment. The processor 1001 and the memory may be interconnected via a bus. The memory may be used to store software programs and modules. The processor 1001 may assist the central processor in implementing various functional applications and data processing by running the software programs and modules stored in the memory under the control of the central processor, such as the data processing method provided in the embodiment of the present application, preprocessing the data, and transmitting the preprocessing results to the image processor via an interface.

[0142] The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store at least one application program, etc.; the data storage area may be used to store data and pre-processing results of the data, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0143] The processor 1001 in the data processor 1000 is used to run the computer instructions or programs stored in the memory and perform the functions in any of the above method embodiments. In some embodiments, the processor 1001 may include one or more processing units, and different processing units may be independent devices or integrated in one or more processors. The processor 1001 may also include a controller, which may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0144] It is understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the acceleration device. In other embodiments of the present application, the acceleration device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0145] Based on the same technical concept as the above embodiment, a computing system is also provided in the embodiment of the present application. The computing system can be deployed in a computing device or a computing device cluster. In some embodiments, the computing system can be understood as a computing device. Figure 1 As shown, the computing system can include a central processing unit, a data processor, and an image processor. The central processing unit, the data processor, and the image processor are communicatively connected. The central processing unit can be used to control the data processor and the image processor to perform artificial intelligence AI tasks; the data processor can be used to preprocess the data, obtain the preprocessing results, and transmit the preprocessing results to the image processor; the image processor can be used to perform AI tasks based on the preprocessing results. Among them, the data processor has at least one high-performance network interface and a flexible and programmable acceleration engine, and the data processor has a large capacity and high bandwidth memory (for example, DDR) that can be used to cache data received from the network. An efficient network interface can be used to receive network data, a flexible and programmable acceleration engine can be used to generate a preprocessing kernel for AI preprocessing calculations, and the memory can be used to cache data received from the network that needs to be preprocessed.

[0146] The method steps in the embodiments of the present application may be implemented by hardware, or by a processor executing a computer program or instruction. The computer program or instruction may constitute a computer program product.

[0147] The present application also provides a computer program product, which includes computer executable instructions. In one embodiment, the computer executable instructions are used to enable a computer to execute the functions of the above method embodiment.

[0148] The computer executable instructions may be stored in a computer readable storage medium. The present application also provides a computer readable storage medium, wherein the computer readable storage medium stores the executable instructions. In one embodiment, the computer executable instructions are used to enable a computer to execute the functions in the above method embodiment.

[0149] The computer-readable storage medium provided in the embodiments of the present application may be a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a mobile hard disk, a CD-ROM, or any other form of computer-readable storage medium known in the art.

[0150] Computer executable instructions may be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium. For example, the computer program or instructions may be transferred from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it may also be an optical medium, such as a digital video disc (DVD); it may also be a semiconductor medium, such as a solid state drive.

[0151] One or more of the above modules or units can be implemented by software, hardware or a combination of the two. When any of the above modules or units is implemented in software, the software exists in the form of computer program instructions and is stored in a memory, and the processor can be used to execute the program instructions and implement the above method flow. The processor may include but is not limited to at least one of the following: a CPU, a microprocessor, a digital signal processor (digital signal processor, DSP), a microcontroller (microcontroller unit, MCU), or an artificial intelligence processor and other types of computing devices that run software, each computing device may include one or more cores for executing software instructions for calculation or processing. The processor may be built into a SoC, a DPU or an ASIC, or it may be an independent semiconductor chip. In addition to the core used to execute software instructions for calculation or processing in the processor, it may further include necessary hardware accelerators, such as FPGA, PLD or logic circuits for implementing dedicated logic operations.

[0152] When the above modules or units are implemented in hardware, the hardware can be any one or any combination of a CPU, a microprocessor, a DSP, an MCU, an artificial intelligence processor, an ASIC, a SoC, an FPGA, a PLD, a dedicated digital circuit, a hardware accelerator or a non-integrated discrete device, which can run the necessary software or not rely on the software to execute the above method flow.

[0153] The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and these modifications or substitutions should be included in the protection scope of the present application.

Claims

1. A data processing method, characterized in that: Applied to a computing system, the computing system includes a central processing unit, an image processor and a data processor; the method includes: The data processor, under the control of the central processor, preprocesses the data to obtain a preprocessing result, and transmits the preprocessing result to the image processor; The image processor performs an artificial intelligence (AI) task based on the preprocessing result.

2. The method according to claim 1, characterized in that The image processor and the data processor are different chips on the same board, or chips on different boards.

3. The method according to claim 1 or 2, characterized in that: Preprocessing the data includes one or more of the following processes: The data is decoded, randomly transformed, scaled, cropped, flipped, or converted into a color space.

4. The method according to any one of claims 1 to 3, characterized in that The data processor includes a plurality of data preprocessing units, and the data is a plurality of data; the preprocessing of the data to obtain a preprocessing result includes: The data processor preprocesses the multiple data in parallel through the multiple data preprocessing units to obtain preprocessing results corresponding to the multiple data respectively.

5. The method according to any one of claims 1 to 4, characterized in that: The transmitting the preprocessing result to the image processor comprises: The data processor acquires a target display memory address corresponding to the preprocessing result; the target display memory address is an address of a storage unit in the display memory of the image processor for storing the preprocessing result; The data processor saves the preprocessing result to the storage unit corresponding to the target video memory address through DMA.

6. The method according to any one of claims 1 to 4, characterized in that: The transmitting the preprocessing result to the image processor comprises: The image processor obtains a first memory address in the data processor for storing the preprocessing result; The image processor reads the preprocessing result from the first memory address in the data processor through DMA.

7. The method according to any one of claims 1 to 4, characterized in that: The transmitting the preprocessing result to the image processor comprises: The data processor obtains a second memory address corresponding to the preprocessing result; the second memory address is an address of a storage unit in the memory of the central processor for storing the preprocessing result; The data processor saves the preprocessing result to the storage unit corresponding to the second memory address through DMA; The central processing unit saves the preprocessing result stored in the storage unit corresponding to the second memory address to the storage unit corresponding to the target video memory address of the preprocessing result through DMA; the target video memory address is the address of the storage unit in the video memory of the image processor for storing the preprocessing result.

8. A data processing method, characterized in that: A data processor applied to a computing system, the computing system comprising a central processing unit, an image processor and a data processor, the method comprising: Under the control of the central processing unit, preprocessing the data to obtain preprocessing results; The preprocessing result is transmitted to the image processor; the image processor is used to perform artificial intelligence (AI) tasks based on the preprocessing result.

9. The method according to claim 8, characterized in that Preprocessing the data includes one or more of the following processes: The data is decoded, randomly transformed, scaled, cropped, flipped, or converted into a color space.

10. The method according to claim 8 or 9, characterized in that: The data processor includes a plurality of data pre-processing units; The data is multiple, and the data is preprocessed to obtain a preprocessing result, including: The multiple data preprocessing units preprocess the multiple data in parallel to obtain preprocessing results corresponding to the multiple data respectively.

11. The method according to any one of claims 8 to 10, characterized in that: The transmitting the preprocessing result to the image processor comprises: Acquire a target video memory address corresponding to the preprocessing result; the target video memory address is an address of a storage unit in the video memory of the image processor for storing the preprocessing result; The preprocessing result is saved to the storage unit corresponding to the target video memory address through DMA.

12. The method according to any one of claims 8 to 10, characterized in that: The transmitting the preprocessing result to the image processor comprises: Acquire a second memory address corresponding to the preprocessing result; the second memory address is an address of a storage unit in the memory of the central processing unit for storing the preprocessing result; The preprocessing result is saved to the storage unit corresponding to the second memory address through DMA, and the preprocessing result is saved to the storage unit corresponding to the target video memory address of the preprocessing result through the central processing unit; the target video memory address is the address of the storage unit in the video memory of the image processor for saving the preprocessing result.

13. A data processing unit, characterized in that: A data processor applied to a computing system, the computing system comprising a central processing unit, an image processor and a data processor, the device comprising: A data preprocessing module, used to preprocess the data under the control of the central processing unit to obtain a preprocessing result; A data transfer module is used to transmit the preprocessing result to the image processor; the image processor is used to perform artificial intelligence (AI) tasks based on the preprocessing result.

14. The data processing unit according to claim 13, characterized in that The data transfer module is specifically used for: Acquire a target video memory address corresponding to the preprocessing result; the target video memory address is an address of a storage unit in the video memory of the image processor for storing the preprocessing result; The preprocessing result is saved to the storage unit corresponding to the target video memory address through DMA.

15. The data processing unit according to claim 13, characterized in that The data transfer module is specifically used for: Acquire a second memory address corresponding to the preprocessing result; the second memory address is an address of a storage unit in the memory of the central processing unit for storing the preprocessing result; The preprocessing result is saved to the storage unit corresponding to the second memory address through DMA, and the preprocessing result is saved to the storage unit corresponding to the target video memory address of the preprocessing result through the central processing unit; the target video memory address is the address of the storage unit in the video memory of the image processor for saving the preprocessing result.

16. A data processing chip, characterized in that: The chip includes a processor and a power supply circuit; the power supply circuit is used to supply power to the processor, and the processor is used to execute a computer program to implement the method according to any one of claims 8 to 12.

17. A data processing card, characterized in that: The method comprises a processor and an interface, wherein the processor is used to execute the method according to any one of claims 8 to 12, and the interface is used for the processor to communicate with a central processing unit and an image processor.

18. A computing system, characterized in that: Including central processing unit, image processor and data processor; The central processor is used to control the data processor and the image processor to perform artificial intelligence AI tasks; The data processor is used to preprocess the data to obtain a preprocessing result, and transmit the preprocessing result to the image processor; The image processor is used to perform the AI ​​task based on the preprocessing result.

19. A computer-readable storage medium, characterized in that: Computer executable instructions are stored, and the computer executable instructions are used to make a computer execute the method according to any one of claims 8 to 12.

Images