Data processing method, accelerator chip, computing device, and storage medium

Abstract

The present application relates to the technical field of artificial intelligence (AI). Disclosed are a data processing method, an accelerator chip, a computing device, and a storage medium. The method is applied to the accelerator chip, which comprises a first computing unit and a second computing unit, wherein with regard to a first operator and a second operator in an AI model, the first computing unit can determine, on the basis of output data of the first operator, numerical statistical information of the output data, and the numerical statistical information can be used for determining the precision of data which is computed by means of the second operator. In this way, adaptive-precision data format configuration is performed on an AI model, and thus the flexibility is high. Moreover, the process of determining numerical statistical information and the process of executing an operator are asynchronously executed on different computing units, such that the computing efficiency can be improved.

Description

Data processing methods, accelerator chips, computing devices, and storage media

[0001] This application claims priority to Chinese Patent Application No. 202410939347.6, filed on July 12, 2024, entitled "Data Processing Method, Accelerator Chip, Computing Device and Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence (AI) technology, and in particular to a data processing method, an acceleration chip, a computing device, and a storage medium. Background Technology

[0003] With the rapid development of AI technology, a series of accelerator chips have emerged to provide computing power for matrices, vectors, and other data, thereby accelerating the computation process of AI models. Currently, many accelerator chips support multiple precision data formats (such as single-precision FP32 and half-precision FP16) to support mixed-precision training of AI models.

[0004] In related technologies, mixed-precision training of AI models is performed as follows: during the training process, a suitable precision data format is selected for calculation based on different layers or operators. For example, for some layers or operators with low precision requirements (such as forward computation and backward computation), a low-precision data format (such as FP16) is selected for calculation, while for some layers or operators with high precision requirements (such as parameter updates), a high-precision data format (such as FP32) is selected for calculation.

[0005] However, the above methods require pre-configuring a fixed mixed precision mode for the AI ​​model, and cannot configure the data format for adaptive precision of the AI ​​model, resulting in low flexibility. Summary of the Invention

[0006] This application provides a data processing method, an acceleration chip, a computing device, and a storage medium, which can configure the data format for AI models with adaptive accuracy and offer high flexibility.

[0007] In a first aspect, this application provides a data processing method applied in scenarios where an accelerator chip is used to execute AI model-related operators. The accelerator chip includes a first computing unit and a second computing unit. The method includes:

[0008] The first computing unit determines the numerical statistics of the output data based on the output data of the first operator in the artificial intelligence (AI) model. The numerical statistics indicate the distribution characteristics of the numerical values ​​in the output data.

[0009] The second computing unit receives the execution instructions of the second operator in the AI ​​model sent by the host. The second operator is used to calculate the first data, and the accuracy of the first data is determined by numerical statistics.

[0010] In the AI ​​model, for the first and second operators, the first computing unit can determine the numerical statistics based on the output data of the first operator. This numerical statistics can be used to determine the precision of the data calculated by the second operator. In this way, the AI ​​model is configured with an adaptive precision data format, which is highly flexible. Furthermore, the process of determining the numerical statistics and the execution process of the operators are executed asynchronously on different computing units, which can improve the computational efficiency.

[0011] In some embodiments, the precision of the first data differs from the precision of the output data. In some scenarios, the precision of the first data is the same as the precision of the output data.

[0012] In some embodiments, the execution instruction carries a target data format for a second operator, the target data format indicating the precision of the first data. The method further includes: the second calculation unit performing format conversion on the input data of the second operator according to the target data format to obtain the first data, wherein the input data of the second operator includes the output data of the first operator.

[0013] In some embodiments, the target data format indicates the data precision of each matrix block in the first data.

[0014] In this way, the target data format of the second operator is determined at the matrix block level, which realizes fine-grained configuration of the data format and maximizes computational efficiency while ensuring accuracy.

[0015] In some embodiments, the first computing unit includes an input register, a computing core, and an output register, wherein the input register is used to store the output data, the computing core is used to determine the numerical statistics based on the output data, and the output register is used to store the numerical statistics.

[0016] In some embodiments, the acceleration chip further includes a storage unit connected to the first computing unit via a chip bus, and the method further includes: the first computing unit storing the numerical statistics information into the storage unit.

[0017] In some embodiments, the second operator refers to an operator in the AI ​​model that is executed after the first operator; the first computing unit determines the numerical statistics of the output data based on the output data of the first operator in the artificial intelligence AI model, including: after the second computing unit executes the first operator, the first computing unit determines the numerical statistics based on the output data of the first operator.

[0018] In some embodiments, the first computing unit determines the numerical statistics of the output data based on the output data of the first operator in the artificial intelligence (AI) model, including: the first computing unit samples the output data to obtain sampled data, and determines the numerical statistics based on the magnitude of the sampled data.

[0019] By using sampling to determine numerical statistics based on the magnitude of the sampled data, the amount of data processing can be reduced, and the overall data distribution characteristics can be approximated with the lowest possible time complexity.

[0020] In some embodiments, the numerical statistics include at least one of normalized norm, mean, and median.

[0021] Secondly, this application provides a data processing method applied to a host, the method comprising:

[0022] An execution instruction for the second operator in the AI ​​model is sent to the acceleration chip. The second operator is used to calculate the first data. The accuracy of the first data is determined by the numerical statistics of the output data of the first operator in the AI ​​model. The numerical statistics indicate the distribution characteristics of the numerical values ​​in the output data.

[0023] In some embodiments, the execution instructions carry a target data format for the second operator, the target data format indicating the precision of the first data, and the method further includes:

[0024] Based on the numerical statistics, the target data format is determined, and the input data of the second operator includes the output data of the first operator.

[0025] In some embodiments, determining the target data format based on the numerical statistical information includes:

[0026] If the numerical statistics are within the first interval, the target data format is determined to be the first data format.

[0027] If the numerical statistics are in the second interval, the target data format is determined to be the second data format.

[0028] The data precision indicated by the first data format is lower than that indicated by the second data format.

[0029] Thirdly, this application provides a data processing method applied to a computing system, the system including a host and an acceleration chip, the acceleration chip including a first computing unit and a second computing unit, the method including:

[0030] The first computing unit determines the numerical statistics of the output data based on the output data of the first operator in the AI ​​model, and the numerical statistics indicate the distribution characteristics of the numerical values ​​in the output data;

[0031] The host sends an execution instruction for the second operator in the AI ​​model to the second computing unit. The second operator is used to calculate the first data, and the accuracy of the first data is determined by the numerical statistics.

[0032] The second computing unit receives the execution instruction and executes the second operator.

[0033] In some embodiments, the execution instructions carry a target data format for the second operator, the target data format indicating the precision of the first data.

[0034] In some embodiments, the method further includes:

[0035] The host determines the target data format based on the numerical statistics.

[0036] The second calculation unit performs format conversion on the input data of the second operator according to the target data format to obtain the first data. The input data of the second operator includes the output data of the first operator.

[0037] Fourthly, this application provides a data processing apparatus configured on an accelerator chip, the accelerator chip including a first computing unit and a second computing unit, the apparatus including at least one functional module for implementing the data processing method provided by the first aspect or any possible implementation thereof.

[0038] Fifthly, this application provides an acceleration chip, which includes a communication interface, a first computing unit, and a second computing unit. The communication interface is used to provide program instructions and / or data, and the acceleration chip is used to implement the data processing method provided by the first aspect or any possible implementation of the first aspect.

[0039] In a sixth aspect, this application provides a computing device including a host and an acceleration chip, wherein the host is used to control the acceleration chip, and the acceleration chip is used to implement the data processing method provided by the first aspect or any possible implementation thereof.

[0040] In a seventh aspect, this application provides a computer-readable storage medium for storing at least one piece of program code, which, when executed by an acceleration chip, causes the acceleration chip to implement the data processing method provided by the first aspect or any possible implementation thereof. The storage medium includes, but is not limited to, volatile memory, such as random access memory, and non-volatile memory, such as flash memory, hard disk drive (HDD), and solid-state drive (SSD).

[0041] Eighthly, this application provides a computer program product that, when run on an accelerator chip, causes the accelerator chip to implement the data processing method provided by the first aspect or any possible implementation thereof. The computer program product can be a software installation package; when the aforementioned data processing method needs to be implemented, the computer program product can be downloaded and executed on the accelerator chip. Attached Figure Description

[0042] Figure 1 is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0043] Figure 2 is a schematic diagram of the hardware structure of a computing device provided in an embodiment of this application;

[0044] Figure 3 is a schematic diagram of the hardware structure of an acceleration chip provided in an embodiment of this application;

[0045] Figure 4 is a schematic diagram illustrating the principle of a data processing method provided in an embodiment of this application;

[0046] Figure 5 is a flowchart of a data processing method provided in an embodiment of this application;

[0047] Figure 6 is a schematic diagram of a data sampling process provided in this application;

[0048] Figure 7 is a schematic diagram of determining a target data format according to an embodiment of this application;

[0049] Figure 8 is a flowchart of another data processing method provided in an embodiment of this application;

[0050] Figure 9 is a flowchart of another data processing method provided in an embodiment of this application;

[0051] Figure 10 is a flowchart of another data processing method provided in an embodiment of this application;

[0052] Figure 11 is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings. It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application are authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the AI ​​model-related data, operator execution instructions, data formats, etc., involved in this application were all obtained with full authorization.

[0054] To facilitate understanding, the key terms and concepts involved in this application will be explained below.

[0055] Artificial intelligence (AI) models are a class of mathematical algorithm models that use machine learning concepts to solve practical problems. Typically, AI models include a large number of parameters and calculation formulas (or calculation rules).

[0056] Acceleration chips, also known as accelerators, acceleration devices, or accelerator cards, are a type of specialized hardware device or computer system designed to accelerate computation processes in AI and HPC scenarios. In the embodiments of this application, acceleration chips may be, for example, graphics processing units (GPUs), neural network processing units (XPUs), intelligent processing units (IPUs), tensor processing units (TPUs), domain-specific architecture (DSA) chips, and so on, but are not limited to these.

[0057] An operator (OP) is a computational unit or function that runs on a computing device. In the field of deep learning, neural network layers and even the entire model are composed of operators, which correspond to the computational logic within the neural network layers. For example, a convolutional layer is an operator; the weight summation process in a fully-connected layer (FC layer) is also an operator.

[0058] Data format refers to the way data is stored and represented in a computer, such as integers, floating-point numbers (FP), characters, and strings. For example, integers include int8, int16, int32, int64, uint8, uint16, uint32, and uint64, while floating-point numbers include single-precision floating-point numbers (FP32), double-precision floating-point numbers (FP64), half-precision floating-point numbers (FP16), and lower-precision floating-point numbers such as FP8. Illustratively, different data formats correspond to different precisions of data. For example, FP32 can represent approximately 7 significant digits, while FP64 can represent approximately 16 significant digits. Higher precision allows for a wider range of represented values ​​and more accurate representation, but it also consumes more storage space and computing resources.

[0059] A normalized norm is an operation that standardizes or normalizes the norm of a vector or matrix. Common norms include the L1 norm (Manhattan norm) and the L2 norm (Euclidean norm). The purpose of normalization is usually to eliminate the influence of dimensions, making data of different dimensions or magnitudes comparable.

[0060] The application scenarios and implementation environment of this application are described below.

[0061] This application can be applied to scenarios where accelerator chips are used to execute AI model-related operators. These AI model-related operators can be operators used during AI model training, such as those in the forward computation or backward computation process, or they can be operators used during AI model inference; this application does not limit the specific operators. Currently, many accelerator chips support multiple precision data formats (such as single-precision FP32 and half-precision FP16) to support mixed-precision training of AI models. Based on this, this application provides a data processing method for executing AI model-related operators using accelerator chips, enabling adaptive precision data format configuration for AI models. The implementation environment of this application is described below with reference to Figure 1.

[0062] Figure 1 is a schematic diagram of an implementation environment provided in an embodiment of this application. As shown in Figure 1, the implementation environment includes a computing system 100, which includes a host 101 and an acceleration chip 102, and the host 101 and the acceleration chip 102 are communicatively connected.

[0063] Host 101 is used to provide AI services. In this embodiment, host 101 can control acceleration chip 102 to execute relevant operators of the AI ​​model. For example, in response to a training request from the AI ​​model, host 101 sends execution instructions for each forward computation operator to acceleration chip 102 during forward computation and execution instructions for each backward computation operator during backward computation. As another example, in response to an inference request from the AI ​​model, host 101 sends execution instructions for each operator in the AI ​​model to acceleration chip 102. This process can also be understood as loading the training and inference tasks of the AI ​​model into acceleration chip 102 for execution. Furthermore, the number of hosts 101 can be one or more, and this application does not limit this.

[0064] Accelerator chip 102 provides computing power for the training or inference tasks of AI models, thereby accelerating the training or inference process of the AI ​​model. For example, accelerator chip 102 can be a GPU, XPU, IPU, TPU, DSA chip, etc., and this application is not limited to these. Furthermore, the number of accelerator chips 102 can be one or more, and this application does not limit this. In the embodiments of this application, accelerator chip 102 can determine the numerical statistics of the output data of the first operator in the AI ​​model. This numerical statistics can serve as the basis for configuring the data format of the second operator, thereby realizing adaptive precision data format configuration for the AI ​​model. The architecture of accelerator chip 102 is shown in Figure 3 below and will not be repeated here.

[0065] The host 101 and acceleration chip 102 described above can be integrated into a single computing device or configured separately; this application does not limit this. Illustratively, taking the integration of host 101 and acceleration chip 102 into a single computing device as an example, host 101 and acceleration chip 102 are connected via a peripheral component interconnect express (PCIe) link, and data exchange between host 101 and acceleration chip 102 occurs through the PCIe link. The computing device described above can be a terminal or a server. A terminal can be, for example, at least one of a smartphone, game console, in-vehicle terminal, smart home device, desktop computer, augmented reality terminal, tablet computer, e-book reader, and laptop computer. A server can be, for example, a standalone physical server, a server cluster composed of multiple physical servers, a distributed file system, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Taking cloud servers as an example, computing devices can also be referred to as cloud platforms (short for cloud computing platforms). These platforms provide computing, networking, and storage capabilities based on hardware and software resources. Through the network "cloud," massive amounts of data are processed and analyzed remotely before being returned to the user. They are characterized by large scale, distributed nature, virtualization, high availability, scalability, on-demand service, and security. Cloud platforms can rapidly deploy and publish configurable computing resources with relatively low management costs or low interaction complexity between users and service providers.

[0066] Furthermore, the networks mentioned above include, but are not limited to, data center networks, storage area networks (SANs), local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), mobile, wired or wireless networks, private networks, or virtual private networks (VPNs). In some implementations, technologies and / or formats, including Hypertext Markup Language (HTML) and Extensible Markup Language (XML), are used to represent data exchanged over the network. Additionally, conventional encryption technologies such as Secure Sockets Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or part of the links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0067] The hardware structure of the above computing system is described below.

[0068] This application provides a computing device that can be configured as the aforementioned computing system. Referring to FIG2, FIG2 is a schematic diagram of the hardware structure of a computing device provided in an embodiment of this application. As shown in FIG2, the computing device 200 includes a memory 201, a processor 202, a communication interface 203, a bus 204, and at least one acceleration chip 102. The memory 201, processor 202, communication interface 203, and at least one acceleration chip 102 are interconnected via the bus 204.

[0069] Memory 201 refers to a device for storing data, which may be a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code having an instruction or data structure form and accessible by a computer, but not limited thereto. Indicatively, memory 201 is used to store at least a piece of program code. When the program code stored in memory 201 is executed by processor 202, processor 202 performs the steps executed by the host in the following method embodiments.

[0070] The processor 202 can be a network processor (NP), a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), or an integrated circuit used to control the execution of the program in this application. The processor 202 can be a single-core processor or a multi-core processor. There can be one or more processors 202. The memory 201 and the processor 202 can be separately configured or integrated together.

[0071] The communication interface 203 uses a transceiver module, such as a transceiver, to enable communication between the computing device 200 and other devices or communication networks. For example, data can be acquired through the communication interface 203.

[0072] Bus 204 may include a pathway for transmitting information between various components of computing device 200 (e.g., memory 201, processor 202, communication interface 203, at least one accelerator chip 102).

[0073] For any acceleration chip 102, the hardware structure of the acceleration chip 102 is shown in FIG3, which is a schematic diagram of the hardware structure of an acceleration chip provided in an embodiment of this application. As shown in FIG3, the acceleration chip 102 includes a communication interface 1021, a first computing unit 1022, a second computing unit 1023, a storage unit 1024, and a chip bus 1025. The communication interface 1021, the first computing unit 1022, the second computing unit 1023, and the storage unit 1024 are interconnected through the chip bus 1025.

[0074] The communication interface 1021 is used to provide program instructions and / or data. The communication interface 1021 includes a PCIe communication interface, other general peripheral interfaces, etc., which are not limited in this application. For example, the accelerator chip 102 exchanges data with the host through the PCIe communication interface. Or, for example, the accelerator chip 102 communicates with other devices or communication networks through peripheral interfaces.

[0075] The first calculation unit 1022 is used to provide a function for determining numerical statistical information, wherein the numerical statistical information is used to characterize the distribution characteristics of the numerical values ​​in the data. For example, the numerical statistical information includes at least one of normalized norm, mean, and median, and this application is not limited thereto. In some embodiments, the first calculation unit 1022 is also called a numerical distribution calculation unit. Schematically, the first calculation unit 1022 includes an input register, a calculation kernel, and an output register, wherein the input register is used to store data for which the numerical statistical information to be determined is to be determined, the calculation kernel is used to determine the numerical statistical information of the data based on the data stored in the input register, and the output register is used to store the numerical statistical information output by the calculation kernel. The number of input registers, calculation kernels, and output registers can be one or more, and this application does not limit this. The calculation kernel is, for example, an arithmetic logic unit (ALU) or a control unit (CU), and this application does not limit this. In addition, the number of first computing units 1022 can be one or more, and this application does not limit this. For example, if there are multiple first computing units 1022, the multiple first computing units 1022 are used to calculate the numerical statistical information of multiple data in parallel to improve the overall processing efficiency.

[0076] The second computing unit 1023 provides operator execution functionality, where the operator refers to an operator in the AI ​​model, such as the Matmul matrix multiplication operator. In some embodiments, the second computing unit 1023 is also referred to as a matrix calculation unit. Schematally, the second computing unit 1023 includes an input register, a matrix calculation kernel, and an output register. The input register stores the input data of the operator to be executed, the matrix calculation kernel performs calculations on the input data, and the output register stores the output data. In some embodiments, the second computing unit 1023 further includes a precision conversion unit, which performs format conversion on the input data of the operator according to the data format of the operator, thereby converting the precision of the input data to obtain converted input data. The matrix calculation kernel then performs calculations based on the converted input data. Furthermore, the number of second computing units 1023 can be one or more; this application does not limit this. For example, if there are multiple second computing units 1023, they can be used to execute multiple operators in parallel to improve overall processing efficiency.

[0077] Storage unit 1024 provides storage space for acceleration chip 102. Storage unit 1024 includes high bandwidth memory (HBM), double data rate memory (DDR), static random-access memory (SRAM), or other types of dynamic storage devices capable of storing information and instructions. It may also include any other medium capable of carrying or storing desired program code in the form of instructions or data structures, accessible by a computer, but is not limited thereto. In embodiments of this application, storage unit 1024 stores AI models, input data of various operators in the AI ​​model, output data, etc. In some embodiments, storage unit 1024 also stores numerical statistical information; that is, after determining the numerical statistical information, the first calculation unit 1022 stores the numerical statistical information in storage unit 1024.

[0078] The chip bus 1025 may include a path for transmitting information between various components of the acceleration chip 102 (e.g., communication interface 1021, first computing unit 1022, second computing unit 1023, and storage unit 1024).

[0079] It should be noted that Figure 3 above is only a hardware structure diagram of an acceleration chip provided in this application. In some embodiments, the acceleration chip 102 may also include other components to achieve more functions. For example, the acceleration chip 102 may also include a task scheduler (TS), etc. This application is not limited to this.

[0080] This application also provides a computing device cluster, which includes at least one computing device, and the computing devices in the cluster can be connected to each other via a network. The network can be a wide area network (WAN) or a local area network (LAN), etc.

[0081] Based on the content shown in Figures 1 to 3 above, the application scenarios and implementation environment of this application have been introduced. The data processing method provided by this application will be introduced below.

[0082] As described above, the acceleration chip provided in this application can determine the numerical statistics of the output data of the first operator in the AI ​​model. This numerical statistics can serve as the basis for configuring the data format of the second operator, thereby enabling adaptive precision data format configuration for the AI ​​model. Referring to Figure 4 and the architecture of the aforementioned acceleration chip, the principle of the data processing method provided in this application will be explained below.

[0083] Figure 4 is a schematic diagram illustrating the principle of a data processing method provided in an embodiment of this application. As shown in Figure 4, the method is applied to a computing system, which includes a host and an accelerator chip, and the host and the accelerator chip are connected in communication. Taking the host controlling the accelerator chip to execute the training task of an AI model as an example, an instruction and data dispatch module runs in the computing system. This module is controlled by the host and is used to realize the distribution and flow of instructions and / or data in the computing system.

[0084] In illustrative terms, in response to the training request of the AI ​​model, the host sends execution instructions for each operator to the acceleration chip in sequence through the instruction and data dispatch module. It should be understood that there are usually dependencies between the operators in the AI ​​model. For example, the output data a of operator A and the output data b of operator B serve as the input data c of operator C. This is only an example and does not constitute a limitation of this application.

[0085] For the second computing unit on the accelerator chip, the second computing unit receives the execution instructions of each operator through the instruction and data dispatch module, and obtains the input data of each operator from the storage unit. Taking any operator as an example, when the execution instruction of the operator carries a data format, the second computing unit uses a precision conversion unit to convert the input data of the operator according to the data format carried by the execution instruction, thereby converting the precision of the input data and obtaining the converted input data (taking the matrix multiplication operator as an example, the second computing unit may include a first register and a second register, which are used to store the left matrix and the right matrix of the matrix multiplication operator, respectively). Through the matrix calculation kernel, calculation is performed based on the converted input data, and the output data of the operator is temporarily stored in the output register. Then, the second computing unit stores the output data of the operator in the storage unit so as to execute the next operator.

[0086] For the first computing unit on the accelerator chip, the first computing unit retrieves the output data of each operator from the storage unit through the instruction and data dispatch module. Taking any operator as an example, if the output data of the operator has already been stored in the storage unit, the first computing unit reads the output data of the operator into the input register, calculates the numerical distribution of the output data through the computing core, obtains the numerical statistics of the output data, and temporarily stores the numerical statistics in the output register. Then, the first computing unit stores the numerical statistics in the storage unit so as to calculate the numerical distribution of the next output data. This process means that the first computing unit, as a separate accompanying computing unit, performs pipeline optimization asynchronously with the second computing unit to ensure that the numerical distribution calculation does not lead to efficiency loss.

[0087] For the host computer, the instruction and data dispatch module reads the numerical statistics of each output data from the storage unit. Based on the operator to be executed, the host computer uses solvers configured on the host (including data formats of different precisions, precision solvers, and operator solvers) to determine the numerical statistics of the input data of the operator, and then determines the data format of the operator. This process is equivalent to determining which data format is suitable for the current operator based on the distribution characteristics of the numerical values ​​indicated by the numerical statistics, thus achieving adaptive data format configuration. Next, the host computer generates the execution instruction for the operator and sends it to the accelerator chip. For example, the execution instruction may include the data format of the operator, so that the accelerator chip can convert the input data of the operator according to the specified data format.

[0088] As can be seen, by adding a hardware unit, namely the first computing unit, to the acceleration chip, the numerical statistics of each operator in the AI ​​model can be automatically determined based on the output data of each operator. Based on this, when the host issues the execution instruction of the operator, it can configure a suitable data format for the operator according to the numerical statistics of the input data corresponding to the operator. In this way, the data format configuration of the AI ​​model can be adapted to the precision, which is highly flexible and has the ability to be applied in multiple scenarios. Furthermore, the process of determining the numerical statistics is executed on a separate accompanying computing unit, which accelerates the processing and improves the computational efficiency while optimizing the convergence and accuracy of AI model training.

[0089] The above data processing method will be described in detail below through several method implementation examples.

[0090] Figure 5 is a flowchart of a data processing method provided in an embodiment of this application. As shown in Figure 5, the method is applied to a computing system, which includes a host and an acceleration chip. The acceleration chip includes a first computing unit and a second computing unit. Schematically, the data processing method includes the following steps 501 to 511.

[0091] 501. In response to the execution request of the AI ​​model, the host sends the execution instruction of the first operator to the second computing unit of the acceleration chip.

[0092] In the embodiments of this application, the execution request of the AI ​​model can be an inference request of the AI ​​model or a training request of the AI ​​model, and this application does not limit it in this regard. The first operator refers to any operator of the AI ​​model, and this application does not limit the type of the first operator or the position of the first operator in the AI ​​model.

[0093] Indicatively, in response to the execution request of the AI ​​model, the host computer compiles operators based on the AI ​​model's framework, i.e., generates operators for the AI ​​model. For example, operator generation involves three processes: input tensor description, weight data transformation, and output tensor description. Specifically, in the input tensor description, information such as the input dimension and memory size of each operator is calculated, and the format of the operator's input data is defined. In the weight data transformation, the weight parameters used by the operator undergo data format, shape transformation, and data compression. In the output tensor description, information such as the output dimension and memory size of the operator is calculated. After compiling operators based on the AI ​​model's framework, the host computer adapts the operators to the backend device interface (i.e., adapts the upper-layer operators to the lower-layer hardware devices), identifies the operator attributes, and sends the operator execution instructions to the acceleration chip based on the identified operators.

[0094] 502. The second computing unit of the acceleration chip receives the execution instructions of the first operator.

[0095] 503. The second computing unit of the acceleration chip executes the first operator based on the input data of the first operator and stores the output data of the first operator in the storage unit.

[0096] In this embodiment, the second computing unit obtains the input data of the first operator from the storage unit based on the execution instructions of the first operator, executes the first operator based on the input data, and stores the output data of the first operator in the storage unit. The method by which the second computing unit executes the operator is as described above and will not be repeated here.

[0097] 504. The first computing unit of the acceleration chip obtains the output data of the first operator from the storage unit.

[0098] In the embodiments of this application, the first computing unit and the storage unit are connected via a chip bus. In some embodiments, the first computing unit reads data from the storage unit at first time intervals. If the data in the storage unit is updated, the updated data, i.e., the output data of the first operator, is obtained. The first time interval is a preset time interval, which is not limited in this application. In other embodiments, after executing the first operator, the second computing unit sends a notification message to the first computing unit. Based on the notification message sent by the second computing unit, the first computing unit obtains the output data of the first operator from the storage unit. In still other embodiments, after the host sends the execution instruction of the first operator to the second computing unit, it sends a notification message to the first computing unit. Based on the notification message sent by the host, the first computing unit obtains the output data of the first operator from the storage unit. It should be understood that this application does not limit the timing of the first computing unit obtaining the output data of the first operator from the storage unit. The first computing unit may obtain the output data of the executed operator from the storage unit on its own during the execution of the AI ​​model according to the initial configuration of the computing system, or it may obtain the output data of the executed operator from the storage unit according to the notification message sent by the host or the second computing unit, etc. This application is not limited to this.

[0099] 505. The first computing unit of the acceleration chip determines the numerical statistics of the output data based on the output data of the first operator. The numerical statistics indicate the distribution characteristics of the numerical values ​​in the output data.

[0100] In this embodiment, numerical statistical information indicating the distribution characteristics of numerical values ​​in the output data means reflecting the distribution of numerical values ​​in the output data through numerical statistical information. For example, it could describe the relationship between the maximum and minimum values ​​and zero, or the range of numerical values ​​in the output data. Indicatively, numerical statistical information includes at least one of normalized norm, mean, and median. It should be understood that numerical statistical information can include at least one type of statistical information. For example, when numerical statistical information includes multiple types of statistical information, it can reflect the distribution characteristics of numerical values ​​from multiple dimensions; this application does not limit this.

[0101] In some embodiments, the first computing unit samples the output data of the first operator to obtain sampled data, and determines numerical statistics based on the magnitude of the sampled data. For example, referring to Figure 6, which is a schematic diagram of a data sampling process provided in this application, as shown in Figure 6, taking the output data of the first operator as part of the input data (i.e., the left matrix) of the subsequent second operator as an example, the first computing unit samples the two diagonals of the matrix to obtain sampled data. Furthermore, the first and second operators are dependent on each other; the second operator refers to the operator in the AI ​​model whose execution order is after the first operator. The input data of the second operator includes the output data of the first operator. It should be understood that, taking the matrix multiplication operator as an example in the figure, the input data of the second operator includes not only the output data of the first operator but also the output data of the third operator. The execution process of the third operator is similar to that of the first operator, and therefore will not be described in detail here. Moreover, the figures are merely illustrative examples. In some embodiments, other sampling strategies can be used, such as using the outer perimeter of the matrix and the two diagonals of the matrix as sampled data, or using a Z-shaped sampling strategy, etc. This application does not limit the scope of these methods. By using sampling to determine numerical statistics based on the magnitude of the sampled data, the amount of data processing can be reduced, and the overall data distribution characteristics can be approximated with the lowest possible time complexity.

[0102] Schematic, taking the normalized norm as an example of numerical statistical information, the norm can be, for example, the L1 norm (Manhattan norm), the L2 norm (Euclidean norm), etc., where the L1 norm is the sum of the absolute values ​​of the matrix elements, and the L2 norm is the square root of the sum of the squares of the matrix elements. Taking the L2 norm as an example, the first calculation unit can use the following formula (1) to determine the numerical statistical information of the sampling matrix:

[0103] Here, norm(T) represents the normalized L2 norm of the sampling matrix T, ||T||² represents the L2 norm of the sampling matrix T, and #(T) represents the number of matrix elements in the sampling matrix. Illustratively, the normalized norm value is typically between 0 and 1, or within a specific normalization range. This normalized norm value can be used to evaluate the numerical distribution characteristics of the matrix. For example, a smaller normalized norm value indicates that the element values ​​in the matrix are relatively small or more concentrated, while a larger normalized norm value indicates that the element values ​​are larger or more dispersed. Furthermore, describing the numerical distribution based on the normalized norm of the samples has generalizability for model types and data formats, and the computational cost is far less than that of matrix calculations, without affecting overall computational efficiency.

[0104] In some embodiments, the first computing unit determines the numerical statistics corresponding to each matrix block in the output data based on the output data of the first operator. Accordingly, the numerical statistics of the output data indicate the distribution characteristics of the numerical values ​​of each matrix block in the output data. This application does not limit the granularity of matrix block division; for example, the size of the matrix block can be the smallest adaptive unit of the matrix computing unit, etc. In this way, numerical statistics can be determined at the matrix block level, providing technical support for providing more accurate data format configurations subsequently.

[0105] 506. The first computing unit of the acceleration chip stores the statistical information of this value into the storage unit.

[0106] In some embodiments, the first computing unit stores the numerical statistics and data identifier in the storage unit. The data identifier is used to identify that the numerical statistics are obtained based on the output data of the first operator. In some embodiments, a memory space is pre-allocated in the storage unit for storing the numerical statistics of the operator output data; this application does not limit this.

[0107] 507. The host obtains the numerical statistics from the storage unit, and determines the target data format of the second operator based on the numerical statistics. The second operator is used to calculate the first data, and the target data format indicates the precision of the first data.

[0108] In this embodiment, the first operator and the second operator are dependent on each other. The second operator refers to the operator that executes after the first operator in the AI ​​model. The input data of the second operator includes the output data of the first operator. It should be understood that, taking the matrix multiplication operator as an example, the input data of the second operator includes not only the output data of the first operator but also the output data of the third operator. The execution process of the third operator is the same as the aforementioned execution process of the first operator, and therefore will not be described again here. In some embodiments, the host reads the numerical statistics stored in the storage unit every second time interval to obtain the numerical statistics of the output data of the first operator. The second time interval is a preset time interval, and this application does not limit it. In other embodiments, the acceleration chip sends the updated numerical statistics from the storage unit to the host every third time interval. The third time interval is a preset time interval, and this application does not limit the method by which the host obtains the numerical statistics.

[0109] The second operator is used to calculate the first data. The precision of the first data may be the same as or different from the precision of the output data of the first operator. It should be understood that the target data format of the second operator can indicate the precision of the first data. Since the target data format may not be the same as the data format corresponding to the first operator, the precision of the first data may not be the same as the precision of the output data of the first operator.

[0110] Schematic, the target data format of the second operator indicates the precision of the first data, which means that when the second operator is executed, the data format of the data processed by the second operator is as follows. In this application, the type of the target data format is not limited. For example, it can be int8, int16, int32, int64, FP32, FP64, FP16, FP8, etc.

[0111] In some embodiments, the host determines the target data format of the second operator based on the numerical statistics of the output data of the first operator, including the following cases: Case 1: If the numerical statistics are in a first interval, the target data format is determined to be the first data format; Case 2: If the numerical statistics are in a second interval, the target data format is determined to be the second data format; wherein, the data precision indicated by the first data format is lower than the data precision indicated by the second data format. The first interval and the second interval can be set according to business requirements. For example, referring to Figure 7, Figure 7 is a schematic diagram of determining the target data format provided by an embodiment of this application. As shown in Figure 7, taking the numerical statistics as a normalized norm as an example, the first interval is represented as (τ... lb ,τ ub The second interval is represented as (inf, τ). lb ]∪[τ ub If the numerical statistics fall within the first interval, the target data format is determined to be the first data format, such as FP8, which is a low-precision data format. If the numerical statistics fall within the second interval, the target data format is determined to be the second data format, such as FP16, which is a high-precision data format. It should be noted that the content shown in Figure 7 is merely illustrative and does not constitute a limitation of this application. The relationship between numerical statistics and data format can be configured according to business needs. In some embodiments, the host can also determine the target data format of the second operator by considering underflow and precision loss. For example, the host determines the reference data format of the second operator as FP8 based on the numerical statistics. However, if underflow occurs when using FP8, the data format is switched, and the target data format of the second operator is determined to be FP16. This application does not limit this; in practical applications, the strategy for determining the target data format by the host can be adjusted according to business needs.

[0112] Furthermore, as can be seen from step 505 above, in some embodiments, the numerical statistics of the first operator's output data can indicate the distribution characteristics of the numerical values ​​of each matrix block in the output data. Accordingly, in this step, the host can determine the target data format of each matrix block in the first data based on the numerical statistics. This target data format indicates the precision of each matrix block in the first data. In this way, by determining the target data format of the second operator at the matrix block level, fine-grained configuration of the data format is achieved, maximizing computational efficiency while ensuring precision.

[0113] It should be noted that in some embodiments, the host may also offload the process of determining the target data format based on numerical statistical information to the acceleration chip for execution, and this application does not limit this.

[0114] 508. The host sends the execution instruction of the second operator to the second computing unit of the acceleration chip. The execution instruction carries the target data format.

[0115] 509. The second computing unit of the acceleration chip receives the execution instructions of the second operator.

[0116] 510. The second computing unit of the acceleration chip performs format conversion on the input data of the second operator according to the target data format to obtain the first data.

[0117] In this embodiment of the application, the second computing unit obtains the input data of the second operator from the storage unit based on the execution instructions of the second operator, and performs format conversion on the output data of the first operator in the input data of the second operator according to the target data format to obtain the first data, the precision of which conforms to the target data format.

[0118] 511. The second computing unit of the acceleration chip executes the second operator based on the first data and stores the output data of the second operator into the storage unit.

[0119] In some embodiments, if the input data of the second operator also includes the output data of the third operator, the second calculation unit similarly performs format conversion on the output data of the third operator to obtain the second data. Accordingly, in this step, the second calculation unit executes the second operator based on the first data and the second data.

[0120] In addition, after the second computing unit executes the second operator, it stores the output data of the second operator in the storage unit. Then, based on the same process as steps 504 to 508 mentioned above, the first computing unit determines the numerical statistics of the output data of the second operator so that the host can send the execution instructions of other operators after the second operator to the acceleration chip according to the numerical statistics. This will not be described in detail here.

[0121] It should be noted that the data processing method described in steps 501 to 511 above is based on configuring the data format of the second operator. It should be understood that the data format of the first operator can be configured similarly, and will not be elaborated upon here. Furthermore, if the first operator is the first operator corresponding to the execution request of the AI ​​model, then the input data of the first operator is the model input data of the AI ​​model. The host can then send a numerical distribution calculation instruction to the first computing unit. Based on this numerical distribution calculation instruction, the first computing unit determines the numerical statistics of the model input data, and then, based on this numerical statistics, sends the execution instruction of the first operator to the second computing unit.

[0122] In summary, the data processing method provided in this application adds a hardware unit, namely the first computing unit, to the acceleration chip. This unit can determine the numerical statistics of the first operator of the AI ​​model based on its output data. Based on this, when the host issues the execution instruction of the second operator, it can configure a suitable data format for the second operator based on the numerical statistics of the input data corresponding to the second operator. In this way, the AI ​​model is configured with adaptive precision data format, which is highly flexible. Furthermore, by placing the determination process of numerical statistics and the execution process of the operator on different computing units and executing them asynchronously, the computational efficiency can be improved.

[0123] In the embodiment shown in Figure 5 above, when the host sends the execution instruction of the second operator to the acceleration chip, the execution instruction carries the target data format of the second operator. The second computing unit of the acceleration chip performs format conversion on the input data of the second operator according to the target data format to obtain the first data. In other embodiments, the host may also perform format conversion on the input data of the second data; or, the acceleration chip has multiple second computing units, and the second computing units that do not execute the second operator perform the format conversion on the input data of the second data; or, the computing system includes multiple acceleration chips, and the acceleration chips that do not execute the second operator perform the format conversion on the input data of the second data, etc. This application does not limit this.

[0124] Referring to Figure 8, another data processing method provided in this application will be introduced, taking the format conversion of the host's input data to the second operator as an example.

[0125] Figure 8 is a flowchart of another data processing method provided in an embodiment of this application. As shown in Figure 8, the method is applied to a computing system, which includes a host and an acceleration chip. The acceleration chip includes a first computing unit and a second computing unit. Schematically, the data processing method includes the following steps 801 to 811.

[0126] 801. In response to the execution request of the AI ​​model, the host sends the execution instruction of the first operator to the acceleration chip.

[0127] 802. The second computing unit of the acceleration chip receives the execution instruction of the first operator.

[0128] 803. The second computing unit of the acceleration chip executes the first operator based on the input data of the first operator and stores the output data of the first operator in the storage unit.

[0129] 804. The first computing unit of the acceleration chip obtains the output data of the first operator from the storage unit.

[0130] 805. The first computing unit of the acceleration chip determines the numerical statistics of the output data based on the output data of the first operator. The numerical statistics indicate the distribution characteristics of the numerical values ​​in the output data.

[0131] 806. The first computing unit of the acceleration chip stores the statistical information of this value into the storage unit.

[0132] 807. The host obtains the numerical statistics from the storage unit, and determines the target data format of the second operator based on the numerical statistics. The second operator is used to calculate the first data, and the target data format indicates the precision of the first data.

[0133] Steps 801 to 807 described above are the same as steps 501 to 507 in the embodiment shown in Figure 5 above, so they will not be repeated here.

[0134] 808. The host performs format conversion on the input data of the second operator according to the target data format to obtain the first data.

[0135] 809. The host sends the first data and the second operator execution instructions to the acceleration chip.

[0136] The host obtains the input data of the second operator from the storage unit, which includes the output data of the first operator. It then performs format conversion on the output data of the first operator to obtain the first data and sends the first data and the execution instructions of the second operator to the acceleration chip.

[0137] 810. The second computing unit of the acceleration chip receives the first data and the execution instructions of the second operator.

[0138] 811. The second computing unit of the acceleration chip executes the second operator based on the first data.

[0139] The above method can save computing resources in the second computing unit.

[0140] Referring to Figure 9 below, another data processing method provided in this application is introduced by taking an acceleration chip with multiple second computing units, where the second computing unit (hereinafter referred to as the target computing unit) that does not execute the second operator performs format conversion on the input data of the second data as an example.

[0141] Figure 9 is a flowchart of another data processing method provided in an embodiment of this application. As shown in Figure 9, the method is applied to a computing system, which includes a host and an acceleration chip. The acceleration chip includes a first computing unit and a second computing unit. Schematically, the data processing method includes the following steps 901 to 911.

[0142] 901. In response to the execution request of the AI ​​model, the host sends the execution instruction of the first operator to the acceleration chip.

[0143] 902. The second computing unit of the acceleration chip receives the execution instruction of the first operator.

[0144] 903. The second computing unit of the acceleration chip executes the first operator based on the input data of the first operator and stores the output data of the first operator in the storage unit.

[0145] 904. The first computing unit of the acceleration chip obtains the output data of the first operator from the storage unit.

[0146] 905. The first computing unit of the acceleration chip determines the numerical statistics of the output data based on the output data of the first operator. The numerical statistics indicate the distribution characteristics of the numerical values ​​in the output data.

[0147] 906. The first computing unit of the acceleration chip stores the statistical information of this value into the storage unit.

[0148] 907. The host obtains the numerical statistics from the storage unit, and determines the target data format of the second operator based on the numerical statistics. The second operator is used to calculate the first data, and the target data format indicates the precision of the first data.

[0149] Steps 901 to 907 described above are the same as steps 501 to 507 in the embodiment shown in Figure 5 above, so they will not be repeated here.

[0150] 908. The host sends a format conversion instruction to the target computing unit of the acceleration chip. This format conversion instruction carries the target data format.

[0151] The acceleration chip has multiple second computing units, and the target computing unit is the second computing unit that does not execute the second operator among the multiple second computing units.

[0152] 909. The target computing unit of the acceleration chip receives the format conversion instruction, performs format conversion on the input data of the second operator according to the target data format, obtains the first data, and stores the first data in the storage unit.

[0153] 910. The host sends the execution instruction of the second operator to the second computing unit of the acceleration chip.

[0154] 911. The second computing unit of the acceleration chip receives the execution instruction of the second operator, retrieves the first data from the storage unit, and executes the second operator based on the first data.

[0155] It should be noted that this application does not limit the execution order of steps 908 and 910. For example, the host can simultaneously send the execution instruction of the second operator and the format conversion instruction. Alternatively, the host can first send the execution instruction of the second operator to the second computing unit, and then send the format conversion instruction to the target computing unit. After the target computing unit stores the first data in the storage unit, it notifies the second computing unit to execute the second operator.

[0156] The above method can save computing resources in the second computing unit.

[0157] Referring to Figure 10, another data processing method provided in this application is described below, with the example of a computing system including multiple acceleration chips, where the input data of the second data is formatted by an acceleration chip that does not execute the second operator (hereinafter referred to as the second acceleration chip).

[0158] Figure 10 is a flowchart of another data processing method provided in an embodiment of this application. As shown in Figure 10, the method is applied to a computing system, which includes a host and multiple acceleration chips. The first acceleration chip includes a first computing unit and a second computing unit. Schematically, the data processing method includes the following steps 1001 to 1011.

[0159] 1001. In response to the execution request of the AI ​​model, the host sends the execution instruction of the first operator to the first acceleration chip.

[0160] 1002. The second computing unit of the first acceleration chip receives the execution instruction of the first operator.

[0161] 1003. The second computing unit of the first acceleration chip executes the first operator based on the input data of the first operator and stores the output data of the first operator in the storage unit.

[0162] 1004. The first computing unit of the first acceleration chip obtains the output data of the first operator from the storage unit.

[0163] 1005. The first computing unit of the first acceleration chip determines the numerical statistics of the output data based on the output data of the first operator. The numerical statistics indicate the distribution characteristics of the numerical values ​​in the output data.

[0164] 1006. The first computing unit of the first acceleration chip stores the statistical information of the value into the storage unit.

[0165] 1007. The host obtains the numerical statistics from the storage unit, and determines the target data format of the second operator based on the numerical statistics. The second operator is used to calculate the first data, and the target data format indicates the precision of the first data.

[0166] Steps 1001 to 1007 described above are the same as steps 501 to 507 in the embodiment shown in Figure 5 above, so they will not be repeated here.

[0167] 1008. The host sends a format conversion instruction to the second acceleration chip, which carries the target data format.

[0168] The computing system includes multiple acceleration chips. The structure of the second acceleration chip may be the same as or different from that of the first acceleration chip. This application does not limit this.

[0169] 1009. The second acceleration chip receives a format conversion instruction, obtains the input data of the second operator from the first acceleration chip, performs format conversion on the input data of the second operator according to the target data format, obtains the first data, and sends the first data to the first acceleration chip.

[0170] 1010. The host sends the execution instruction of the second operator to the first acceleration chip.

[0171] 1011. The second computing unit of the first acceleration chip receives the execution instruction of the second operator and executes the second operator based on the first data sent by the second acceleration chip.

[0172] It should be noted that this application does not limit the execution order of steps 1008 and 1010. For example, the host can simultaneously send the execution instruction of the second operator and the format conversion instruction. Alternatively, the host can first send the execution instruction of the second operator to the first acceleration chip, and then send the format conversion instruction to the second acceleration chip. After the second acceleration chip stores the first data in the storage unit, it sends the first data back to the first acceleration chip, so that the first acceleration chip can execute the second operator.

[0173] The above method can save computing resources of the first acceleration chip.

[0174] It should be noted that Figures 8 to 10 above are only illustrative examples. In practical applications, the execution entity for format conversion of operators in AI models can be configured according to business needs, and this application does not limit this.

[0175] Based on the foregoing method embodiments, this application also provides a data processing device, which can implement some or all of the functions of the aforementioned acceleration chip through software, hardware, or a combination of both. Schematic, FIG11 is a structural diagram of a data processing device provided in an embodiment of this application. As shown in FIG11, the device is configured on the acceleration chip, and the device includes a determining module 1101 and a receiving module 1102.

[0176] The determination module 1101 is used to determine the numerical statistics of the output data based on the output data of the first operator in the artificial intelligence (AI) model. The numerical statistics indicate the distribution characteristics of the numerical values ​​in the output data.

[0177] The receiving module 1102 is used to receive the execution instructions of the second operator in the AI ​​model sent by the host. The second operator is used to calculate the first data, and the accuracy of the first data is determined by numerical statistics.

[0178] In some embodiments, the precision of the first data differs from the precision of the output data.

[0179] In some embodiments, the execution instructions carry a target data format for a second operator, the target data format indicating the precision of the first data, and the apparatus further includes:

[0180] The format conversion module is used to convert the input data of the second operator according to the target data format to obtain the first data. The input data of the second operator includes the output data of the first operator.

[0181] In some embodiments, the target data format indicates the data precision of each matrix block in the first data.

[0182] In some embodiments, the device further includes a storage module for storing numerical statistical information.

[0183] In some embodiments, the second operator refers to the operator in the AI ​​model that is executed after the first operator; the determining module 1101 is used to determine numerical statistics based on the output data of the first operator after the first operator is executed.

[0184] In some embodiments, the determining module 1101 is used to sample the output data to obtain sampled data, and determine numerical statistical information based on the numerical magnitude of the sampled data.

[0185] In some embodiments, the numerical statistics include at least one of the normalized norm, mean, and median.

[0186] It should be noted that the data processing apparatus provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the apparatus can be divided into different functional modules to complete all or part of the functions described above. In addition, the data processing apparatus and data processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0187] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items with substantially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor does it limit the quantity or execution order. It should also be understood that although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are merely used to distinguish one element from another. For example, without departing from the various examples described, a first operator can be referred to as a second operator, and similarly, a second operator can be referred to as a first operator. Both the first and second operators can be operators, and in some cases, they can be separate and distinct operators.

[0188] In this application, the term "at least one" means one or more, and the term "multiple" means two or more. For example, multiple operators means two or more operators.

[0189] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0190] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of program structure information. This program structure information includes one or more program instructions. When these program instructions are loaded and executed on a computing device, the processes or functions according to the embodiments of this application are generated, in whole or in part.

[0191] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0192] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A data processing method, characterized in that, Applied to an acceleration chip, the acceleration chip including a first computing unit and a second computing unit, the method includes: The first computing unit determines the numerical statistics of the output data based on the output data of the first operator in the artificial intelligence (AI) model. The numerical statistics indicate the distribution characteristics of the numerical values ​​in the output data. The second computing unit receives the execution instruction of the second operator in the AI ​​model sent by the host. The second operator is used to calculate the first data, and the accuracy of the first data is determined by the numerical statistics.

2. The method according to claim 1, characterized in that, The precision of the first data is different from the precision of the output data.

3. The method according to claim 1 or 2, characterized in that, The execution instruction carries the target data format of the second operator, the target data format indicating the precision of the first data, and the method further includes: The second calculation unit performs format conversion on the input data of the second operator according to the target data format to obtain the first data. The input data of the second operator includes the output data of the first operator.

4. The method according to claim 3, characterized in that, The target data format indicates the data precision of each matrix block in the first data.

5. The method according to any one of claims 1 to 4, characterized in that, The first computing unit includes an input register, a computing core, and an output register. The input register is used to store the output data, the computing core is used to determine the numerical statistics based on the output data, and the output register is used to store the numerical statistics.

6. The method according to any one of claims 1 to 5, characterized in that, The acceleration chip further includes a storage unit, which is connected to the first computing unit via a chip bus. The method further includes: The first calculation unit stores the numerical statistics information into the storage unit.

7. The method according to any one of claims 1 to 6, characterized in that, The second operator refers to the operator in the AI ​​model that is executed after the first operator; The first computing unit determines the numerical statistics of the output data based on the output data of the first operator in the artificial intelligence (AI) model, including: after the second computing unit executes the first operator, the first computing unit determines the numerical statistics based on the output data of the first operator.

8. The method according to any one of claims 1 to 7, characterized in that, The first computing unit determines the numerical statistics of the output data based on the output data of the first operator in the artificial intelligence (AI) model, including: The first calculation unit samples the output data to obtain sampled data, and determines the numerical statistics based on the magnitude of the sampled data.

9. The method according to any one of claims 1 to 8, characterized in that, The numerical statistical information includes at least one of the following: normalized norm, mean, and median.

10. A data processing method, characterized in that, Applied to a host, the method includes: An execution instruction for the second operator in the AI ​​model is sent to the acceleration chip. The second operator is used to calculate the first data. The accuracy of the first data is determined by the numerical statistics of the output data of the first operator in the AI ​​model. The numerical statistics indicate the distribution characteristics of the numerical values ​​in the output data.

11. The method according to claim 10, characterized in that, The execution instruction carries the target data format of the second operator, the target data format indicating the precision of the first data, and the method further includes: Based on the numerical statistics, the target data format is determined, and the input data of the second operator includes the output data of the first operator.

12. The method according to claim 11, characterized in that, Determining the target data format based on the numerical statistical information includes: If the numerical statistics are within the first interval, the target data format is determined to be the first data format. If the numerical statistics are in the second interval, the target data format is determined to be the second data format. The data precision indicated by the first data format is lower than that indicated by the second data format.

13. A data processing method, characterized in that, Applied to a computing system, the system including a host and an acceleration chip, the acceleration chip including a first computing unit and a second computing unit, the method includes: The first computing unit determines the numerical statistics of the output data based on the output data of the first operator in the AI ​​model, and the numerical statistics indicate the distribution characteristics of the numerical values ​​in the output data; The host sends an execution instruction for the second operator in the AI ​​model to the second computing unit. The second operator is used to calculate the first data, and the accuracy of the first data is determined by the numerical statistics. The second computing unit receives the execution instruction and executes the second operator.

14. The method according to claim 13, characterized in that, The execution instruction carries the target data format of the second operator, and the target data format indicates the precision of the first data.

15. The method according to claim 14, characterized in that, The method further includes: The host determines the target data format based on the numerical statistics. The second calculation unit performs format conversion on the input data of the second operator according to the target data format to obtain the first data. The input data of the second operator includes the output data of the first operator.

16. An accelerator chip, characterized in that, The acceleration chip includes a communication interface, a first computing unit, and a second computing unit. The communication interface is used to provide program instructions and / or data. The acceleration chip is used to implement the data processing method as described in any one of claims 1 to 9.

17. A computing device, characterized in that, It includes a host and an acceleration chip, wherein the host is used to control the acceleration chip, and the acceleration chip is used to implement the data processing method as described in any one of claims 1 to 9.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store at least one piece of program code, which, when executed by the acceleration chip, causes the acceleration chip to implement the data processing method as described in any one of claims 1 to 9.

19. A computer program product, characterized in that, When the computer program product is run on the acceleration chip, the acceleration chip performs the data processing method as described in any one of claims 1 to 9.

Images