Data processing device and operation method, computing device, storage medium

Abstract

The present disclosure provides a data processing device and an operating method thereof, a computing device and a storage medium. The data processing device comprises a multiply-add operation array and a configuration unit, wherein the multiply-add operation array comprises a plurality of multiply-add calculation units, the configuration unit is coupled with the multiply-add operation array, and the configuration unit has at least a first configuration state and a second configuration state. When the configuration unit is in the first configuration state, the plurality of multiply-add calculation units are divided into N groups, and M multiply-add calculation units in each group are configured to be coupled with each other for accumulating M calculation results in each group. When the configuration unit is in the second configuration state, the plurality of multiply-add calculation units are divided into Q groups, and P multiply-add calculation units in each group are configured to be coupled with each other for accumulating P calculation results in each group. The data processing device can transform the configuration state of the multiply-add operation array, so that the parameters in the neural network algorithm can be more fully mapped to the hardware units, thereby improving the utilization rate of the hardware units.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to a data processing apparatus and an operating method, a computing apparatus, and a storage medium. BACKGROUND

[0002] At present, in the field of artificial intelligence, deep learning technology has achieved very good results in image recognition, speech recognition, automatic driving and other applications. Deep learning involves, for example, convolutional neural networks (CNN), deep neural networks (DNN), recurrent neural networks (RNN), etc. Neural networks abstract the neural network of the human brain from the perspective of information processing, establish a certain simple model, and form different networks according to different connection modes. In order to speed up the operation, the neural network processor will adopt the method of multiply-add operation array to realize it, and map the dimensions in the neural network algorithm to each dimension of the multiply-add operation array.

[0003] DISCLOSURE

[0004] Some embodiments of the present disclosure provide a data processing apparatus, comprising a multiply-add operation array and a configuration unit, wherein the multiply-add operation array comprises a plurality of multiply-add calculation units, the configuration unit is coupled with the multiply-add operation array, and the configuration unit has at least a first configuration state and a second configuration state. When the configuration unit is in the first configuration state, the plurality of multiply-add calculation units are controlled to be divided into N groups, and M multiply-add calculation units in each group are configured to be coupled with each other for accumulating M calculation results in each group. When the configuration unit is in the second configuration state, the plurality of multiply-add calculation units are controlled to be divided into Q groups, and P multiply-add calculation units in each group are configured to be coupled with each other for accumulating P calculation results in each group, wherein M, N, P and Q are all integers greater than 1, and N and Q are not equal.

[0005] For example, in the data processing apparatus provided by some embodiments of the present disclosure, M is a positive integer multiple of 3.

[0006] For example, in the data processing apparatus provided by some embodiments of the present disclosure, P is a positive integer power of 2.

[0007] For example, in the data processing apparatus provided by some embodiments of the present disclosure, the control unit is configured to obtain a calculation parameter, and control the configuration state of the configuration unit according to the calculation parameter.

[0008] For example, in the data processing apparatus provided by some embodiments of the present disclosure, the control unit is further configured to control the N groups of multiply-add calculation units to perform parallel calculation in the first configuration state, and the Q groups of multiply-add calculation units to perform parallel calculation in the second configuration state.

[0009] For example, the data processing apparatus provided by some embodiments of the present disclosure further comprises a plurality of adders, wherein the configuration unit is coupled with the plurality of adders, and in the first configuration state, at least one adder is allocated to each of the N groups of multiply-add calculation units to accumulate M calculation results in each group, and in the second configuration state, at least one adder is allocated to each of the Q groups of multiply-add calculation units to accumulate P calculation results in each group.

[0010] Some embodiments of the present disclosure further provide a computing apparatus comprising the data processing apparatus of any of the above embodiments.

[0011] Some embodiments of the present disclosure further provide an operation method of a data processing apparatus, for the data processing apparatus of any of the above embodiments, the operation method comprising: receiving an input matrix and a convolution kernel; controlling the configuration unit to configure the multiply-add operation array according to a calculation parameter; and performing convolution calculation on the input matrix and the convolution kernel using the configured multiply-add operation array.

[0012] For example, in the operation method of the data processing apparatus provided by some embodiments of the present disclosure, the step of controlling the configuration unit to configure the multiply-add operation array comprises: in response to the configuration state of the configuration unit switching from the first configuration state to the second configuration state, selecting at least one group of the N groups of multiply-add calculation units as a target split group and other groups as to-be-allocated groups, and allocating the multiply-add calculation units in the target split group to the to-be-allocated groups, so that the plurality of multiply-add calculation units are divided into Q groups, or in response to the configuration state of the configuration unit switching from the second configuration state to the first configuration state, selecting at least one multiply-add calculation unit in each of the Q groups of multiply-add calculation units as a target split unit, and merging the target split units into at least one new group, so that the plurality of multiply-add calculation units are divided into N groups.

[0013] Some embodiments of the present disclosure further provide a computer readable storage medium storing computer instructions, wherein the computer instructions, when executed by a processor, are used to implement the following steps: receiving an input matrix and a convolution kernel; controlling the configuration unit to configure the multiply-add operation array according to a calculation parameter; and performing convolution calculation on the input matrix and the convolution kernel using the configured multiply-add operation array. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are merely related to some embodiments of the present disclosure, but not limit the present disclosure.

[0015] Figure 1AThe input and output of a neuron in an exemplary convolutional neural network are shown.

[0016] Figure 1A A schematic diagram of a multi-channel convolution operation of an exemplary convolutional layer is shown.

[0017] Figure 1A An architecture schematic diagram of a neural network processor is shown.

[0018] Figure 1A A structure schematic diagram of a data processing apparatus provided by at least one embodiment of the present disclosure is shown.

[0019] Figure 1A A configuration state schematic diagram of a data processing apparatus provided by at least one embodiment of the present disclosure is shown.

[0020] Figure 1A Another configuration state schematic diagram of a data processing apparatus provided by at least one embodiment of the present disclosure is shown.

[0021] Figure 1A An operation method flow schematic diagram of a data processing apparatus provided by at least one embodiment of the present disclosure is shown.

[0022] Figure 1A A structure schematic diagram of an electronic device provided by some embodiments of the present disclosure; and

[0023] Figure 1A A schematic diagram of a storage medium provided by some embodiments of the present disclosure. DETAILED DESCRIPTION

[0024] In order for those skilled in the art to better understand the technical solutions of the present disclosure, the embodiments of the present disclosure will be further described in detail below in conjunction with the accompanying drawings. The specific embodiments and the accompanying drawings described herein are only used to explain the present disclosure, and not to limit the embodiments of the present disclosure. In the case of no conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.

[0025] For ease of description, only parts related to the embodiments of the present disclosure are shown in the accompanying drawings of the embodiments of the present disclosure, and parts unrelated to the embodiments of the present disclosure are not shown in the accompanying drawings. Each unit and module involved in the embodiments of the present disclosure can only correspond to one entity structure, or can be composed of multiple entity structures, or multiple units and modules can also be integrated into one entity structure. In the case of no conflict, the functions and steps marked in the flowcharts and block diagrams of the embodiments of the present disclosure can occur in an order different from that marked in the accompanying drawings.

[0026] The flowcharts and block diagrams of the embodiments of the present disclosure show the architecture, functions and operations of possible implementations of the systems, apparatuses, devices, methods according to the embodiments of the present disclosure. Each block in the flowcharts or block diagrams can represent a unit, a module, a program segment, code, which contains executable instructions for implementing the specified functions. Moreover, each block or combination of blocks in the block diagrams and flowcharts can be implemented by a hardware-based system for implementing the specified functions, or by a combination of hardware and computer instructions.

[0027] In order to keep the following description of the embodiments of the present disclosure clear and brief, detailed description of known functions and known components can be omitted. When any component of the embodiments of the present disclosure appears in more than one figure, the component is denoted by the same or similar reference numeral in each figure.

[0028] A neural network is a mathematical calculation model inspired by the structure of brain neurons and the principle of neural conduction. The way of intelligent calculation based on such a model is called brain-inspired calculation. For example, a neural network includes various forms of network structures, such as a back propagation (BP) neural network, a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network (LSTM), etc. For example, a convolutional neural network can be further divided into a fully convolutional network, a deep convolutional network, a U-shaped network (U-Net), etc.

[0029] For example, a general convolutional neural network generally includes an input end, an output end and a plurality of processing layers. For example, the input end is used to receive data to be processed, such as an image to be processed, etc., the output end is used to output the processing result, such as the processed image, etc., and the processing layers can include convolutional layers, pooling layers, batch normalization layers (BN), fully connected layers, etc. According to the structure of the convolutional neural network, the processing layers can include different contents and combination modes. After the input data is input into the convolutional neural network, the corresponding output is obtained through a plurality of processing layers. For example, the input data can complete convolution, upsampling, downsampling, standardization, full connection, flattening, etc. through a plurality of processing layers.

[0030] A convolutional layer is the core layer of a convolutional neural network, which applies several filters to the input data (input image or input feature map) to extract various types of features from the input data. The result of applying one filter to the input data is called a feature map, and the number of feature maps is equal to the number of filters. The output feature maps of a convolutional layer can be input to the next level of convolutional layer for further processing to obtain new feature maps. A pooling layer is an intermediate layer sandwiched between consecutive convolutional layers, which is used to reduce the size of the input data and to some extent reduce the overfitting phenomenon. There are many ways to implement pooling, including but not limited to: max-pooling, avg-pooling, random pooling, decimation (e.g., selecting fixed pixels), demuxout (splitting the input image into multiple smaller images), etc. Usually, the last subsampling layer or convolutional layer is connected to one or more fully connected layers, and the output of the fully connected layer is the final output, which can be a one-dimensional matrix, i.e., a vector.

[0031] Figure 1A The input and output of a neuron in a convolutional neural network are shown abstractly. As shown in Figure 1A C1, C2, etc. refer to different signal channels, and different filters are used to convolve the data on the C1, C2, etc. signal channels of a local receptive field (which contains multiple channels) to obtain the convolution result, which is input to the stimulus node, which calculates according to the corresponding function to obtain the feature information. For example, a convolutional neural network is usually a deep convolutional neural network, which can include at least five convolutional layers, for example, a VGG-16 neural network has 16 layers, and a GoogLeNet neural network has 22 layers, of course, other neural network structures can have more processing layers, the above is only an exemplary introduction to the neural network, and the present disclosure does not limit the structure of the neural network.

[0032] Figure 1A A schematic diagram of a convolutional layer performing multi-channel convolution operation is shown. As shown in Figure 1A M groups of R×S convolution kernels with C channels are used to convolve N groups of H×W input images (or input feature maps) with C channels to obtain N groups of E×F output feature maps with M channels, so the output feature maps include F / E / M dimensions in total.

[0033] Convolutional operations are characterized by high parallelism, meaning that multiple convolutional kernels can operate simultaneously with multiple input feature maps. Furthermore, due to the computational complexity of neural networks, especially for convolutional layers with large input feature maps, it's often necessary to decompose the computational operations within a convolutional layer. For example, convolutional operations for different parts of the same convolutional layer can be performed independently. These decomposed tasks are then processed in parallel by multiple processing units. The results are then combined to obtain the final result of the entire convolutional layer, which can then be used as input for the next convolutional layer.

[0034] Figure 1A A schematic diagram of a neural network processor architecture is shown. (For example...) Figure 1A As shown, the neural network processor includes a processing unit (PE) array 110, a global cache 120, and memory 130. The processing unit array 110 includes multiple rows and columns (e.g., 12 rows × 12 columns) of processing units, which are coupled to each other via on-chip interconnects and share the global cache 120, such as a network-on-chip (NoC). Each processing unit has computational capabilities and may also have its own local cache, such as including a multiply-accumulate (MAC) unit and a cache or register array for caching input vectors (or matrices). Each PE can access other surrounding PEs, its own local cache, and the global cache. The global cache 120 is further coupled to memory 130 via, for example, a bus.

[0035] During operation, data such as the convolutional kernel (Flt) and input feature map (Ifm) required for computation in a network layer (e.g., a convolutional layer) are read from memory 130 into global cache 120. Then, from global cache 120, the convolutional kernel (Flt) and input image (Img) are input into processing unit array 110 for computation. Computation tasks for different image pixels are assigned to different processing units (i.e., mapping is performed). The partial sum (Psum1) generated during the computation process is temporarily stored in global cache. If subsequent computations require further accumulation operations on the previously generated partial sum (Psum1), the required partial sum (Psum2) can be read from global cache 120 into processing unit array 110 for computation. The output feature map (Ofm) obtained after completing the computation of a convolutional layer can be output from global cache 120 to memory 130 for storage, for example, for use in the computation of the next network layer (e.g., a convolutional layer).

[0036] As described above, in order to speed up the operation, the neural network processor will adopt the method of multiply-add operation array to realize, and map the dimensions in the neural network algorithm to the dimensions of the multiply-add operation array. However, different neural network algorithms and each layer of the same neural network algorithm have different channel numbers and convolution kernel sizes, which may cause the dimension parameters of the neural network to be unable to divide the dimension parameters of the multiply-add operation array, and thus part of the multiply-add operation units are idle.

[0037] For example, the channel number of the input data in the first layer of the convolutional neural network is mostly 3. For example, when the convolutional neural network processes an image, the input data of the RGB image will be divided into three channels for processing, and the convolution kernel of the convolutional neural network usually has a size of 3x3, so that the operation number of the multiply-add operation in the convolutional neural network is usually a positive integer multiple of 3. However, in other types of neural networks such as transformer network model, the number of operation of most multiply-add operations is a positive integer power of 2, and the size of the multiply-add operation array of the neural network processor is usually a positive integer power of 2. For example, the three dimensions of the tensor core of Nvidia are positive integer powers of 2, which can well match most networks outside the first layer, but are not friendly to the first layer. On the one hand, it cannot be divided, and on the other hand, the neural network processor will have a dimension of data to add the product together after multiplication. In order to improve the parallelism of the input feature map, the accumulation dimension is usually made to be 16 or even larger, resulting in low utilization of the multiply-add operation array in the input channel direction of the first layer of the neural network. If the neural network parameters cannot simultaneously satisfy the division relationship with the parameters of the two algorithms when mapped to the parallel computing units of the hardware, part of the hardware resources will not be able to participate in the mapping. These hardware units that cannot participate in the mapping are idle, reducing the hardware utilization.

[0038] At least one embodiment of the present disclosure provides a data processing apparatus, which comprises a multiply-add operation array and a configuration unit. The multiply-add operation array comprises a plurality of multiply-add computing units. The configuration unit is coupled to the multiply-add operation array, and has at least a first configuration state and a second configuration state. When the configuration unit is in the first configuration state, the plurality of multiply-add computing units are divided into N groups, and M multiply-add computing units in each group are coupled to each other to accumulate M calculation results in each group. When the configuration unit is in the second configuration state, the plurality of multiply-add computing units are divided into Q groups, and P multiply-add computing units in each group are coupled to each other to accumulate P calculation results in each group. M, N, P and Q are all integers greater than 1, and N and Q are not equal.

[0039] The above embodiments of the present disclosure can change the configuration state of the configuration unit, so that the data processing apparatus is more matched with the parameters in the neural network algorithm, so that the input graph, feature graph and the like can be more fully mapped to the multiply-add computation units in the operation array, so that as many hardware units as possible are involved in the calculation, thereby obtaining higher hardware utilization.

[0040] The neural network (model) configured by the data processing apparatus to process may, for example, be a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a transformer, etc. The data processing apparatus can also be configured to process other multiply-accumulate computation operations, and embodiments of the present disclosure do not limit this.

[0041] For example, M is a positive integer multiple of 3, and P is a positive integer power of 2. In embodiments of the present disclosure, for example, the factors 3 and 2 are both typical parameters in a convolutional neural network, so introducing factors 3 and 2 in the accumulation dimension of each group can make the parameter mapping to the hardware unit have higher flexibility, reduce idle operation units, and improve hardware utilization. However, it should be noted that in other embodiments of the present disclosure, M and P can also take other numbers as factors, such as 5, 7, etc., and embodiments of the present disclosure do not limit this.

[0042] Figure 1A A structural schematic diagram of a data processing apparatus provided by at least one embodiment of the present disclosure is shown.

[0043] As shown in Figure 1A The data processing apparatus 10 includes a multiply-add operation array 101 and a configuration unit 102, and the data processing apparatus 10 may, for example, serve as a processing unit for processing neural network computation. The multiply-add operation array 101 includes a plurality of multiply-add computation units (MAC) 1011 arranged in an array, and each column of multiply-add computation units 1011 may, for example, be divided into a group. Each group of multiply-add computation units can perform multiplication and accumulate the results of the multiplication, respectively. Each multiply-add computation unit may, for example, include one or more multipliers and one or more adders.

[0044] For example, the configuration unit includes a plurality of selectors (selection switches) connected to the plurality of multiply-add computation units included in the multiply-add operation array 101, respectively. The selectors can perform logical operations under the control of a control signal to combine the plurality of multiply-add computation units, thereby realizing different configuration states.

[0045] In at least one example, the data processing apparatus can further comprise a control unit communicatively connected with the configuration unit and configured to receive control information and send control signals to the configuration unit to control the configuration unit to implement or maintain a certain configuration state. For example, a plurality of data processing apparatuses can share the control unit, i.e., the same control unit controls the configuration units of the plurality of data processing apparatuses.

[0046] Figure 1A A configuration state diagram of the data processing apparatus provided by at least one embodiment of the present disclosure is shown.

[0047] The following will be described in combination with Figure 1A The present disclosure describes how to implement convolution calculation by using the multiply-add operation array.

[0048] As Figure 1A shown, the data processing apparatus comprises a multiply-add operation array 101, a configuration unit 102, and a control unit 103. The multiply-add operation array 101 comprises multiply-add computation units 1011 arranged in 3 rows and 4 columns, and a plurality of adders 1012, wherein each multiply-add computation unit 1011 performs one multiplication operation to obtain one computation result; the configuration unit 102 is coupled with the plurality of adders 1012, and at this time, the configuration unit 102 is in a first configuration state, and 2 adders are allocated to each of the 4 groups of multiply-add computation units to accumulate 3 computation results in each group, for example, when the configuration unit 102 is in a second configuration state, at least one adder is allocated to each of the 3 groups of multiply-add computation units to accumulate 4 computation results in each group (which will be described below with reference to Figure 1A ).

[0049] The control unit 103 is configured to obtain a computation parameter as control information, and can control the configuration state of the configuration unit 102 according to the computation parameter. For example, when the control unit 103 obtains a computation parameter that is a positive integer multiple of 3, the control unit 103 controls the configuration unit 102 to be in the first configuration state, or when the control unit 103 obtains a computation parameter that is a multiple of 2, the control unit 103 controls the configuration unit 102 to be in the second configuration state. For example, Figure 1A shown, the control unit 103 controls the 4 groups of multiply-add computation units in parallel calculation in the first configuration state.

[0050] For example, to Figure 1AThe data processing device shown takes a 4×4 data matrix as input. The convolutional neural network processes this input data using a 3×3 convolution kernel, requiring four convolution operations. Each convolution operation can be broken down into nine multiplication operations, and the results of these nine multiplication operations are then summed. In this case, three multiply-accumulate units 1011 in a column are grouped together. Each group of multiply-accumulate units 1011 performs three multiplication operations, and the results of all multiplication operations performed by this group are summed to obtain the result of one convolution operation. Different groups of multiply-accumulate units 1011 perform different convolution operations. Therefore, all multiply-accumulate units 1011 in the column direction are used in this calculation, resulting in high hardware unit utilization.

[0051] Figure 1A For at least one embodiment of this disclosure, corresponding to such Figure 1A This is a schematic diagram of another configuration state of the multiply-accumulate array shown.

[0052] The following is combined Figure 1A This disclosure explains how to improve hardware utilization by configuring state transitions under different input computation parameters.

[0053] When the number of multiply-accumulate operations is not a positive integer multiple of 3, for example, processing 5×5 input data using a 4×4 convolution kernel, four convolution operations are required. Each convolution operation can be broken down into 16 multiplication operations, and the results of these 16 multiplication operations are then added together. In this case, using three multiply-accumulate units 1011 per group cannot match the 16 multiplication operations, resulting in unequal execution counts for the three multiply-accumulate units 1011 in a group, thus reducing hardware utilization. In this situation, for example, the configuration unit 102 can be controlled by the control unit 103 to change its configuration state from a first configuration state to a second configuration state. The configuration unit 102 then controls the multiply-accumulate array 101 to change from 3 rows and 4 columns to 4 rows and 3 columns, with each column forming a group.

[0054] For example, such as Figure 1A As shown, the three multiply-accumulate units 1011 in the second column are split into three other columns by a selector, resulting in four multiply-accumulate units 1011 in each column (group). Each group of multiply-accumulate units 1011 performs four multiplication operations. Adding the results of all multiplication operations in one group yields the result of one convolution operation. In this case, all multiply-accumulate units 1011 in the column direction are used in this calculation, resulting in high hardware unit utilization.

[0055] Current data processing devices can adapt to two scenarios: the number of multiplication and accumulation operands are positive integer multiples of 3 and positive integer powers of 2. When the number of multiplication and accumulation operands is a positive integer multiple of 3, these 12 operation units are arranged in 3 rows and 4 columns. Data in the same column is added together (corresponding to one group of multiplication and accumulation operations in the neural network algorithm), and data in different columns are not added together (different columns correspond to different groups of multiplication and accumulation operations in the neural network algorithm); when the number of multiplication and accumulation operands is not a positive integer multiple of 3, one column is selected from these 12 operation units ( Figure 1A (Taking the second column as an example) it is split into three other columns, so that the cumulative dimension of these 12 operation units becomes 4 (4 rows), and different columns are not added together.

[0056] It should be noted that, within the same data processing device, depending on the configuration of the selector, expansion can be performed in both column and row directions to create more parallel computing units and improve computing power.

[0057] Figure 1A This is a schematic flowchart illustrating the operation method of a data processing apparatus provided in at least one embodiment of the present disclosure.

[0058] At least one embodiment of this disclosure also provides a method for operating a data processing apparatus, which is applied to the data processing apparatus described in any of the above embodiments, such as... Figure 1A As shown, the operation method includes the following steps S1 to S3:

[0059] Step S1: Receive the input matrix and convolution kernel;

[0060] Step S2: Configure the multiply-accumulate array according to the calculation parameters;

[0061] Step S3: Perform convolution calculation on the input matrix and convolution kernel using the configured multiply-accumulate array.

[0062] For example, computational parameters may include parameters related to the input matrix and the kernel size.

[0063] For example, it can be done through Figure 1A The control unit 103 shown controls the configuration unit to configure the multiply-accumulate array.

[0064] The embodiments disclosed above can adapt to the parameter characteristics of different neural network algorithms, support changes in hardware configuration, enable algorithm parameters to be fully mapped onto hardware, allow more hardware units to participate in the calculation, and improve the utilization rate of hardware units.

[0065] For example, the control configuration unit configures the multiply-add operation array, including: in response to the configuration state of the configuration unit switching from the first configuration state to the second configuration state, selecting at least one group in the N groups of multiply-add calculation units as a target split group, and other groups as to-be-allocated groups, allocating the multiply-add calculation units in the target split group to the to-be-allocated groups, so that the plurality of multiply-add calculation units are divided into Q groups; or, in response to the configuration state of the configuration unit switching from the second configuration state to the first configuration state, selecting at least one multiply-add calculation unit in each of the Q groups of multiply-add calculation units as a target split unit, and merging the target split units into at least one new group, so that the plurality of multiply-add calculation units are divided into N groups.

[0066] For example, the transition from the first configuration state to the second configuration state can adopt a process of splitting a group of multiply-add calculation units to other groups as shown in Figure 1A For example, the transition from the second configuration state to the first configuration state can adopt a process opposite to Figure 1A For example, the transition from the first configuration state to the second configuration state can adopt a process of splitting a group of multiply-add calculation units to other groups as shown in

[0067] The data processing apparatus can be implemented as a processor or any product or component including the processor and being used with the processor. Figure 1A For example, the data processing apparatus can be implemented as a processor or any product or component including the processor and being used with the processor. Figure 1A The computing device can be a terminal device such as a personal computer, a notebook computer, a tablet computer, a mobile phone, etc., or a server device such as a workstation, a server, a cloud service, etc.

[0068] The computing device can be an AI accelerator such as a general-purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), a deep computing unit (DCU), etc., or an electronic device including the AI accelerator.

[0069] Figure 1A The computing device provided by some embodiments of the present disclosure, for example, as shown in Figure 1A The computing device 200 can be a terminal device such as a personal computer, a notebook computer, a tablet computer, a mobile phone, etc., or a server device such as a workstation, a server, a cloud service, etc. Figure 1A The computing device 200 shown is only an example, which does not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0070] As shown in Figure 1AAs shown, the computing device 200 can include a processing device (e.g., one or more central processing units, one or more graphics processing units, etc., the data processing device described above) 210, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 220 or loaded into a random access memory (RAM) 230 from a storage device 280. Various executable programs and data required for the operation of the computing device 200 are also stored in the RAM 230. The processing device 210, the ROM 220, and the RAM 230 are connected to each other through a bus 240. An input / output (I / O) interface 250 is also connected to the bus 240.

[0071] Generally, the following devices can be connected to the I / O interface 250: an input device 260 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, or a gyroscope; an output device 270 including, for example, a liquid crystal display (LCD), a speaker, or a vibrator; a storage device 280 including, for example, a magnetic tape, a hard disk, or the like; and a communication device 290. The communication device 290 can allow the computing device 200 to communicate with other computing devices wirelessly or via wires to exchange data. Although Figure 1A The computing device 200 is shown to include various devices, but it should be understood that not all of the illustrated devices are required, and the computing device 200 can instead implement or have more or fewer devices.

[0072] For example, according to embodiments of the present disclosure, the above-described data processing method can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a non-transitory computer-readable medium, the computer program including program code for executing the above-described data processing method. In such embodiments, the computer program can be downloaded and installed from a network via the communication device 290, or installed from the storage device 280, or installed from the ROM 220. When the computer program is executed by the processing device 210, the functions defined in the data processing method provided by embodiments of the present disclosure can be performed.

[0073] Some embodiments of the present disclosure also provide a storage medium for storing non-transitory computer program executable code (e.g., computer executable instructions) that, when executed by a computer (e.g., including one or more processors), can implement the data processing method of any of the embodiments of the present disclosure; or, when executed by a computer, can implement the data processing method provided by embodiments of the present disclosure.

[0074] Figure 1A A schematic diagram of a storage medium provided by some embodiments of the present disclosure. As Figure 1AAs shown, the storage medium 300 non-transitorily stores computer program executable code 301. For example, the computer program executable code 301, when executed by a computer (for example, comprising one or more processors), can perform the data processing method according to the embodiments of the present disclosure.

[0075] For example, the storage medium 300 can be applied in the data processing apparatus described above. For another example, the storage medium 300 can be the memory 220 in the computing device 200 as shown. Figure 1A For example, the relevant description about the storage medium 300 can refer to the corresponding description of the memory 220 in the computing device 200 as shown. Figure 1A Figure 1A Figure 1A Figure 1A Figure 1A Figure 1A Figure 1A Figure 1A Figure 1A Figure 1A Figure 1A Figure 1A Figure 1A Figure 1A For example, the relevant description about the storage medium 300 can refer to the corresponding description of the memory 220 in the computing device 200 as shown.

[0076] It can be understood that the above implementation is only an exemplary implementation adopted for illustrating the principles of the present disclosure, and the present disclosure is not limited thereto. Various modifications and improvements can be made by those of ordinary skill in the art without departing from the spirit and principle of the present disclosure, and these modifications and improvements are also considered to be within the protection scope of the present disclosure.

[0077] In addition to the above exemplary description, the following points need to be explained for the present disclosure:

[0078] (1) The drawings of the embodiments of the present disclosure only relate to the structures involved in the embodiments of the present disclosure, and other structures can refer to the general design.

[0079] (2) For the purpose of clarity, in the drawings used to describe the embodiments of the present disclosure, the thickness of a layer or region is exaggerated or reduced, that is, the drawings are not drawn according to the actual proportion.

[0080] (3) In the case of no conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0081] The above is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A data processing apparatus comprising a multiply-add array and a configuration unit, wherein, The multiply-add operation array comprises a plurality of multiply-add calculation units, The configuration unit is coupled with the multiply-add operation array, and the configuration unit has at least a first configuration state and a second configuration state, The configuration unit controls the plurality of multiply-add calculation units to be divided into N groups in the first configuration state, and M multiply-add calculation units in each group are configured to be coupled with each other for accumulating M calculation results in each group. The configuration unit controls the plurality of multiply-add calculation units to be divided into Q groups in the second configuration state, and P multiply-add calculation units in each group are configured to be coupled with each other for accumulating P calculation results in each group, wherein M, N, P and Q are all integers greater than 1, and N and Q are not equal. The configuration unit is further configured to: in response to the configuration state being switched from the first configuration state to the second configuration state, select at least one group in the N groups of multiply-add calculation units as a target split group, and other groups as to-be-allocated groups, and allocate the multiply-add calculation units in the target split group to the to-be-allocated groups, so that the plurality of multiply-add calculation units are divided into Q groups, or in response to the configuration state being switched from the second configuration state to the first configuration state, select at least one multiply-add calculation unit in each group in the Q groups of multiply-add calculation units as a target split unit, and combine the target split units into at least one new group, so that the plurality of multiply-add calculation units are divided into N groups.

2. The data processing apparatus of claim 1, wherein, M is a positive integer multiple of 3.

3. The data processing apparatus according to claim 1 or 2, wherein, P is a positive integer power of 2.

4. The data processing apparatus according to claim 1, further comprising a control unit, wherein, The control unit is configured to obtain a calculation parameter to control the configuration state of the configuration unit according to the calculation parameter.

5. The data processing apparatus of claim 4, wherein, The control unit is further configured to control the N groups of multiply-add calculation units to perform parallel calculation in the first configuration state, and control the Q groups of multiply-add calculation units to perform parallel calculation in the second configuration state.

6. The data processing apparatus of claim 1 or 4, further comprising a plurality of adders, wherein, The configuration unit is coupled with the plurality of adders, In the first configuration state, at least one adder is allocated to each of the N groups of multiply-add calculation units to accumulate M calculation results in each group, In the second configuration state, at least one adder is allocated to each of the Q groups of multiply-add calculation units to accumulate P calculation results in each group.

7. A data processing device, comprising the data processing device of any one of claims 1-6.

8. An operation method of a data processing device, for the data processing device of any one of claims 1-6, comprising: receiving an input matrix and a convolution kernel; controlling the configuration unit to configure the multiply-add operation array according to a calculation parameter; performing convolution calculation on the input matrix and the convolution kernel using the configured multiply-add operation array; wherein controlling the configuration unit to configure the multiply-add operation array comprises: in response to the configuration state of the configuration unit being switched from the first configuration state to the second configuration state, selecting at least one group in the N groups of multiply-add calculation units as a target split group, and other groups as to-be-allocated groups, allocating the multiply-add calculation units in the target split group to the to-be-allocated groups, so that the plurality of multiply-add calculation units are divided into Q groups, or in response to the configuration state of the configuration unit switching from the second configuration state to the first configuration state, selecting at least one multiply-add calculation unit in each group in the Q-group multiply-add calculation unit as a target split unit, merging the target split units into at least one new group, so that the plurality of multiply-add calculation units are divided into N groups.

9. A computer readable storage medium storing computer instructions, wherein, The computer instructions, when executed by the processor, are used to implement the operation method of the data processing device as claimed in claim 8.

Images