Data processing apparatus and operation method therefor, and computing apparatus and storage medium

Abstract

A data processing apparatus and an operation method therefor, and a computing apparatus and a storage medium. The data processing apparatus comprises a multiply-accumulate operation array (101) and a configuration unit (102), wherein the multiply-accumulate operation array (101) comprises a plurality of multiply-accumulate computing units (1011); the configuration unit (102) is coupled to the multiply-accumulate operation array (101) and has at least a first configuration state and a second configuration state; in the first configuration state, the configuration unit (102) controls the plurality of multiply-accumulate computing units (1011) to be divided into N groups, and M multiply-accumulate computing units (1011) in each group are configured to couple to each other so as to accumulate M computation results in the group; and in the second configuration state, the configuration unit controls the plurality of multiply-accumulate computing units (1011) to be divided into Q groups, and P multiply-accumulate computing units (1011) in each group are configured to couple to each other so as to accumulate P computation results in the group. By means of changing a configuration state of a multiply-accumulate operation array (101), a data processing apparatus makes parameters in a neural network algorithm more sufficiently mapped to hardware units, thereby improving the utilization rate of the hardware units.

Description

Data processing device and operation method, computing device, and storage medium

[0001] This application claims priority to Chinese Patent Application No. 202410089267.6 filed on January 22, 2024, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0002] Embodiments of the present disclosure relate to a data processing device and operating method, a computing device, and a storage medium. Background Art

[0003] Currently, deep learning technology in the field of artificial intelligence has achieved remarkable results in applications such as image recognition, speech recognition, and autonomous driving. Deep learning involves methods such as convolutional neural networks (CNNs), deep neural networks (DNNs), and recurrent neural networks (RNNs). Neural networks abstract the human brain's neural networks from an information processing perspective, establishing a simple model that forms different networks based on different connection methods. To accelerate computational speed, neural network processors often use a multiply-add array approach, mapping the dimensions of the neural network algorithm to the dimensions of the multiply-add array.

[0004] Summary of the Invention

[0005] Some embodiments of the present disclosure provide a data processing device, which includes a multiplication-addition operation array and a configuration unit, wherein the multiplication-addition operation array includes multiple multiplication-addition calculation units, the configuration unit is coupled to the multiplication-addition operation array, and the configuration unit has at least a first configuration state and a second configuration state, the configuration unit controls the multiple multiplication-addition calculation units to be divided into N groups in the first configuration state, and the M multiplication-addition calculation units in each group are configured to be coupled to each other for accumulating the M calculation results in each group, and the configuration unit controls the multiple multiplication-addition calculation units to be divided into Q groups in the second configuration state, and the P multiplication-addition calculation units in each group are configured to be coupled to each other for accumulating the P calculation results in each group, wherein M, N, P and Q are all integers greater than 1, and N and Q are not equal.

[0006] For example, in the data processing apparatus provided in some embodiments of the present disclosure, M is a positive integer multiple of 3.

[0007] For example, in the data processing apparatus provided in some embodiments of the present disclosure, P is a positive integer power of 2.

[0008] For example, in the data processing apparatus provided in some embodiments of the present disclosure, the control unit is configured to obtain calculation parameters to control the configuration state of the configuration unit according to the calculation parameters.

[0009] For example, in the data processing device provided in some embodiments of the present disclosure, the control unit is further configured to control the parallel calculation of N groups of the multiplication and addition computing units in the first configuration state, and the parallel calculation of Q groups of the multiplication and addition computing units in the second configuration state.

[0010] For example, the data processing device provided in some embodiments of the present disclosure also includes multiple adders, wherein the configuration unit is coupled to the multiple adders, and in the first configuration state, at least one adder is allocated to each of the N groups of multiplication and addition calculation units to accumulate M calculation results in each group, and in the second configuration state, at least one adder is allocated to each of the Q groups of multiplication and addition calculation units to accumulate P calculation results in each group.

[0011] Some embodiments of the present disclosure further provide a computing device, which includes the data processing device described in any of the above embodiments.

[0012] Some embodiments of the present disclosure also provide an operating method for a data processing device, which is used for the data processing device described in any of the above embodiments, and the operating method includes: receiving an input matrix and a convolution kernel; controlling the configuration unit to configure the multiplication and addition operation array according to calculation parameters; and using the configured multiplication and addition operation array to perform convolution calculation on the input matrix and the convolution kernel.

[0013] For example, in the operating method of the data processing device provided in some embodiments of the present disclosure, the control of the configuration unit to configure the multiplication-addition operation array includes: in response to the configuration state of the configuration unit switching from the first configuration state to the second configuration state, selecting at least one group among the N groups of multiplication-addition calculation units as the target split group, and the other groups as the to-be-allocated groups, and allocating the multiplication-addition calculation units in the target split group to the to-be-allocated groups so that the multiple multiplication-addition calculation units are divided into Q groups, or, in response to the configuration state of the configuration unit switching from the second configuration state to the first configuration state, selecting at least one multiplication-addition calculation unit in each group of the Q groups of multiplication-addition calculation units as the target split unit, and merging the target split units into at least one new group so that the multiple multiplication-addition calculation units are divided into N groups.

[0014] Some embodiments of the present disclosure also provide a computer-readable storage medium, which stores computer instructions, wherein the computer instructions are used to implement the following steps when executed by a processor: receiving an input matrix and a convolution kernel; controlling the configuration unit to configure the multiplication-addition operation array according to calculation parameters; and using the configured multiplication-addition operation array to perform convolution calculation on the input matrix and the convolution kernel. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, rather than limiting the present disclosure.

[0016] FIG1A shows the input and output of a neuron in an exemplary convolutional neural network;

[0017] FIG1B is a schematic diagram showing an exemplary convolutional layer performing a multi-channel convolution operation;

[0018] FIG1C shows a schematic diagram of the architecture of a neural network processor;

[0019] FIG2 shows a schematic structural diagram of a data processing device provided by at least one embodiment of the present disclosure;

[0020] FIG3 shows a schematic diagram of a configuration state of a data processing device provided by at least one embodiment of the present disclosure;

[0021] FIG4 shows another configuration state diagram of the data processing device provided by at least one embodiment of the present disclosure;

[0022] FIG5 is a schematic flow chart showing an operating method of a data processing device provided by at least one embodiment of the present disclosure;

[0023] FIG6 is a schematic structural diagram of an electronic device provided by some embodiments of the present disclosure; and

[0024] FIG7 is a schematic diagram of a storage medium provided in some embodiments of the present disclosure. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the embodiments of the present disclosure will be further described in detail below in conjunction with the accompanying drawings. The specific embodiments and drawings described herein are only used to explain the present disclosure, rather than to limit the disclosed embodiments. Unless there is a conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0026] For ease of description, the drawings of the embodiments of the present disclosure only show the parts related to the embodiments of the present disclosure, and the parts not related to the embodiments of the present disclosure are not shown in the drawings. Each unit and module involved in the embodiments of the present disclosure may correspond to only one physical structure, or may be composed of multiple physical structures, or multiple units and modules may be integrated into one physical structure. In the absence of conflict, the functions and steps marked in the flowcharts and block diagrams of the embodiments of the present disclosure may occur in an order different from that marked in the drawings.

[0027] The flowcharts and block diagrams of the embodiments of the present disclosure illustrate the possible architectures, functions, and operations of the systems, devices, equipment, and methods according to the embodiments of the present disclosure. Each box in the flowchart or block diagram may represent a unit, module, program segment, or code, which contains executable instructions for implementing the specified functions. Moreover, each box or combination of boxes in the block diagram and flowchart may be implemented by a hardware-based system that implements the specified functions, or by a combination of hardware and computer instructions.

[0028] In order to keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and known components may be omitted. When any component of the embodiments of the present disclosure appears in more than one drawing, the component is represented by the same or similar reference numerals in each drawing.

[0029] Neural networks are mathematical computational models inspired by the structure of neurons in the brain and the principles of neural conduction. The method of implementing intelligent computing based on this model is called brain-inspired computing. For example, neural networks include various network structures, such as back propagation (BP) neural networks, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long short-term memory networks (LSTMs). For example, convolutional neural networks can be further divided into fully convolutional networks, deep convolutional networks, and U-net networks.

[0030] For example, a common convolutional neural network usually includes an input end, an output end, and multiple processing layers. For example, the input end is used to receive data to be processed, such as an image to be processed, and the output end is used to output processing results, such as a processed image. These processing layers may include convolution layers, pooling layers, batch normalization layers (Batch Normalization, abbreviated as BN), fully connected layers, etc. Depending on the structure of the convolutional neural network, the processing layers may include different contents and combinations. After the input data is input into the convolutional neural network, it passes through several processing layers to obtain the corresponding output. For example, the input data can pass through several processing layers to complete operations such as convolution, upsampling, downsampling, standardization, full connection, and flattening.

[0031] The convolutional layer is the core layer of a convolutional neural network. It applies several filters to the input data (input image or input feature map), which is then used to extract various types of features. The result of applying a filter to the input data is called a feature map, and the number of feature maps is equal to the number of filters. The feature map output by a convolutional layer can be fed into the next convolutional layer for further processing to produce a new feature map. The pooling layer is an intermediate layer sandwiched between consecutive convolutional layers, used to reduce the size of the input data and, to a certain extent, mitigate overfitting. There are many ways to implement pooling, including but not limited to max-pooling, avg-pooling, random pooling, decimation (e.g., selecting fixed pixels), and demux (splitting the input image into multiple smaller images). Typically, the last subsampling layer or convolutional layer is connected to one or more fully connected layers, the output of which is the final output, resulting in a one-dimensional matrix, or vector.

[0032] Figure 1A abstractly shows the input and output of a neuron in a convolutional neural network. As shown in Figure 1A, C1, C2, etc. refer to different signal channels. For a certain local receptive field (the local receptive field contains multiple channels), different filters are used to convolve the data on the C1, C2, etc. signal channels of the local receptive field. The convolution result is input into the stimulation node, and the stimulation node is calculated according to the corresponding function to obtain feature information. For example, a convolutional neural network is usually a deep convolutional neural network, which can include at least five convolution layers. For example, the VGG-16 neural network has 16 layers, and the GoogLeNet neural network has 22 layers. Of course, other neural network structures can have more processing layers. The above content is only an exemplary introduction to the neural network. The present disclosure does not limit the structure of the neural network.

[0033] Figure 1B shows a schematic diagram of a convolutional layer performing a multi-channel convolution operation. As shown in Figure 1B, M sets of R×S convolution kernels with C channels are used to perform convolution operations on N sets of H×W input images (or input feature maps) with C channels, respectively. N sets of E×F output feature maps with M channels are obtained, so the output feature maps have the dimensions F / E / M.

[0034] Convolution operations are characterized by high parallelism, which means that multiple convolution kernels can operate simultaneously with multiple input feature maps. Moreover, due to the large amount of computation required by neural networks, especially for convolution layers with large-sized input feature maps, it is usually necessary to decompose the computational operations of a convolution layer in the neural network. For example, the convolution operations of different parts of the same convolution layer can be performed independently of each other. These decomposed tasks are assigned to multiple processing units to perform calculations in parallel, and then the calculation results of these processing units are merged to obtain the calculation results of the entire convolution layer. The calculation results of this network layer can then be used as input to the next convolution layer.

[0035] Figure 1C shows a schematic diagram of the architecture of a neural network processor. As shown in Figure 1C, the neural network processor includes a processing element (PE) array 110, a global cache 120, and a memory 130. The processing element array 110 includes multiple rows and columns (e.g., 12 rows × 12 columns) of processing elements, which are coupled to each other via an on-chip interconnect 111 and share a global cache 120. The on-chip interconnect 111 is, for example, a network on chip (NoC). Each processing element has a computing function and, for example, may also have its own local cache, such as a cache or register array including a multiply-accumulator (MAC) and a vector (or matrix) for caching inputs. Each PE can access other PEs around it, the PE's own local cache, and the global cache. The global cache 120 is further coupled to the memory 130 via, for example, a bus.

[0036] During operation, for example, the convolution kernel (Flt) and input feature map (Ifm) data required for calculations of a network layer (e.g., a convolutional layer) are read from memory 130 into global cache 120. The convolution kernel (Flt) and input image (Img) are then input from global cache 120 into processing unit array 110 for calculation. Computational tasks for different image pixels are assigned to different processing units (i.e., mapping is performed). The partial cumulative sum (Psum1) generated during the calculation process is temporarily stored in the global cache. If a subsequent calculation requires further accumulation of the previously generated partial cumulative sum (Psum1), the required partial cumulative sum (Psum2) can be read from global cache 120 and returned to processing unit array 110 for calculation. The output feature map (Ofm) obtained after completing the calculations of a convolutional layer can be output from global cache 120 to memory 130 for storage, for example, to be used for calculations of the next network layer (e.g., a convolutional layer).

[0037] As mentioned above, neural network processors typically use a multiply-add array approach to accelerate computational speed, mapping the dimensions of the neural network algorithm to the dimensions of the multiply-add array. However, different neural network algorithms, and each layer within the same neural network algorithm, have different numbers of channels and convolution kernel sizes. This can lead to situations where the neural network's dimensional parameters cannot evenly divide the dimensional parameters of the multiply-add array, resulting in some multiply-add units being idle.

[0038] For example, the number of input channels for the parameters of a convolutional neural network in the first layer is mostly 3. For example, when a convolutional neural network processes an RGB image, the input data is divided into three channels for processing, and the convolution kernel of the convolutional neural network often uses a 3×3 size. This means that the number of multiplication and accumulation operations in the convolutional neural network is usually a positive integer multiple of 3. However, in other types of neural networks, such as transformer network models, the number of operands in most multiplication and accumulation operations is a positive integer power of 2, and the multiplication and accumulation array size of the neural network processor is usually a positive integer power of 2. For example, the three dimensions of NVIDIA's tensor core are all positive integer powers of 2. This is a good match for most networks outside the first layer, but it is not friendly to the first layer. On the one hand, it is not divisible, and on the other hand, the neural network processor accumulates the data in one dimension after the multiplication. To increase the parallelism of the input feature map, the accumulation dimension is usually set to 16 or even larger, resulting in low utilization of the multiplication and accumulation array in the input channel direction of the first layer of the neural network. If the neural network parameters are not divisible by both algorithm parameters when mapped to the hardware's parallel computing units, some hardware resources will be unable to participate in the mapping. These unavailable hardware units will become idle, reducing hardware utilization.

[0039] At least one embodiment of the present disclosure provides a data processing device, which includes a multiplication-addition operation array and a configuration unit, wherein the multiplication-addition operation array includes a plurality of multiplication-addition calculation units, the configuration unit is coupled to the multiplication-addition operation array, and the configuration unit has at least a first configuration state and a second configuration state. In the first configuration state, the configuration unit controls the plurality of multiplication-addition calculation units to be divided into N groups, and the M multiplication-addition calculation units in each group are configured to be coupled to each other for accumulating the M calculation results in each group. In the second configuration state, the configuration unit controls the plurality of multiplication-addition calculation units to be divided into Q groups, and the P multiplication-addition calculation units in each group are configured to be coupled to each other for accumulating the P calculation results in each group. M, N, P, and Q are all integers greater than 1, and N and Q are not equal.

[0040] The above-mentioned embodiments of the present disclosure can make the data processing device more compatible with the parameters in the neural network algorithm by transforming the configuration state of the configuration unit, so that the input graph, feature graph, etc. can be more fully mapped to the multiplication and addition calculation unit in the operation array, allowing as many hardware units as possible to participate in the calculation, thereby obtaining a higher utilization rate of the hardware units.

[0041] The neural network (model) that the data processing device is configured to process can be, for example, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a transformation neural network (transformer), etc. The data processing device can also be configured to process other multiplication and accumulation operations, which is not limited by the embodiments of the present disclosure.

[0042] For example, M is a positive integer multiple of 3, and for example, P is a positive integer power of 2. In the embodiments of the present disclosure, for example, factors 3 and 2 are both commonly used parameters in convolutional neural networks. Therefore, introducing factors 3 and 2 in the cumulative dimension of each group can enable greater flexibility when mapping parameters to hardware units, reduce idle computing units, and improve hardware utilization. However, it should be noted that in other embodiments of the present disclosure, M and P can also take other numbers as factors, such as 5, 7, etc., and the embodiments of the present disclosure are not limited to this.

[0043] FIG2 is a schematic diagram of the structure of a data processing device provided by at least one embodiment of the present disclosure.

[0044] As shown in FIG2 , the data processing device 10 includes a multiplication-addition operation array 101 and a configuration unit 102 . The data processing device 10 can be used as a processing unit, for example, to process neural network calculations. The multiplication-addition operation array 101 includes a plurality of multiplication-addition computing units (MACs) 1011 arranged in an array. For example, each column of multiplication-addition computing units 1011 can be grouped together, and each group of multiplication-addition computing units can perform multiplication calculations separately and accumulate the results of the multiplication calculations. Each multiplication-addition computing unit, for example, includes one or more multipliers and one or more adders.

[0045] For example, the configuration unit includes multiple selectors (selection switches), which are respectively connected to multiple multiplication and addition calculation units included in the multiplication and addition operation array 101. Logical operations can be performed under the control of control signals to combine multiple multiplication and addition calculation units, thereby achieving different configuration states.

[0046] In at least one example, the data processing device may further include a control unit that is communicatively coupled to the configuration unit and configured to receive control information and send control signals to the configuration unit to control the configuration unit to achieve or maintain a certain configuration state. For example, multiple data processing devices may share the control unit, i.e., the same control unit controls the configuration units of multiple data processing devices.

[0047] FIG3 is a schematic diagram of the configuration status of a data processing device provided by at least one embodiment of the present disclosure.

[0048] The following describes how the present disclosure implements convolution calculations through a multiplication-addition operation array in conjunction with FIG3 .

[0049] As shown in FIG3 , the data processing apparatus includes a multiplication-addition operation array 101, a configuration unit 102, and a control unit 103. The multiplication-addition operation array 101 includes multiplication-addition calculation units 1011 arranged in three rows and four columns, and a plurality of adders 1012, wherein each multiplication-addition calculation unit 1011 performs a multiplication calculation to obtain a calculation result. The configuration unit 102 is coupled to the plurality of adders 1012. At this time, the configuration unit 102 is in a first configuration state, and two adders are allocated to each of the four groups of multiplication-addition calculation units to accumulate the three calculation results in each group. For example, in a second configuration state, the configuration unit 102 allocates at least one adder to each of the three groups of multiplication-addition calculation units to accumulate the four calculation results in each group (described below with reference to FIG4 ).

[0050] The control unit 103 is configured to obtain a calculation parameter as control information and to control the configuration state of the configuration unit 102 based on the calculation parameter. For example, when the calculation parameter obtained by the control unit 103 is a positive integer multiple of 3, the configuration state of the configuration unit 102 is controlled to be the first configuration state. Alternatively, when the calculation parameter obtained by the control unit 103 is a multiple of 2, the configuration state of the configuration unit 102 is controlled to be the second configuration state. For example, as shown in FIG3 , the control unit 103 controls four groups of multiplication and addition calculation units in the first configuration state to perform parallel calculations.

[0051] For example, if a 4×4 data matrix is ​​input to the data processing device shown in FIG3 and the convolutional neural network uses a 3×3 convolution kernel to process the input data, four convolution operations are required. Each convolution operation can be split into nine multiplication operations, and the results of the nine multiplication operations are added together. In this case, the three multiplication-addition computing units 1011 in a column are grouped together. One group of multiplication-addition computing units 1011 is selected to perform three multiplication operations respectively. The results of all the multiplication operations performed by this group of multiplication-addition computing units 1011 are then added together to obtain the result of a single convolution operation. Different groups of multiplication-addition computing units 1011 perform different convolution operations. In this case, all the multiplication-addition computing units 1011 in the column direction are used in this calculation, which improves the utilization rate of the hardware units.

[0052] FIG. 4 is a schematic diagram of another configuration state of the multiplication-addition operation array shown in FIG. 3 , provided by at least one embodiment of the present disclosure.

[0053] 4 , the following describes how the present disclosure improves hardware utilization when different computing parameters are input through configuration state conversion.

[0054] When the number of multiplication-accumulation operations is not a positive integer multiple of 3, for example, when a 4×4 convolution kernel is used to process the input 5×5 data, 4 convolution operations are required. Each convolution operation can be split into 16 multiplication operations, and the results of the 16 multiplication operations are then added together. In this case, using three multiplication-addition computing units 1011 per group will not be able to match the 16 multiplication operations. When performing multiplication operations, the three multiplication-addition computing units 1011 in a group will be executed unequally, reducing hardware utilization. In this case, for example, the control unit 103 can control the configuration unit 102 to convert the configuration state of the configuration unit 102 from a first configuration state to a second configuration state. The configuration unit 102 controls the multiplication-addition operation array 101 from 3 rows and 4 columns to 4 rows and 3 columns, with each column as a group.

[0055] For example, as shown in FIG4 , the three multiplication-addition units 1011 in the second column are split into three other columns by a selector, resulting in four multiplication-addition units 1011 in each column (group). In this case, each group of multiplication-addition units 1011 performs four multiplication operations, and the results of all multiplication operations of a group of multiplication-addition units 1011 are added together to obtain the result of a single convolution operation. In this case, all multiplication-addition units 1011 in the column direction are used in this calculation, resulting in high hardware unit utilization.

[0056] The current data processing device can adapt to two scenarios: the accumulation operands are positive integer multiples of 3 and positive integer powers of 2. When the number of multiplication and accumulation operands is a positive integer multiple of 3, the 12 operation units are arranged into 3 rows and 4 columns, and the data in the same column are added (corresponding to a group of multiplication and accumulation operations in the neural network algorithm), and the data in different columns are not added (different columns correspond to different groups of different multiplication and accumulation operations in the neural network algorithm); when the number of multiplication and accumulation operands is not a positive integer multiple of 3, one column selected from the 12 operation units (taking the second column as an example in Figure 4) is split into another three columns, so that the accumulation dimension of the 12 operation units becomes 4 (4 rows), and different columns are not added.

[0057] It should be noted that, in the same data processing device, according to the configuration of the selector, expansion can be performed in the column direction and the row direction to expand more parallel computing units to improve computing power.

[0058] FIG5 is a flowchart illustrating an operating method of a data processing device provided by at least one embodiment of the present disclosure.

[0059] At least one embodiment of the present disclosure further provides an operating method for a data processing device. The operating method is applied to the data processing device described in any of the above embodiments. As shown in FIG5 , the operating method includes the following steps S1 to S3:

[0060] Step S1, receiving the input matrix and convolution kernel;

[0061] Step S2: controlling the configuration unit to configure the multiplication and addition operation array according to the calculation parameters;

[0062] Step S3: Use the configured multiplication-addition operation array to perform convolution calculation on the input matrix and the convolution kernel.

[0063] For example, computation parameters may include parameters related to the input matrix and the convolution kernel size.

[0064] For example, the control unit 103 shown in FIG3 may control the configuration unit to configure the multiplication and addition operation array.

[0065] The above-mentioned embodiments of the present disclosure can adapt to the parameter characteristics of different neural network algorithms and support the transformation of hardware configuration, so that the algorithm parameters can be fully mapped to the hardware, allowing more hardware units to participate in the calculation and improving the utilization rate of the hardware units.

[0066] For example, controlling the configuration unit to configure the multiplication-addition operation array includes: in response to the configuration state of the configuration unit switching from the first configuration state to the second configuration state, selecting at least one group among N groups of multiplication-addition calculation units as the target split group, and the other groups as the to-be-allocated groups, and allocating the multiplication-addition calculation units in the target split group to the to-be-allocated groups, so that the multiple multiplication-addition calculation units are divided into Q groups; or, in response to the configuration state of the configuration unit switching from the second configuration state to the first configuration state, selecting at least one multiplication-addition calculation unit in each group of the Q groups of multiplication-addition calculation units as the target split unit, merging the target split units into at least one new group, so that the multiple multiplication-addition calculation units are divided into N groups.

[0067] For example, the conversion from the first configuration state to the second configuration state can be carried out by splitting a group of multiplication and addition computing units into other groups as shown in Figure 4. For example, the conversion from the second configuration state to the first configuration state can be carried out by adopting the opposite process of Figure 4, selecting one multiplication and addition computing unit from each column of the 4 rows and 3 columns of multiplication and addition computing units to form a new column, so as to become 3 rows and 4 columns.

[0068] At least one embodiment of the present disclosure further provides a computing device, including the data processing device described in any of the above embodiments, such as the data processing device shown in Figure 3 or Figure 4. The computing device may be, for example, a processor, or any other product or component including the processor and its associated use. For another example, the computing device may be a server device or a terminal device, such as a mobile phone or laptop computer.

[0069] Some embodiments of the present disclosure further provide a computing device comprising the aforementioned data processing device for neural network computing. The computing device may be implemented as an AI accelerator, such as a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), or a deep computing unit (DCU), or as an electronic device comprising the aforementioned AI accelerator.

[0070] FIG6 illustrates a computing device provided in some embodiments of the present disclosure. For example, as shown in FIG6 , the computing device 200 is used to implement the data processing method provided in any embodiment of the present disclosure. For example, the computing device 200 may be a terminal device such as a personal computer, a laptop computer, a tablet computer, a mobile phone, or a workstation, a server, a cloud service, etc. It should be noted that the computing device 200 shown in FIG6 is merely an example and does not impose any limitations on the functionality and scope of use of the embodiments of the present disclosure.

[0071] As shown in FIG6 , the computing device 200 may include a processing device (e.g., one or more central processing units, one or more graphics processing units, etc., or the aforementioned data processing device) 210, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 220 or a program loaded from a storage device 280 into a random access memory (RAM) 230. Various executable programs and data required for the operation of the computing device 200 are also stored in the RAM 230. The processing device 210, the ROM 220, and the RAM 230 are connected to each other via a bus 240. An input / output (I / O) interface 250 is also connected to the bus 240.

[0072] Typically, the following devices may be connected to I / O interface 250: input device 260, including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, or gyroscope; output device 270, including, for example, a liquid crystal display (LCD), speaker, or vibrator; storage device 280, including, for example, a magnetic tape, hard disk, etc.; and communication device 290. Communication device 290 may allow computing device 200 to communicate with other computing devices wirelessly or by wire to exchange data. Although FIG6 illustrates computing device 200 as including various devices, it should be understood that implementation or presence of all illustrated devices is not required, and computing device 200 may alternatively implement or be provided with more or fewer devices.

[0073] For example, according to an embodiment of the present disclosure, the above-mentioned data processing method can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the above-mentioned data processing method. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 290, or installed from the storage device 280, or installed from the ROM 220. When the computer program is executed by the processing device 210, the functions defined in the data processing method provided in the embodiment of the present disclosure can be executed.

[0074] Some embodiments of the present disclosure also provide a storage medium for storing non-transitory computer program executable code (e.g., computer executable instructions). When the non-transitory computer program executable code is executed by a computer (e.g., including one or more processors), it can implement the data processing method of any embodiment of the present disclosure; or, when the non-transitory computer program executable code is executed by a computer, it can implement the data processing method provided by the embodiments of the present disclosure.

[0075] Figure 7 is a schematic diagram of a storage medium provided in some embodiments of the present disclosure. As shown in Figure 7, storage medium 300 non-transitorily stores computer program executable code 301. For example, when computer program executable code 301 is executed by a computer (e.g., including one or more processors), it can perform the data processing method provided in accordance with embodiments of the present disclosure.

[0076] For example, the storage medium 300 can be used in the above-mentioned data processing device. For another example, the storage medium 300 can be the memory 220 in the computing device 200 shown in FIG6 . For example, the relevant description of the storage medium 300 can refer to the corresponding description of the memory 220 in the computing device 200 shown in FIG6 , and will not be repeated here.

[0077] It is understood that the above embodiments are merely exemplary embodiments for illustrating the principles of the present disclosure, and the present disclosure is not limited thereto. Those skilled in the art may make various modifications and improvements without departing from the spirit and substance of the present disclosure, and such modifications and improvements are also considered to be within the scope of protection of the present disclosure.

[0078] In addition to the above exemplary description, the following points need to be explained for this disclosure:

[0079] (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure. Other structures may refer to conventional designs.

[0080] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present disclosure, the thickness of layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale.

[0081] (3) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to form new embodiments.

[0082] The above description is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A data processing device, comprising a multiply-accumulate operation array and a configuration unit, wherein, The multiplication-addition operation array includes a plurality of multiplication-addition calculation units, The configuration unit is coupled to the multiplication-addition operation array, and the configuration unit has at least a first configuration state and a second configuration state. The configuration unit controls the plurality of multiplication and addition computing units to be divided into N groups in the first configuration state, and the M multiplication and addition computing units in each group are configured to be coupled to each other for accumulating the M calculation results in each group. In the second configuration state, the configuration unit controls the multiple multiplication and addition computing units to be divided into Q groups, and the P multiplication and addition computing units in each group are configured to be coupled to each other for accumulating the P calculation results in each group, wherein M, N, P and Q are all integers greater than 1, and N and Q are not equal.

2. The data processing device according to claim 1, wherein M is a positive integer multiple of 3.

3. The data processing device according to claim 1 or 2, wherein, P is a positive integer power of 2.

4. The data processing device according to any one of claims 1-3 further includes a control unit, wherein, The control unit is configured to obtain calculation parameters to control the configuration state of the configuration unit according to the calculation parameters.

5. The data processing device according to claim 4, wherein, The control unit is further configured to control the N groups of the multiplication and addition computing units to perform parallel calculations in the first configuration state, and to control the Q groups of the multiplication and addition computing units to perform parallel calculations in the second configuration state.

6. The data processing device according to any one of claims 1-4 further comprises a plurality of adders, wherein, The configuration unit is coupled to the plurality of adders, In the first configuration state, at least one adder is allocated to each of the N groups of multiplication and addition calculation units to accumulate the M calculation results in each group. In the second configuration state, at least one adder is allocated to each of the Q groups of multiplication and addition calculation units to accumulate the P calculation results in each group.

7. A computing device comprising the data processing device according to any one of claims 1 to 6.

8. A method for operating a data processing device, used in the data processing device according to any one of claims 1 to 6, comprising: Receive input matrix and convolution kernel; Controlling the configuration unit to configure the multiplication and addition operation array according to calculation parameters; The configured multiplication-addition operation array is used to perform convolution calculation on the input matrix and the convolution kernel.

9. The operating method of the data processing device according to claim 8, wherein, Controlling the configuration unit to configure the multiplication-addition operation array includes: In response to the configuration state of the configuration unit being switched from the first configuration state to the second configuration state, at least one group among the N groups of multiplication-addition computing units is selected as a target split group, and the other groups are selected as groups to be allocated, Allocate the multiplication-addition computing units in the target split group to the to-be-allocated group, so that the plurality of multiplication-addition computing units are divided into Q groups, or, In response to the configuration state of the configuration unit being switched from the second configuration state to the first configuration state, at least one multiplication-addition computing unit is selected from each of the Q groups of multiplication-addition computing units as a target splitting unit, The target split units are merged into at least one new group, so that the multiple multiplication and addition computing units are divided into N groups.

10. A computer-readable storage medium stores computer instructions, wherein, When the computer instructions are executed by a processor, they are used to implement the following steps: Receive input matrix and convolution kernel; Controlling the configuration unit to configure the multiplication and addition operation array according to calculation parameters; The configured multiplication-addition operation array is used to perform convolution calculation on the input matrix and the convolution kernel.

Images