A data processing method, apparatus and device

Abstract

A data processing method, device and equipment are disclosed, and relate to the field of machine learning. A processing device acquires first data to be processed, and processes the first data by using a data processing model deployed in the processing device, and outputs a processing result. The data processing model includes a grouped convolution layer, the data volume of input data to be processed by the grouped convolution layer each time matches the computing power of a matrix calculation unit included in the processing device, avoiding the problem of waste of computing power of the processing device due to mismatch of computing power, leading to a decrease in processing efficiency. Moreover, since the data to be processed by the grouped convolution layer matches the computing power provided by the matrix calculation unit, the grouped convolution layer can more effectively utilize the computing resources included in the processing device, reducing idle computing resources in the processing device, and improving the resource utilization rate of the processing device. In the case that the data processing model can utilize more computing resources provided by the processing device, the model processing efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of machine learning, and in particular to a data processing method, apparatus, and device. Background Technology

[0002] Neural network models are widely used in fields such as computer vision (CV), speech recognition, and natural language processing (NLP). For example, processors often use models containing grouped convolutional layers to process data and obtain model predictions. Grouped convolution refers to adjusting the number of groups to determine the reduction in network accuracy and computational cost. However, the smaller the number of groups in a convolutional layer, the greater the optimal ratio of memory bandwidth to on-chip computing units required by the processor. In other words, a smaller number of convolutional layers necessitates either increasing the optimal memory bandwidth or decreasing the optimal number of computing units. This results in a significant amount of idle computing resources on the processing device, impacting model processing efficiency. Therefore, providing a more efficient model processing method is a pressing issue that needs to be addressed. Summary of the Invention

[0003] This application provides a data processing method, apparatus, and device that solves the problems of low efficiency of grouped convolution and large amount of idle computing resources in processing devices, thereby improving model processing efficiency.

[0004] In a first aspect, embodiments of this application provide a data processing method. This method can be executed by a processing device, which includes a matrix computation unit and a data processing model deployed within it. The method includes: the processing device acquiring first data to be processed and inputting the first data into the data processing model to obtain a processing result for the first data. Subsequently, the processing device outputs the processing result. The data processing model includes a grouped convolutional layer, wherein the amount of input data to be processed by the grouped convolutional layer each time is matched to the computational power of the matrix computation unit, and the input data to be processed by the grouped convolutional layer each time includes at least one of the first data and intermediate data input to the grouped convolutional layer after the data processing model processes the first data.

[0005] Compared to the inconsistent data volume read from memory by the processing device each time, which leads to unstable efficiency in grouping and convolution of the read data and a large amount of idle computing resources, the processing device of this application matches the amount of input data to be processed by the grouped convolution layer with the computing power of the matrix calculation unit each time, thus avoiding the problem of wasted computing power and decreased processing efficiency caused by computing power mismatch.

[0006] Furthermore, because the data processed by the grouped convolutional layers matches the computing power provided by the matrix computation units in the processing device, and these matrix computation units are part of the computing resources contained in the processing device, the grouped convolutional layers can more effectively utilize the computing resources contained in the processing device when the data processed by each grouped convolutional layer matches this portion of computing resources. This reduces idle computing resources in the processing device and improves the resource utilization rate of the processing device during data processing. Additionally, when the data processing model can make greater use of the computing resources provided by the processing device, it also helps to improve the efficiency of the data processing model in processing data.

[0007] In one possible implementation, the matrix computation unit comprises M rows × N columns of computation nodes, where M and N are both positive integers. The matrix computation unit supports the computation of one or more preset first convolutional kernels. The amount of input data processed by the processing device each time is M+c, where c is the same as the width of the first convolutional kernel. This achieves the effect of correlated between the amount of input data stored, the number of rows in the matrix computation unit, and the width of the preset first convolutional kernel supported by the matrix computation unit. This ensures that the amount of data processed by the grouped convolutional layer each time matches the computational power provided by the matrix computation unit, avoiding the waste of processing power and the resulting decrease in processing efficiency caused by a mismatch in computational power.

[0008] In one possible implementation, the first data and intermediate data in the above embodiments are stored in a memory. For example, the processing device writes at least one of the first data and intermediate data into the memory; the data stored in the memory includes the input data to be processed each time. Thus, the processing device processes the input data retrieved from the memory each time based on a grouped convolutional layer to obtain the processing result. By writing the first data and intermediate data into the memory, and processing the data retrieved from the memory based on the grouped convolutional layer, the processing device can retrieve the data required by the grouped convolutional layer from the memory, avoiding the problem of low processing efficiency caused by the varying amount of data read by the processing device each time during data processing.

[0009] In one possible example, the above memory is located in the processing device.

[0010] In another possible example, the memory described above is a dedicated storage device connected to the processing device.

[0011] Regarding the process of a processing device acquiring the input data to be processed by each grouped convolutional layer, this application embodiment provides a possible implementation: the processing device confirms that the i-th computing node supports the computation of one or more preset first convolutional kernels, where the i-th computing node is the i-th computing node in the first column of an M row × N column of computing nodes. Then, the processing device inputs c data points from memory into the i-th computing node, where c is the same width as the first convolutional kernel.

[0012] In this embodiment, the matrix computation unit transmits M+c-1 data items from the second cache unit each time. The i-th computation node of the matrix computation unit retrieves a total of c data items from the c sub-units of the second cache unit. Thus, the processing device correlates the width of the first convolution kernel supported by the matrix computation unit with the amount of input data, ensuring that the computing power of the matrix computation unit matches the amount of data transmitted from the memory. This avoids the problem of low processing efficiency caused by varying amounts of data read by the processing device each time during data processing. Furthermore, since only c data items are transmitted, it avoids transmitting all the data stored in the memory to a single computation node of the matrix computation unit, reducing the amount of data transmitted between the memory and the matrix computation unit. This, in turn, avoids excessive wiring between the second cache unit and the matrix computation unit, preventing wiring congestion.

[0013] In one possible implementation, the grouped convolutional layer includes one or more second convolutional kernels. For any one of the one or more second convolutional kernels, the processing device determines whether the second convolutional kernel matches any one of the one or more preset first convolutional kernels. If the second convolutional kernel does not match any of the first convolutional kernels, the processing device processes the second convolutional kernel to obtain multiple convolutional kernels to be processed, which match at least one of the one or more preset first convolutional kernels. Thus, the processing device segments the source data according to the multiple convolutional kernels to be processed to obtain n first data points, where n is the same as the number of convolutional kernels to be processed.

[0014] In this embodiment, the processing device divides the second convolution kernel to obtain the convolution kernel to be processed. Since the convolution kernel to be processed matches the first convolution kernel, the matrix computing unit can effectively utilize the computing resources of the matrix computing unit when processing the convolution kernel to be processed, thereby reducing the idle computing resources in the processing device and improving the resource utilization rate of the processing device.

[0015] Regarding the acquisition process of the convolutional kernels to be processed, this application embodiment provides a possible implementation: the processing device segments the second convolutional kernel based on one or more first segmentation windows to obtain multiple convolutional kernels to be processed. Each first segmentation window corresponds to a preset first convolutional kernel.

[0016] In this embodiment, the processing device divides the second convolution kernel to obtain multiple convolution kernels to be processed, so that the divided second convolution kernels are matched with the first convolution kernels supported by the matrix computing unit. This effectively utilizes the computing resources of the matrix computing unit, reduces idle computing resources in the processing device, and improves the resource utilization rate of the processing device.

[0017] In one possible scenario, for a matching first segmentation window and a preset first convolutional kernel, the size of the first segmentation window and the size of the first convolutional kernel are the same.

[0018] For example, the size of the first segmentation sliding window includes the height of the convolution kernel.

[0019] For example, the size of the first segmentation sliding window includes the width of the convolution kernel.

[0020] For example, the dimensions of the first segmented sliding window include the height and width of the convolution kernel.

[0021] In this embodiment, the processing device sets the size of the first segmentation window and the size of the first convolution kernel to be the same, so that the size of the convolution kernel to be processed obtained by the first segmentation window is the same as the size of the first convolution kernel. This makes the second convolution kernel after segmentation match the first convolution kernel supported by the matrix calculation unit, thereby effectively utilizing the computing resources of the matrix calculation unit, reducing the idle computing resources in the processing device, and improving the resource utilization rate of the processing device.

[0022] As a feasible example, regarding the acquisition process of the first data, this application embodiment also provides a possible implementation: the processing device segments the source data based on one or more second segmentation windows to obtain n first data points. For example, the size of each second window is determined based on at least one of the source data, the size of the aforementioned preset first convolution kernel, and the size of the second convolution kernel.

[0023] In this embodiment, the processing device obtains the size of the second segmentation window based on the source data and the aforementioned preset sizes of the first and second convolutional kernels. It then uses this second segmentation window to segment the source data, obtaining segmented data such that the number of segmented data is consistent with the number of convolutional kernels to be processed. This achieves a correspondence between the convolutional kernels to be processed and the segmented data, thereby enabling the data processing model to more effectively utilize the computational resources of the matrix calculation unit, reducing idle computational resources in the processing device, and improving the resource utilization rate of the processing device.

[0024] Secondly, embodiments of this application provide a data processing apparatus. This data processing apparatus includes modules for executing the data processing method in the first aspect or any possible implementation of the first aspect. Exemplarily, the data processing apparatus includes: an acquisition unit, a data processing unit, and a transceiver unit. The acquisition unit is used to acquire first data to be processed. The data processing unit is used to input the first data into a data processing model and obtain the processing result of the first data; wherein the data processing model includes a grouped convolutional layer, the amount of input data to be processed by the grouped convolutional layer each time is matched with the computing power of the matrix calculation unit, and the input data to be processed by the grouped convolutional layer each time includes at least one of the first data and intermediate data input to the grouped convolutional layer after processing by the data processing model. The transceiver unit is used to output the processing result.

[0025] It is worth noting that the acquisition unit, data processing unit, and transceiver unit described above can also be used to implement the data processing method in any possible implementation of the first aspect, which will not be elaborated here.

[0026] Thirdly, embodiments of this application provide a processing device. The processing device includes a processor and an interface circuit. The interface circuit is used to receive signals from other devices outside the processing device and transmit them to the processor, or to send signals from the processor to other devices outside the processing device. The processor implements the operational steps of the data processing method in the first aspect or any possible implementation of the first aspect through logic circuits or executing code instructions.

[0027] Fourthly, embodiments of this application provide a data processing system. The data processing apparatus includes a memory and a processing device provided in the third aspect. The memory is used to store first data to be processed. The processing device is used to implement the operational steps of the data processing method in the first aspect or any possible implementation of the first aspect.

[0028] Fifthly, embodiments of this application provide a computer-readable storage medium. This storage medium stores a computer program or instructions that, when executed by a processing device, implement the operational steps of the data processing method in the first aspect or any possible implementation of the first aspect.

[0029] Optionally, when the computer program or instructions are executed by the data processing system provided by the fourth aspect, the operation steps of the data processing method in the first aspect or any possible implementation of the first aspect may also be implemented.

[0030] Sixthly, embodiments of this application provide a computer program product. When the computer program product is run on a computing device, it causes the computing device to perform the operational steps of the method described in the first aspect or any possible implementation of the first aspect. Here, the computing device may refer to the processing device provided in the third aspect, or to a data processing system including the processing device provided in the third aspect.

[0031] The beneficial effects achievable by aspects two through six above can be referred to the description of any implementation method in aspect one, and will not be repeated here. Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of a neural network structure;

[0033] Figure 2 This is a schematic diagram of a convolutional neural network structure;

[0034] Figure 3 This is a schematic diagram illustrating an application scenario of the data processing method provided in the embodiments of this application;

[0035] Figure 4 An architecture diagram of a data processing device provided in an embodiment of this application;

[0036] Figure 5 This application provides a schematic diagram of the structure of a computing node.

[0037] Figure 6 This is a schematic diagram of the structure of a child node provided in an embodiment of this application;

[0038] Figure 7 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0039] Figure 8 This is a schematic diagram illustrating an example of input data M+c provided in an embodiment of this application.

[0040] Figure 9 This is a diagram illustrating the processing procedure of the grouped convolutional layer on the input data provided in an embodiment of this application.

[0041] Figure 10 This is a schematic diagram of computing node operations provided in an embodiment of this application;

[0042] Figure 11 A flowchart illustrating a segmentation method provided in an embodiment of this application;

[0043] Figure 12 A schematic diagram illustrating a specific embodiment of the segmentation method provided in this application;

[0044] Figure 13 A schematic diagram illustrating a specific embodiment of computing node operations provided in this application;

[0045] Figure 14 This is a schematic diagram of data transmission when the computing node is a multiply-accumulator.

[0046] Figure 15 This is a schematic diagram of data transmission when the computing node is a vector multiplier-accumulator.

[0047] Figure 16 for Figure 15 Data flow diagram when the computation node is a vector multiplier-accumulator;

[0048] Figure 17 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Detailed Implementation

[0049] This application provides a data processing method executed by a processing device. The processing device acquires first data to be processed, processes the first data using a data processing model deployed within the device, and outputs the processing result. The data processing model includes a grouped convolutional layer. The amount of input data processed by each grouped convolutional layer is matched to the computing power of the matrix computation units included in the processing device, avoiding the waste of computing power caused by a mismatch in computing power, which leads to a decrease in processing efficiency. Furthermore, because the data to be processed by the grouped convolutional layer is matched to the computing power provided by the matrix computation units, the grouped convolutional layer can more effectively utilize the computing resources included in the processing device, reducing idle computing resources and improving the resource utilization rate of the processing device. When the data processing model can make greater use of the computing resources provided by the processing device during data processing, it is beneficial to improve the efficiency of the data processing model in processing data.

[0050] To ensure clarity and brevity in the description of the following embodiments, a brief introduction to the relevant technologies is given first.

[0051] (1) Neural Network

[0052] Neural networks can be composed of neurons, and neurons can refer to neurons that are denoted by x. s The arithmetic unit takes an intercept of 1 as input. The output of this arithmetic unit satisfies the following formula:

[0053]

[0054] Where s = 1, 2, ..., n, n is a natural number greater than 1, W s For x sThe weights are denoted by b, where b is the neuron's bias. f is the neuron's activation function, used to introduce nonlinearity into the neural network, converting the input signal into the output signal. The output signal of this activation function can be used as the input to the next layer. A neural network is a network formed by connecting multiple individual neurons. The weights characterize the strength of the connections between different neurons and determine the influence of the input on the output.

[0055] Figure 1 This is a schematic diagram of a neural network structure. The neural network 100 includes X processing layers, where X is an integer greater than or equal to 3. The first layer of the neural network 100 is the input layer 110, responsible for receiving input signals. The last layer of the neural network 100 is the output layer 130, responsible for outputting the processing results of the neural network. The remaining layers, excluding the first and last layers, are intermediate layers 140. These intermediate layers 140 together form the hidden layer 120. Each intermediate layer 140 in the hidden layer 120 can both receive input signals and output signals. The hidden layer 120 is responsible for processing the input signals. Each layer represents a logical level of signal processing; through multiple layers, data signals can undergo multi-level logical processing.

[0056] (2) Convolutional Neural Network

[0057] A convolutional neural network (CNN) is a deep neural network with a convolutional structure. A CNN contains a feature extractor consisting of convolutional layers and subsampling layers. This feature extractor can be viewed as a filter, and the convolution process can be seen as performing convolution between a trainable filter and an input image or feature map. A convolutional layer can output several feature maps, which can refer to intermediate results in the CNN's computation. Neurons within the same feature map share weights; these shared weights are the convolutional kernels. Within the same convolutional layer, multiple convolutional kernels can be used to extract different image information. Generally, the more convolutional kernels, the richer the image information reflected by the convolution operation.

[0058] For example, Figure 2 This is a schematic diagram of a convolutional neural network structure. The convolutional neural network 200 may include an input layer 210, a convolutional / pooling layer 220 (where the pooling layer is optional), and a neural network layer 230.

[0059] The convolutional / pooling layer 220 may include layers 221 to 226.

[0060] In one possible example, layer 221 is a convolutional layer, layer 222 is a pooling layer, layer 223 is a convolutional layer, layer 224 is a pooling layer, layer 225 is a convolutional layer, and layer 226 is a pooling layer.

[0061] In another possible example, layers 221 and 222 are convolutional layers, layer 223 is a pooling layer, layers 224 and 225 are convolutional layers, and layer 226 is a pooling layer.

[0062] The following section uses convolutional layer 221 as an example to introduce the internal working principle of a convolutional layer. Convolutional layer 221 can include many convolution operators, also known as convolution kernels. In image processing, the convolution kernel acts as a filter to extract specific information from the input image matrix. A convolution operator is essentially a weight matrix, which is usually predefined. During the convolution operation, the weight matrix typically processes the input image pixel by pixel (or two pixels by two pixels, depending on the stride) along the horizontal direction, thus extracting specific features from the image. The depth dimension of the weight matrix is ​​the same as the depth dimension of the input image. During the convolution operation, the weight matrix extends to the entire depth of the input image. Different weight matrices are used to extract different features from the image. For example, one weight matrix is ​​used to extract image edge information, another to extract specific colors, and yet another to blur unwanted noise in the image.

[0063] After processing by the convolutional / pooling layers 220, the convolutional neural network 200 is still insufficient to output the required information. As mentioned earlier, the convolutional / pooling layers 220 extract features and reduce the parameters introduced by the input image. However, in order to generate the final output information (the required class information or other relevant information), the convolutional neural network 200 needs to utilize neural network layer 230 to generate one or a set of required class numbers of output. Therefore, neural network layer 230 may include multiple hidden layers (such as... Figure 2 Layers 231, 232 to 23n, and output layer 240 are shown. The parameters contained in these multiple hidden layers can be pre-trained based on relevant training data for specific task types. For example, task types include image recognition, image classification, image super-resolution reconstruction, etc.

[0064] After multiple hidden layers in neural network layer 230, the final layer of the entire convolutional neural network 200 is the output layer 240. This output layer 240 has a loss function similar to classification cross-entropy, specifically used to calculate the prediction error. Once the entire convolutional neural network 200 has undergone forward propagation (e.g., ...), ... Figure 2Propagation from layer 210 to layer 240 is called forward propagation, and backward propagation (such as...) is completed. Figure 2 The propagation from layer 240 to layer 210 (backpropagation) will begin to update the weight values ​​and biases of the aforementioned layers to reduce the loss of the convolutional neural network 200 and the error between the output of the convolutional neural network 200 through the output layer and the ideal result.

[0065] It should be noted that, as Figure 2 The convolutional neural network 200 shown is only an example of a convolutional neural network. In specific applications, convolutional neural networks can also exist in the form of other network models.

[0066] (3) Grouped convolution

[0067] Grouped convolution is the process of grouping the input data (Cin*H*W*C) in a convolutional neural network and then performing convolution, where Cin is the number of channels in the input data, H is the height of the input data, W is the width of the input data, and C is the number of input data. For example, the input data can be at least one of images, audio, and text.

[0068] Grouped convolution can replace convolutional neural networks by sacrificing network precision for a reduction in computational cost.

[0069] When using traditional convolution: there are Y convolution kernels, the number of output feature maps is the same as the number of convolution kernels, also Y, the size of each convolution kernel is C*K*K, and the total number of parameters of N convolution kernels is Y*C*K*K.

[0070] When using grouped convolution: Let the number of groups be g. Then the number of input feature maps in each group is C / g, the number of output feature maps in each group is Y / g, the size of each convolutional kernel is C / g*K*K, and the total number of convolutional kernels remains Y. The number of convolutional kernels in each group is Y / g. Therefore, the total number of parameters when using grouped convolution is Y*C / g*K*K. The total number of parameters in grouped convolution is 1 / g of that in traditional convolution.

[0071] The above text describes the relevant technical terms used in this application. The following text describes the application scenarios in conjunction with the accompanying drawings. Figure 3 This diagram illustrates an application scenario of the data processing method provided in this application. The data processing method can be applied to scenarios such as face recognition, face payment, smart access control, tracking, autonomous driving, and speech and semantic recognition. The processing device processes one or more of the image, sound, or text information generated in the above scenarios to obtain the processing result.

[0072] In one possible example, image, sound, and text information can be collected by a terminal device and transmitted to a processing device for processing to obtain the processing result.

[0073] For example, the terminal device can be a mobile phone, tablet computer, virtual reality (VR) terminal device, augmented reality (AR) terminal device, wireless terminal in industrial control, wireless terminal in self-driving, wireless terminal in smart grid, wireless terminal in transportation safety, wireless terminal in smart city, or wireless terminal in smart home.

[0074] In another possible example, images, sounds, and text information can also be stored in a storage device and transmitted to a processing device for processing to obtain the processing result.

[0075] For example, the storage device can be a disk, such as a hard disk drive or a solid-state drive.

[0076] For example, the processing device may be a processor or a computer or mobile phone with processing capabilities.

[0077] The following is a detailed illustration of the architecture of the processing device in this application, using a processor as an example. Figure 4 This is an architectural diagram of a data processing device provided in an embodiment of this application. The processing device 400 includes a first cache unit 410, a second cache unit 420, a third cache unit 430, a matrix calculation unit 440, an output cache unit 450, and a control unit.

[0078] In one possible example, the control unit can invoke a data processing model to process the data. For instance, the processing device 400 deploys a data processing model that may include grouped convolutional layers, such as one or more second convolutional kernels (R*C), where R is the height of the second convolutional kernel and C is the width of the second convolutional kernel. During data processing, the matrix calculation unit 440 supports the implementation of the grouped convolution function of the data processing model.

[0079] For example, the data processing model in this example may include, but is not limited to: object recognition model, target detection model, image classification model, etc., or other artificial intelligence (AI) models that meet user needs and are obtained based on samples collected by the processing device.

[0080] The components included in the processing device 400 will be described below.

[0081] The output of the first cache unit 410 is connected to the inputs of the second cache unit 420 and the third cache unit 430, respectively. The first cache unit 410 is used to store first data or intermediate data determined by the data processing model.

[0082] In one possible example, the first or intermediate data can be audio, image, or text information. The size of the first or intermediate data is (Cin*H*W), where Cin is the number of channels, H is the height, and W is the width.

[0083] In another possible example, the intermediate data is the data input to the grouped convolutional layer after the first data has been processed by the data processing model. For example, the data processing model includes at least one preprocessing layer placed before the grouped convolutional layer. The preprocessed result (such as image segmentation) obtained after the first data input to the data processing model is processed by the preprocessing layer is the intermediate data.

[0084] The output of the second buffer unit 420 is connected to the input of the matrix calculation unit 440.

[0085] In one possible example, the second cache unit 420 and the matrix calculation unit 440 are connected through a multicast network, thereby realizing a one-to-many connection between the second cache unit 420 and the matrix calculation unit 440.

[0086] The second buffer unit 420 is used to store input data, which may be data obtained by the control unit reading the first data line by line.

[0087] In one possible example, the second cache unit 420 could be a line cache unit.

[0088] The output of the third buffer unit 430 is connected to the input of the matrix calculation unit 440.

[0089] In one possible example, the first cache unit 410, the second cache unit 420, and the third cache unit 430 may be cache, solid state drives (SSDs), hard disk drives (HDDs), storage class memory (SCMs), memory, or other storage media, for example, storage particles that store a specific number of bits: single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), or quad-level cell (QLC), etc.

[0090] The output of the matrix calculation unit 440 is connected to the input of the output buffer unit 450. The matrix calculation unit 440 supports the calculation of one or more preset first convolution kernels (r*c), where r is the height of the first convolution kernel and c is the width of the first convolution kernel.

[0091] The matrix calculation unit 440 includes M rows × N columns of calculation nodes, where M and N are both positive integers.

[0092] In one possible example, the individual computing nodes can be connected via a two-dimensional pulsating array. For example... Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of a computing node provided in an embodiment of this application. The computing node 50 includes a shift register 51 and at least one child node 52.

[0093] For example, the shift register 51 of computing node 50 is used to store data obtained from the previous computing node 50 or the second cache unit 420, and the obtained data is transferred to the child node 52 connected to it after shifting. The computing nodes 50 located in the same row as the matrix calculation unit 440 obtain the same data.

[0094] For example, compute node 50 is located in the first column, and the shift register 51 of compute node 50 obtains data from the second cache unit 420.

[0095] For example, compute node 50 is located in other columns, and the shift register 51 of compute node 50 obtains data from the previous compute node 50.

[0096] For example, the child nodes 52 of computing node 50 can be connected via a one-dimensional pulsating array or a broadcast network. Figure 6 As shown, Figure 6 This is a schematic diagram of a child node structure provided in an embodiment of this application. The child node 52 may include at least one multiplier 61, an addition tree 62, and an output buffer 63. The multiplier 61 is used to receive data sent by the shift register 51, and the multiplier 61 can independently choose whether to receive the value transmitted by the shift register 51.

[0097] The output terminal of the output buffer unit 450 is connected to the input terminal of the first buffer unit 410. The output buffer unit 450 is used to store the results calculated by the matrix calculation unit 440.

[0098] In one possible example, the output buffer unit 450 may be a register set or a static memory set with multiple storage spaces.

[0099] above Figures 4 to 6This application is merely illustrative of possible implementations of the processing device and does not limit the implementation of the processing device architecture. For example, the architecture can be implemented using one or more of Field Programmable Gate Array (FPGA), Programmable Array Logic (PLA), and / or integrated circuits (such as Application Specific Integrated Circuit (ASIC)).

[0100] In summary Figures 4 to 6 The following describes the processing method provided in this application: The processing device 400 stores input data in a second cache unit 420 and transmits the input data to a matrix calculation unit 440 connected to the second cache unit 420 via a multicast network for processing to obtain the processing result. Thus, by matching the amount of input data with the computing power of the matrix calculation unit 440, the processing device 400 avoids the problem of low processing efficiency caused by varying amounts of data read each time. Furthermore, the grouped convolutional layer can more effectively utilize the computing resources contained in the processing device 400, reducing idle computing resources and improving the resource utilization rate of the processing device 400.

[0101] The above text combined Figures 4 to 6 The architecture diagram of the processing device 400 of this application is described below. The data processing method of this application is explained below with reference to the accompanying drawings.

[0102] like Figure 7 As shown, Figure 7 A flowchart illustrating a data processing method provided in this application embodiment, which can be applied to... Figure 4 The processing device 400 shown is an example. Exemplarily, this data processing method is... Figure 4 The processing device 400 performs the processing. The data processing method provided in this embodiment includes the following steps S710 to S730.

[0103] S710, The processing device acquires the first data to be processed.

[0104] In a first possible example, the first data is the source data (Cin0*H0*W0) collected by the processing device, where Cin0 is the number of channels in the source data, H0 is the height of the source data, and W0 is the width of the source data. The source data may include, but is not limited to, images, audio, and text.

[0105] In a second possible example, the first data is the data obtained by the processing device after preprocessing the source data. The process by which the processing device obtains the first data after preprocessing the source data can be referred to the following... Figure 11 The description of that will not be repeated here.

[0106] The above two examples are merely two methods for obtaining the first data provided in this embodiment and should not be construed as limiting the first data in this application. In some optional cases, the first data is source data preprocessed by other processing devices, and the preprocessed data is sent to the processing device. In other words, the data received by the processing device from other processing devices is the first data described above.

[0107] S720, the processing device inputs the first data into the data processing model and obtains the processing result of the first data.

[0108] In one possible example, the result of processing the first data is the output of the data processing model after the first data is input into the data processing model.

[0109] For information on data processing model types, please refer to the above. Figure 4 The relevant descriptions will not be repeated here.

[0110] In this embodiment, the data processing model includes a grouped convolutional layer, wherein the amount of input data to be processed by the grouped convolutional layer each time is matched with the computing power of the matrix calculation unit.

[0111] In one possible scenario, the control unit sets a node count threshold for the number of hardware nodes that the data processing model can use. This threshold represents a portion or all of the nodes included in the processing device (which can be determined by setting an upper limit or proportion of the node count). Matching the input data volume with the computational power of the matrix calculation unit means that the input data volume is consistent with the computational power (or computational resources) provided by the computing nodes corresponding to the node count threshold.

[0112] In another possible scenario, the amount of input data to be processed by the grouped convolutional layer each time is matched with the computing power of the matrix computation unit, including: the amount of input data to be processed each time is M+c, where c is the same as the width of the first convolutional kernel.

[0113] For example, the control unit controls the first cache unit 410 to transfer the first data or intermediate data to the second cache unit 420 in rows M+c columns. Figure 8 This is a schematic diagram illustrating an example of input data M+c provided in this application.

[0114] For example, taking a first convolution kernel with a size of 3*4 as an example, the control unit controls the first cache unit 410 to transmit the first data or intermediate data to the second cache unit 420 in one row and M+4 columns.

[0115] Combination Figure 4 To briefly explain this example, the processing device 400 controls the amount of data stored in the second cache unit 420 by controlling the amount of data transmitted from the first cache unit 410 to the second cache unit 420. This achieves a correlation between the amount of input data stored in the second cache unit 420, the number of rows in the matrix calculation unit 440, and the width of the preset first convolutional kernel supported by the matrix calculation unit 440. This ensures that the amount of data processed by the grouped convolutional layer each time matches the computing power provided by the matrix calculation unit, avoiding the problem of low processing power caused by the processing device reading different amounts of data each time during data processing.

[0116] In one possible scenario, the processing device inputs first data into a data processing model to obtain a processing result for the first data, including: the processing device writing at least one of the first data and intermediate data processed by the data processing model into a memory connected to the processing device; thereby, the processing device processes each input data to be processed retrieved from the memory based on a grouped convolutional layer. The processing device processes each input data to be processed retrieved from the memory based on a grouped convolutional layer as follows: the processing device reads the input data to be processed each time from the memory, inputs it into the grouped convolutional layer, and uses the output of the grouped convolutional layer as the processing result.

[0117] Optionally, in this case, the memory may refer to the above. Figure 4 The second cache unit 420.

[0118] It is worth noting that the input data processed by the above grouped convolutional layers each time includes at least one of the following: first data and intermediate data input to the grouped convolutional layers after the first data has been processed by the data processing model. For details regarding the intermediate data, please refer to the relevant description of the processing device 400 above, which will not be elaborated upon here.

[0119] Combination Figure 4 The data processing process of the grouped convolutional layer is briefly described as follows: the processing device 400 writes at least one of the first data and intermediate data into the second cache unit 420. The data stored in the second cache unit 420 includes the input data to be processed each time. The processing device 400 processes the data obtained from the second cache unit 420 based on the grouped convolutional layer.

[0120] The processing device 400 writes at least one of the first data and intermediate data into the second cache unit 420. The processing device 400 processes the data obtained from the second cache unit 420 based on the grouped convolutional layer, so that the processing device 400 can obtain the data required by the grouped convolutional layer from the second cache unit 420. This avoids the problem of low processing efficiency of the processing device 400 due to the different amount of data read each time during the data processing process. Figure 9 This is a diagram illustrating the processing procedure of the grouped convolutional layer on the input data provided in this embodiment. The processing procedure of the grouped convolutional layer on the input data can be referred to below. Figure 9 The description of that will not be repeated here.

[0121] In one possible scenario, the processing device 400 processes the input data to be processed each time, retrieved from memory, based on a grouped convolutional layer. This includes: the processing device 400 confirms, based on the number of rows and columns of the matrix computation unit, one or more preset first convolutional kernels supported by the i-th computation node 50, and inputs c data points from memory into the i-th computation node 50, where c is the same as the width of the first convolutional kernel. The i-th computation node 50 is the i-th computation node 50 in the first column of an M rows × N columns of computation nodes.

[0122] In one possible scenario, the storage addresses of the above c data items are contiguous. For example, the c data items are stored in c consecutive storage locations within the second cache unit.

[0123] Regarding the implementation process of the processing device 400 acquiring the grouped convolutional layer, the following is a possible example: The processing device 400 controls the second cache unit 420 to transfer c of the input data stored therein to a computation node 50 located in the first column of the matrix computation unit 440. The N computation nodes 50 in each row of the matrix computation unit 440 acquire the same data.

[0124] Taking a first convolution kernel with a size of 3*4 as an example: the control unit controls the second cache unit 420 to transmit 4 of the input data stored therein to a calculation node 50 located in the first column of the matrix calculation unit 440. The calculation node 50 then transmits the obtained data to the other N-1 calculation nodes 50 located in the same row.

[0125] For example, consider a compute node (0, 0) containing four child nodes, each with a value of 52. Figure 10 This is a schematic diagram of computing node operations provided in an embodiment of this application. Figure 10 In the first clock cycle, the four child nodes 52 in the computation node (0,0) acquire A0-A3 after shifting through shift register 51. The computation nodes 50 in the first row of the matrix computation unit 440 all obtain the data A0-A3.

[0126] For example, the calculation nodes 50 of each column of the matrix calculation unit 440 obtain the same weight data. In one possible embodiment, the third cache unit 430 first provides all the weight data in one group, then provides all the weight data in the next group, until all the weights have been provided.

[0127] For example, consider compute node 50, which contains four child nodes 52. Figure 10 In the first clock cycle, the third buffer unit 430 first provides the data of ker0 and ker1 in g0 to the child node (0) and child node (1) respectively, and then provides the data of ker0 and ker1 in g1 to the child node (2) and child node (3) respectively.

[0128] After acquiring the input data and the weight data stored in the convolution kernel, the computation node 52 of the matrix computation unit 440 performs calculations based on the input data and the weight data to obtain the calculation result.

[0129] For example, child node (0): A0*W00+A1*W01+A2*W02+A3*W03.

[0130] Child node (1): A0*W04+A1*W05+A2*W06+A3*W07.

[0131] Child node (2): A0*V00+A1*V01+A2*V02+A3*V03.

[0132] Child node (3): A0*V00+A1*V01+A2*V02+A3*V03.

[0133] As an optional scenario, the memory sends data to the computation nodes 50 located in adjacent rows of the first column of the matrix computation unit 440 in two consecutive transactions. For example, the second cache unit 440 sends c data to the computation node 50 in the (i-1)th row of the first column in the i-th transaction, and the second cache unit 440 sends c data to the computation node 50 in the i-1th transaction in the i-th transaction.

[0134] Optionally, the interval between two consecutive data transmissions from the memory is s data points, where s is a positive integer greater than 1. For example, the second buffer unit 420 sends Ai-A(i+3) to the matrix calculation unit 440 for the i-th time, and sends A(s+i)-A(s+i+3) to the matrix calculation unit 440 for the (i+1)-th time.

[0135] The processing device 400 transmits c data points from its second cache unit 420 to its matrix computation unit 440. This correlates the width of the first convolution kernel supported by the matrix computation unit 440 with the amount of input data, ensuring that the computational power of the matrix computation unit 440 matches the amount of data transmitted by the second cache unit 420. This avoids the problem of low processing efficiency caused by varying amounts of data read by the processing device 400 during data processing. Furthermore, because the data processed by the grouped convolutional layer matches the computational power provided by the matrix computation unit 440, the grouped convolutional layer can more effectively utilize the computational resources contained in the processing device 400, reducing idle computational resources and improving the resource utilization rate of the processing device 400.

[0136] S730, the processing device outputs the processing result obtained from S720.

[0137] In the first possible example, the processing device writes the processing result to memory. For example... Figure 4 As shown, the matrix calculation unit 440 writes the calculation result into the first cache unit 410. The memory can also be accessed by other devices / nodes, enabling other devices / nodes to retrieve the data stored in the memory and display / play the processed data.

[0138] In a second possible example, the processing device sends the processing result to another device. This other device may include, but is not limited to, computers, mobile phones, virtual reality devices, and terminal devices with receiving capabilities.

[0139] exist Figure 7 In the illustrated embodiment, the processing device acquires first data to be processed, processes the first data using a data processing model deployed within the processing device, and outputs the processing result. This data processing model includes grouped convolutional layers. The amount of input data processed by each grouped convolutional layer is matched to the computing power of the matrix computation units contained in the processing device, thus avoiding the waste of computing power caused by a mismatch in computing power, which leads to a decrease in processing efficiency.

[0140] Furthermore, because the data processed by the grouped convolutional layers matches the computational power provided by the matrix computation units, the grouped convolutional layers can more effectively utilize the computational resources contained in the processing device, reducing idle computational resources and improving the resource utilization rate of the processing device during data processing. Additionally, when the data processing model can make greater use of the computational resources provided by the processing device, it also helps to improve the efficiency of the data processing model in processing data.

[0141] As a feasible example, the processing device 400 stores k*(M+c) data in the second cache unit 420, where k is a positive integer, and transfers c data in the second cache unit 420 to a computing node 50 located in the first column of the matrix computing unit 440.

[0142] In this example, the processing device 400 associates the height and width of the first convolution kernel supported by the matrix computing unit 440 with the amount of data stored in the second cache unit 420 and the amount of data transmitted to the matrix computing unit 440. This makes the computing power of the matrix computing unit 440 match the amount of data to be processed by the grouped convolutional layer, effectively utilizing the computing resources of the processing device 400, reducing idle computing resources in the processing device 400, and improving the resource utilization rate of the processing device 400.

[0143] Figure 11 A flowchart of a segmentation method provided in an embodiment of this application is shown below. Figure 11 The possible implementation methods for the processing device to acquire the first data are described in detail, including the following steps S1101 to S1104.

[0144] S1101, the processing device determines whether a second convolution kernel matches any one of a set of one or more preset first convolution kernels for any one of the one or more second convolution kernels.

[0145] Matching the first and second convolution kernels can mean that the height of the first and second convolution kernels are the same, and the width of the first and second convolution kernels are the same.

[0146] S1102, if the second convolution kernel matches any of the first convolution kernels, the processing device uses the source data as the first data.

[0147] S1103, if the second convolution kernel does not match any of the first convolution kernels, the processing device processes the second convolution kernel to obtain multiple convolution kernels to be processed; the convolution kernels to be processed match one or more preset first convolution kernels.

[0148] In one possible scenario, the second convolutional kernel may not match any of the first convolutional kernels. This could mean that the height and width of the second convolutional kernel are smaller than the height and width of the first convolutional kernel. In this case, the second convolutional kernel is extended in both the width and height directions using elements with values ​​equal to zero, so that the second convolutional kernel matches the first convolutional kernel.

[0149] In another possible scenario, the second convolutional kernel may not match any of the first convolutional kernels. This could mean that the height and / or width of the second convolutional kernel is smaller than the height and / or width of the first convolutional kernel. In this case, the processing device divides the second convolutional kernel to obtain multiple convolutional kernels to be processed, which are matched with one or more preset first convolutional kernels.

[0150] The following describes in detail the segmentation method of the processing device to obtain multiple convolution kernels to be processed by segmenting the second convolution kernel. The segmentation method includes the following steps: the processing device segments the second convolution kernel based on one or more first segmentation sliding windows to obtain multiple convolution kernels to be processed.

[0151] In this context, each first segmentation sliding window corresponds to a preset first convolutional kernel. For example, Figure 12 A schematic diagram illustrating a specific embodiment of the segmentation method provided in this application is shown below. Figure 12 As shown, the size of the first convolution kernel is 3*4, so the size of the first segmentation sliding window is 3*4.

[0152] Optionally, the size of the first segmentation sliding window is consistent with the size of a preset first convolution kernel.

[0153] For example, the size of the first segmentation sliding window includes the height of the first convolution kernel.

[0154] For example, the size of the first segmented sliding window includes the width of the first convolution kernel.

[0155] For example, the dimensions of the first segmented sliding window include the height and width of the first convolution kernel.

[0156] The segmentation process of the second convolutional kernel is described below, with dimensions including the height and width of the kernel. The segmentation process of the second convolutional kernel includes the following first and second steps.

[0157] Step 1: The processing device divides the second convolution kernel in the width direction of the second convolution kernel.

[0158] The segmentation process of the second convolution kernel in the width direction may include the following three steps: Step ① to Step ③.

[0159] Step ①: The first segmentation sliding window first selects the element located at the leftmost end of the second convolution kernel, and the processing device obtains the first convolution kernel to be processed.

[0160] Step ②: The first segmentation window moves c steps along the width direction of the second convolutional kernel, and the processing device obtains the second convolutional kernel to be processed. Repeat the above operation until the second convolutional kernel is completely segmented in the width direction.

[0161] Step 3: If, during the final segmentation, the elements in the second convolution kernel cannot completely fill the first segmentation window, the processing device extends the second convolution kernel in the width direction so that the elements in the second convolution kernel completely fill the first segmentation window, resulting in a convolution kernel to be processed.

[0162] For example, the processing device uses elements with values ​​equal to zero to extend the second convolution kernel in the width direction.

[0163] Step 2: The processing device divides the second convolution kernel along the height direction of the second convolution kernel.

[0164] The difference between the height and width directions in the convolutional kernel segmentation process lies in the stride size: the stride size in the height direction is different from that in the width direction, and the stride size in the height direction is r. The rest of the height-direction convolutional kernel segmentation process is the same as that in the width direction, and will not be repeated here.

[0165] S1104, the processing device segments the source data according to multiple convolutional kernels to be processed, obtains segmented data with the same number of kernels to be processed, and uses the segmented data as the first data.

[0166] The processing device obtains multiple segmented data based on one or more second segmentation sliding window source data.

[0167] In this case, one segment of data corresponds to one second convolution kernel.

[0168] Optionally, the size of the second sliding window is determined based on at least one of the source data, the size of a first convolution kernel, and the size of a second convolution kernel. Several possible examples are given below to illustrate the size of the second sliding window.

[0169] Example 1: The size of the second sliding window includes the height of the source data.

[0170] Example 2: The size of the second sliding window includes the width of the source data.

[0171] Example 3: The size of the second sliding window includes the height of the aforementioned first convolution kernel.

[0172] Example 3: The size of the second sliding window includes the width of the aforementioned first convolution kernel.

[0173] Example 4: The size of the second sliding window includes the height of the second convolution kernel.

[0174] Example 5: The size of the second sliding window includes the width of the second convolution kernel.

[0175] Example 6: The size of the second sliding window is calculated based on the source data, a first convolution kernel, and a second convolution kernel. For example, the height of the second segmentation sliding window is (H0 - R + r), and the width is (W0 - C + c). Figure 12 As shown, the size of the source data is 6*6, the size of the first convolution kernel is 3*4, the size of the second convolution kernel is 5*5, and the size of the second segmentation window is 4*5.

[0176] The above six examples are merely possible implementations provided in this embodiment and should not be construed as limiting this application.

[0177] The following example illustrates the process of processing equipment segmenting source data. The process of segmenting source data may include the following first and second steps.

[0178] Step 1: The processing device segments the source data in the width direction.

[0179] The process of segmenting the source data in the width direction may include the following three steps: Step ① to Step ③.

[0180] Step ①: The second segmentation sliding window first selects the element located at the leftmost end of the source data, and the processing device obtains the first segmented data.

[0181] Step ②: The second segmentation window moves c steps in the direction of the first data width, and the processing device obtains the second segmented data. The processing device repeats the above operation until the source data is completely segmented.

[0182] Step 3: If, during the final segmentation, the elements in the source data cannot completely fill the second segmentation window, the processing device extends the source data in the width direction until the elements in the source data completely fill the second segmentation window, resulting in a segmented data set.

[0183] For example, the processing device extends the source data in the width direction using elements with a value of zero.

[0184] Step 2: The processing device segments the source data along the height direction.

[0185] The difference between the source data segmentation process in the height and width directions lies in the step size: the step size in the height direction is different from that in the width direction, and the step size in the height direction is r. The rest of the source data segmentation process in the height direction is the same as that in the width direction, and will not be repeated here.

[0186] like Figure 12 As shown, the first convolution kernel size is 3*4, the second convolution kernel size is 5*5, and the source data size is 6*6. The specific segmentation process includes the following first and second steps.

[0187] Step 1: Divide the second convolutional kernel to obtain multiple convolutional kernels to be processed, including the following four steps: Step 1 to Step 4.

[0188] Step ①: The first segmentation sliding window (3*4) selects the elements from the first row and first column to the fourth row and fourth column of the second convolution kernel (5*5), and the processing device obtains the convolution kernel (0) to be processed.

[0189] Step ②: The first segmentation window (3*4) moves 4 steps in the width direction of the second convolution kernel (5*5) based on the first segmentation. Since the second convolution kernel (5*5) has a width dimension of 5, it can only fill the first column of the first segmentation window (3*4). The processing device extends the second convolution kernel (5*5) by 3 dimensions in the width direction using elements with a value of 0. The first segmentation window (3*4) selects the first row and fifth column to the fourth row and eighth column of the extended second convolution kernel (5*8), and the processing device obtains the convolution kernel (1) to be processed.

[0190] Step 3: The first segmentation window (3*4) moves 3 steps in the height direction of the second convolution kernel (5*5) based on the first segmentation. Since the size of the second convolution kernel (5*5) in the height direction is 5, at this time, the second convolution kernel (5*5) can only fill the first row and the second row of the first segmentation window (3*4). The processing device extends the second convolution kernel (5*5) by 1 dimension in the height direction using an element with a value of 0. The first segmentation window (3*4) selects the 4th row and 1st column to the 6th row and 4th column of the extended second convolution kernel (6*5), and the processing device obtains the convolution kernel (2) to be processed.

[0191] Step 4: The first segmentation window (3*4) moves 3 steps in the height direction of the second convolution kernel (5*5) based on the second segmentation. Since the size of the second convolution kernel (5*5) in the height direction is 5, at this time, the second convolution kernel (5*5) can only fill the first row and first column to the second row and first column of the first segmentation window (3*4). The processing device extends the second convolution kernel (5*5) by 1 dimension in the height direction and 3 dimensions in the width direction using elements with a value of 0. The first segmentation window (3*4) selects the fourth row and fifth column to the sixth row and eighth column of the extended second convolution kernel (6*8), and the processing device obtains the convolution kernel (3) to be processed.

[0192] Step 2: Segment the source data (6*6) to obtain segmented data that matches the number of convolution kernels to be processed. This includes the following four steps: Step 1 to Step 4.

[0193] Step ①: Use the second splitting sliding window (4*5) to select the first row and first column of the source data (6*6) to the fourth row and fifth column to obtain the splitting data (0).

[0194] Step ②: The second segmentation window (4*5) moves 4 steps in the width direction of the source data (6*6). Since the size of the source data (6*6) in the width direction is 6, at this time, the source data (6*6) can only fill the first and second columns of the second segmentation window (4*5). The processing device extends the source data (6*6) by 3 dimensions in the width direction using elements with a value of 0. The second segmentation window (4*5) selects the first row and fifth column to the fourth row and ninth column of the extended source data (6*9), and the processing device obtains the segmented data (1).

[0195] Step 3: The second segmentation window (4*5) moves 3 steps in the height direction of the source data (6*6) based on the first segmentation. Since the size of the source data (6*6) in the height direction is 6, at this time, the source data (6*6) can only fill the first to third rows of the second segmentation window (4*5). The processing device extends the source data (6*6) by 1 size in the height direction using elements with a value of 0. The second segmentation window (4*5) selects the fourth row and first column to the seventh row and fifth column of the extended source data (7*6), and the processing device obtains the segmented data (2).

[0196] Step 4: The second segmentation window (4*5) moves 3 steps in the height direction of the source data (6*6) based on the second segmentation. Since the source data (6*6) has a height dimension of 6, it can only fill the first row and first column to the third row and second column of the second segmentation window (4*5). The processing device extends the source data (6*6) by 1 dimension in the height direction and 3 dimensions in the width direction using elements with a value of 0. The second segmentation window (4*5) selects the fourth row and fifth column to the seventh row and ninth column of the extended source data (7*9), and the processing device obtains the segmented data (3).

[0197] In one possible example, the high-order bits of the input data in the second cache unit can be provided to the corresponding child node through the shift register of the computing node, enabling the processing device to reread the data stored in the second cache unit.

[0198] For example, Figure 10 In the first clock cycle, input data A0-A3 is read into compute node (0,0). Child nodes (0) and (1) of compute node (0,0) use the data corresponding to group g0 of A0-A3 for calculation. In the second clock cycle, input data A0-A3 is read into compute node (0,0) again. Child nodes (2) and (3) of compute node (0,0) use the data corresponding to group g1 of A0-A3 for calculation. The processing device can reread the data stored in the second buffer unit without setting a shift register for each child node or every few child nodes, thus reducing the number of shift registers.

[0199] In one possible example, the bandwidth of the second cache unit can match the computational power of a matrix computation unit with two channels per group (i.e., cpg = 2, channels per group) and one convolutional kernel. This bandwidth matching ensures that each child node of the matrix computation unit is computed every clock cycle, and that the second cache unit can provide new data every clock cycle.

[0200] For example, Figure 13 This is a schematic diagram illustrating a specific embodiment of the computing node operation provided in this application. For example... Figure 13 As shown, with cpg=2 and a convolution kernel of 2, the data row buffer unit writes new data to the child nodes every clock cycle (the gray rectangle shows the data). Each child node performs one to two VCC operations per clock cycle. In this embodiment, each child node reads a maximum of 1 row and 4 columns of input data per clock cycle. Each multiplier in the child node can independently choose to perform operations, or simply store the input data in its own input buffer without performing operations. The addition tree of the child node can choose to perform operations or not. There are 3 sets of output buffers (BUF), and the result of each operation is stored in one of the BUFs. The processing device adds the data in the output buffer (BUF) to the current VCC result to obtain the output accumulation. Output accumulation can be enabled or disabled. The fourth buffer unit writes the output data back to the first buffer unit for the next layer of convolution.

[0201] US2020 / 0233803 computes grouped convolutions by reusing computational units (PEs) within matrix multiplication units. The input feature map (IFMAP) is fed row by row into the input buffer IRAM and then sent to the matrix computational units composed of PEs via a shift register (SFT). Figure 14 This is a schematic diagram of data transmission when the computing node is a multiply-accumulator, such as... Figure 14 As shown, the computing power is low when PE is a multiply-accumulator, and PE is calculating every clock cycle. Figure 15 This is a schematic diagram illustrating data transmission when the computation node is a vector multiplier-adder. For example... Figure 15 As shown, the increased computing power of PE means that PE is not calculating every clock cycle, resulting in computational downtime and low utilization of PE. Figure 16 for Figure 15 The data flow graph when the computation node is a vector multiplier-accumulator is shown below. Figure 16 As shown, computational downtime occurred during cyc3. Increasing the computing power of the PE (Programmable Execution Unit) can lead to computational downtime. However, in this application, because the amount of input data to be processed each time is matched with the computing power of the matrix computation unit, computational downtime does not occur, thus improving the utilization efficiency of computing resources.

[0202] The above text combined Figures 1 to 16The data processing method provided in this application is described below, and will be combined with... Figure 17 This application describes the data processing apparatus provided.

[0203] Figure 17 This is a schematic diagram of a data processing apparatus provided in an embodiment of this application. The data processing apparatus 1700 can be used to implement the functions of the processing device in the above method embodiments, and therefore can also achieve the beneficial effects of the above method embodiments.

[0204] like Figure 17 As shown, the data processing device 1700 includes an acquisition unit 1710, a data processing unit 1720, and a transceiver unit 1730. The data processing unit 1720 is coupled to the acquisition unit 1710 and the transceiver unit 1730. It can be understood that the acquisition unit 1710 and the transceiver unit 1730 can be transceivers or input / output interfaces.

[0205] The acquisition unit 1710 is used to acquire the first data to be processed. For example, the acquisition unit 1710 is used to perform... Figure 7 Step 710.

[0206] The data processing unit 1720 is used to input the first data into the data processing model and obtain the processing result of the first data. The data processing model includes a grouped convolutional layer. The amount of input data to be processed by the grouped convolutional layer each time is matched with the computing power of the matrix calculation unit. The input data to be processed each time includes at least one of the first data and intermediate data that has been processed by the data processing model and then input into the grouped convolutional layer. For example, the data processing unit 1720 is used to execute... Figure 7 Step 720.

[0207] In one possible example, the matrix computation unit comprises M rows × N columns of computation nodes, where M and N are both positive integers. The matrix computation unit supports the computation of one or more predefined first convolutional kernels. The amount of input data processed by each grouped convolutional layer is matched to the computational power of the matrix computation unit, including: the amount of input data processed each time is M+c, where c is the same as the width of the first convolutional kernel.

[0208] Optionally, the data processing unit 1720 is specifically configured to write at least one of the first data and intermediate data determined by the data processing model into a memory connected to the processing device. The data stored in the memory includes the input data to be processed each time. The data processing unit 1720 is specifically configured to process the input data to be processed each time retrieved from the memory based on the grouped convolutional layer, and obtain the processing result.

[0209] Optionally, the data processing unit 1720 is specifically used to confirm that the i-th computing node supports the computation of one or more preset first convolution kernels, wherein the i-th computing node is the i-th computing node in the first column of an M-row × N-column computing node. The data processing unit 1720 is specifically used to input c data points from the memory into the i-th computing node, where c is the same width as the first convolution kernel.

[0210] Optionally, the grouped convolutional layer includes one or more second convolutional kernels. The acquisition unit 1710 is specifically used to determine, for any one of the one or more second convolutional kernels, whether the processing device matches any one of the one or more preset first convolutional kernels.

[0211] In one possible example, if the second convolutional kernel does not match any of the first convolutional kernels, the acquisition unit 1710 is further specifically configured to: segment the second convolutional kernel to obtain multiple convolutional kernels to be processed. The convolutional kernels to be processed match at least one of one or more preset first convolutional kernels. The acquisition unit 1710 segments the second convolutional kernel based on one or more first segmentation windows to obtain multiple convolutional kernels to be processed. Each first segmentation window corresponds to one preset first convolutional kernel.

[0212] Optionally, the size of the first segmentation sliding window is the same as the size of a preset first convolution kernel. Further details regarding the size can be found in the foregoing method embodiments and will not be repeated here.

[0213] Optionally, the acquisition unit 1710 is further specifically used to: segment the source data according to multiple convolutional kernels to be processed to obtain n first data points, where n is the same as the number of multiple convolutional kernels to be processed. The acquisition unit 1710 segments the source data based on one or more second segmentation sliding windows to obtain n first data points.

[0214] Optionally, the size of the second sliding window is determined based on at least one of the source data, the size of a first convolution kernel, and the size of a second convolution kernel. Further details regarding the size can be found in the foregoing method embodiments and will not be repeated here.

[0215] In another possible example, if the second convolutional kernel matches any of the first convolutional kernels, the acquisition unit 1710 is further specifically used to: use the source data as the first data. The transceiver unit 1730 is used to output the processing result obtained by the data processing unit 1720 to other devices. For example, the transceiver unit 1730 is used to perform... Figure 7 Step 730.

[0216] Optionally, the data processing apparatus 1700 may further include a storage unit 1740 for storing instructions executed by the data processing unit 1720 or storing input data required for the execution instructions of the data processing unit 1720 or storing data generated after the execution instructions of the data processing unit 1720.

[0217] The data processing apparatus provided by this application has been described above; the data processing system provided by this application is described below. The data processing system includes a memory and a processing device. The memory is used to store first data from the method embodiments described above. For information on the type of memory, please refer to the above description. Figure 4 The relevant description will not be repeated here. This processing device is used to execute the above based on the first data. Figure 7 The data processing method described above. The processing device may be as described above. Figure 4 The processing equipment described herein, and related descriptions of the processing equipment can be found in [reference needed]. Figure 4 The relevant descriptions will not be repeated here.

[0218] This application also provides a chip system including a processor for implementing the functions of the processing device in the above-described method. In one possible example, the chip system further includes a memory for storing program instructions and / or data. This chip system may be composed of chips or may include chips and other discrete devices.

[0219] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are performed entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD).

[0220] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method is executed by a processing device, which includes a matrix computation unit and a data processing model deployed therein. The data processing model includes a grouped convolutional layer. The matrix computation unit comprises M rows × N columns of computation nodes, where M and N are both positive integers. Each computation node supports computation of one or more preset first convolutional kernels. The grouped convolutional layer includes one or more second convolutional kernels. The method includes: Get the first data to be processed; The acquisition of the first data to be processed includes: For any one of the one or more second convolution kernels, determine whether the second convolution kernel matches any one of the one or more preset first convolution kernels; If the second convolutional kernel does not match any of the first convolutional kernels, the second convolutional kernel is split to obtain multiple convolutional kernels to be processed; the convolutional kernels to be processed match at least one of the one or more preset first convolutional kernels. Based on the multiple convolutional kernels to be processed, n first data points are obtained by segmenting the source data, where n is the same as the number of the multiple convolutional kernels to be processed; The first data is input into the data processing model to obtain the processing result of the first data; wherein, the amount of input data to be processed by the grouped convolutional layer each time is matched with the computing power of the matrix calculation unit, and the input data to be processed each time includes at least one of the first data and intermediate data that is processed by the data processing model and then input into the grouped convolutional layer; Output the processing result.

2. The method according to claim 1, characterized in that, The matrix calculation unit includes: M rows × N columns of calculation nodes, where M and N are both positive integers; the matrix calculation unit supports the calculation of one or more preset first convolution kernels; The amount of input data to be processed by the grouped convolutional layer each time is matched with the computing power of the matrix calculation unit, including: the amount of input data to be processed each time is M+c, where c is the same as the width of the first convolutional kernel.

3. The method according to claim 1 or 2, characterized in that, The step of inputting the first data into the data processing model and obtaining the processing result of the first data includes: Write at least one of the first data and the intermediate data determined by the data processing model into the memory; the data stored in the memory includes the input data to be processed each time; The processing result is obtained by processing the input data to be processed each time, which is obtained from the memory, based on the grouped convolutional layer.

4. The method according to claim 3, characterized in that, The matrix calculation unit includes: M rows × N columns of calculation nodes, where M and N are both positive integers; The process of processing the input data to be processed each time based on the grouped convolutional layer obtained from the memory includes: Confirm that the i-th computing node supports the computation of one or more preset first convolutional kernels, wherein the i-th computing node is the i-th computing node in the first column of the M rows × N columns of computing nodes; The c data points in the memory are input into the i-th computation node, where c is the same as the width of the first convolution kernel.

5. The method according to claim 1, characterized in that, The process of obtaining the first data to be processed also includes: If the second convolution kernel matches any of the first convolution kernels, then the source data is used as the first data.

6. The method according to claim 1 or 5, characterized in that, The process of segmenting the second convolutional kernel to obtain multiple convolutional kernels to be processed includes: The second convolutional kernel is segmented based on one or more first segmentation sliding windows to obtain multiple convolutional kernels to be processed; In this context, each first segmentation sliding window corresponds to a preset first convolutional kernel.

7. The method according to claim 6, characterized in that, The first segmentation sliding window corresponds to a preset first convolutional kernel, including: the size of the first segmentation sliding window is the same as the size of the preset first convolutional kernel, and the size includes at least one of the height and width of the convolutional kernel.

8. The method according to claim 1 or 5, characterized in that, The step of obtaining n first data points based on the segmentation of the source data using the plurality of unprocessed convolutional kernels includes: The source data is segmented based on one or more second segmentation sliding windows to obtain n first data; The size of the second sliding window is determined based on at least one of the source data, the size of the first convolution kernel, and the size of the second convolution kernel.

9. A data processing apparatus, characterized in that, The device is applied to a processing equipment, which includes a matrix computation unit and a data processing model deployed therein. The data processing model includes a grouped convolutional layer. The matrix computation unit includes M rows × N columns of computation nodes, where M and N are both positive integers. Each computation node supports computation of one or more preset first convolutional kernels. The grouped convolutional layer includes one or more second convolutional kernels. The device includes: The acquisition unit is used to acquire the first data to be processed. The acquisition unit is specifically used to determine whether the second convolution kernel matches any one of the one or more preset first convolution kernels for any one of the one or more second convolution kernels; The acquisition unit is further specifically configured to, if the second convolutional kernel does not match any of the first convolutional kernels, divide the second convolutional kernel to obtain multiple convolutional kernels to be processed; the convolutional kernels to be processed match at least one of the one or more preset first convolutional kernels. The acquisition unit is further specifically used to obtain n first data points based on the segmentation of the source data according to the plurality of convolutional kernels to be processed, wherein n is consistent with the number of the plurality of convolutional kernels to be processed; A data processing unit is used to input the first data into a data processing model and obtain the processing result of the first data; wherein, the amount of input data to be processed by the grouped convolutional layer each time is matched with the computing power of the matrix calculation unit, and the input data to be processed each time includes at least one of the first data and intermediate data that is processed by the data processing model and then input into the grouped convolutional layer; The transceiver unit is used to output the processing result.

10. The apparatus according to claim 9, characterized in that, The matrix calculation unit includes: M rows × N columns of calculation nodes, where M and N are both positive integers; the matrix calculation unit supports the calculation of one or more preset first convolution kernels; The amount of input data to be processed by the grouped convolutional layer each time is matched with the computing power of the matrix calculation unit, including: the amount of input data to be processed each time is M+c, where c is the same as the width of the first convolutional kernel.

11. The apparatus according to claim 9 or 10, characterized in that, The data processing unit is specifically configured to write at least one of the first data and the intermediate data determined by the data processing model into a memory connected to the processing device; the data stored in the memory includes the input data to be processed each time; The data processing unit is specifically used to process the input data to be processed each time obtained from the memory based on the grouped convolutional layer, and to obtain the processing result.

12. The apparatus according to claim 11, characterized in that, The matrix calculation unit includes: M rows × N columns of calculation nodes, where M and N are both positive integers; The data processing unit is specifically used to confirm one or more preset first convolution kernels that the i-th computing node supports for computation, wherein the i-th computing node is the i-th computing node in the first column of the M rows × N columns of computing nodes; The data processing unit is specifically used to input c data from the memory into the i-th computing node, where c is the same as the width of the first convolution kernel.

13. The apparatus according to claim 9, characterized in that, The acquisition unit is further specifically used to: if the second convolution kernel matches any of the first convolution kernels, use the source data as the first data.

14. The apparatus according to claim 9 or 13, characterized in that, The acquisition unit is further specifically used to: segment the second convolutional kernel based on one or more first segmentation sliding windows to acquire multiple convolutional kernels to be processed; In this context, each first segmentation sliding window corresponds to a preset first convolutional kernel.

15. The apparatus according to claim 14, characterized in that, The first segmentation sliding window corresponds to a preset first convolutional kernel, including: the size of the first segmentation sliding window is the same as the size of the preset first convolutional kernel, and the size includes at least one of the height and width of the convolutional kernel.

16. The apparatus according to claim 9 or 13, characterized in that, The acquisition unit is further specifically used to: segment the source data based on one or more second segmentation sliding windows to obtain n first data; The size of the second sliding window is determined based on at least one of the source data, the size of the first convolution kernel, and the size of the second convolution kernel.

17. A processing apparatus, characterized in that, include: Processor and interface circuitry; The interface circuit is used to receive signals from other devices outside the processing device and transmit them to the processor, or to send signals from the processor to other devices outside the processing device. The processor is used to implement the method of any one of claims 1 to 8 through logic circuits or execution code instructions.

18. A data processing system, characterized in that, include: The memory and the processing device of claim 17; The memory is used to store the first data; The processing device is used to perform the method of any one of claims 1 to 8 based on the first data.

19. A computer-readable storage medium, characterized in that, The storage medium stores a computer program or instructions, which, when executed by a processing device, implement the method of any one of claims 1 to 8.

Images