Data processing method and device, chip, electronic equipment and storage medium

Abstract

The present disclosure provides a data processing method and device, a chip, an electronic equipment and a storage medium, wherein the method comprises: obtaining to-be-processed data; based on operation parameters of a first convolution operation on the to-be-processed data, extracting sub-data corresponding to multiple second convolution operations from the to-be-processed data; the first convolution operation step is greater than the corresponding step of the second convolution operation; using the operation number corresponding to the first convolution operation to process the sub-data corresponding to the multiple second convolution operations respectively by the second convolution operation to obtain result data corresponding to the multiple second convolution operations respectively; and performing data reorganization on the result data corresponding to the multiple second convolution operations respectively to obtain target result data of the first convolution operation on the to-be-processed data.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of deep learning, and in particular, to a data processing method and device, a chip, an electronic device, and a storage medium. BACKGROUND

[0002] In a neural network, convolution is a common operator, and dilated convolution is a special convolution in which a hyperparameter called dilation rate is introduced, which defines the spacing of values when the convolution kernel processes data, and is usually represented as "hole" or "dilation" in the operator parameters. Generally, a multiplier unit for performing efficient convolution operation is built into an artificial intelligence (AI) chip. However, due to the limitation of hardware resources, the values of "hole" and "pad" cannot be set to a large parameter in dilated convolution in the convolution operation, resulting in the problem that the size of the data to be processed is limited when the multiplier unit is used to perform dilated convolution.

[0003] To solve the above problem, in related technologies, the convolution kernel is usually decomposed into multiple 1x1 convolution operations, and then the convolution results of the 1x1 convolution operations are accumulated to obtain the final result of the dilated convolution. However, this method has the problem of low calculation accuracy. SUMMARY

[0004] The present disclosure provides at least a data processing method, device, chip, electronic device, and storage medium.

[0005] In a first aspect, the present disclosure provides a data processing method, comprising:

[0006] obtaining to-be-processed data;

[0007] extracting, from the to-be-processed data, sub-data corresponding to a plurality of second convolution operations respectively based on operation parameters of a first convolution operation on the to-be-processed data; the step length of the first convolution operation is greater than the step length corresponding to the second convolution operation;

[0008] performing, using operation parameters corresponding to the first convolution operation, the second convolution operation on the sub-data corresponding to the plurality of second convolution operations respectively to obtain result data corresponding to the plurality of second convolution operations respectively;

[0009] reorganizing the result data corresponding to the plurality of second convolution operations respectively to obtain target result data of the first convolution operation on the to-be-processed data.

[0010] In an optional implementation, the obtaining to-be-processed data comprises:

[0011] obtain original to-be-processed data;

[0012] perform data padding processing on the original to-be-processed data by using the padding parameter corresponding to the original to-be-processed data, to obtain to-be-processed data.

[0013] In an optional implementation, the first convolution operation includes a dilated convolution, and the second convolution operation includes a normal convolution; the operation parameter includes a dilated size of the dilated convolution; and the operation number corresponding to the first convolution operation includes a convolution kernel of the dilated convolution.

[0014] In an optional implementation, the extracting, based on the operation parameter of the first convolution operation on the to-be-processed data, of the sub data corresponding to each of the second convolution operations, includes:

[0015] determining an operation identifier of the second convolution operation based on the dilated size of the dilated convolution;

[0016] for each of the second convolution operations, extracting, based on the operation identifier of the second convolution operation and the size information of the to-be-processed data, the sub data corresponding to the second convolution operation from the to-be-processed data.

[0017] In an optional implementation, the dilated size of the dilated convolution includes a first size in a height direction of the to-be-processed data and a second size in a width direction of the to-be-processed data.

[0018] The determining, based on the dilated size of the dilated convolution, of the operation identifier of the second convolution operation includes constructing the operation identifier of the second convolution operation based on the first size and the second size.

[0019] In an optional implementation, the extracting, for each of the second convolution operations, of the sub data corresponding to the second convolution operation from the to-be-processed data based on the order of the second convolution operation in the plurality of second convolution operations and the size information of the to-be-processed data, includes:

[0020] in response to the operation identifier of the current second convolution operation being i*j, taking a first data element located at the ith row and the jth column in the to-be-processed data as a first data element at a head of the sub data corresponding to the current second convolution operation;

[0021] extracting, based on a position of the first data element at the head in the to-be-processed data, a step length of extraction in the height direction of the first size, and a step length of extraction in the width direction of the second size, other first data elements in the sub data except the first data element at the head;

[0022] construct sub data of the current second convolution operation based on the first first data element and the other first data elements extracted from the first data element; wherein i is a positive integer less than or equal to the first size; j is a positive integer less than or equal to the second size.

[0023] In an optional implementation, the data reorganization of the result data corresponding to each of the plurality of second convolution operations to obtain the target result data of the first convolution operation on the to-be-processed data comprises:

[0024] determining position information of each second data element in the result data of each second convolution operation in the target result data based on the operation identifier of the plurality of second convolution operations;

[0025] filling the second data element into the corresponding position of the target result data based on the position information of each second data element in the result data of each second convolution operation in the target result data to obtain the target result data.

[0026] In an optional implementation, the determination of the position information of each second data element in the result data of each second convolution operation in the target result data based on the operation identifier of the plurality of second convolution operations comprises:

[0027] in response to the operation identifier of the current second convolution operation being i*j, determining the position of the i th row and the j th column in the target result data as the position information of the first second data element in the result data corresponding to the current second convolution operation in the target result data;

[0028] based on the position information of the first second data element in the target result data, the filling step size in the height direction of the first size, and the filling step size in the width direction of the second size, determining the position information of the other second data elements in the result data corresponding to the current second convolution operation in the target result data except the first second data element.

[0029] In a second aspect, the embodiments of the present disclosure further provide a data processing apparatus, comprising: a processing unit and an operation unit;

[0030] the processing unit is configured to: obtain to-be-processed data; extract sub data corresponding to each of a plurality of second convolution operations from the to-be-processed data based on operation parameters of a first convolution operation on the to-be-processed data; the second convolution operation is obtained by converting the first convolution operation; and send the sub data to the operation unit;

[0031] The operation unit is configured to, in response to receiving the sub-data, perform second convolution operation processing on the sub-data corresponding to each of the second convolution operations respectively by using the operation number corresponding to the first convolution operation, to obtain result data corresponding to each of the second convolution operations, and to perform data reorganization on the result data corresponding to each of the second convolution operations to obtain target result data of the first convolution operation on the to-be-processed data.

[0032] In an optional implementation, the processing unit is configured to, when obtaining the to-be-processed data:

[0033] obtain original to-be-processed data;

[0034] perform data padding processing on the original to-be-processed data by using padding parameters corresponding to the original to-be-processed data to obtain the to-be-processed data.

[0035] In an optional implementation, the first convolution operation includes a dilated convolution, the second convolution operation includes a normal convolution, the operation parameter includes a dilated size of the dilated convolution, and the operation number corresponding to the first convolution operation includes a convolution kernel of the dilated convolution.

[0036] In an optional implementation, the processing unit is configured to, when extracting the sub-data corresponding to each of the second convolution operations from the to-be-processed data based on the operation parameter of the first convolution operation on the to-be-processed data:

[0037] determine the operation identifier of the second convolution operation based on the dilated size of the dilated convolution;

[0038] for each of the second convolution operations, extract the sub-data corresponding to the second convolution operation from the to-be-processed data based on the operation identifier of the second convolution operation and the size information of the to-be-processed data.

[0039] In an optional implementation, the dilated size of the dilated convolution includes a first size in a height direction of the to-be-processed data and a second size in a width direction of the to-be-processed data.

[0040] The processing unit is configured to, when determining the operation identifier of the second convolution operation based on the dilated size of the dilated convolution, construct the operation identifier of the second convolution operation based on the first size and the second size.

[0041] In an optional implementation, the processing unit is configured to, for each of the second convolution operations, extract the sub-data corresponding to the second convolution operation from the to-be-processed data based on the order of the second convolution operation in the second convolution operations and the size information of the to-be-processed data.

[0042] in response to the operation identifier of the current second convolution operation being i*j, taking a first data element located at the ith row and the jth column in the to-be-processed data as a first data element of the sub-data corresponding to the current second convolution operation;

[0043] based on a position of the first data element in the to-be-processed data, an extraction step length in a height direction of the first size, and an extraction step length in a width direction of the second size, extracting other first data elements of the sub-data from the to-be-processed data, except for the first data element;

[0044] based on the first data element and the other first data elements, constructing the sub-data of the current second convolution operation; wherein i is a positive integer less than or equal to the first size; and j is a positive integer less than or equal to the second size.

[0045] In an optional implementation, when the operation unit performs data reorganization on the result data corresponding to each of the multiple second convolution operations to obtain target result data of the first convolution operation on the to-be-processed data, the operation unit is configured to:

[0046] based on the operation identifiers of the multiple second convolution operations, determining position information of each second data element in the result data of each second convolution operation in the target result data;

[0047] based on the position information of each second data element in the result data of each second convolution operation in the target result data, filling the second data element into a corresponding position of the target result data to obtain the target result data.

[0048] In an optional implementation, when the operation unit determines the position information of each second data element in the result data of each second convolution operation in the target result data based on the operation identifiers of the multiple second convolution operations, the operation unit is configured to:

[0049] in response to the operation identifier of the current second convolution operation being i*j, determining a position of the target result data located at the ith row and the jth column as position information of a first second data element in the result data corresponding to the current second convolution operation in the target result data;

[0050] based on the position information of the first second data element in the target result data, and a filling step length in a height direction of the first size and a filling step length in a width direction of the second size, determining position information of other second data elements in the result data corresponding to the current second convolution operation in the target result data, except for the first second data element.

[0051] In a third aspect, the embodiments of the present disclosure further provide a data processing apparatus, comprising:

[0052] an acquisition module, configured to acquire to-be-processed data;

[0053] an extraction module, configured to extract, based on operation parameters of a first convolution operation on the to-be-processed data, sub-data corresponding to a plurality of second convolution operations from the to-be-processed data; the second convolution operation is obtained by converting the first convolution operation;

[0054] a processing module, configured to perform, by using operation parameters of the first convolution operation, the second convolution operation on the sub-data corresponding to the plurality of second convolution operations to obtain result data corresponding to the plurality of second convolution operations;

[0055] a reorganization module, configured to perform data reorganization on the result data corresponding to the plurality of second convolution operations to obtain target result data of the first convolution operation on the to-be-processed data.

[0056] In a fourth aspect, the embodiments of the present disclosure further provide a data processing chip, comprising the data processing apparatus as described in the second aspect.

[0057] In a fifth aspect, the embodiments of the present disclosure further provide an electronic device, comprising a processor and a memory, wherein the memory stores machine readable instructions executable by the processor, the processor is configured to execute the machine readable instructions stored in the memory, and the machine readable instructions, when executed by the processor, cause the processor to perform the steps of the first aspect or any possible implementation manner of the first aspect.

[0058] In a sixth aspect, the optional implementation manner of the present disclosure further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program, when executed, performs the steps of the first aspect or any possible implementation manner of the first aspect.

[0059] The effects of the data processing apparatus, the chip, the electronic device, and the computer readable storage medium are described above in the description of the data processing method, and will not be repeated here.

[0060] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, rather than limiting the technical solutions of the present disclosure.

[0061] The data processing method provided by the embodiment of the present disclosure reduces the number of rounding operations on the convolution result in the convolution operation process, thereby reducing the problem of low accuracy of the dilated convolution result caused by rounding the convolution result, and improving the processing accuracy of the dilated convolution result.

[0062] In order to make the above objectives, characteristics and advantages of the present disclosure more apparent, clear and easy to understand, the following will specifically describe a preferred embodiment in conjunction with the accompanying drawings, and the detailed description is as follows. BRIEF DESCRIPTION OF DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings needed to be used in the embodiments. The drawings herein are incorporated into the specification and form a part of the specification, which show the embodiments consistent with the present disclosure, and are used to explain the technical solutions of the present disclosure together with the specification. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0064] Figure 1 A flow chart of a data processing method provided by the embodiment of the present disclosure is shown;

[0065] Figure 2 A specific example of filling the processing of the to-be-processed data provided by the embodiment of the present disclosure is shown;

[0066] Figure 3 A specific example of data extraction of the to-be-processed data provided by the embodiment of the present disclosure is shown;

[0067] Figure 4 Another specific example of data extraction of the to-be-processed data provided by the embodiment of the present disclosure is shown;

[0068] Figure 5 A specific example of ordinary convolution processing of the extracted sub-data provided by the embodiment of the present disclosure is shown;

[0069] Figure 6 A specific example of data extraction of the target result corresponding to the second convolution operation provided by the embodiment of the present disclosure is shown;

[0070] Figure 7 A schematic diagram of a data processing device provided by the embodiment of the present disclosure is shown;

[0071] Figure 8 FIG. 1 shows a schematic diagram of another data processing apparatus provided by an embodiment of the present disclosure;

[0072] Figure 9 FIG. 1 shows a schematic diagram of another data processing apparatus provided by an embodiment of the present disclosure; DETAILED DESCRIPTION

[0073] In order to make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure and are not all the embodiments. The components of the embodiments of the present disclosure described and shown herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the claimed present disclosure, but only represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present disclosure.

[0074] It is found through research that a multiplier unit for performing high-efficiency convolution operation is built in an AI chip. When the multiplier unit is used to implement a dilated convolution, the convolution kernel is usually disassembled into multiple 1x1 convolution operations, and then the convolution results of the obtained 1x1 convolution operations are accumulated to obtain the final result of the dilated convolution. However, in the process of accumulating the 1x1 convolution results multiple times, due to the limited number of bits of the data to be processed and the convolution kernel, rounding off the 1x1 convolution results will cause the final result of the dilated convolution to have a low precision problem.

[0075] Based on the above research, the present disclosure provides a data processing method, which converts the dilated convolution operation on the data to be processed into a normal convolution operation on sub-data composed of the data to be processed before performing the dilated convolution, thereby reducing the number of times of rounding off the convolution results in the convolution operation process, thereby reducing the problem of low precision of the dilated convolution result caused by rounding off the convolution results, and improving the processing precision of the dilated convolution result.

[0076] The defects of the above solutions are the result of the inventors' practice and careful research, and therefore the discovery process of the above problems and the solutions proposed by the present disclosure to solve the above problems in the following should be the contribution of the inventors to the present disclosure.

[0077] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0078] For the convenience of understanding the present embodiment, first, a data processing method disclosed by the present embodiment is introduced in detail, the execution subject of the data processing method provided by the present embodiment is generally an electronic device with certain computing capability, which for example includes a terminal device or a server or other processing device, and the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the data processing method can be realized by a processor calling computer readable instructions stored in a memory.

[0079] The data processing method provided by the present embodiment is described below.

[0080] Referring to FIG. 1, Figure 1 The flowchart of the data processing method provided by the present embodiment is shown, and the method includes steps S101-S104, wherein:

[0081] S101: obtaining to-be-processed data;

[0082] S102: based on operation parameters of a first convolution operation on the to-be-processed data, extracting sub-data corresponding to multiple second convolution operations from the to-be-processed data respectively; the step length of the first convolution operation is greater than the step length corresponding to the second convolution operation;

[0083] S103: using operation numbers corresponding to the first convolution operation to perform second convolution operation processing on the sub-data corresponding to the multiple second convolution operations respectively, to obtain result data corresponding to the multiple second convolution operations respectively;

[0084] S104: performing data reorganization on the result data corresponding to the multiple second convolution operations respectively, to obtain target result data of the first convolution operation on the to-be-processed data.

[0085] In the embodiments of the present disclosure, after obtaining the to-be-processed data, sub-data corresponding to a plurality of second convolution operations are extracted from the to-be-processed data based on operation parameters of the first convolution operation on the to-be-processed data, and the sub-data corresponding to the plurality of second convolution operations are processed by the second convolution operation by using operation numbers corresponding to the first convolution operation, to obtain result data corresponding to the plurality of second convolution operations, and then the result data corresponding to the plurality of second convolution operations are reorganized to obtain target result data of the first convolution operation on the to-be-processed data. In the case where the first convolution operation is a dilated convolution and the second convolution operation is a normal convolution, the dilated convolution operation on the to-be-processed data can be converted into a normal convolution operation on sub-data based on the to-be-processed data, so as to reduce the number of rounding operations on the convolution result in the convolution operation process, thereby reducing the problem of low precision of the dilated convolution result caused by rounding the convolution result, and improving the processing precision of the dilated convolution result.

[0086] The above S101-S104 will be described in detail below.

[0087] For the above S101, in the embodiments of the present disclosure, the data processing method can be applied in different scenarios; for example, applied in the scenario of training a neural network, or applied in the scenario of executing an inference task by using a neural network. In a network layer in the neural network, a network layer corresponding to the first convolution operation can be included. The network layer is used to perform the first convolution operation on the to-be-processed data.

[0088] The to-be-processed data is based on the different functions of the neural network. For example, if the neural network is used for image classification, target recognition and the like, the to-be-processed data can include image data; if the neural network is used for speech data recognition or conversion into text, the to-be-processed data is audio data; if the neural network is used for processing text data, the to-be-processed data is data obtained by encoding the text data. In addition, the to-be-processed data can be original data input into the neural network, such as original image data or original audio data; or can be feature data obtained by performing certain feature extraction processing on the image data or the audio data. For example, a feature map corresponding to the image data, feature data corresponding to the audio data, and the like.

[0089] In the embodiments of the present disclosure, the to-be-processed data is taken as an example of a feature map. In the feature map, a plurality of data channels are included, each data channel includes a plurality of data elements, and the plurality of data elements of the data channel form a data matrix, that is, a feature sub-map corresponding to the data channel.

[0090] The to-be-processed data can be an original feature map (also referred to as original to-be-processed data) input into the neural network, or a feature map formed by data padding on the original feature map.

[0091] In the examples of the present disclosure, a specific example of obtaining to-be-processed data is provided, including:

[0092] Obtaining original to-be-processed data;

[0093] Padding the original to-be-processed data by using a padding parameter corresponding to the original to-be-processed data to obtain to-be-processed data.

[0094] For example, the padding parameter for padding the original to-be-processed data includes a padding direction and / or a padding size.

[0095] For example, the padding direction includes at least one of the following positions: top, down, left, and right of the original to-be-processed data.

[0096] For example, the padding size includes a width value of width padding.

[0097] For example, if the padding direction is top or down, the width value can also be referred to as the number of rows of padding; if the padding direction is left or right, the width value can also be referred to as the number of columns of padding.

[0098] For example, referring to Figure 2 In the specific example of padding the original to-be-processed data shown in the figure, the original to-be-processed data includes one data channel. The gray shaded part represents the original to-be-processed data bottomtensor. The top, bottom, left, and right of the original to-be-processed data are respectively padded: top_pad (top padding), down_pad (bottom padding), left_pad (left padding), and right_pad (right padding). The obtained to-be-processed data is represented as: paddedbottomtensor.

[0099] For example, the padded value is a preset value, such as 0.

[0100] For example, assuming that the size of the original to-be-processed data is n*m dimensions, which represents the original to-be-processed data including n rows and m columns of data elements, as follows:

[0101]

[0102] For example, the padded value is a preset value, such as 0.

[0103]

[0104] As shown in Figure 2 , a specific example of data padding on original to-be-processed data is provided, where the original to-be-processed data is a bottom tensor with a size of {b, c, h, w}, where b represents a batch, i.e., a data processing batch in the to-be-processed data, c represents a data channel number, e.g., a number of sub-feature maps included in a feature map, h represents a height of a data channel, i.e., a number of rows of data elements in a data matrix corresponding to the data channel, and w represents a width of a data channel, i.e., a number of columns of data elements in a data matrix corresponding to the data channel.

[0105] The original to-be-processed data is padded, and in the padding parameters, the padding directions include a top, a bottom, a left side, and a right side of the original to-be-processed data; where the padding sizes of the top and the bottom are pad_h respectively; and the padding sizes of the left side and the right side are pad_w respectively.

[0106] Then, the original to-be-processed data bottom tensor is processed using the above padding parameters to obtain to-be-processed data, which is represented as padded bottom tensor with a size of {b, c, h’, w’}, and h’ = h + 2 x pad_h and w’ = w + 2 x pad_w; where the extra data element positions of the to-be-processed data padded bottom tensor compared with the original to-be-processed data bottom tensor are padded with data 0.

[0107] After obtaining the to-be-processed data based on the above method, the to-be-processed data can be processed using the above S102-S104.

[0108] For S102, in the embodiments of the present disclosure, the first convolution operation, for example, includes a dilated convolution; the second convolution operation, for example, includes a normal convolution; in this case, the operation parameters of the first convolution operation, for example, include a convolution kernel corresponding to the dilated convolution.

[0109] Here, the embodiment of the present disclosure is to convert the atrous convolution into the normal convolution by processing the to-be-processed data without adjusting the convolution kernel of the atrous convolution. In addition, the data processing method provided by the embodiment of the present disclosure can also be applied to other types of first convolution operation and second convolution operation; the first convolution operation and the second convolution operation satisfy the following condition: the first convolution operation is converted into the second convolution operation based on adjusting the to-be-processed data corresponding to the first convolution operation. The type of the first convolution operation and the second convolution operation is not limited by the present disclosure. For example, the first convolution operation can also be a depth separable convolution, and the corresponding second convolution operation is a normal convolution; the to-be-processed data corresponding to the depth separable convolution can be split into sub-data corresponding to multiple normal convolutions, and the sub-data is processed by using the convolution kernel corresponding to the depth separable convolution.

[0110] The embodiment of the present disclosure takes the first convolution operation including the atrous convolution operation and the second convolution operation including the normal convolution operation as an example, and the operation parameters corresponding to the first convolution operation include the atrous size of the atrous convolution. According to the atrous size, the sub-data corresponding to the multiple normal convolutions is extracted from the to-be-processed data.

[0111] The atrous size of the atrous convolution is the atrous step in the height direction and the width direction. That is, the position difference between the positions of the data elements corresponding to the two adjacent weight elements in the convolution kernel of the atrous convolution in the to-be-processed data. The atrous size includes: a first size in the height direction of the to-be-processed data, and a second size in the width direction of the to-be-processed data. At least one of the first size and the second size is an integer greater than 1. If the first size and the second size are both 1, the atrous convolution is equivalent to the normal convolution.

[0112] Suppose the convolution kernel of a certain atrous convolution is represented as:

[0113] The to-be-processed data is:

[0114]

[0115] The above to-be-processed data is processed by using the above convolution kernel for atrous convolution, and the corresponding atrous size is that the first size in the height direction of the to-be-processed data is 2, and the second size in the width direction of the to-be-processed data is 2. When the to-be-processed data is processed by using the convolution kernel of the atrous convolution, the distance between the data elements corresponding to the two adjacent weight elements in the convolution kernel is 2 in the height direction and 2 in the width direction. If the convolution element w 11 The corresponding data element in the to-be-processed data is a 11 , and the corresponding relationship between w 12 ~ w 33 and the data element is:

[0116] w 12 corresponding a 13 ; w 13 corresponding a 15 ; w 21 corresponding a 31 ; w 22 corresponding a 33 ; w 23 corresponding a 35 ; w 31 corresponding a 51 ; w 32 corresponding a 53 ; w 33 corresponding a 55 .

[0117] The sub data corresponding to the second convolution operation is represented as sub bottom tensor, and the number of the second convolution operation is equal to the product of the first size and the second size, for example.

[0118] Exemplarily, the hole size of the atrium convolution includes a first size (hole h) in the height direction of the padded bottom tensor and a second size (hole w) in the width direction of the padded bottom tensor to be processed.

[0119] When the padded bottom tensor is subjected to the above-mentioned dilated convolution, the position difference between data elements corresponding to two adjacent convolution elements in the width direction of the convolution kernel is hole h, and the position difference between data elements corresponding to two adjacent convolution elements in the height direction of the convolution kernel is hole w. After the dilated convolution is converted into the ordinary convolution, the position difference between data elements corresponding to two adjacent convolution elements in the height direction of the convolution kernel of the ordinary convolution is 1, and the position difference between data elements corresponding to two adjacent convolution elements in the width direction of the convolution kernel of the ordinary convolution is also 1. Therefore, when the dilated convolution is converted into the ordinary convolution, data elements satisfying the ordinary convolution need to be extracted from the padded bottom tensor according to the dilated size, and reorganized into sub-data corresponding to one ordinary convolution operation. When extracted each time, the position interval of two adjacent data elements in the width direction of the sub-data in the padded bottom tensor is hole w, and the position interval of two adjacent data elements in the height direction of the sub-data in the padded bottom tensor is hole h. Then, hole h x hole w times of extraction can be performed to realize that all data elements in the padded bottom tensor are extracted into corresponding sub-data, and each data element in the padded bottom tensor only appears in one group of sub-data. Then, hole h x hole w times of extraction is performed to obtain hole h x hole w groups of sub-data, that is, the number of times of the corresponding second convolution operation is hole h x hole w.

[0120] For example, the embodiment of the disclosure also provides a specific way of extracting sub-data corresponding to a second convolution operation from the to-be-processed data a plurality of times based on operation parameters of a first convolution operation on the to-be-processed data, comprising:

[0121] Based on the dilated size of the dilated convolution, determining the operation identifier of the second convolution operation;

[0122] For each second convolution operation, based on the operation identifier of the second convolution operation and the size information of the to-be-processed data, extracting sub-data corresponding to the second convolution operation from the to-be-processed data.

[0123] In a specific implementation, when the operation identifier of the second convolution operation is determined based on the dilated size of the dilated convolution, for example, the operation identifier of the second convolution operation can be constructed based on the first size and the second size.

[0124] Suppose 1<=i<=hole_h and 1<=j<=hole_w; the operation identifier can be represented as i*j, where the operation identifier represents that the first data element in the sub-data corresponding to the operation identifier is a data element located at the i-th row in the h direction and the j-th column in the w direction of the data in the padded bottom tensor to be processed, and the data element is represented as a ij .

[0125] It can be seen that when the i-th=1 and j-th=1 extraction operation is performed, the operation identifier can be represented as 1*1, that is, the first data element in the sub-data corresponding to the operation identifier is a data element located at the 1-st row and the 1-st column in the padded bottom tensor to be processed, and the data element is represented as a 11 .

[0126] For example, when the i-th=2 and j-th=1 extraction operation is performed, the operation identifier can be represented as 2*1, that is, the first data element in the sub-data corresponding to the operation identifier is a data element located at the 2-nd row and the 1-st column in the padded bottom tensor to be processed.

[0127] …

[0128] Until the i-th=hole_h and j-th=hole_w extraction operation is performed, the operation identifier can be represented as hole_h*hole_w, and the first data element corresponding to the operation identifier is a data element located at the hole_h-th row and the hole_w-th column in the padded bottom tensor to be processed, and the data element is represented as a (hole_h)(hole_w) .

[0129] In a specific implementation, the height of the padded bottom tensor to be processed is h', and the width is w', where the data element at the top left corner is the first data element in the padded bottom tensor to be processed, and the data element is taken as the starting point of the extraction. The first data element is also the first data element in the sub-data corresponding to the second convolution operation with the operation identifier 1*1.

[0130] For the second convolution operation with operation identifier i*j, when the sub data sub bottom tensor_(i*j) corresponding to the second convolution operation with operation identifier i*j is extracted from the to-be-processed data padded bottom tensor, there are multiple data elements in the sub data sub bottom tensor_(i*j). The sub bottom tensor_(i*j) represents the sub data corresponding to the second convolution operation with operation identifier i*j.

[0131] For each second convolution operation, the sub data corresponding to the second convolution operation can be extracted from the to-be-processed data based on the order of the second convolution operation in the plurality of second convolution operations and the size information of the to-be-processed data in the following manner:

[0132] In response to the operation identifier of the current second convolution operation being i*j, the first data element located at the i-th row and the j-th column in the to-be-processed data is taken as the first data element of the sub data corresponding to the current second convolution operation;

[0133] Based on the position of the first data element in the to-be-processed data, the extraction step length in the height direction of the first size, and the extraction step length in the width direction of the second size, the other first data elements in the sub data except the first data element are extracted from the to-be-processed data;

[0134] Based on the extracted first data element and the other first data elements, the sub data of the current second convolution operation is constructed.

[0135] In a specific implementation, for the second convolution operation with operation identifier i*j, the first data element located at the i-th row and the j-th column in the to-be-processed data is taken as the first data element of the corresponding sub data, that is, the data element located at the upper left corner of the corresponding sub data. Then, according to the order from top to bottom in the height direction and from left to right in the width direction, the first data element in the corresponding sub data is taken as the reference point, and the other data elements in the sub data except the first data element are sequentially extracted from the to-be-processed data. When extracting, the extraction step length in the height direction is the first size in the hole size, and the extraction step length in the width direction is the second size in the hole size. Then, according to the relative position between the extracted first data elements and the first data element in the sub data, the first data elements and the extracted other data elements are organized together to form the corresponding sub data.

[0136] For example, for the second convolution operation with the operation identifier i*j, the data element at position (i, j) in the data to be processed is taken as the starting point (where i represents the row number of the data element and j represents the column number of the data element) as the first data element. This first data element is extracted from the data to be processed and stored in the top left corner of the subbottom tensor. The stored first data element is then used as the reference first data element. The data element in the padded bottom tensor of the data to be processed with a distance of n*hole_h from the reference first data element in the h direction is taken as the other first data element adjacent to the current reference first data element in the subbottom tensor in the h direction. Here, 1≤n≤Floor((hi) / hole_h)-1, and n is an integer; where Floor represents rounding down (hi) / hole_h. h represents the row number of the data to be processed.

[0137] In the padded bottom tensor of the data to be processed, the data element with a distance of m*hole_w between it and the current reference first data element in the w direction will be regarded as another first data element in the sub-bottom tensor that is adjacent to the current reference first data element in the w direction, where 1≤m≤Floor((wj) / hole_w)-1, and m is an integer, where Floor represents the floor function of (wj) / hole_w, and w represents the number of columns of the data to be processed.

[0138] The positions of the other first data elements besides the first first data element in the data to be processed can be represented as: (i+n*hole_h, j+m*hole_w).

[0139] Then, according to the positional relationship between each other first data element and the reference first data element, each other first data element is placed into the sub-data, except for the top left corner, to form the sub-data.

[0140] Based on the above method, the positions of all extracted first data elements in the padded bottom tensor and the positions of the extracted first data elements stored in the sub-bottom tensors after extraction can be determined. The size of the sub-bottom tensor is {b,c,h”,w”}, where h” is the height value of the first data element in the corresponding sub-bottom tensor, i.e., the number of rows, and w” is the width value of the first data element in the corresponding sub-bottom tensor, i.e., the number of columns.

[0141] As shown in the example of Figure 3 , an example of the specific process of extracting the sub data sub bottom tensor from the padded bottom tensor is shown, where the first size is 4 and the second size is 8. As can be seen, when the operation identifier is 1*1, the first first data in the to-be-processed data padded bottom tensor, which is the data element located at the first row and the first column in the to-be-processed data, will be extracted and stored in the first position in the sub data sub bottom tensor as the first first data element. Then, each first data element extracted in the above manner is located at the position in the to-be-processed data as shown in a of Figure 3 , and the sub data formed as shown in b of Figure 3 .

[0142] As shown in the example of Figure 4 , an example of the specific process of extracting the sub data sub bottom tensor from the to-be-processed data padded bottom tensor is shown, where the first size is 4 and the second size is 8. As can be seen, when the operation identifier is 4*8, the data element located at the fourth row and the eighth column in the to-be-processed data padded bottom tensor will be extracted as the first first data in the sub data sub bottom tensor, which will be extracted and stored in the first position in the sub data sub bottom tensor. Then, each first data element extracted in the above manner is located at the position in the to-be-processed data as shown in a of Figure 4 , and the sub data formed as shown in b of Figure 4 .

[0143] For the above S103, in the case where the first convolution operation is a hole convolution operation and the second convolution operation is a normal convolution operation, the operation number corresponding to the first convolution operation is a hole convolution kernel;

[0144] The second convolution operation processing on the sub data corresponding to the plurality of second convolution operations respectively includes: performing convolution calculation on the sub data sub bottom tensor corresponding to the plurality of second convolution operations respectively obtained in S102 to obtain result data sub top tensor corresponding to the plurality of convolution calculations respectively.

[0145] In a specific implementation, as shown in Figure 5As shown, a convolution calculation is performed on the sub data sub bottom tensor with a size of {b, c, h", w"} to obtain a result data sub top tensor with a size of {b, Co, H', W'}; wherein h" represents a height value of a first data element in the sub data sub bottom tensor, that is, a row number, w" represents a width value of the first data element in the sub data sub bottom tensor, that is, a column number; Co represents a channel number of the result data sub top tensor corresponding to the second convolution operation after the sub data sub bottom tensor is subjected to the convolution; H' is a height value of the result data sub top tensor corresponding to the second convolution operation after the sub data sub bottom tensor is subjected to the convolution; W' is a width value of the result data sub top tensor corresponding to the second convolution operation after the sub data sub bottom tensor is subjected to the convolution; a convolution kernel size used for the convolution calculation is {Co, c, kernel_h, kernel_w}, wherein Co represents a number of convolution kernels, which represents a channel number of the result data; c represents a channel number of each convolution kernel, which is consistent with a channel number of the sub data; and kernel_h and kernel_w are a length and a width direction step of the convolution kernel.

[0146] For the above S104: after obtaining the result data corresponding to the multiple second convolution operations respectively through the above S103, the result data corresponding to the multiple second convolution operations respectively can be subjected to data reorganization to obtain target result data of the first convolution operation on the to-be-processed data.

[0147] Here, the embodiments of the present disclosure are to place each second data element in the result data corresponding to the multiple ordinary convolutions respectively into a corresponding position in the target result data by reorganizing the result data corresponding to the multiple ordinary convolutions respectively, so as to obtain the same result data as that of directly performing the atrous convolution on the original data bottom tensor.

[0148] When the data reorganization is performed, for example, data elements in the result data sub top tensor corresponding to the multiple second convolution operations respectively are stored in the target result data top tensor in a scattered manner.

[0149] In specific implementation, the disclosure embodiment provides a specific manner of data reorganization on the result data corresponding to each of the plurality of second convolution operations, to obtain target result data of the first convolution operation on the to-be-processed data, including: determining the position information of each second data element in the result data of each second convolution operation in the target result data based on the operation identifier of the plurality of second convolution operations;

[0150] Based on the position information of each second data element in the result data of each second convolution operation in the target result data, the second data element is filled into the corresponding position of the target result data to obtain the target result data.

[0151] For example, the position information of each second data element in the result data of each second convolution operation in the target result data can be determined based on the operation identifier of the plurality of second convolution operations in the following manner:

[0152] For the second convolution operation with operation identifier i*j, the position of the target result data located in the i-th row and the j-th column is determined as the position information of the first second data element in the result data corresponding to the current second convolution operation in the target result data.

[0153] Based on the position information of the first second data element in the target result data, and the filling step length in the height direction of the first size and the filling step length in the width direction of the second size, the position information of other second data elements in the result data corresponding to the current second convolution operation in the target result data is determined.

[0154] In specific implementation, for the second convolution operation with operation identifier i*j, the first second data element in the sub-data corresponding to the second convolution operation is taken as a reference second data element, and the data elements adjacent to it in the h direction are stored in the target result data top tensor. Compared with the position of the current reference second data element in the top tensor, the number of columns in the w direction is the same, the row spacing in the h direction is hole_h and below the position of the current reference second data in the top tensor; and the data elements adjacent to it in the w direction are stored in the top tensor. Compared with the position of the current reference second data element in the top tensor, the number of rows in the h direction is the same, the column spacing in the w direction is hole_w and to the right of the position of the current reference second data in the top tensor.

[0155] Based on the above method, each time the second data element is stored, it is taken as the current reference second data, and the data elements adjacent to it in the h direction and the w direction are stored in the target result data top tensor according to the above method, until all the second data elements in the corresponding result data sub top tensor composed of the data elements with the operation identifier i*j are stored in the target result data top tensor.

[0156] For example, the operation identifier i*j of the multiple second convolution operations is obtained, and according to the data identifier i*j, the position of the i-th row and the j-th column in the target result data top tensor is taken as the position of the first second data element in the corresponding result data sub top tensor of the second convolution operation.

[0157] Then, taking the position as the reference position, the positions of the other second data elements in the target result data except the first second data element in the second convolution operation are determined.

[0158] Among the sub data corresponding to the second convolution operation, the position of the second data element at position (r, t) in the target result data satisfies (r*hole_h, t*hole_w), where 1

[0159] Then, according to the position, the other second data elements are scattered and stored in the corresponding positions of the target result data.

[0160] After the sub data corresponding to all the second convolution operations are scattered and stored in the corresponding positions of the target result data, the target result data obtained by performing the atrous convolution on the to-be-processed data using the convolution kernel is obtained.

[0161] As shown in the example of FIG. 8, the process of scattering and storing the second data elements in the result data sub top tensor of the second convolution operation into the target result data top tensor is shown when the operation identifiers of the second convolution operations are 1*1 and hole_h*hole_w respectively. Figure 6

[0162] Those skilled in the art can understand that in the above method of the specific embodiment, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process, and the specific execution order of each step should be determined by its function and possible internal logic.

[0163] ​Based on the same inventive concept, the embodiments of the present disclosure also provide a data processing device corresponding to the data processing method. Since the principle of the device solving the problem in the embodiments of the present disclosure is similar to the data processing method described above, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described here.

[0164] Referring to Figure 7 Fig. 1 is a schematic diagram of a data processing device provided by the embodiments of the present disclosure, which comprises a processing unit 71 and an operation unit 72.

[0165] The processing unit 71 is configured to obtain to-be-processed data, extract sub-data corresponding to a plurality of second convolution operations respectively from the to-be-processed data based on operation parameters of a first convolution operation on the to-be-processed data, the second convolution operation being obtained by converting the first convolution operation, and send the sub-data to the operation unit.

[0166] The operation unit 72 is configured to, in response to receiving the sub-data, perform second convolution operation processing on the sub-data corresponding to the plurality of second convolution operations respectively by using operation numbers corresponding to the first convolution operation, obtain result data corresponding to the plurality of second convolution operations respectively, and perform data reorganization on the result data corresponding to the plurality of second convolution operations respectively to obtain target result data of the first convolution operation on the to-be-processed data.

[0167] In the embodiments of the present disclosure, the data processing device is, for example, an AI chip or other device for data processing. Taking the data processing device as an AI chip for example, the operation unit comprises, for example, a processing engine (PE) array in the AI chip, and the PE array comprises, for example, a multiply-accumulator array.

[0168] In an optional implementation, when obtaining to-be-processed data, the processing unit 71 is configured to:

[0169] obtain original to-be-processed data.

[0170] perform data padding processing on the original to-be-processed data by using padding parameters corresponding to the original to-be-processed data to obtain the to-be-processed data.

[0171] In an optional implementation, the first convolution operation comprises a dilated convolution, the second convolution operation comprises a normal convolution, the operation parameters comprise a dilated size of the dilated convolution, and the operation numbers corresponding to the first convolution operation comprise a convolution kernel of the dilated convolution.

[0172] In an alternative implementation, the processing unit 71, when extracting, from the to-be-processed data, sub-data corresponding to each of the second convolution operations based on operation parameters of a first convolution operation on the to-be-processed data, is configured to:

[0173] determine operation identifiers of the second convolution operations based on a dilation size of the dilated convolution;

[0174] extract, for each of the second convolution operations, sub-data corresponding to the second convolution operation from the to-be-processed data based on the operation identifier of the second convolution operation and size information of the to-be-processed data.

[0175] In an alternative implementation, the dilation size of the dilated convolution includes a first size in a height direction of the to-be-processed data and a second size in a width direction of the to-be-processed data.

[0176] The processing unit 71, when determining the operation identifiers of the second convolution operations based on the dilation size of the dilated convolution, is configured to construct the operation identifiers of the second convolution operations based on the first size and the second size.

[0177] In an alternative implementation, the processing unit 71, when extracting, for each of the second convolution operations, sub-data corresponding to the second convolution operation from the to-be-processed data based on the order of the second convolution operation in the plurality of second convolution operations and the size information of the to-be-processed data, is configured to:

[0178] in response to the operation identifier of the current second convolution operation being i*j, take a first data element located at the ith row and the jth column in the to-be-processed data as a first data element at a head of the sub-data corresponding to the current second convolution operation;

[0179] extract, based on a position of the first data element at the head in the to-be-processed data, an extraction step size in the height direction of the first size, and an extraction step size in the width direction of the second size, other first data elements in the sub-data except the first data element at the head;

[0180] construct the sub-data of the current second convolution operation based on the first data element at the head and the other first data elements; wherein i is a positive integer less than or equal to the first size, and j is a positive integer less than or equal to the second size.

[0181] In an alternative implementation, the operation unit 72, when performing data reorganization on result data corresponding to the plurality of second convolution operations respectively to obtain target result data of the first convolution operation on the to-be-processed data, is configured to:

[0182] based on the operation identifier of the plurality of second convolution operations, determine position information of each second data element in the result data of each second convolution operation in the target result data respectively;

[0183] based on the position information of each second data element in the result data of each second convolution operation in the target result data respectively, fill the second data element into the corresponding position of the target result data to obtain the target result data.

[0184] In an optional implementation, when the operation unit 72 determines the position information of each second data element in the result data of each second convolution operation in the target result data respectively based on the operation identifier of the plurality of second convolution operations, the operation unit 72 is configured to:

[0185] in response to the operation identifier of the current second convolution operation being i*j, determine the position of the i-th row and the j-th column in the target result data as the position information of the first second data element in the result data corresponding to the current second convolution operation in the target result data;

[0186] based on the position information of the first second data element in the target result data, and the filling step length in the height direction of the first size and the filling step length in the width direction of the second size, determine the position information of other second data elements in the result data corresponding to the current second convolution operation in the target result data respectively.

[0187] Referring to Figure 8 The embodiments of the present disclosure also provide a data processing apparatus, which comprises:

[0188] The obtaining module 81 is configured to obtain to-be-processed data.

[0189] The extracting module 82 is configured to extract, from the to-be-processed data, sub-data corresponding to a plurality of second convolution operations respectively based on operation parameters of a first convolution operation performed on the to-be-processed data; the second convolution operation is obtained by converting the first convolution operation;

[0190] The processing module 83 is configured to perform a second convolution operation on the sub-data corresponding to the plurality of second convolution operations respectively by using operation parameters corresponding to the first convolution operation, to obtain result data corresponding to the plurality of second convolution operations respectively.

[0191] The reorganizing module 84 is configured to reorganize the result data corresponding to the plurality of second convolution operations respectively to obtain target result data of the first convolution operation performed on the to-be-processed data.

[0192] In a possible implementation, the obtaining module, when obtaining the to-be-processed data, is configured to:

[0193] obtain original to-be-processed data;

[0194] perform data padding processing on the original to-be-processed data by using the padding parameter corresponding to the original to-be-processed data, to obtain the to-be-processed data.

[0195] In a possible implementation, the first convolution operation includes a dilated convolution, and the second convolution operation includes a normal convolution; the operation parameter includes a dilation size of the dilated convolution; and the operation number corresponding to the first convolution operation includes a convolution kernel of the dilated convolution.

[0196] In a possible implementation, the extracting module 82, when extracting, based on the operation parameter of the first convolution operation on the to-be-processed data, the sub data corresponding to each of the second convolution operations from the to-be-processed data, is configured to:

[0197] determine the operation identifier of the second convolution operation based on the dilation size of the dilated convolution;

[0198] for each of the second convolution operations, extract, based on the operation identifier of the second convolution operation and the size information of the to-be-processed data, the sub data corresponding to the second convolution operation from the to-be-processed data.

[0199] In a possible implementation, the dilation size of the dilated convolution includes a first size in a height direction of the to-be-processed data and a second size in a width direction of the to-be-processed data.

[0200] When determining the operation identifier of the second convolution operation based on the dilation size of the dilated convolution, the extracting module 82 is configured to: construct the operation identifier of the second convolution operation based on the first size and the second size.

[0201] In a possible implementation, the extracting module 82, when, for each of the second convolution operations, extracting, based on the order of the second convolution operation in the plurality of second convolution operations and the size information of the to-be-processed data, the sub data corresponding to the second convolution operation from the to-be-processed data, is configured to:

[0202] in response to the operation identifier of the current second convolution operation being i*j, taking a first data element located at the i th row and the j th column in the to-be-processed data as a first data element of the sub data corresponding to the current second convolution operation;

[0203] extract, based on a position of the first data element in the to-be-processed data, an extraction step length in a height direction of the first size, and an extraction step length in a width direction of the second size, the other first data elements in the sub-data from the to-be-processed data, except for the first data element;

[0204] construct the sub-data of the current second convolution operation based on the extracted first data element and the other first data elements, where i is a positive integer less than or equal to the first size, and j is a positive integer less than or equal to the second size.

[0205] In a possible implementation, the reorganization module 84 is configured to, when reorganizing the result data corresponding to each of the multiple second convolution operations to obtain the target result data of the first convolution operation on the to-be-processed data, perform the following operations:

[0206] determine, based on the operation identifiers of the multiple second convolution operations, position information of each second data element in the result data of each second convolution operation in the target result data;

[0207] fill the second data elements into corresponding positions of the target result data based on the position information of each second data element in the result data of each second convolution operation in the target result data, to obtain the target result data.

[0208] In a possible implementation, the reorganization module 84 is configured to, when determining, based on the operation identifiers of the multiple second convolution operations, the position information of each second data element in the result data of each second convolution operation in the target result data, perform the following operations:

[0209] in response to the operation identifier of the current second convolution operation being i*j, determine, as the position information of the first second data element in the result data corresponding to the current second convolution operation in the target result data, a position of an i th row and a j th column in the target result data;

[0210] determine, based on the position information of the first second data element in the target result data, a filling step length in a height direction of the first size, and a filling step length in a width direction of the second size, the position information of the other second data elements in the result data corresponding to the current second convolution operation in the target result data, except for the first second data element.

[0211] The description of the processing procedure of each module in the apparatus and the interaction procedure between the modules can refer to the related description in the method embodiments, which will not be repeated here.

[0212] The chip provided in the embodiments of the present disclosure includes the apparatus as described above.Figure 7 The data processing apparatus described in the corresponding embodiments.

[0213] The embodiments of the present disclosure also provide an electronic device, such as Figure 9 As shown in the structural schematic diagram of the electronic device provided by the embodiments of the present disclosure, the electronic device comprises:

[0214] The processor 91 and the memory 92; the memory 92 stores machine readable instructions executable by the processor 91, and the processor 91 is configured to execute the machine readable instructions stored in the memory 92, and when the machine readable instructions are executed by the processor 91, the processor 91 executes the following steps:

[0215] Obtaining to-be-processed data;

[0216] Based on operation parameters of a first convolution operation on the to-be-processed data, sub-data corresponding to a plurality of second convolution operations are extracted from the to-be-processed data respectively; the second convolution operation is obtained by converting the first convolution operation;

[0217] Using operation parameters of the first convolution operation, the sub-data corresponding to the plurality of second convolution operations are processed by the second convolution operation to obtain result data corresponding to the plurality of second convolution operations respectively;

[0218] The result data corresponding to the plurality of second convolution operations are reorganized to obtain target result data of the first convolution operation on the to-be-processed data.

[0219] The memory 92 includes the memory 921 and the external memory 922; the memory 921 is also called an internal memory, and is used to temporarily store operation data in the processor 91 and exchange data with the external memory 922 such as a hard disk, and the processor 91 exchanges data with the external memory 922 through the memory 921.

[0220] The specific execution process of the instructions can refer to the steps of the data processing method described in the embodiments of the present disclosure, which will not be described here.

[0221] The embodiments of the present disclosure also provide a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to execute the steps of the data processing method described in the method embodiments. The storage medium can be a volatile or non-volatile computer readable storage medium.

[0222] The embodiments of the present disclosure also provide a computer program product, and the computer program product carries program codes, and the instructions included in the program codes can be used to execute the steps of the data processing method described in the method embodiments. For details, refer to the method embodiments, which will not be described here.

[0223] The computer program product can be implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied in a computer storage medium. In another optional embodiment, the computer program product is embodied in a software product, such as a software development kit (SDK) or the like.

[0224] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here. In several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interface, and can be electrical, mechanical or other forms.

[0225] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment.

[0226] In addition, each functional unit in each embodiment of the present disclosure can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit.

[0227] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a nonvolatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present disclosure essentially or the part of the prior art that contributes to the present disclosure or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for making an electronic device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present disclosure. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0228] Finally, it should be noted that: the above-described embodiments are only specific embodiments of the present disclosure, used to illustrate the technical solutions of the present disclosure, and not to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art who is familiar with the technical field of the present disclosure can make modifications or easily think of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present disclosure, or make equivalent replacements to some of the technical features. The modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A data processing method, characterized in that, The method is executed by an AI chip, which includes a dedicated convolution multiplier unit, and the method includes: Acquire the data to be processed; the data to be processed includes image data, audio data, text data, or feature data corresponding to any of the above types of data; Based on the operation parameters of the first convolution operation on the data to be processed, sub-data corresponding to multiple second convolution operations are extracted from the data to be processed; the stride of the first convolution operation is greater than the stride of the corresponding second convolution operation. Using the operands corresponding to the first convolution operation, the dedicated convolution multiplier unit performs second convolution operations on the sub-data corresponding to multiple second convolution operations respectively, to obtain the result data corresponding to the multiple second convolution operations respectively; wherein, the second convolution operation is a normal convolution. The result data corresponding to multiple second convolution operations are reorganized to obtain the target result data for performing the first convolution operation on the data to be processed.

2. The method according to claim 1, characterized in that, The acquisition of data to be processed includes: Obtain the raw data to be processed; Using the padding parameters corresponding to the original data to be processed, the original data to be processed is padded to obtain the data to be processed.

3. The method according to claim 1 or 2, characterized in that, The first convolution operation includes dilated convolution; the second convolution operation includes ordinary convolution; the operation parameters include the hole size of the dilated convolution; the operands corresponding to the first convolution operation include the convolution kernel of the dilated convolution.

4. The method according to claim 3, characterized in that, The step of extracting sub-data corresponding to multiple second convolution operations from the data to be processed based on the operation parameters of the first convolution operation on the data to be processed includes: Based on the hole size of the dilated convolution, determine the operation identifier of the second convolution operation; For each second convolution operation, based on the operation identifier of the second convolution operation and the size information of the data to be processed, sub-data corresponding to the second convolution operation is extracted from the data to be processed.

5. The method according to claim 4, characterized in that, The hole size of the dilated convolution includes: a first dimension in the height direction of the data to be processed, and a second dimension in the width direction of the data to be processed; The step of determining the operation identifier of the second convolution operation based on the hole size of the dilated convolution includes: constructing the operation identifier of the second convolution operation based on the first size and the second size.

6. The method according to claim 4, characterized in that, For each second convolution operation, based on the order of this second convolution operation among multiple second convolution operations and the size information of the data to be processed, sub-data corresponding to this second convolution operation is extracted from the data to be processed, including: The operation flag in response to the current second convolution operation is The first data element located in the i-th row and j-th column of the data to be processed is taken as the first data element of the sub-data corresponding to the current second convolution operation. Based on the position of the first first data element in the data to be processed, the extraction step size in the height direction with the first size as the extraction step size in the width direction, other first data elements except the first first data element are extracted from the sub-data in the data to be processed; Based on the extracted first data element and the other first data elements, the sub-data of the current second convolution operation is formed; where i is a positive integer less than or equal to the first size; j is a positive integer less than or equal to the second size.

7. The method according to any one of claims 4-6, characterized in that, The step of reorganizing the result data corresponding to multiple second convolution operations to obtain the target result data for performing the first convolution operation on the data to be processed includes: Based on the operation identifiers of multiple second convolution operations, determine the position information of each second data element in the target result data in the result data of each second convolution operation; Based on the position information of each second data element in the target result data in the result data of each second convolution operation, the second data elements are filled into the corresponding positions in the target result data to obtain the target result data.

8. The method according to claim 7, characterized in that, The method of determining the position information of each second data element in the target result data based on the operation identifier of multiple second convolution operations includes: The operation flag in response to the current second convolution operation is The position of the first second data element in the target result data located in the i-th row and j-th column is determined as the position information of the first second data element in the result data corresponding to the current second convolution operation in the target result data. Based on the position information of the first second data element in the target result data, and the padding step with the first size as the height direction and the padding step with the second size as the width direction, the position information of the other second data elements in the result data corresponding to the current second convolution operation, excluding the first second data element, in the target result data is determined.

9. A data processing apparatus, characterized in that, include: The system comprises a processing unit and a computation unit; the computation unit includes at least a dedicated convolution multiplier unit. The processing unit is configured to acquire data to be processed; extract sub-data corresponding to multiple second convolution operations from the data to be processed based on the operation parameters of the first convolution operation on the data to be processed; the second convolution operation is obtained by transforming the first convolution operation; and send the sub-data to the computing unit; the data to be processed includes image data, audio data, text data, or feature data corresponding to any of the above types of data. The computation unit is configured to, in response to receiving the sub-data, use the operands corresponding to the first convolution operation to perform second convolution operations on the sub-data corresponding to multiple second convolution operations through the dedicated convolution multiplier unit, to obtain result data corresponding to the multiple second convolution operations, and to reassemble the result data corresponding to the multiple second convolution operations to obtain the target result data for performing the first convolution operation on the data to be processed; wherein, the second convolution operation is a normal convolution.

10. A data processing chip, characterized in that, include: Dedicated convolution multiplier unit; The chip executes the data processing method as described in any one of claims 1-8 during operation.

11. An electronic device, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, the processor executing the machine-readable instructions stored in the memory, wherein when the machine-readable instructions are executed by the processor, the processor performs the steps of the data processing method as described in any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by an electronic device, performs the steps of the data processing method as described in any one of claims 1 to 8.

Images