Method, electronic device and storage medium for implementing deep convolution operator in chip

Abstract

A method, electronic device and storage medium for implementing a deep convolution operator in a chip. The method for implementing a deep convolution operator includes: selecting M data channels from a plurality of data channels, loading N data groups in each of the M data channels into N target registers in a one-to-one correspondence, and selecting M weight channels from a plurality of weight channels, loading N weights in each of the M weight channels into N target registers in a one-to-one correspondence; in the N target registers, performing multiplication and addition operations on the N data groups in the kth data channel in the M data channels and the N weights in the kth weight channel in the M weight channels to obtain a first operation result. The method for implementing a deep convolution operator can realize parallel operations of deep convolution of multiple groups of data in multiple data channels, thereby improving the execution efficiency of deep convolution calculation tasks and improving the computing performance of the chip.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a method for implementing a deep convolution operator in a chip, an electronic device, and a storage medium. Background Art

[0002] The field of deep learning involves a large number of convolution operations, and convolution operation computing tasks can be performed by convolutional neural networks (CNNs). The computing tasks of convolutional neural networks need to be deployed on different artificial intelligence chips, such as graphics processing units (GPUs), general-purpose computing on graphics processing units (GPGPUs), etc. Artificial intelligence chips complete various computing functions by executing programs. Summary of the invention

[0003] At least one embodiment of the present disclosure provides a method for implementing a deep convolution operator in a chip, the chip comprising N target registers, N being a positive integer, input data of the deep convolution operator comprising a plurality of data channels, a plurality of data elements in each of the plurality of data channels being divided into a plurality of data groups, weight data of the deep convolution operator comprising a plurality of weight channels, each of the plurality of weight channels comprising a plurality of weights, the method for implementing the deep convolution operator comprising: selecting M data channels from the plurality of data channels, loading N data groups in each of the M data channels into the N target registers in a one-to-one correspondence, and selecting M weight channels from the plurality of weight channels, loading N weights in each of the M weight channels into the N target registers in a one-to-one correspondence, wherein M is a positive integer; in the N target registers, performing a multiplication-addition operation on N data groups in a kth data channel from the M data channels and N weights in a kth weight channel from the M weight channels to obtain a first operation result, wherein k = 1, 2, …, M.

[0004] For example, in the method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure, the multiplication and addition operations are performed on the N data groups in the kth data channel in the M data channels and the N weights in the kth weight channel in the M weight channels to obtain a first operation result, including: performing multiplication and addition operations on the N data groups in the kth data channel and the N weights in the kth weight channel one by one to obtain N first multiplication and addition results; and accumulating the N first multiplication and addition results to obtain the first operation result.

[0005] For example, in the method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure, in response to the data types of the input data and the weight data being the first data type, before performing multiplication and addition operations on the N data groups in the kth data channel and the N weights in the kth weight channel in a one-to-one correspondence, performing multiplication and addition operations on the N data groups in the kth data channel in the M data channels and the N weights in the kth weight channel in the M weight channels to obtain a first operation result, further includes: converting the data types of the N data groups in the kth data channel and the N weights in the kth weight channel from the first data type to a second data type.

[0006] For example, in the method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure, the first data type is a BF16 data type, and the second data type is a FP32 data type.

[0007] For example, in the method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure, in response to the N data groups in the kth data channel including the i+1th data group to the i+Nth data group in the kth data channel, and the N weights in the kth weight channel including the i+1th weight to the i+Nth weight in the kth weight channel, the method for implementing the deep convolution operator also includes: in the N target registers, accumulating the first operation result and the second operation result to obtain a third operation result, wherein the second operation result is the multiplication and addition operation result of the 1st data group to the i-th data group in the kth data channel and the 1st weight to the i-th weight in the kth weight channel, wherein i is a positive integer.

[0008] For example, in the method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure, the input data of the deep convolution operator includes multiple basic block data, the multiple basic block data include target basic block data, the target basic block data includes Y data groups in the kth data channel, wherein Y is an integer greater than or equal to N, and in response to i+N=Y, the third operation result is used as the deep convolution operation result of the Y data groups corresponding to the target basic block data in the kth data channel.

[0009] For example, in a method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure, in response to the data types of the input data and the weight data being a first data type, and the deep convolution operation result being a second data type, the method for implementing the deep convolution operator further includes: converting the deep convolution operation result from the second data type to the first data type.

[0010] For example, the method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure also includes: obtaining first output data of the M data channels based on at least one deep convolution operation result of at least one data group of the M data channels; and outputting the first output data.

[0011] For example, the method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure further includes: storing the output first output data in a shared memory inside the chip.

[0012] For example, in the method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure, the input data of the deep convolution operator includes multiple basic block data, the multiple basic block data include target basic block data, the target basic block data includes Y data groups in the Kth data channel, wherein Y is an integer greater than or equal to N, and before selecting M data channels from the multiple data channels, the method for implementing the deep convolution operator also includes: loading and storing the multiple basic block data into a shared memory inside the chip.

[0013] For example, in the method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure, before selecting M weight channels from the multiple weight channels, the method for implementing the deep convolution operator also includes: loading and storing the weight data of the deep convolution operator into a shared memory inside the chip.

[0014] For example, in the method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure, the deep convolution operator is a non-stride deep convolution operator, and the deep convolution operator is implemented in a vector processing core of the chip.

[0015] At least one embodiment of the present disclosure further provides an electronic device, comprising: one or more processors; one or more memories, including one or more computer program modules; wherein the one or more computer program modules are stored in the one or more memories and are configured to be executed by the one or more processors, and the one or more computer program modules are used to implement the method for implementing a deep convolution operator in a chip provided in any embodiment of the present disclosure.

[0016] At least one embodiment of the present disclosure further provides a storage medium storing non-temporary computer-readable instructions, which, when executed by a computer, implements the method for implementing a deep convolution operator in a chip provided by any embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, but are not intended to limit the present disclosure.

[0018] Figure 1 A schematic diagram of a method for executing a deep convolution computing task;

[0019] Figure 2 An exemplary flow chart of a method for implementing a deep convolution operator in a chip provided for at least one embodiment of the present disclosure;

[0020] Figure 3 A schematic diagram of an example of a method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure;

[0021] Figure 4 A schematic diagram of another example of a method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure;

[0022] Figure 5A A schematic diagram of another example of a method for implementing a deep convolution operator in a chip provided for at least one embodiment of the present disclosure;

[0023] Figure 5B for Figure 5A A schematic diagram of an example of how data groups and weights are grouped in ;

[0024] Figure 6 A schematic block diagram of a device for implementing a deep convolution operator in a chip provided for at least one embodiment of the present disclosure;

[0025] Figure 7 A schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure;

[0026] Figure 8 A schematic block diagram of another electronic device provided for at least one embodiment of the present disclosure; and

[0027] Fig. 9 A schematic diagram of a storage medium provided for at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0029] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, similar words such as "one", "one" or "the" do not indicate quantity restrictions, but indicate that there is at least one. Similar words such as "include" or "comprise" mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Similar words such as "connect" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0030] The present disclosure is described below through several specific embodiments. In order to keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and known components may be omitted. When any component of the embodiments of the present disclosure appears in more than one figure, the component is represented by the same or similar reference numeral in each figure.

[0031] Convolution is a common operation in deep learning, and is often used to process data such as images, speech, and text. The standard convolution calculation uses a weight matrix to achieve the joint mapping of spatial dimension and channel dimension features, but at the cost of high computational complexity, high memory overhead, and a large number of weight coefficients. Depthwise convolution is an algorithm that improves the standard convolution calculation in convolutional neural networks. It reduces the number of parameters required for convolution calculation by splitting the correlation between spatial dimension and channel dimension, and can improve the efficiency of convolution kernel parameters to a certain extent.

[0032] The deep convolution computing task can be implemented by the deep convolution operator. In the neural network, the operator corresponds to the computing logic of the layer or node. When implementing the neural network, the operator can be implemented by software (for example, by computer program) or by hardware (for example, by circuit). The deep convolution operator can be deployed on the artificial intelligence chip to perform the deep convolution computing task by combining software and hardware.

[0033] Figure 1 A schematic diagram of a deep convolution computing task execution method.

[0034] For example, Figure 1As shown, the data corresponding to the input data includes multiple channels (Channel), input_c represents the number of channels of the input data; in each channel of the input data, multiple data elements are arranged in two dimensions to form an array, the height of the data in each channel in the vertical dimension is input_h, input_h represents the number of data elements of the array along the column direction, and the width of the data in each channel in the horizontal dimension is input_w, input_w represents the number of data elements of the array along the row direction.

[0035] For example, Figure 1 As shown, similar to the input data, the data corresponding to the weight data and the output data also include multiple channels, kernel_c represents the number of channels of the weight data, and output_c represents the number of channels of the output data; in each channel of the weight data, the height of the data in the vertical dimension is kernel_h, and the width of the data in the horizontal dimension is kernel_w; in each channel of the output data, the height of the data in the vertical dimension is output_h, and the width of the data in the horizontal dimension is output_w.

[0036] For example, Figure 1 As shown, in the deep convolution, the 4 channels of weight data correspond one-to-one with the 4 channels of input data, and the convolution operation is performed in a two-dimensional plane. For example, specifically, the first channel of the input data is convolved with the first channel of the weight data to obtain the output data of the first channel; the second channel of the input data is convolved with the second channel of the weight data to obtain the output data of the second channel; ...; the fourth channel of the input data is convolved with the fourth channel of the weight data to obtain the output data of the fourth channel. For example, the 4 channels of the calculated output data correspond one-to-one with the 4 channels of the input data and the weight data.

[0037] In the traditional AI chip architecture, tensor cores are used to perform deep convolution computing tasks. The tensor cores of AI chips have alignment requirements on the channels, and the size of deep convolution on the channels is very small (for example, the size of deep convolution in the channel direction is generally 1); if the tensor core is used directly to perform deep convolution operations, padding is required in the channel direction, which reduces the computing efficiency. For example, taking the tensor core's channel alignment as 64 as an example, if the tensor core is used directly to perform deep convolution operations with a channel size of 1, it is necessary to fill the remaining 63 channels with 0, resulting in a computing efficiency of only 1 / 64 of the peak computing power, which greatly reduces the computing efficiency.

[0038] At least one embodiment of the present disclosure provides a method for implementing a deep convolution operator in a chip, wherein the chip includes N target registers, N is a positive integer; the input data of the deep convolution operator includes multiple data channels, and multiple data elements in each of the multiple data channels are divided into multiple data groups; the weight data of the deep convolution operator includes multiple weight channels, and each of the multiple weight channels includes multiple weights. The method for implementing the deep convolution operator includes: selecting M data channels from the multiple data channels, loading the N data groups in each of the M data channels into the N target registers in a one-to-one correspondence, and selecting M weight channels from the multiple weight channels, loading the N weights in each of the M weight channels into the N target registers in a one-to-one correspondence, wherein M is a positive integer; in the N target registers, performing multiplication and addition operations on the N data groups in the kth data channel in the M data channels and the N weights in the kth weight channel in the M weight channels to obtain a first operation result, wherein k = 1, 2, …, M.

[0039] At least one embodiment of the present disclosure further provides an electronic device and a storage medium for implementing the method of implementing a deep convolution operator in a chip according to the above embodiment.

[0040] The method, apparatus, electronic device and storage medium provided by at least one embodiment of the present disclosure can realize parallel operations of deep convolution of multiple groups of data in multiple data channels by grouping the input data and weight data of the deep convolution operator using the target register in the chip, thereby improving the execution efficiency of the deep convolution calculation tasks and enhancing the computing performance of the chip.

[0041] Hereinafter, at least one embodiment of the present disclosure will be described in detail with reference to the accompanying drawings. It should be noted that the same reference numerals in different drawings will be used to refer to the same elements described.

[0042] In some examples, the deep convolution operator can be implemented in the vector processing core of the chip. Since the vector processing core has no alignment requirement on the channel, using the vector processing core to perform deep convolution calculation tasks does not require padding in the channel direction, thereby improving the computational efficiency of the deep convolution.

[0043] In some examples, the depth convolution operator provided by at least one embodiment of the present disclosure may be a non-stride depth convolution operator, which performs a non-stride depth convolution operation, and the stride of the non-stride depth convolution is 1. For example, the stride of the kernel function of the depth convolution operator provided by at least one embodiment of the present disclosure may also be greater than 1, which may be selected according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0044] Figure 2 An exemplary flow chart of a method for implementing a deep convolution operator in a chip provided for at least one embodiment of the present disclosure.

[0045] For example, Figure 2 As shown, at least one embodiment of the present disclosure provides a method for implementing a deep convolution operator in a chip. For example, the chip includes N target registers, where N is a positive integer; the input data of the deep convolution operator includes multiple data channels, and multiple data elements in each of the multiple data channels are divided into multiple data groups; the weight data of the deep convolution operator includes multiple weight channels, and each of the multiple weight channels includes multiple weights.

[0046] For example, Figure 2 As shown, the method for implementing a deep convolution operator in a chip may include the following steps S110 to S130.

[0047] Step S110: Select M data channels from the plurality of data channels, and load N data groups in each of the M data channels into N target registers one by one.

[0048] Step S120: select M weight channels from the multiple weight channels, and load the N weights in each of the M weight channels into N target registers one by one, where M is a positive integer.

[0049] For example, in steps S110 and S120, the M data channels include the 1st data channel to the Mth data channel, and the M weight channels include the 1st weight channel to the Mth weight channel. For example, the M data channels and the M weight channels are used to perform a one-to-one deep convolution operation (for example, the specific operation principle of the deep convolution is referred to in Figure 1 For example, in some examples, M = 2 and N = 2.

[0050] For example, the number of selected data channels and weight channels (that is, the value of M) depends on the capacity of the target register. For example, taking the target register as a thread local register (TLR), when the data type of the input data and weight data is BF16 type, the target register can store 2 BF16 type data of 2 data channels and 2 weight channels (that is, M = 2). It should be noted that the type, capacity and number of the target register, as well as the size and data type of the input data and weight data, can all be selected according to actual conditions, and the embodiments of the present disclosure are not limited to this.

[0051] For example, the number of data elements in each data group of each data channel depends on the number of threads in the chip's warp. For example, when the chip's processor allocates deep convolution computing tasks, a task can be represented by a dispatch grid, and a dispatch grid can be divided into multiple workgroups; threads in the same workgroup can be grouped according to scheduling units, and then dispatched to the hardware for execution in groups. The scheduling unit is called a warp. For example, when the chip's warp includes n threads, each data group of each data channel includes n data elements corresponding to the n threads one by one, where n is a positive integer. It should be noted that the number of threads in the warp in the chip may depend on the specific application scenario, and the embodiments of the present disclosure are not limited to this.

[0052] For example, when the input data is an input picture, the data elements in each data group of each data channel may be pixels; the input data and the data elements therein may also be selected as other types according to the actual application scenario, and the embodiments of the present disclosure are not limited to this.

[0053] Step S130: In N target registers, perform multiplication and addition operations on N data groups in the kth data channel in the M data channels and N weights in the kth weight channel in the M weight channels to obtain a first operation result, where k = 1, 2, …, M.

[0054] For example, in step S130, the value of the number of data groups in each data channel and the value of the number of weights in each weight channel are consistent with N (that is, the value of the number of target registers), which depends on the capacity of the chip and the specific hardware architecture (for example, the specific setting method of the target registers inside the chip, etc.), and the embodiments of the present disclosure are not limited to this.

[0055] For example, in step S130, the calculation logic of the multiplication and addition operation can be implemented by an efficient dot product (DP) multiplication and addition instruction. For the case of calculating N data groups in parallel in N target registers, the multiplication and addition instruction is a DP-N multiplication and addition instruction (for example, when N=2, the multiplication and addition instruction is a DP2 multiplication and addition instruction). For example, the calculation logic of the multiplication and addition operation can also be selected as other instruction types according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0056] It should be noted that the M data channels can be any M data channels of the multiple data channels of the input data, that is, any M data channels can be calculated in the manner of the above steps S110~S130; the N data groups in each data channel can be any N data groups in the corresponding data channel, that is, any N data groups can be calculated in the manner of the above steps S110~S130; the N weights in each weight channel can be any N weights in the corresponding weight channel, that is, any N weights can be calculated in the manner of the above steps S110~S130; the selection method of M data channels, N data groups, and N weights can be selected according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0057] For example, for the input data and weight data of the deep convolution operator, they can be grouped and calculated in N target registers from the first data channel to the last data channel, and from the first weight channel to the last weight channel in accordance with the above steps S110 to S130 until all data elements in all data channels of the input data are calculated to obtain the final deep convolution operation result.

[0058] In some examples, Figure 2 The step S130 may further include the following steps S131~S132.

[0059] Step S131: performing multiplication and addition operations on the N data groups in the kth data channel and the N weights in the kth weight channel in a one-to-one correspondence to obtain N first multiplication and addition results;

[0060] Step S132: Accumulate the N first multiplication and addition results to obtain a first operation result.

[0061] For example, specifically, in step S131, in the first target register, a plurality of data elements in the first data group in the kth data channel are respectively multiplied and added with the first weight in the kth weight channel, and then the plurality of products are accumulated to obtain a first multiplication and addition result of the first data group and the first weight; in the second target register, a plurality of data elements in the second data group in the kth data channel are respectively multiplied and added with the second weight in the kth weight channel, and then the plurality of products are accumulated to obtain a first multiplication and addition result of the second data group and the second weight; ...; in the Nth target register, a plurality of data elements in the Nth data group in the kth data channel are respectively multiplied and added with the Nth weight in the kth weight channel, and then the plurality of products are accumulated to obtain a first multiplication and addition result of the Nth data group and the Nth weight.

[0062] For example, further in step S132, the N first multiplication and addition results obtained in step S131 are accumulated to obtain a first operation result of the kth data channel and the kth weight channel.

[0063] In some examples, in response to the data type of the input data and the weight data being the first data type, before executing step S131 , step S130 may further include the following step S133 .

[0064] Step S133: Convert the data types of the N data groups in the kth data channel and the data types of the N weights in the kth weight channel from the first data type to the second data type.

[0065] For example, in some examples, the first data type may be a BF16 data type, and the second data type may be an FP32 data type. For example, when the data types of the input data and weight data are BF16 data types, if the BF16 data type is directly used for deep convolution operations, the intermediate results in the calculation process may have low precision, thereby causing the final calculation result to have fuzzy precision; since the FP32 data type has high computational precision, by executing step S133, the FP32 data type may be used for deep convolution operations, thereby improving the precision of deep convolution operations.

[0066] It should be noted that if the data type of the input data and the weight data is itself an FP32 data type, the data type conversion in step S133 may not be performed; in addition, the first data type and the second data type are not limited to BF16 and FP32 data types, and may be selected as other data types according to actual needs, and the embodiments of the present disclosure do not impose any restrictions on this.

[0067] Figure 3 A schematic diagram of an example of a method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure. For example, Figure 3 for Figure 2 A specific example of a method for implementing deep convolution in a chip is shown.

[0068] For example, Figure 3 As shown, the chip includes N target registers (target register 0, target register 1 ... target register (N-1)), where N is a positive integer; the input data of the deep convolution operator includes multiple data channels, and multiple data elements in each of the multiple data channels are divided into multiple data groups; the weight data of the deep convolution operator includes multiple weight channels, and each of the multiple weight channels includes multiple weights.

[0069] For example, Figure 3 As shown, in Figure 2In step S110, M data channels (data channels C0, C1, ... C(M-1)) are selected from the multiple data channels, where M is a positive integer, and the N data groups in each data channel are loaded into the N target registers one by one. For example, specifically, the first data group (G0_C0) of the first data channel C0, the first data group (G0_C1) of the second data channel C1 ... the first data group (G0_C(M-1)) of the Mth data channel C(M-1) are loaded into the target register 0 respectively; the second data group (G1_C0) of the first data channel C0, the second data group (G1_C1) of the second data channel C1 ... the Mth data channel C(M-1) are loaded into the target register 0 respectively. The second data group (G1_C(M-1)) of data channel C(M-1) is loaded into target register 1 respectively; ...; the Nth data group (G(N-1)_C0) of the first data channel C0, the second data group (G(N-1)_C1) of the second data channel C1...the Nth data group (G(N-1)_C(M-1)) of the Mth data channel C(M-1) is loaded into target register (N-1) respectively.

[0070] For example, Figure 3 As shown, in Figure 2 In step S120, M weight channels (weight channels C0, C1, ... C(M-1)) are selected from the multiple weight channels, where M is a positive integer, and the N weights in each weight channel are loaded into the N target registers one by one. For example, specifically, the first weight (W0_C0) of the first weight channel C0, the first weight (W0_C1) of the second weight channel C1...the first weight (W0_C(M-1)) of the Mth weight channel C(M-1) are respectively loaded into the target register 0; the second weight (W1_C0) of the first weight channel C0, the second weight (W1_C1) of the second weight channel C1...the second weight (W1_C(M-1)) of the Mth weight channel C(M-1) are respectively loaded into the target register 1; ...; the Nth weight (W(N-1)_C0) of the first weight channel C0, the second weight (W(N-1)_C1) of the second weight channel C1...the Nth weight (W(N-1)_C(M-1)) of the Mth weight channel C(M-1) are respectively loaded into the target register (N-1).

[0071] For example, in Figure 2In step S130, step S131 is executed, and for the N data groups in the k-th data channel and the N weights in the k-th weight channel, in the target register 0, a multiplication and addition operation is performed on the multiple data elements in the data group (G0_C(k-1)) and the weight (W0_C(k-1)) to obtain the corresponding first multiplication and addition result; in the target register 1, a multiplication and addition operation is performed on the multiple data elements in the data group (G1_C(k-1)) and the weight (W1_C(k-1)) to obtain the corresponding first multiplication and addition result; ...; in the target register (N-1), a multiplication and addition operation is performed on the multiple data elements in the data group (G(N-1)_C(k-1)) and the weight (W(N-1)_C(k-1)) to obtain the corresponding first multiplication and addition result. For example, further, step S132 is executed, and the N first multiplication and addition results obtained above are accumulated to obtain the first operation result Result1_C(k-1) of the k-th data channel and the k-th weight channel. For example, the first operation result Result1_C(k-1) can be expressed by the following formula (1):

[0072] Result1_C(k-1) = G0_C(k-1)×W0_C(k-1) + G1_C(k-1)×W1_C(k-1) + …+ G(N-1)_C(k-1)×W(N-1)_C(k-1) Formula (1)

[0073] For example, if the data types of the input data and the weight data are BF16 data types, step S133 may be performed to convert the data types of the N data groups in the kth data channel and the N weights in the kth weight channel from BF16 data types to FP32 data types to improve the accuracy of the deep convolution operation. For example, in this case, the first operation result Result1_C(k-1) may be expressed by the following formula (2):

[0074] Result1_C(k-1) = cast_to_FP32(G0_C(k-1)×W0_C(k-1)) + cast_to_FP32(G1_C(k-1)×W1_C(k-1)) +……+ cast_to_FP32(G(N-1)_C(k-1)×W(N-1)_C(k-1)) Formula (2)

[0075] It should be noted that Figure 3 The target registers and the grouping of data groups and weights shown are only for illustration; the number and specific implementation of the target registers, as well as the grouping and specific implementation of data groups and weights, can all be selected according to the actual application scenario, and the embodiments of the present disclosure do not limit this.

[0076] At least one embodiment of the present disclosure provides a method for implementing a deep convolution operator in a chip. By using a target register within the chip to group the input data and weight data of the deep convolution operator, parallel operations of deep convolution of multiple groups of data in multiple data channels can be achieved, thereby improving the execution efficiency of the deep convolution calculation tasks and enhancing the computing performance of the chip.

[0077] In some examples, in response to the N data groups in the kth data channel including the i+1th data group to the i+Nth data group in the kth data channel, and the N weights in the kth weight channel including the i+1th weight to the i+Nth weight in the kth weight channel, the method for implementing a deep convolution operator provided by at least one embodiment of the present disclosure may further include the following step S140.

[0078] Step S140: In the N target registers, the first operation result and the second operation result are accumulated to obtain a third operation result.

[0079] For example, in step S140, the second operation result is the multiplication and addition result of the 1st to ith data groups in the kth data channel and the 1st to ith weights in the kth weight channel, where i is a positive integer.

[0080] For example, in step S140, when the N data groups in the kth data channel include the i+1th data group to the i+Nth data group in the kth data channel, and the N weights in the kth weight channel include the i+1th weight to the i+Nth weight in the kth weight channel, since i is greater than or equal to 1, before the N data groups, i data groups (i.e., the 1st data group to the ith data group) in the kth data channel have completed the multiplication and addition operation and obtained the second operation result. For example, in step S140, the first operation result of the N data groups is accumulated with the second operation results of the i data groups before it to obtain the third operation result.

[0081] It should be noted that the second operation result can also be obtained through steps S110 to S140 as described above; when the i+Nth data group is not the last data group of the kth data channel, the third operation result obtained in step S140 can be used as the second operation result of the next data group, and further accumulated with the first operation result of the next data group, until all data groups in the kth data channel are fully calculated.

[0082] For example, Figure 3 For example, in step S140, if before the data group G0_C0, there are i data groups in the k-th data channel that have completed the multiplication and addition operation and obtained the second operation result, then Figure 3After the first operation result Result1_C(k-1) of the N data groups is obtained, the first operation result Result1_C(k-1) is accumulated with the second operation results Result2_C(k-1) of the previous i data groups to obtain the third operation result Result3_C(k-1). For example, the third operation result Result3_C(k-1) can be expressed by the following formula (3):

[0083] Result3_C(k-1) = Result1_C(k-1) + Result2_C(k-1) Formula (3)

[0084] For example, in some examples, the input data of the deep convolution operator includes a plurality of basic block data, the plurality of basic block data includes target basic block data, the target basic block data includes Y data groups in the k-th data channel, where Y is an integer greater than or equal to N. For example, in response to i+N=Y, the third operation result is used as the deep convolution operation result of the Y data groups corresponding to the target basic block data in the k-th data channel.

[0085] For example, in step S140, when i+N = Y, the i+Nth data group is the last data group among the Y data groups corresponding to the target basic block data in the kth data channel, indicating that the Y data groups corresponding to the target basic block data in the kth data channel have all been calculated; therefore, the third operation result can be used as the deep convolution operation result of the Y data groups corresponding to the target basic block data in the kth data channel.

[0086] For example, in some examples, in response to the data type of the input data and the weight data being a first data type, and the result of the deep convolution operation being a second data type, the method for implementing a deep convolution operator provided by at least one embodiment of the present disclosure may further include the following step S150.

[0087] Step S150: converting the depthwise convolution operation result from the second data type into the first data type.

[0088] For example, in step S150, if the data types of the input data and the weight data are the first data type, and the result of the deep convolution operation is the second data type, the result of the deep convolution operation can be converted from the second data type to the first data type; that is, the input data and the weight data are converted during the operation, and after the operation is completed, the data type of the operation result can be converted to be consistent with the input data and the weight data. For example, the first data type can be a BF16 data type, and the second data type can be a FP32 data type. The first data type and the second data type can also be selected as other data types according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0089] For example, in some examples, the method for implementing a depthwise convolution operator provided by at least one embodiment of the present disclosure may further include the following steps S161 to S162.

[0090] Step S161: obtaining first output data of the M data channels based on at least one depth convolution operation result of at least one data group of the M data channels;

[0091] Step S162: Output the first output data.

[0092] For example, in step S161, by executing the method for implementing the depth convolution operator provided by at least one embodiment of the present disclosure as described above, one or more depth convolution operation results of one or more data groups of M data channels can be obtained; when there is only one depth convolution operation result, the depth convolution operation result can be used as the first output data of the M data channels; when there are multiple depth convolution operation results, the first output data of the M data channels can be obtained by performing a merging operation (for example, an accumulation operation, etc.) on one or more depth convolution operation results.

[0093] For example, further in step S162, the first output data obtained in step S161 is output from a computing device such as a target register.

[0094] It should be noted that at least one data group of the M data channels may be all data groups of the M data channels, or may be part of the data groups of the M data channels, depending on the number, size and computing power of the target registers, which is not limited in the embodiments of the present disclosure.

[0095] For example, in some examples, the method for implementing a depthwise convolution operator provided by at least one embodiment of the present disclosure may further include the following step S170.

[0096] Step S170: storing the output first output data into the shared memory inside the chip.

[0097] For example, in step S170, the shared memory is a hardware resource inside the chip; since the access speed of the shared memory inside the chip is much faster than the external memory (global memory) outside the chip, the first output data output in step S162 can be first stored in the shared memory inside the chip, so as to be stored in the external memory together with the first output data calculated from other channels or other data groups, thereby reducing the number of times data is repeatedly stored in the external memory 130, thereby improving the storage efficiency of the output data.

[0098] For example, in some examples, the input data of the deep convolution operator includes a plurality of basic block data, the plurality of basic block data includes target basic block data, the target basic block data includes Y data groups in the kth data channel, where Y is an integer greater than or equal to N. For example, in Figure 2 Before step S110, the method for implementing a depthwise convolution operator provided by at least one embodiment of the present disclosure may further include the following step S180.

[0099] Step S180: Load and store multiple basic block data into a shared memory inside the chip.

[0100] For example, in step S180, the number and size of the multiple basic block data depends on the size of the shared memory, which can be set according to actual needs, and the embodiments of the present disclosure do not limit this. For example, since the access speed of the shared memory inside the chip is much faster than the external memory (for example, more than ten times faster), storing the multiple basic block data in the shared memory can facilitate the target register of the subsequent chip and other computing devices to repeatedly load the basic block data to perform the deep convolution operation provided by at least one embodiment of the present disclosure, reducing the number of repeated loading of data from the external memory, thereby improving the access efficiency of the input data and improving the execution efficiency of the deep convolution calculation task.

[0101] It should be noted that the target basic block data can refer to any basic block data among multiple basic block data, and any N data groups in each data channel in the target basic block data can perform deep convolution operations using the method for implementing a deep convolution operator provided by at least one embodiment of the present disclosure.

[0102] For example, in some examples, Figure 2 Before step S120, the method for implementing a depthwise convolution operator provided by at least one embodiment of the present disclosure may further include the following step S190.

[0103] Step S190: Load and store the weight data of the depthwise convolution operator into the shared memory inside the chip.

[0104] For example, in step S190, since the size of the weight data is small, all the weight data can be stored in a shared memory. For example, since the access speed of the shared memory inside the chip is much faster than that of the external memory (for example, more than ten times faster), storing all the weight data in the shared memory can facilitate the target register of the subsequent chip and other computing devices to repeatedly load the weight data to perform the deep convolution operation provided by at least one embodiment of the present disclosure, avoiding repeated loading of weight data from the external memory, thereby improving the access efficiency of the weight data and improving the execution efficiency of the deep convolution calculation task.

[0105] For example, in some examples, block data loading can be performed through tensor memory accelerator (TMA) instructions, that is, multiple basic block data of input data and weight data are loaded. For example, TMA instructions can directly load data from external memory to computing devices such as target registers inside the chip without using thread resources, thereby reducing the delay from external memory to shared memory and from shared memory to target registers, and improving the computing efficiency of the chip.

[0106] Figure 4 A schematic diagram of another example of a method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure. For example, Figure 4 This is a specific example of steps S170 to S190 described above.

[0107] For example, in Figure 4 In the example, the input data, weight data, and output data of the deep convolution operator are as follows: Figure 1 In the example shown, the input data, weight data and output data of the deep convolution operator are all stored in the external memory 130 outside the chip. For example, the specific introduction of the input data, weight data and output data can be referred to Figure 1 The description will not be repeated here.

[0108] For example, Figure 4 As shown, the input data of the deep convolution operator includes multiple basic block data. For example, in step S180, multiple basic block data are loaded from the external memory 130 and stored in the shared memory 120 inside the chip; in step S190, all weight data are loaded and stored in the shared memory 120 inside the chip. For example, since the access speed of the shared memory 120 inside the chip is much faster than that of the external memory 130, storing the basic block data and the weight data in the shared memory 120 can facilitate the target register 110 of the chip to repeatedly load the basic block data to perform the deep convolution operation provided by at least one embodiment of the present disclosure, reducing the number of repeated loading of data from the external memory 130, thereby improving the execution efficiency of the deep convolution calculation task.

[0109] For example, multiple basic block data include target basic block data, and the target basic block data includes Y data groups in the Kth data channel, where Y is an integer greater than or equal to N; the target basic block data can refer to any basic block data in the multiple basic block data, and any N data in each data channel in the target basic block data can use the method for implementing a deep convolution operator provided by at least one embodiment of the present disclosure to perform a deep convolution operation in the target register 110. It should be noted that the number of target registers 110 can be one or more; the specific method of performing a deep convolution operation in the target register 110 can refer to Figure 2~Figure 3 The relevant description will not be repeated here.

[0110] For example, Figure 4 As shown, for the first output data of the M data channels of the target basic block data, in step S170, the first output data can be stored as a basic block of output data in the shared memory 120 inside the chip; when the calculation of multiple basic block data of the input data is completed, the obtained multiple first output data are stored from the shared memory 120 to the external memory 130. For example, since the access speed of the shared memory 120 inside the chip is much higher than the external memory 130 outside the chip, storing the first output data in the shared memory 120 first can reduce the number of repeated data storage times in the external memory 130, thereby improving the storage efficiency of the output data.

[0111] It should be noted that Figure 4 The input data, weight data, output data, target register 110, shared memory 120 and external memory 130 shown are only for illustration, and their specific implementation methods can be selected according to the actual application scenario, and the embodiments of the present disclosure are not limited to this.

[0112] Figure 5A A schematic diagram of another example of a method for implementing a deep convolution operator in a chip provided for at least one embodiment of the present disclosure; Figure 5B for Figure 5A A diagram showing an example of how the data sets and weights are grouped in . For example, Figure 5A and Figure 5B for Figure 2 A specific example of a method for implementing deep convolution in a chip is shown; Figure 5A The loading and storage methods of input data, weight data, and output data in the example can be referred to Figure 4 For example, Figure 5A The specific calculation process in the example can be found in Figure 4 This is performed in the target register 110 shown.

[0113] For example, in Figure 5A and Figure 5B In the example, M = 2, N = 2, and Y = 9. For example, the chip includes multiple target registers, and each two target registers (target register j and target register j+1) can perform a group of parallel multiplication and addition operations, where j is an integer greater than or equal to 0; in the target basic block data of the input data loaded each time, each data channel includes 9 data groups; in the 2 weight channels loaded, each weight channel includes 9 weights.

[0114] For example, in Figure 2 In step S110, two data channels (C0 and C1) are selected, and two data groups in each of the two data channels are loaded into the target register j and the target register j+1 in a one-to-one correspondence. For example, specifically, Figure 5B As shown, the j-th data group (Gj_C0) of the first data channel C0 and the j-th data group (Gj_C1) of the second data channel C1 are respectively loaded into the target register j; the j+1-th data group (G(j+1)_C0) of the first data channel C0 and the j+1-th data group (G(j+1)_C1) of the second data channel C1 are respectively loaded into the target register j+1.

[0115] For example, in Figure 2 In step S120, two weight channels (C0 and C1) are selected, and two weights in each of the two weight channels are loaded into the target register j and the target register j+1 in a one-to-one correspondence. For example, specifically, Figure 5B As shown, the jth weight (Wj_C0) of the first data channel C0 and the jth weight (Wj_C1) of the second data channel C1 are respectively loaded into the target register j; the j+1th weight (W(j+1)_C0) of the first data channel C0 and the j+1th weight (W(j+1)_C1) of the second data channel C1 are respectively loaded into the target register j+1.

[0116] For example, in Figure 2 In step S130, taking data channel C0 and weight channel C0 as an example, in the target register j, a multiplication and addition operation is performed on multiple data elements in the data group (Gj_C0) and the weight (Wj_C0) to obtain the corresponding first multiplication and addition result; in the target register j+1, a multiplication and addition operation is performed on multiple data elements in the data group (G(j+1)_C0) and the weight (W(j+1)_C0) to obtain the corresponding first multiplication and addition result; further, the two first multiplication and addition results obtained above are accumulated to obtain the first operation result Result1_C0 of the data channel C0 and the weight channel C0. For example, the first operation result Result1_C0 can be expressed by the following formula (4):

[0117] Result1_C0 = Gj_C0×Wj_C0 + G(j+1)_C0×W(j+1)_C0 Formula (4)

[0118] For example, if the data types of the input data and weight data are BF16 data types, the data types of the two data groups of the data channel C0 and the two weights in the weight channel C0 can be converted from BF16 data types to FP32 data types to improve the accuracy of the deep convolution operation. For example, in this case, the first operation result Result1_C0 can be expressed by the following formula (5):

[0119] Result1_C0 = cast_to_FP32(Gj_C0×Wj_C0) + cast_to_FP32(G(j+1)_C0×W(j+1)_C0) Formula (5)

[0120] For example, in step S140, if before the data group Gj_C0, there are i data groups in the data channel C0 that have completed the multiplication and addition operation and obtained the second operation result, then after obtaining the first operation result Result1_C0, the first operation result Result1_C0 is accumulated with the second operation results Result2_C0 of the i data groups before it to obtain the third operation result Result3_C0. For example, the third operation result Result3_C0 can be expressed by the following formula (6):

[0121] Result3_C0= Result1_C0 + Result2_C0 formula (6)

[0122] For example, the calculation method of the data channel C1 and the weight channel C1 is similar to the calculation method of the data channel C0 and the weight channel C0 described above, and will not be repeated here.

[0123] For example, Figure 5AAs shown, the thread bundle of the chip includes 32 threads, and each data group includes 32 data elements corresponding to the 32 threads one by one; the depth convolution operator performs non-stride depth convolution operation, and the stride of the non-stride depth convolution is 1 (stride = 1). For example, for the first target basic block data loaded into the target register, in the data channels C0 and C1, the first data group contains data elements p-1~p30, the second data group contains data elements p0~p31, ..., and the ninth data group contains data elements p37~p68. For example, in the two weight channels C0 and C1, the data size of each weight channel is 3×3 (that is, each weight channel includes 9 weights w0~w8), and each weight performs multiplication and addition operations with a corresponding data group, so that 9 data groups can be calculated in parallel.

[0124] For example, Figure 5A As shown, since every two target registers (target register j and target register j+1) can perform a set of parallel multiplication and addition operations, the multiplication and addition operations of each weight and the corresponding data group can be performed in the corresponding target register. For example, the multiplication and addition operations in 2 target registers can be represented as burst2, and the multiplication and addition operations in 1 target register can be represented as burst1. For example, in Figure 5A In the example, every two data groups in the 1st to 8th data groups and every two weights in weights w0 to w8 perform burst2 multiplication and addition operations in two target registers in a one-to-one correspondence, and the 9th data group and the 9th weight w9 perform burst1 multiplication and addition operations in one target register.

[0125] For example, Figure 5A As shown, specifically, taking data channel C0 and weight channel C0 as an example, in target register 0, a multiplication and addition operation is performed on multiple data elements (p-1~p30) in the first data group and weight (w0_C0) to obtain a corresponding first multiplication and addition result; in target register 1, a multiplication and addition operation is performed on multiple data elements (p0~p31) in the second data group and weight (w1_C0) to obtain a corresponding first multiplication and addition result; further, the two first multiplication and addition results obtained above are accumulated to obtain the first first operation result Result1_C0-0 of data channel C0 and weight channel C0. For example, since there are no other data groups that have completed the multiplication and addition operation in data channel C0 before the first data group, the accumulation operation of step S140 may not be performed, and the first operation result Result1_C0-0 may be used as the second operation result Result2_C0-1 of the next data group.

[0126] For example, Figure 5AAs shown, further, in the target register 2, a multiplication and addition operation is performed on the multiple data elements (p1~p32) in the third data group and the weight (w2_C0) to obtain a corresponding first multiplication and addition result; in the target register 3, a multiplication and addition operation is performed on the multiple data elements (p17~p48) in the fourth data group and the weight (w3_C0) to obtain a corresponding first multiplication and addition result; further, the two first multiplication and addition results obtained above are accumulated to obtain the second first operation result Result1_C0-1 of the data channel C0 and the weight channel C0. For example, since before the third data group, the first and second data groups of the data channel C0 have completed the multiplication and addition operation and obtained the second operation result Result2_C0-1, the first operation result Result1_C0-1 and the second operation result Result2_C0-1 of the first data group are accumulated to obtain the third operation result Result3_C0-1; and the third operation result Result3_C0-1 can be used as the second operation result Result2_C0-2 of the next data group.

[0127] For example, Figure 5A As shown, further, in the target register 4, a multiplication and addition operation is performed on the multiple data elements (p18~p49) in the fifth data group and the weight (w4_C0) to obtain a corresponding first multiplication and addition result; in the target register 5, a multiplication and addition operation is performed on the multiple data elements (p19~p50) in the sixth data group and the weight (w5_C0) to obtain a corresponding first multiplication and addition result; further, the two first multiplication and addition results obtained above are accumulated to obtain the third first operation result Result1_C0-2 of the data channel C0 and the weight channel C0. For example, since before the 5th data group, the 1st to 4th data groups of the data channel C0 have completed the multiplication and addition operation and obtained the second operation result Result2_C0-2, the first operation result Result1_C0-2 and the second operation result Result2_C0-2 of the 1st data group are accumulated to obtain the third operation result Result3_C0-2; and the third operation result Result3_C0-2 can be used as the second operation result Result2_C0-3 of the next data group.

[0128] For example, Figure 5AAs shown, further, in the target register 6, a multiplication and addition operation is performed on the multiple data elements (p35~p66) in the 7th data group and the weight (w6_C0) to obtain the corresponding first multiplication and addition result; in the target register 7, a multiplication and addition operation is performed on the multiple data elements (p36~p67) in the 8th data group and the weight (w7_C0) to obtain the corresponding first multiplication and addition result; further, the two first multiplication and addition results obtained above are accumulated to obtain the fourth first operation result Result1_C0-3 of the data channel C0 and the weight channel C0. For example, since before the 7th data group, the 1st to 6th data groups of the data channel C0 have completed the multiplication and addition operation and obtained the second operation result Result2_C0-3, the first operation result Result1_C0-3 and the second operation result Result2_C0-3 of the 1st data group are accumulated to obtain the third operation result Result3_C0-2; and the third operation result Result3_C0-3 can be used as the second operation result Result2_C0-4 of the next data group.

[0129] For example, Figure 5A As shown, further, in the target register 8, a multiplication and addition operation is performed on multiple data elements (p37~p68) and weights (w8_C0) in the 9th data group to obtain a corresponding first multiplication and addition result, which can be used as the 5th first operation result Result1_C0-4 of the data channel C0 and the weight channel C0. For example, since the 1st to 8th data groups of the data channel C0 have completed the multiplication and addition operation before the 9th data group, and obtained the second operation result Result2_C0-4, the first operation result Result1_C0-4 is accumulated with the second operation result Result2_C0-4 of the 1st data group to obtain the third operation result Result3_C0-4. For example, since the 9th data group is the last data group among the 9 data groups, the third operation result Result3_C0-4 can be used as the depth convolution operation result of the 9 data groups corresponding to the target basic block data in the data channel C0.

[0130] For example, the calculation method of the data channel C1 and the weight channel C1 is similar to the calculation method of the data channel C0 and the weight channel C0 described above, and will not be repeated here.

[0131] For example, Figure 5AAs shown, further, assuming that the data type of the input data and the weight data is the first data type BF16, and the data type of the deep convolution operation result of the calculated target basic block data is the second data type FP32, the deep convolution operation result can be converted from the second data type FP32 to the first data type BF16 to make the data type of the output data consistent with the data type of the input data and the weight data.

[0132] For example, Figure 5A As shown, further, the next target basic block data of the input data can be loaded into the target register, and the weights in the two weight channels C0 and C1 loaded into the target register (each weight channel includes 9 weights w0~w8) are reused to perform multiplication and addition operations similar to the first target basic block data described above. For example, for the second target basic block data loaded into the target register, in the data channels C0 and C1, the first data group contains data elements p31~p62, the second data group contains data elements p32~p63,..., and the ninth data group contains data elements p69~p100. For example, the multiplication and addition operations between the 9 data groups of the second target basic block data and the reused weight data can refer to the description of the first target basic block data above, and will not be repeated here.

[0133] For example, Figure 5A As shown, further, the two depth convolution operation results of the 18 data groups of the data channels C0 and C1 are merged (for example, accumulation operation, etc.), and the first output data of the data channels C0 and C1 can be obtained; then the first output data is output, and the output first output data is stored in the shared memory inside the chip.

[0134] It should be noted that Figure 5A and Figure 5B The scenario and implementation method shown are only examples, and the specific scenario and implementation method may also be selected according to actual conditions, which is not limited in the embodiments of the present disclosure.

[0135] At least one embodiment of the present disclosure provides a method for implementing a deep convolution operator in a chip. By using a target register within the chip to group the input data and weight data of the deep convolution operator, parallel operations of deep convolution of multiple groups of data in multiple data channels can be achieved, thereby improving the execution efficiency of the deep convolution calculation tasks and enhancing the computing performance of the chip.

[0136] Figure 6 A schematic block diagram of an apparatus for implementing a deep convolution operator in a chip is provided for at least one embodiment of the present disclosure.

[0137] For example, Figure 6As shown, at least one embodiment of the present disclosure provides a device for implementing a deep convolution operator in a chip. For example, the chip includes N target registers, where N is a positive integer; the input data of the deep convolution operator includes multiple data channels, and multiple data elements in each of the multiple data channels are divided into multiple data groups; the weight data of the deep convolution operator includes multiple weight channels, and each of the multiple weight channels includes multiple weights.

[0138] For example, Figure 6 As shown, the device 200 for implementing the depth convolution operator includes a loading module 210 and a calculation module 220.

[0139] For example, the loading module 210 can be configured to select M data channels from a plurality of data channels, and load the N data groups in each of the M data channels into the N target registers in a one-to-one correspondence. For example, the loading module 210 is also configured to select M weight channels from a plurality of weight channels, and load the N weights in each of the M weight channels into the N target registers in a one-to-one correspondence. For example, the M data channels include the 1st data channel to the Mth data channel, and the M weight channels include the 1st weight channel to the Mth weight channel, where M is a positive integer. That is, the loading module 210 can be configured to execute, for example Figure 2 Steps S110 and S120 are shown.

[0140] For example, the calculation module 220 may be configured to perform a multiplication and addition operation on N data groups in the kth data channel in the M data channels and N weights in the kth weight channel in the M weight channels in the N target registers to obtain a first operation result, where k = 1, 2, …, M. That is, the calculation module 220 may be configured to perform, for example Figure 2 Step S130 is shown.

[0141] In some examples, M = 2, N = 2.

[0142] In some examples, the depthwise convolution operator is a non-stride depthwise convolution operator, and the depthwise convolution operator is implemented in a vector processing core of the chip.

[0143] In some examples, the computing module 220 can also be configured to: perform multiplication and addition operations on the N data groups in the kth data channel and the N weights in the kth weight channel one by one to obtain N first multiplication and addition results; and accumulate the N first multiplication and addition results to obtain a first operation result.

[0144] For example, in response to the data type of the input data and the weight data being a first data type, before performing multiplication and addition operations on the N data groups in the kth data channel and the N weights in the kth weight channel one by one, the calculation module 220 can also be configured to: convert the data type of the N data groups in the kth data channel and the data type of the N weights in the kth weight channel from the first data type to a second data type.

[0145] For example, the first data type is a BF16 data type, and the second data type is a FP32 data type.

[0146] In some examples, in response to the N data groups in the kth data channel including the i+1th data group to the i+Nth data group in the kth data channel, and the N weights in the kth weight channel including the i+1th weight to the i+Nth weight in the kth weight channel, the calculation module 220 can also be configured to: accumulate the first operation result and the second operation result in the N target registers to obtain a third operation result. For example, the second operation result is the multiplication and addition operation result of the 1st data group to the ith data group in the kth data channel and the 1st weight to the ith weight in the kth weight channel, where i is a positive integer.

[0147] For example, the input data of the deep convolution operator includes a plurality of basic block data, the plurality of basic block data includes target basic block data, the target basic block data includes Y data groups in the k-th data channel, where Y is an integer greater than or equal to N. For example, in response to i+N=Y, the third operation result is used as the deep convolution operation result of the Y data groups corresponding to the target basic block data in the k-th data channel.

[0148] For example, in response to the data types of the input data and the weight data being the first data type, and the deep convolution operation result being the second data type, the calculation module 220 may also be configured to convert the deep convolution operation result from the second data type to the first data type.

[0149] For example, the calculation module 220 can also be configured to: obtain first output data of the M data channels based on at least one deep convolution operation result of at least one data group of the M data channels; and output the first output data.

[0150] For example, the apparatus 200 for implementing a deep convolution operator in a chip may further include a storage module 230 (not shown in the figure). For example, the storage module 230 may be configured to store the output first output data in a shared memory inside the chip.

[0151] In some examples, input data of the deep convolution operator includes multiple basic block data, the multiple basic block data includes target basic block data, and the target basic block data includes Y data groups in the Kth data channel, where Y is an integer greater than or equal to N. For example, before selecting M data channels from the multiple data channels, the loading module 210 can also be configured to: load and store the multiple basic block data into a shared memory inside the chip.

[0152] In some examples, before selecting M weight channels from the multiple weight channels, the loading module 210 may also be configured to load and store the weight data of the deep convolution operator into a shared memory inside the chip.

[0153] Since in the above description, for example Figure 2 In the process of implementing the method of the deep convolution operator in the chip shown in FIG. 1 , the details of the operation of the apparatus 200 for implementing the deep convolution operator have been introduced, so for the sake of brevity, they will not be repeated here. For the relevant details, please refer to the above description of Figure 1 to Figure 5B etc.

[0154] It should be noted that Figure 6 Each functional module in the device 200 for implementing the deep convolution operator shown can be configured as software, hardware, firmware or any combination of the above items to perform specific functions. For example, these modules may correspond to dedicated integrated circuits, pure software codes, or modules that combine software and hardware. In addition, it is clear to those skilled in the art that the processing performed by each module can also be performed without any specific module division or clear demarcation between the modules in the device 200 for implementing the deep convolution operator. As an example, refer to Figure 6 The described device 200 for implementing the depthwise convolution operator may be a PC computer, a tablet device, a personal digital assistant, a smart phone, a web application or other device capable of executing program instructions, but is not limited thereto.

[0155] At least one embodiment of the present disclosure further provides an electronic device, which includes one or more processors and one or more memories; the one or more memories include one or more computer program modules; the one or more computer program modules are stored in the one or more memories and are configured to be executed by one or more processors, and the one or more computer program modules include a method for implementing a deep convolution operator in a chip provided by the embodiment of the present disclosure described above. For example, the one or more processors may be a single-core processor or a multi-core processor.

[0156] Figure 7 A schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure.

[0157] For example, Figure 7 As shown, the electronic device 300 includes a processor 310 and a memory 320. For example, the memory 320 is used to store non-transitory computer-readable instructions (e.g., one or more computer program modules). The processor 310 is used to run non-transitory computer-readable instructions, and when the non-transitory computer-readable instructions are run by the processor 310, one or more steps of the method for implementing a deep convolution operator in a chip described above can be executed. The memory 320 and the processor 310 can be interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0158] For example, the processor 310 may be a central processing unit (CPU), a graphics processing unit (GPU), a general purpose graphics processing unit (GPGPU), a digital signal processor (DSP), or other forms of processing units, such as a field programmable gate array (FPGA), etc.; for example, the central processing unit (CPU) may be an X86, RISC-V, or ARM architecture, etc. The processor 310 may be a general-purpose processor or a dedicated processor, and may control other components in the electronic device 300 to perform desired functions.

[0159] For example, the memory 320 may include any combination of one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and the processor 310 may run one or more computer program modules to implement various functions of the electronic device 300. Various applications and various data, as well as various data used and / or generated by the application, etc. may also be stored in the computer-readable storage medium.

[0160] It should be noted that in the embodiments of the present disclosure, the specific functions and technical effects of the electronic device 300 can refer to the above description of the method for implementing a deep convolution operator in a chip provided by at least one embodiment of the present disclosure, and will not be repeated here.

[0161] Figure 8 A schematic block diagram of another electronic device provided for at least one embodiment of the present disclosure.

[0162] For example, Figure 8As shown, the electronic device 400 is suitable for implementing the method for implementing a deep convolution operator in a chip provided by an embodiment of the present disclosure. It should be noted that, Figure 8 The electronic device 400 shown is only an example and does not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0163] For example, Figure 8 As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 41, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 42 or a program loaded from a storage device 48 to a random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44. Generally, the following devices can be connected to the I / O interface 45: an input device 46 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 47 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 48 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 49. The communication device 49 can allow the electronic device 400 to communicate with other electronic devices wirelessly or by wire to exchange data.

[0164] Although Figure 8 The electronic device 400 is shown to have various devices, but it should be understood that it is not required to implement or possess all the devices shown, and the electronic device 400 may alternatively implement or possess more or fewer devices.

[0165] For the detailed description and technical effects of the electronic device 400, please refer to the above description of the method for implementing a deep convolution operator in a chip, which will not be repeated here.

[0166] Fig. 9 A schematic diagram of a storage medium provided for at least one embodiment of the present disclosure.

[0167] For example, Fig. 9 As shown, the storage medium 500 stores non-transitory computer-readable instructions 510. For example, when the non-transitory computer-readable instructions 510 are executed by a computer, one or more steps in the method for implementing a depthwise convolution operator in a chip described above are performed.

[0168] For example, the storage medium 500 can be applied to Figure 7For example, the storage medium 500 may be the memory 320 in the electronic device 300. For example, the description of the storage medium 500 may refer to Figure 7 The corresponding description of the memory 320 in the electronic device 300 is shown and will not be repeated here.

[0169] There are a few points to note about this disclosure:

[0170] (1) In the drawings of the embodiments of the present disclosure, only the structures related to the embodiments of the present disclosure are involved, and other structures can refer to the general design.

[0171] (2) Unless there is a conflict, the features of the same embodiment or different embodiments of the present disclosure may be combined with each other.

[0172] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A method for implementing a deep convolution operator in a chip, characterized in that: The chip includes N target registers, where N is a positive integer. The input data of the depth convolution operator includes a plurality of data channels, and a plurality of data elements in each of the plurality of data channels are divided into a plurality of data groups. In response to the thread warp of the chip including n threads, each data group of each data channel includes n data elements corresponding to the n threads one by one, where n is a positive integer, The weight data of the deep convolution operator includes a plurality of weight channels, each of the plurality of weight channels includes a plurality of weights, The method for implementing a deep convolution operator comprises: Selecting M data channels from the plurality of data channels, loading the N data groups in each of the M data channels into the N target registers one by one, and Selecting M weight channels from the multiple weight channels, and loading N weights in each weight channel of the M weight channels into the N target registers one by one, where M is a positive integer; In the N target registers, multiplication and addition operations are performed on the N data groups in the kth data channel in the M data channels and the N weights in the kth weight channel in the M weight channels to obtain a first operation result, where k=1, 2,…,M.

2. The method for implementing a deep convolution operator in a chip according to claim 1, characterized in that: The performing a multiplication and addition operation on the N data groups in the kth data channel in the M data channels and the N weights in the kth weight channel in the M weight channels to obtain a first operation result includes: Perform multiplication and addition operations on the N data groups in the kth data channel and the N weights in the kth weight channel in a one-to-one correspondence to obtain N first multiplication and addition results; The N first multiplication and addition results are accumulated to obtain the first operation result.

3. The method for implementing a deep convolution operator in a chip according to claim 2, characterized in that: In response to the data types of the input data and the weight data being the first data type, before performing multiplication and addition operations on the N data groups in the kth data channel and the N weights in the kth weight channel in a one-to-one correspondence, The performing multiplication and addition operations on the N data groups in the kth data channel in the M data channels and the N weights in the kth weight channel in the M weight channels to obtain a first operation result also includes: The data types of the N data groups in the k-th data channel and the data types of the N weights in the k-th weight channel are converted from the first data type to a second data type.

4. The method for implementing a deep convolution operator in a chip according to claim 3, characterized in that: The first data type is a BF16 data type, and the second data type is a FP32 data type.

5. The method for implementing a deep convolution operator in a chip according to claim 1, characterized in that: In response to the N data groups in the kth data channel including the i+1th data group to the i+Nth data group in the kth data channel, and the N weights in the kth weight channel including the i+1th weight to the i+Nth weight in the kth weight channel, The method for implementing a deep convolution operator also includes: In the N target registers, the first operation result and the second operation result are accumulated to obtain a third operation result, The second operation result is the multiplication and addition result of the 1st data group to the ith data group in the kth data channel and the 1st weight to the ith weight in the kth weight channel, wherein i is a positive integer.

6. The method for implementing a deep convolution operator in a chip according to claim 5, characterized in that: The input data of the deep convolution operator includes a plurality of basic block data, the plurality of basic block data includes target basic block data, the target basic block data includes Y data groups in the k-th data channel, wherein Y is an integer greater than or equal to N, In response to i+N=Y, the third operation result is used as the depth convolution operation result of the Y data groups corresponding to the target basic block data in the kth data channel.

7. The method for implementing a deep convolution operator in a chip according to claim 6, characterized in that: In response to the data types of the input data and the weight data being the first data type, and the depth convolution operation result being the second data type, The method for implementing a deep convolution operator also includes: The depthwise convolution operation result is converted from the second data type to the first data type.

8. The method for implementing a deep convolution operator in a chip according to claim 6, characterized in that: The method for implementing a deep convolution operator also includes: Based on at least one depth convolution operation result of at least one data group of the M data channels, obtaining first output data of the M data channels; The first output data is output.

9. The method for implementing a deep convolution operator in a chip according to claim 8, characterized in that: The method for implementing a deep convolution operator also includes: The outputted first output data is stored in a shared memory inside the chip.

10. The method for implementing a deep convolution operator in a chip according to claim 1, characterized in that: The input data of the deep convolution operator includes a plurality of basic block data, the plurality of basic block data includes target basic block data, the target basic block data includes Y data groups in the k-th data channel, wherein Y is an integer greater than or equal to N, Before selecting M data channels from the multiple data channels, the method for implementing a depth convolution operator further includes: The plurality of basic block data are loaded and stored into a shared memory inside the chip.

11. The method for implementing a deep convolution operator in a chip according to claim 1, characterized in that: Before selecting M weight channels from the multiple weight channels, the method for implementing a deep convolution operator further includes: The weight data of the deep convolution operator is loaded and stored into the shared memory inside the chip.

12. The method for implementing a deep convolution operator in a chip according to any one of claims 1 to 11, characterized in that: The depth convolution operator is a non-strided depth convolution operator, and the depth convolution operator is implemented in the vector processing core of the chip.

13. An electronic device, characterized in that: The electronic device comprises: one or more processors; one or more memories including one or more computer program modules; The one or more computer program modules are stored in the one or more memories and configured to be executed by the one or more processors, and the one or more computer program modules are used to implement the method for implementing a deep convolution operator in a chip as described in any one of claims 1-12.

14. A storage medium, characterized in that: The storage medium stores non-transitory computer-readable instructions, and when the non-transitory computer-readable instructions are executed by a computer, the method for implementing a deep convolution operator in a chip as described in any one of claims 1-12 is implemented.

Images