Convolution operator deployment method and device, equipment, storage medium and program product

Abstract

The invention provides a convolution operator deployment method and device, equipment, a storage medium and a program product, and relates to the technical field of computation.The convolution operator deployment method comprises the steps that a weight value in a convolution kernel is split, and convolution kernel splitting information is obtained; the convolution kernel splitting information comprises N initial weight value combinations, and copying each initial weight value combination to obtain at least one copied weight value combination corresponding to each initial weight value combination; deploying each initial weight value combination and the corresponding at least one copy weight value combination on a target array in a computing array of the chip; one target array comprises N sub-arrays; one weight value combination is deployed in one column of the subarray, any two adjacent columns of weight value combinations are deployed in a dislocation mode, the row occupied space of a calculation array of the chip is reduced, the number of times of repeatedly reading input data is reduced, and the storage function requirement and operation power consumption of the chip are reduced.

Description

Technical Field

[0001] This invention relates to the field of computing technology, and in particular to a method, apparatus, device, storage medium, and program product for deploying convolution operators. Background Technology

[0002] With the development and application of deep learning technology, in-memory computing chips have emerged to address the problems of frequent, slow, and energy-intensive data movement between computing and storage units. These chips shift from a computation-centric architecture to a data-centric architecture, organically integrating storage and computing functions and directly utilizing storage units for data processing. In-memory computing chips fundamentally solve the "memory wall" and "power wall" problems inherent in traditional von Neumann architecture chips.

[0003] In the current technological development roadmap of in-memory computing chips, in-memory computing chips are developing rapidly because they have significant advantages over traditional chips in terms of energy efficiency, area efficiency, computing speed, sparsity support, manufacturing cost, and adaptability to artificial intelligence (AI) algorithm models. A large part of the computing function of in-memory computing chips is to support the operation of convolution operators. Currently popular deep learning network models, especially vision-related algorithms, rely heavily on convolution operators.

[0004] Existing in-memory computing chips on the market deploy convolution weights by converting the convolution operators in standard deep learning models into a two-dimensional rectangular format supported by the chip, and directly using this matrix to perform convolution operations within memory units using in-memory computing mode, reducing data movement and improving energy efficiency. Their convolution operator conversion methods all use planar complete two-dimensional rectangular conversions, and the input data is rearranged according to the convolution unfolding method. This conversion method encounters the problem of repeated data reading and calculation during computation.

[0005] Existing methods for deploying convolution operators in in-memory computing chips involve unfolding the weights of the convolution operator by channel and assembling them into a two-dimensional computation matrix. This weight matrix is ​​then directly programmed into the in-memory computing chip as a computation array. During computation, the input data from the sliding window is rearranged and input to a designated location on the computation array of the in-memory computing chip, and the output result is retrieved from the bottom of the computation array. When parallel computation of multiple sliding windows is desired, the weight values ​​need to be copied to the lower layer for parallel computation. This method consumes too many rows of the computation array on the in-memory computing chip and wastes columns. Furthermore, the sliding window computation method can lead to repeated data readings, increasing computation time and memory usage, and reducing computational efficiency. Summary of the Invention

[0006] This invention provides a method, apparatus, device, storage medium, and program product for deploying convolution operators, in order to solve the problems of large memory usage and high power consumption of chips in existing convolution calculation processes.

[0007] To address the aforementioned technical problems, the embodiments of the present invention provide the following technical solutions:

[0008] In a first aspect, embodiments of the present invention provide a method for deploying a convolution operator, the method comprising:

[0009] The weights in a convolution kernel are split to obtain convolution kernel splitting information; wherein, a convolution kernel includes K sub-convolution kernels of different dimensions, and each sub-convolution kernel of different dimensions includes weights in N rows and M columns; the convolution kernel splitting information includes N initial weight value combinations after splitting, each initial weight value combination includes M weight value sub-combinations, and each weight value sub-combination includes the weights in the nth row and mth column of each sub-convolution kernel of different dimensions;

[0010] Each of the initial weight value combinations is copied to obtain at least one copied weight value combination corresponding to each initial weight value combination;

[0011] Each of the initial weight value combinations and the corresponding at least one replicated weight value combination are deployed on a target array in the chip's computing array;

[0012] One of the target arrays comprises N subarrays;

[0013] An initial weight value combination and a corresponding at least one replicated weight value combination are deployed in one of the subarrays, and each weight value combination in the initial weight value combination and the corresponding at least one replicated weight value combination is deployed sequentially in each column of the subarray, and a weight value combination is deployed in one column of the subarray, and any two adjacent columns of weight value combinations are deployed in a staggered manner;

[0014] Where K is a positive integer, N and M are both positive integers greater than 1, n is a positive integer greater than 0 and less than or equal to N, and m is a positive integer greater than 0 and less than or equal to M.

[0015] Optionally, in one of the weight value sub-combinations, the weight values ​​of the nth row and mth column in the sub-convolution kernel of each dimension are arranged in a row in sequence;

[0016] In one of the initial weight value combinations, the weight values ​​in each of the weight value sub-combinations are arranged sequentially in the same row.

[0017] Optionally, each of the initial weight value combinations is copied to obtain at least one copied weight value combination corresponding to each initial weight value combination, including:

[0018] For each of the initial weight value combinations, N-1 copies are performed to obtain N-1 copied weight value combinations corresponding to each initial weight value combination.

[0019] Optionally, one of the target arrays comprises N subarrays arranged from top to bottom;

[0020] In this configuration, each of the N initial weight value combinations and its corresponding at least one replicated weight value combination is sequentially deployed in each of the N subarrays; one initial weight value combination and its corresponding at least one replicated weight value combination are deployed in each subarray.

[0021] Optionally, deploying each of the initial weight value combinations and the corresponding at least one replicated weight value combination on a target array within the chip's computing array includes:

[0022] Transpose one row of weight values ​​in one of the initial weight value combinations, and transpose one row of weight values ​​in each of the corresponding at least one replicated weight value combinations to obtain at least two columns of transposed weight values ​​corresponding to one of the initial weight value combinations.

[0023] Each of the at least two columns of transposed weight values ​​is sequentially deployed in at least two columns of a subarray, wherein one column of the transposed weight values ​​is deployed in one column of a subarray, and any two adjacent columns of transposed weight values ​​are deployed in a dislocation manner.

[0024] Optionally, the number of rows of one of the subarrays is Q, where Q = (2M-1) × K, and the number of columns of one of the subarrays is equal to M;

[0025] In one of the subarrays, the first weight value of the transposed weight value of the first target column is deployed in the i-th row of the subarray, and the first weight value of the transposed weight value of the second target column is deployed in the j-th row of the subarray, K = ji, i < j, i is a positive integer greater than 0 and less than Q, and j is a positive integer greater than 0 and less than Q; wherein, the transposed weight values ​​of the first target column and the transposed weight values ​​of the second target column are the transposed weight values ​​of any two adjacent columns in one of the subarrays;

[0026] The first transpose weight value in the first column of the subarray is deployed in the first row and first column of the subarray.

[0027] Optionally, a value of 0 may be deployed at the positions in the subarray where the transpose weight value is not deployed.

[0028] Optionally, the number of convolution kernels is P;

[0029] The computing array includes P target arrays, which are arranged from left to right in the computing array.

[0030] Where P is a positive integer.

[0031] Optionally, the initial weight value combination and the at least one replicated weight value combination corresponding to each of the P convolutional kernels are sequentially deployed on each of the P target arrays;

[0032] The initial weight value combination corresponding to the convolution kernel and the at least one duplicate weight value combination are deployed on the target array.

[0033] Optionally, the method further includes:

[0034] The input data combination is obtained from the input feature map; wherein the input feature map includes K-dimensional sub-input feature maps, each sub-input feature map includes at least one row and at least one column of input data; the input data combination includes N sub-input data combinations, and one sub-input data combination includes one row and X columns of input data from each dimension of the K-dimensional sub-input feature map;

[0035] Each of the input data in the input data combination is deployed on a column in the input channel array.

[0036] Optionally, in one column of the input channel array, N sub-input data combinations are arranged sequentially from top to bottom;

[0037] The nth sub-input data combination among the N sub-input data combinations includes the first X input data in the nth row of the sub-input feature map for each dimension;

[0038] The nth sub-input data combination includes X first input data combinations arranged in sequence;

[0039] The xth first input data combination in the nth sub-input data combination includes the xth input data in the nth row of the sub-input feature map arranged sequentially in each dimension;

[0040] Where n is a positive integer greater than 0 and less than or equal to N; x is a positive integer greater than 0 and less than or equal to X.

[0041] Optionally, the method further includes:

[0042] According to the matrix calculation method, the output data corresponding to the target array is obtained based on the transpose weight values ​​deployed in the N subarrays of the target array and the input data in a column of the input channel array.

[0043] Secondly, embodiments of the present invention also provide a convolution operator deployment apparatus, the apparatus comprising:

[0044] The first processing module is used to split the weight values ​​in a convolutional kernel to obtain convolutional kernel splitting information; wherein, a convolutional kernel includes K sub-convolutional kernels of different dimensions, and each sub-convolutional kernel of different dimensions includes N rows of weight values ​​and M columns of weight values; the convolutional kernel splitting information includes N initial weight value combinations after splitting, each initial weight value combination includes M weight value sub-combinations, and each weight value sub-combination includes the weight values ​​of the nth row and mth column in each sub-convolutional kernel of different dimensions;

[0045] The second processing module is used to copy each of the initial weight value combinations to obtain at least one copied weight value combination corresponding to each of the initial weight value combinations.

[0046] The third processing module is used to deploy each of the initial weight value combinations and the corresponding at least one replicated weight value combination on a target array in the chip's computing array;

[0047] One of the target arrays comprises N subarrays;

[0048] An initial weight value combination and a corresponding at least one replicated weight value combination are deployed in a subarray. Each weight value combination in the initial weight value combination and the corresponding at least one replicated weight value combination is deployed sequentially in each column of the subarray. A weight value combination is deployed in one column of the subarray, and any two adjacent columns of weight value combinations are deployed in a staggered manner.

[0049] Where P is a positive integer, N and M are both positive integers greater than 1, n is a positive integer greater than 0 and less than or equal to N, and m is a positive integer greater than 0 and less than or equal to M.

[0050] Thirdly, embodiments of the present invention also provide a convolution operator deployment device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the convolution operator deployment method as described in any one of the first aspects.

[0051] Fourthly, embodiments of the present invention also provide a readable storage medium storing a program that, when executed by a processor, implements the steps of the convolution operator deployment method as described in any one of the first aspects.

[0052] Fifthly, embodiments of the present invention also provide a computer program product, including computer instructions, which, when executed by a processor, implement the steps in the convolution operator deployment method as described in any one of the first aspects.

[0053] The beneficial effects of this invention are:

[0054] The convolution operator deployment method provided by this invention involves splitting the weight values ​​in a convolution kernel comprising a sub-convolution kernel with k sub-dimensions to obtain convolution kernel splitting information. Each sub-convolution kernel of a given dimension includes N rows and M columns of weight values. This splitting information includes N initial weight value combinations, where each initial weight value combination includes M weight value sub-combinations. Each weight value sub-combination includes the weight values ​​in the nth row and mth column of each sub-convolution kernel. Each initial weight value combination is copied to obtain at least one copied weight value combination corresponding to each initial weight value combination. Each initial weight value combination and its corresponding at least one copied weight value combination are then deployed on a target array within a chip's computing array. The standard array includes N subarrays. During deployment, an initial weight value combination and at least one corresponding duplicate weight value combination are deployed in each subarray. Each weight value combination in the initial weight value combination and at least one corresponding duplicate weight value combination is deployed sequentially in each column of the subarray, and a weight value combination is deployed in one column of the subarray. Furthermore, any two adjacent columns of weight value combinations are deployed in a staggered manner. By copying and deploying the weight values ​​in the convolution kernel as described above, compared with the existing method of placing the copied weight values ​​in the lower layer, the row space occupied by the chip's computing array can be significantly reduced. Moreover, when performing convolution calculations with weight values ​​deployed in the above manner, the number of times input data is repeatedly read can be reduced, thereby reducing the chip's storage requirements and computational power consumption. Attached Figure Description

[0055] Figure 1 A schematic diagram illustrating the operation method of the single-channel convolution operator provided by the present invention;

[0056] Figure 2 This is a schematic diagram showing the expanded sliding window data and weight values ​​provided by the present invention;

[0057] Figure 3 This is a schematic diagram showing the expanded sliding window data and the copied weight values ​​provided by the present invention;

[0058] Figure 4 A flowchart illustrating the convolution operator deployment method provided in this embodiment of the invention;

[0059] Figure 5 This is a schematic diagram of one of the structures of the convolution kernel provided in an embodiment of the present invention;

[0060] Figure 6 This is the second schematic diagram of the structure of the convolution kernel provided in the embodiment of the present invention;

[0061] Figure 7 One of the schematic diagrams illustrating a subarray after deployment according to an embodiment of the present invention;

[0062] Figure 8 One of the schematic diagrams showing the target array after deployment according to an embodiment of the present invention;

[0063] Figure 9 This is the second schematic diagram illustrating the subarray after deployment according to an embodiment of the present invention.

[0064] Figure 10 This is the second schematic diagram illustrating the target array after deployment according to an embodiment of the present invention;

[0065] Figure 11 This is the third schematic diagram illustrating the structure of the convolution kernel provided in this embodiment of the invention;

[0066] Figure 12 This is the third schematic diagram illustrating the target array after deployment, as provided in this embodiment of the invention.

[0067] Figure 13 This represents one of the structural schematic diagrams of the input feature map provided in an embodiment of the present invention;

[0068] Figure 14 This is the second schematic diagram illustrating the structure of the input feature map provided in this embodiment of the invention.

[0069] Figure 15 This is one of the structural schematic diagrams of the input channel array provided in an embodiment of the present invention;

[0070] Figure 16 This is the second schematic diagram of the structure of the input channel array provided in the embodiment of the present invention;

[0071] Figure 17 This diagram illustrates the matrix calculation provided in an embodiment of the present invention.

[0072] Figure 18 This is a schematic diagram of the structure of the convolution operator deployment device provided in an embodiment of the present invention;

[0073] Figure 19 This is a schematic diagram showing the structure of the convolution operator deployment device provided in an embodiment of the present invention. Detailed Implementation

[0074] To make the technical problems, technical solutions, and advantages of this application clearer, a detailed description will be provided below in conjunction with the accompanying drawings and specific embodiments. In the following description, specific details such as particular configurations and components are provided merely to aid in a comprehensive understanding of the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Furthermore, for clarity and brevity, descriptions of known functions and structures have been omitted.

[0075] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0076] In the various embodiments of this application, it should be understood that the sequence number of each process described below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0077] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, without limiting the number of objects; for example, the first object can be one or more. Furthermore, "or" in this application indicates at least one of the connected objects. For example, "A or B" covers three scenarios: Scenario 1: including A but not B; Scenario 2: including B but not A; Scenario 3: including both A and B. The character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0078] The term "instruction" in this application can be either a direct instruction (or explicit instruction) or an indirect instruction (or implicit instruction). A direct instruction can be understood as one in which the sender explicitly informs the receiver of specific information, the operation to be performed, or the requested result, etc., in the instruction sent. An indirect instruction can be understood as one in which the receiver determines the corresponding information based on the instruction sent by the sender, or makes a judgment and determines the operation to be performed or the requested result, etc., based on the judgment result.

[0079] Before describing the specific embodiments of the present invention, the following will be explained first:

[0080] Figure 1 This is a schematic diagram illustrating the operation method of a single-channel convolution operator. Each calculation involves extracting sliding window data from the input feature map. The initial sliding window data can be selected by sliding the window (represented by the red box) within the input feature map. Figure 1 The numbers included in the blue matrix corresponding to "In1" represent... Figure 1 The letters in the white matrix represent the weights in the convolutional kernel, which is a one-dimensional 3x3 matrix (kernel 3x3). The kernel has one input channel (in_channel = 1) and one output channel (out_channel = 1). During the initial computation, it is performed according to... Figure 1 The calculation method shown, "O1 = 1A + 2B + 3C + 4D + 5E + 6F + 7G + 8H + 9I", yields the result O1. Then, sliding the window one unit to the right retrieves the sliding window data represented by the numbers in the matrix corresponding to the red box "In2". Figure 1 The calculation method shown is "O2=2A+3B+10C+5D+6E+11F+8G+9H+12I". The result is O2. Then, the slider moves one unit to the right to obtain the slider data represented by the numbers in the matrix corresponding to the red box "In3". This is done according to... Figure 1 The calculation method shown is "O3=3A+10B+13C+6D+11E+14F+9G+12H+15I", which calculates the result O3... After each calculation result is obtained, the calculation result is written into the output feature map in sequence.

[0081] According to such Figure 1 The sliding window calculation method shown can result in repeated reading of input data, i.e. Figure 1 The numbers included in the yellow matrix represent the input data that is repeatedly read.

[0082] Figure 1 The corresponding single calculation involves expanding the sliding window data and weight values ​​before calculation. A diagram illustrating the expanded sliding window data and weight values ​​is shown below. Figure 2 As shown in the diagram. During parallel computation, the expanded diagram of the sliding window data and the copied weight values ​​is shown below. Figure 3 As shown.

[0083] To address the issues of high memory consumption and high power consumption in existing convolution calculation processes, this invention provides a convolution operator deployment method, apparatus, device, storage medium, and program product.

[0084] like Figure 4 As shown, this embodiment of the invention provides a method for deploying a convolution operator, the method comprising:

[0085] Step 401: Split the weight values ​​in a convolution kernel to obtain convolution kernel splitting information; wherein, a convolution kernel includes K sub-convolution kernels of different dimensions, and each sub-convolution kernel of different dimensions includes weight values ​​in N rows and M columns; the convolution kernel splitting information includes N initial weight value combinations after splitting, each initial weight value combination includes M weight value sub-combinations, and each weight value sub-combination includes the weight values ​​in the nth row and mth column of each sub-convolution kernel of different dimensions.

[0086] Where K is a positive integer, N and M are both positive integers greater than 1, n is a positive integer greater than 0 and less than or equal to N, and m is a positive integer greater than 0 and less than or equal to M.

[0087] It should be noted that the convolution kernel can also be called a convolution operator. This convolution kernel is the one to be reconstructed.

[0088] In this step, the kernel size information of the convolution kernel is obtained. This kernel size information is used to indicate the size of the sub-convolution kernel in each dimension of the convolution kernel, the number of input channels of the convolution kernel, and the number of output channels of the convolution kernel. The number of input channels of the convolution kernel and the number of dimensions of the convolution kernel are the same. That is, when the number of input channels of the convolution kernel is K, a convolution kernel includes K sub-convolution kernels in sub-dimensions, and the size of each sub-convolution kernel is the same. Each sub-convolution kernel in each dimension includes N rows and M columns of weight values. It can be understood that the number of rows N and the number of columns N in the weight values ​​of each sub-convolution kernel can be understood as the size of the sub-convolution kernel.

[0089] For example, the structure of a convolutional kernel with K equal to 1, 1 input channel (in_channel = 1), and 1 output channel (out_channel = 1) is as follows: Figure 5 As shown, the sub-convolutional kernel includes 3 rows × 3 columns of weight values ​​(Kernel3x3). Figure 5 The letter "AI" in the red matrix represents the weight value.

[0090] For another example, the structure of the 20 sub-convolutional kernels in a convolutional kernel with K greater than 1, such as K=20, 20 input channels (in_channel=20), and 1 output channel (out_channel=1), is as follows: Figure 6As shown, each sub-kernel includes 3 rows × 3 columns of weight values ​​(Kernel 3x3). Figure 6 In the matrix, weight values ​​are represented by combinations of letters and numbers (e.g., "A2", "C20"). In the matrix of the same color, combinations of letters and numbers (e.g., "A2", "C2") represent the weight values ​​included in a sub-convolution kernel of one dimension of an input channel. Different colored matrices represent different sub-convolution kernels.

[0091] In this step, after splitting a convolutional kernel, convolutional kernel splitting information is obtained. This information includes N initial weight value combinations. It can be understood that the convolutional kernel is split in a row-major logical order. That is, each row of the convolutional kernel (which includes each corresponding row of the sub-convolutional kernels, such as the first row of the convolutional kernel including the first row of each sub-convolutional kernel, the second row of the convolutional kernel including the second row of each sub-convolutional kernel, and so on) corresponds to an initial weight value combination.

[0092] An initial weight value combination comprises M weight value sub-combinations, each of which includes the weight values ​​of the nth row and mth column of each sub-convolutional kernel in each dimension. It can be understood that each of the N initial weight value combinations comprises M weight value sub-combinations. For example, the first weight value sub-combination in the initial weight value combination corresponding to the first row of the convolutional kernel includes the weight values ​​of the first column of the first row of each sub-convolutional kernel; the second weight value sub-combination in the initial weight value combination corresponding to the first row of the convolutional kernel includes the weight values ​​of the second column of the first row of each sub-convolutional kernel; and so on, the third weight value sub-combination in the initial weight value combination corresponding to the third row of the convolutional kernel includes the weight values ​​of the third column of the third row of each sub-convolutional kernel…

[0093] Specifically, in implementation, the weight values ​​of the first column of the first row of each dimension of the convolution kernel can be combined to obtain the first sub-combination of weight values ​​in the initial weight value combination corresponding to the first row of the convolution kernel; the weight values ​​of the second column of the first row of each dimension of the convolution kernel can be combined to obtain the second sub-combination of weight values ​​in the initial weight value combination corresponding to the first row of the convolution kernel; and so on. The weight values ​​of the first column of the third row of each dimension of the convolution kernel can be combined to obtain the first sub-combination of weight values ​​in the initial weight value combination corresponding to the third row of the convolution kernel; and the weight values ​​of the second column of the third row of each dimension of the convolution kernel can be combined to obtain the second sub-combination of weight values ​​in the initial weight value combination corresponding to the third row of the convolution kernel; and so on.

[0094] For example, when K equals 1, the structure of the convolution kernel is as follows: Figure 5 As shown, for Figure 5After splitting the convolution kernel shown, three initial weight value combinations are obtained. Each initial weight value combination includes three weight value sub-combinations. Following the above logic, the first weight sub-combination in the initial weight value combination corresponding to the first row of the convolution kernel includes A, the second weight sub-combination in the initial weight value combination corresponding to the first row of the convolution kernel includes B, and the third weight sub-combination in the initial weight value combination corresponding to the first row of the convolution kernel includes C. Therefore, the initial weight value combinations corresponding to the first row of the convolution kernel include A, B, and C, and the first weight sub-combination in the initial weight value combination corresponding to the second row of the convolution kernel includes D, ... If the second weight sub-combination in the initial weight value combination corresponding to the second row of the kernel includes E, and the third weight sub-combination in the initial weight value combination corresponding to the second row of the convolution kernel includes F, then the initial weight value combination corresponding to the second row of the convolution kernel includes D, E, and F. If the first weight sub-combination in the initial weight value combination corresponding to the third row of the convolution kernel includes G, the second weight sub-combination in the initial weight value combination corresponding to the third row of the convolution kernel includes H, and the third weight sub-combination in the initial weight value combination corresponding to the third row of the convolution kernel includes I, then the initial weight value combination corresponding to the second row of the convolution kernel includes G, H, and I.

[0095] For example, when K=20, the structure of the convolution kernel is as follows: Figure 6 As shown, for Figure 6After the convolution kernel shown is split, three initial weight value combinations are obtained. Each initial weight value combination includes three weight value sub-combinations. Following the above logic, the first weight sub-combination in the first row of the convolution kernel includes A1, A2, A3…A20; the second weight sub-combination includes B1, B2, B3…B20; and the third weight sub-combination includes C1, C2, C3…C20. Therefore, the initial weight value combinations in the first row of the convolution kernel are A1, A2, A3…A20, B1, B2, B3…B20, C1, C2, C3…C20. The first weight sub-combination in the second row of the convolution kernel is D1, D2, D3…D20. The second weight sub-combination in the initial weight value combination includes E1, E2, E3...E20. The third weight sub-combination in the initial weight value combination corresponding to the second row of the convolution kernel includes F1, F2, F3...F20. Therefore, the initial weight value combination corresponding to the second row of the convolution kernel includes D1, D2, D3...D20, E1, E2, E3...E20, F1, F2, F3...F20. The first weight sub-combination in the initial weight value combination corresponding to the third row of the convolution kernel includes G1, G2, G3. The second weight sub-combination in the initial weight value combination corresponding to the third row of the convolution kernel includes H1, H2, H3. The third weight sub-combination in the initial weight value combination corresponding to the third row of the convolution kernel includes I1, I2, I3. Therefore, the initial weight value combination corresponding to the second row of the convolution kernel includes G1, G2, G3, H1, H2, H3, I1, I2, I3.

[0096] Step 402: Copy each of the initial weight value combinations to obtain at least one copied weight value combination corresponding to each initial weight value combination.

[0097] In this step, each initial weight value combination after splitting is copied at least once, resulting in at least one copied weight value combination corresponding to each initial weight value combination. In other words, copying an initial weight value combination at least once yields at least one copied weight value combination corresponding to that initial weight value combination.

[0098] The number of times each initial weight value combination is copied is the same.

[0099] It should also be noted that the copied weight value combination obtained after copying the initial weight value combination is exactly the same as the weight values ​​in the original initial weight value combination in terms of specific values ​​and order.

[0100] Step 403: Deploy each of the initial weight value combinations and the corresponding at least one replicated weight value combination on a target array in the chip's computing array.

[0101] One of the target arrays comprises N subarrays;

[0102] An initial weight value combination and a corresponding at least one replicated weight value combination are deployed in a subarray. Each weight value combination in the initial weight value combination and the corresponding at least one replicated weight value combination is deployed sequentially in each column of the subarray. A weight value combination is deployed in one column of the subarray. Furthermore, any two adjacent columns of weight value combinations are deployed in a staggered manner.

[0103] It should be noted that the chip's computing array includes at least one target array, and the number of target arrays is the same as the number of convolution kernels, with one convolution kernel corresponding to one target array.

[0104] In this embodiment of the invention, the chip is an in-memory computing chip.

[0105] In this step, deploying each initial weight value combination and the corresponding at least one replicated weight value combination on a target array in the chip's computing array can be understood as deploying the weight values ​​obtained from a split convolution kernel (i.e., the weight values ​​included in each initial weight value combination) and its corresponding replicated weight values ​​(i.e., the weight values ​​included in each replicated weight value combination) on a target array to obtain the deployed target array.

[0106] After each target array is deployed, a completed computation array is obtained, which is then used for subsequent convolution calculations.

[0107] A target array consists of N subarrays. It can be understood that an initial weight value combination and its corresponding replicated weight value combination are deployed in a subarray.

[0108] Specifically, during deployment, within a subarray, each weight value combination (including each initial weight value combination and each replicated weight value combination) of an initial weight value combination and its corresponding at least one replicated weight value combination is deployed in different columns of the subarray, wherein a weight value combination is deployed in one column of the subarray. Furthermore, during deployment, any two adjacent columns of weight value combinations within a subarray are deployed in a staggered manner.

[0109] Compared to traditional parallel processing methods that copy weights, copying and deploying the weight values ​​from the convolution kernel to the chip using the steps described above can save on the row space occupied by the in-memory computing chip's computing array. Furthermore, after deploying the weight values ​​to the chip using these steps, parallel computation of the convolution sliding window can be achieved during convolution calculations. This allows for parallel processing of sliding window data larger than the convolution kernel size (i.e., the ratio of the size of the processing sliding window data to the convolution kernel size is greater than 1), reducing the number of times input data is repeatedly read, thus lowering the chip's storage requirements and power consumption.

[0110] In some embodiments, in a weight value sub-combination, the weight values ​​of the nth row and mth column in the sub-convolution kernel of each dimension are arranged in a row.

[0111] Here, "arranged sequentially in a row" can be understood as arranged from left to right in a row or arranged sequentially along the column direction in a row. Specifically, in a weight value sub-combination, the weight values ​​of the nth row and mth column in each dimension of the sub-convolution kernel are arranged sequentially in a row. This can be understood as arranging the weight values ​​of the nth row and mth column sequentially in a row according to the order of the input channels to which the weight values ​​belong or the arrangement order of the dimensions of the sub-convolution kernel to which the weight values ​​belong, thus obtaining a weight value sub-combination.

[0112] It should be noted that in the weight values ​​arranged in a row, there is no empty space or column between any two adjacent weight values.

[0113] For example, when K equals 1, each weight value sub-combination includes only one weight value, so that one weight value is in one row, and a weight value in another weight value sub-combination is in another row.

[0114] As another example, when K=20, the structure of the convolution kernel is as follows: Figure 6 As shown, for Figure 6After the convolution kernel is split, three initial weight value combinations are obtained. Each initial weight value combination includes three weight value sub-combinations. After splitting according to the above logic, the first weight sub-combination in the first row of the initial weight value combination of the convolution kernel includes A1, A2, A3...A20, and A1, A2, A3...A20 are arranged in a row from left to right according to the order of their respective input channels, that is, the order is maintained as A1, A2, A3...A20. The second weight sub-combination in the first row of the initial weight value combination of the convolution kernel includes B1, B2, B3...B20, and B1, B2, B3...B20 are arranged in a row from left to right according to the order of their respective input channels. In the first row of the convolution kernel, the initial weight values ​​are arranged in the following order: B1, B2, B3...B20. The third weight sub-combination in the initial weight value combination corresponding to the first row of the convolution kernel includes C1, C2, C3...C20, and C1, C2, C3...C20 are arranged in a row from left to right according to the order of their respective input channels. The third weight sub-combination in the second row of the convolution kernel includes F1, F2, F3...F20, and F1, F2, F3...F20 are arranged in a row from left to right according to the order of their respective input channels.

[0115] In one of the initial weight value combinations, the weight values ​​in each of the weight value sub-combinations are arranged sequentially in the same row.

[0116] The phrase "the weight values ​​in each sub-group of weight values ​​are arranged sequentially in the same row" can be understood as the weight values ​​in each sub-group of weight values ​​being arranged from left to right in the same row, or as the weight values ​​in each sub-group of weight values ​​being arranged sequentially in the same row along the column direction. Specifically, in an initial weight value combination, the weight values ​​arranged in a row in each sub-group of weight values ​​are arranged sequentially in the same row according to their row positions in the sub-convolution kernel, thus obtaining an initial weight value combination.

[0117] For example, when K equals 1, the structure of the convolution kernel is as follows: Figure 5 As shown, for Figure 5After the convolution kernel is split, three initial weight value combinations are obtained. Each initial weight value combination includes three weight value sub-combinations. Following the above logic, the first weight sub-combination in the first row of the convolution kernel includes A, the second weight sub-combination includes B, and the third weight sub-combination includes C. Therefore, the initial weight value combinations in the first row of the convolution kernel include A, B, and C, and A, B, and C are arranged in the same row from left to right according to the order of their positions in the sub-convolution kernels, i.e., the order is maintained as A, B, C. The first weight sub-combination in the second row of the convolution kernel includes D, the second weight sub-combination includes D, and the third weight sub-combination includes C. If the third weight sub-combination in the initial weight value combination corresponding to the second row of the convolution kernel includes E, then the initial weight value combination corresponding to the second row of the convolution kernel includes D, E, and F. Furthermore, A, B, and C are arranged in the same row from left to right according to the order of their positions in the sub-convolution kernel, i.e., the order remains: D, E, F. If the first weight sub-combination in the initial weight value combination corresponding to the third row of the convolution kernel includes G, the second weight sub-combination in the initial weight value combination corresponding to the third row of the convolution kernel includes H, and the third weight sub-combination in the initial weight value combination corresponding to the third row of the convolution kernel includes I, then the initial weight value combination corresponding to the second row of the convolution kernel includes G, H, and I. Furthermore, G, H, and I are arranged in the same row from left to right according to the order of their positions in the sub-convolution kernel, i.e., the order remains: G, H, I.

[0118] As another example, when K=20, the structure of the convolution kernel is as follows: Figure 6 As shown, for Figure 6After splitting the convolutional kernel shown, three initial weight value combinations are obtained. Each initial weight value combination includes three weight value sub-combinations. Following the above logic, the first weight sub-combination in the initial weight value combination corresponding to the first row of the convolutional kernel includes A1, A2, A3...A20 arranged in order in a row; the second weight sub-combination in the initial weight value combination corresponding to the first row of the convolutional kernel includes B1, B2, B3...B20 arranged in order in a row; and the initial weight value combination corresponding to the first row of the convolutional kernel... The third weight sub-combination in the convolution kernel includes C1, C2, C3...C20 arranged in order in a row. Therefore, the initial weight value combination corresponding to the first row of the convolution kernel includes A1, A2, A3...A20, B1, B2, B3...B20, C1, C2, C3...C20. Furthermore, A1, A2, A3...A20, B1, B2, B3...B20, and C1, C2, C3...C20 are arranged in the same row from left to right according to the order of their positions in the sub-convolution kernel, thus preserving the order of arrangement. Given A1, A2, A3…A20, B1, B2, B3…B20, C1, C2, C3…C20, the first weight sub-combination in the initial weight value combination corresponding to the second row of the convolution kernel includes D1, D2, D3…D20, the second weight sub-combination includes E1, E2, E3…E20, and the third weight sub-combination includes F1, F2, F3…F20. Then, the first weight sub-combination of the convolution kernel… The initial weight value combinations corresponding to the two rows include D1, D2, D3...D20, E1, E2, E3...E20, F1, F2, F3...F20, and D1, D2, D3...D20, E1, E2, E3...E20, F1, F2, F3...F20 are arranged in the same row from left to right according to the order of the weight values ​​in the sub-convolution kernel, that is, the arrangement order is D1, D2, D3...D20, E1, E2, E3...E20, F1, F2, F3...F20, ....

[0119] It should also be noted that, after sorting the weight values ​​in the initial weight value combination according to the order in this embodiment, in the at least one replicated weight value combination obtained by copying the initial weight value combination, the order of the weight values ​​in each replicated weight value combination is the same as the order of the weight values ​​in the initial weight value combination, and the specific value of the weight value in each replicated weight value combination is the same as the specific value of the weight value in the initial weight value combination.

[0120] For example, in at least one replicated weight value combination corresponding to an initial weight value combination including A, B, and C arranged in sequence, each replicated weight value combination includes weight values ​​including A, B, and C arranged in sequence. As another example, in at least one replicated weight value combination corresponding to an initial weight value combination including D1, D2, D3…D20, E1, E2, E3…E20, F1, F2, F3…F20, each replicated weight value combination includes weight values ​​including D1, D2, D3…D20, E1, E2, E3…E20, F1, F2, F3…F20 arranged in sequence.

[0121] In some embodiments, each of the initial weight value combinations is copied to obtain at least one copied weight value combination corresponding to each initial weight value combination, including:

[0122] For each of the initial weight value combinations, N-1 copies are performed to obtain N-1 copied weight value combinations corresponding to each initial weight value combination.

[0123] Specifically, when copying the initial weight value combination, (N-1) copies are performed for each initial weight value combination, and each copy results in a copied weight value combination. That is, one initial weight value combination corresponds to (N-1) copied weight value combinations.

[0124] Specifically, in this embodiment, such as Figure 5 The convolutional kernel shown includes a sub-kernel with 3 rows × 3 columns of weight values, where N equals 3 and M equals 3. The split convolutional kernel then yields three initial weight value combinations. Each initial weight value combination is then copied twice, resulting in two copied weight value combinations for each of the three initial weight value combinations. For example, copying an initial weight value combination consisting of A, B, and C arranged in sequence twice yields two copied weight value combinations, each consisting of A, B, and C arranged in sequence.

[0125] As another example, in this embodiment, such as Figure 6As shown, each of the 20 sub-convolutional kernels in the convolutional kernel includes 3 rows × 3 columns of weight values, i.e., N equals 3 and M equals 3. The split convolutional kernel then yields 3 initial weight value combinations. Each initial weight value combination is then copied twice, resulting in 2 copied weight value combinations for each of the 3 initial weight value combinations. For example, copying the initial weight value combination D1, D2, D3…D20, E1, E2, E3…E20, F1, F2, F3…F20 twice yields 2 copied weight value combinations, each of which includes the sequentially ordered D1, D2, D3…D20, E1, E2, E3…E20, F1, F2, F3…F20.

[0126] In some embodiments, one of the target arrays comprises N subarrays arranged from top to bottom;

[0127] In this context, a target array includes N subarrays arranged from top to bottom. This can be understood as N subarrays arranged sequentially along the column direction in a target array, and when N is greater than or equal to 2, there is no gap between any two adjacent subarrays. Alternatively, the N subarrays arranged from top to bottom in a target array can be understood as N subarrays arranged along the column direction in a target array, and when N is greater than or equal to 2, there is no gap between any two adjacent subarrays.

[0128] In this configuration, each of the N initial weight value combinations and its corresponding at least one replicated weight value combination is sequentially deployed in each of the N subarrays; one initial weight value combination and its corresponding at least one replicated weight value combination are deployed in each subarray.

[0129] In this embodiment, when N initial weight value combinations and their corresponding replicated weight value combinations are deployed to N subarrays of a target array, the initial weight value combinations and their corresponding N-1 replicated weight value combinations are deployed to the N subarrays in the order of the weight values ​​in the initial weight value combinations in the sub-convolution kernels. Each subarray deploys one initial weight value combination and its corresponding N-1 replicated weight value combinations.

[0130] For example, such as Figure 5The convolutional kernel shown includes a sub-kernel with 3 rows × 3 columns of weight values, where N equals 3 and M equals 3. The target array consists of 3 sub-arrays. The split convolutional kernels yield 3 initial weight value combinations. The first row of the convolutional kernel (which includes the first row of each sub-convolutional kernel) contains A, B, and C in sequence, or can be understood as the weight values ​​in the sequentially arranged A, B, and C combination being in the first row of the sub-convolutional kernel. This sequentially arranged A, B, and C initial weight value combination and its corresponding duplicate weight values ​​are deployed in the first sub-array of the target array from top to bottom. The second row of the convolutional kernel (which includes the second row of each sub-convolutional kernel) contains D, E, and F in sequence, or can be understood as including... If the weight values ​​in the initial weight value combination of D, E, and F arranged in order are in the second row of the sub-convolution kernel, then this initial weight value combination of D, E, and F arranged in order, along with the corresponding copied weight values, is deployed in the second sub-array from top to bottom of the target array. The initial weight value combination corresponding to the third row of the convolution kernel (which includes the third row of each sub-convolution kernel) includes G, H, and I arranged in order. Alternatively, it can be understood that if the weight values ​​in the initial weight value combination of G, H, and I arranged in order are in the third row of the sub-convolution kernel, then this initial weight value combination of G, H, and I arranged in order, along with the corresponding copied weight values, is deployed in the third sub-array from top to bottom of the target array.

[0131] For example, such as Figure 6The convolutional kernel shown includes 20 sub-convolutional kernels, each containing 3 rows × 3 columns of weight values, where N equals 3 and M equals 3. The target array comprises 3 sub-arrays. The split convolutional kernel yields 3 initial weight value combinations. The initial weight value combination corresponding to the first row of the convolutional kernel (which includes the first row of each sub-convolutional kernel) includes the following ordered combinations: A1, A2, A3…A20, B1, B2, B3…B20, C1, C2, C3…C20. Alternatively, it can be understood as the weight values ​​in the initial weight value combination of the ordered combinations A1, A2, A3…A20, B1, B2, B3…B20, C1, C2, C3…C20, which are located in the first row of the sub-convolutional kernel. Therefore, the initial weight value combination of the ordered combinations A, B, and C… The corresponding replicated weights are deployed in the first subarray from top to bottom of the target array. The initial weight combination corresponding to the second row of the convolution kernel (which includes the second row of each sub-convolution kernel) includes D1, D2, D3...D20, E1, E2, E3...E20, F1, F2, F3...F20 in the following order, and D1, D2, D3...D20, E1, E2, E3...E20, F1, F2, F3...F20, or can be understood as including D1, D2, D3...D20, E1, E2, E3...E20, F1, F2, F3...F20 in the following order, and D1, D2, D3...D20...E1, E2, E3...E20, F1, F2, F3...F20 ...D20...E1, E2, E3...E20, F1, F2, F3...F20, and D1, D2, D3...D20...E1, E2, E3...D20...E1, E2, E3...D20...E1, E2, E3...D20...E1, E2, E3...D20...E1, E2, E3...D20...E1, E2, E3...D20...E1, E2, E3...D20...E1, E2, E3...D If the weight values ​​in the initial weight combination of ...D20, E1, E2, E3...E20, F1, F2, F3...F20 are located in the second row of the sub-convolutional kernel, then the initial weight combination of D1, D2, D3...D20, E1, E2, E3...E20, F1, F2, F3...F20, and the corresponding duplicate weight values ​​of D1, D2, D3...D20, E1, E2, E3...E20, F1, F2, F3...F20, along with their corresponding duplicate weight values, and the third row of the convolutional kernel (which includes the third row of each sub-convolutional kernel) is used as the initial weight value. The weight value combination includes G1, G2, G3...G20, H1, H2, H3...H20, I1, I2, I3...I20 in sequence, or it can be understood as including the weight values ​​in the initial weight value combination of G1, G2, G3...G20, H1, H2, H3...H20, I1, I2, I3...I20 in sequence, which are located in the third row of the sub-convolution kernel. Then, the initial weight value combination of G1, G2, G3...G20, H1, H2, H3...H20, I1, I2, I3...I20 and the corresponding replicated weight values ​​are deployed in the third sub-array from top to bottom of the target array.

[0132] In some embodiments, deploying each of the initial weight value combinations and the corresponding at least one replicated weight value combination on a target array within the chip's computing array includes:

[0133] Transpose one row of weight values ​​in one of the initial weight value combinations, and transpose one row of weight values ​​in each of the corresponding at least one replicated weight value combinations to obtain at least two columns of transposed weight values ​​corresponding to one of the initial weight value combinations.

[0134] Each of the at least two columns of transposed weight values ​​is sequentially deployed in at least two columns of a subarray, wherein one column of the transposed weight values ​​is deployed in one column of a subarray, and any two adjacent columns of transposed weight values ​​are deployed in a dislocation manner.

[0135] It should be noted that since the weights in each initial weight value combination and each replicated weight value combination are deployed in the same row, and each weight value combination in an initial weight value combination and its corresponding at least one replicated weight value combination needs to be deployed in a column of a subarray, it is necessary to transpose one row of weight values ​​in an initial weight value combination to obtain a column of transposed weight values ​​corresponding to that initial weight value combination. Furthermore, each weight value combination in the at least one replicated weight value combination corresponding to that initial weight value combination needs to be transposed to obtain at least one column of transposed weight values. Therefore, an initial weight value combination corresponds to at least two columns of transposed weight values.

[0136] When deploying at least two columns of weight values ​​corresponding to an initial weight value combination, the two columns of weight values ​​are deployed in at least two columns of a subarray, and one column of transposed weight values ​​is deployed in each column, and during deployment, any two adjacent columns of transposed weight values ​​are deployed at a dislocation.

[0137] Specifically, during deployment, an initial weight value combination and at least one corresponding replica row weight combination are deployed in a subarray. During deployment, each row weight value (each row weight value includes a row of initial weight values ​​and each row of replica weight values) in the initial weight value combination and the corresponding at least one replica weight value combination is transposed to obtain at least two columns of transposed weight values ​​(each column of transposed weight values ​​includes a column of initial weight values ​​after transposing a row of initial weight values ​​and a column of replica weight values ​​after transposing each row of replica weight values). Then, each of the at least two columns of transposed weight values ​​is deployed in the subarray. During deployment, one column of transposed weight values ​​is deployed in one column of a subarray. During deployment, any two adjacent columns of transposed weight values ​​in a subarray are deployed in a staggered manner.

[0138] It should be noted that the transposed weight values ​​included in each column of transposed weight values ​​are the same as the weight values ​​in the initial weight value combination or the weight values ​​in the duplicate weight value combination before transposition. The only difference is that they are arranged sequentially in a row and then sequentially in a column.

[0139] For example, if an initial weight value combination or its corresponding replicated weight value combination includes A, B, and C arranged in order, then the transposed weight values ​​in a transposed column will also be arranged in the order of A, B, and C, that is, A is in the first row of the column, B is in the second row of the column, and C is in the third row of the column.

[0140] For another example, if an initial weight value combination or its corresponding replicated weight value combination includes G1, G2, G3...G20, H1, H2, H3...H20, I1, I2, I3...I20 in sequence, then the order of the transposed weight values ​​in the transposed column is also G1, G2, G3...G20, H1, H2, H3...H20, I1, I2, I3...I20. That is, G1 is in the first row of a column, G2 is in the second row of the same column, G3 is in the third row of the same column, H1 is in the twenty-first row of a column, H2 is in the twenty-second row of a column, H3 is in the twenty-third row of a column...H20 is in the fortieth row of a column, I1 is in the forty-first row of a column, I2 is in the forty-second row of a column, I3 is in the forty-third row of a column...I20 is in the sixtieth row of a column.

[0141] Furthermore, the number of rows of one of the subarrays is Q, where Q = (2M-1) × K, and the number of columns of one of the subarrays is M;

[0142] In one of the subarrays, the first weight value of the transposed weight value of the first target column is deployed in the i-th row of the subarray, and the first weight value of the transposed weight value of the second target column is deployed in the j-th row of the subarray, K = ji, i < j, i is a positive integer greater than 0 and less than Q, and j is a positive integer greater than 0 and less than Q; wherein, the transposed weight values ​​of the first target column and the transposed weight values ​​of the second target column are the transposed weight values ​​of any two adjacent columns in one of the subarrays;

[0143] The first transpose weight value in the first column of the subarray is deployed in the first row and first column of the subarray.

[0144] In a subarray, when deploying at least two columns of transposed weight values, the first transposed weight value in the first column (since the order and specific values ​​of the weight values ​​in the initial weight value combination and the replicated weight value combination are the same, the order and specific values ​​of the transposed weight values ​​in each column are also the same; the first transposed weight value can be any one of the two columns of transposed weight values) is deployed in the first row and first column of the subarray. Then, each transposed weight value after the first transposed weight value in the first column is deployed sequentially in each row after the first column, and there are no empty rows or empty positions between any two adjacent rows of transposed weight values ​​in the first column. Next, for each of the remaining transposed weight values ​​from at least two columns of transposed weight values, select one column and deploy the first transposed weight value of that column in the (1+K)th row and the second column. Then, deploy each transposed weight value after the first transposed weight value in that column sequentially in each row after the (1+K)th row of the second column, ensuring that there are no blank rows or spaces between any two adjacent rows of transposed weight values ​​in that second column. Then, following the above order, deploy the remaining transposed weight values ​​in each column sequentially to obtain the deployed subarray.

[0145] The following is combined with Figure 7 and Figure 8 Specifically, this describes the process of deploying the split initial weight value combination and the corresponding replicated weight value combination onto a subarray of the target array:

[0146] For example, to Figure 5After the convolution kernel is split, three initial weight value combinations are obtained. The initial weight value combinations corresponding to the first row of the convolution kernel include A, B, and C in order; the initial weight value combinations corresponding to the second row of the convolution kernel include D, E, and F in order; and the initial weight value combinations corresponding to the third row of the convolution kernel include G, H, and I in order. Taking the initial weight value combination corresponding to the first row of the convolution kernel as an example, the initial weight value combination corresponding to the first row of the convolution kernel is copied twice to obtain two copied weight value combinations. The initial weight value combination and the corresponding two copied weight value combinations are transposed respectively to obtain three columns of transposed weight values. The three columns of transposed weight values ​​are A, B, and C arranged in order in a column. Specifically, when deploying the initial weight value combination corresponding to the first row of the convolution kernel and its corresponding copied weight value combination on the first subarray of the target array, the first transposed weight value A in the first column of the three transposed weight values ​​is deployed in the first column and the first row of the subarray. Then, the second transposed weight value B in the first column is deployed in the first column and the second row of the subarray. Finally, the third transposed weight value C in the first column is deployed in the first column and the third row of the subarray. Next, the first transposed weight value A from the next column of transposed weight values ​​is placed in the second column and second row (K=1) of the subarray. Then, the second transposed weight value B from this column is placed in the second column and third row of the subarray. Next, the third transposed weight value C from this column is placed in the second column and fourth row of the subarray. Then, the first transposed weight value A from the last column of transposed weight values ​​is placed in the third column and third row of the subarray. Then, the second transposed weight value B from this last column is placed in the third column and fourth row of the subarray. Finally, the third transposed weight value C from this last column is placed in the third column and fifth row of the subarray. The first subarray after deployment is as follows: Figure 7 As shown.

[0147] Following the above principle, the initial weight value combination corresponding to the second row of the convolution kernel and its two corresponding duplicate weight value combinations are sequentially deployed in the second subarray, and the initial weight value combination corresponding to the second row of the convolution kernel and its two corresponding duplicate weight value combinations are deployed in the third subarray. After deployment, the result is as follows: Figure 8 A schematic diagram of one of the target arrays in the computational array shown.

[0148] The following is combined with Figure 9 and Figure 10 To further illustrate the process of deploying the split initial weight value combination and the corresponding replicated weight value combination to a subarray of the target array using another embodiment:

[0149] For example, to Figure 6 After the convolution kernel is split, three initial weight value combinations are obtained. The initial weight value combinations corresponding to the first row of the convolution kernel are A1, A2, A3...A20, B1, B2, B3...B20, C1, C2, C3...C20 in sequence. The initial weight value combinations corresponding to the second row of the convolution kernel are D1, D2, D3...D20, E1, E2, E3...E20, F1, F2, F3...F20 in sequence. The initial weight value combinations corresponding to the third row of the convolution kernel are G1, G2, G3...G20, H1, H2, H3...H20, I1, I2, I3...I20 in sequence. Taking the initial weight value combination corresponding to the first row of the convolution kernel as an example, this initial weight value combination is copied twice to obtain two copied weight value combinations. The initial weight value combination and the corresponding two copied weight value combinations are then transposed, resulting in three columns of transposed weight values. These three columns of transposed weight values ​​are A1, A2, A3…A20, B1, B2, B3…B20, C1, C2, C3…C20, arranged sequentially in a column. Specifically, when deploying the initial weight value combination corresponding to the first row of the convolution kernel and its corresponding copied weight value combination on the first subarray of the target array, the first transposed weight value A1 of the first column of the three transposed weight values ​​is deployed in the first column and first row of the subarray; the second transposed weight value A2 of the first column is deployed in the first column and second row of the subarray; the third transposed weight value A3 of the first column is deployed in the first column and third row of the subarray, and so on. The twentieth transpose weight value A20 is deployed in the first column and twentieth row of the subarray. The twenty-first transpose weight value B1 is deployed in the first column and twentieth row of the subarray. The twenty-second transpose weight value B2 is deployed in the first column and twentieth row of the subarray. The twenty-third transpose weight value B3 is deployed in the first column and twentieth row of the subarray. ... The fortieth transpose weight value B20 is deployed in the first column and twentieth row of the subarray. The forty-first transpose weight value C1 is deployed in the first column and twentieth row of the subarray. The forty-second transpose weight value C2 is deployed in the first column and twentieth row of the subarray. The forty-third transpose weight value C3 is deployed in the first column and twentieth row of the subarray. The first subarray after deployment is as follows: Figure 9 As shown.

[0150] Following the above principle, the initial weight value combination corresponding to the second row of the convolution kernel and its two corresponding duplicate weight value combinations are sequentially deployed in the second subarray, and the initial weight value combination corresponding to the second row of the convolution kernel and its two corresponding duplicate weight value combinations are deployed in the third subarray. After deployment, the result is as follows: Figure 10 A schematic diagram of one of the target arrays in the computational array shown.

[0151] In some embodiments, a 0 value is deployed at the location in the subarray where the transpose weight value is not deployed.

[0152] It is understandable that zero values ​​are deployed at all positions in the target array where no transposed weight values ​​are deployed. Specifically, zero values ​​are... Figure 7 -to Figure 10 Not shown in the image.

[0153] In some embodiments, the number of convolution kernels is P;

[0154] The computation array includes P target arrays, that is, one convolution kernel corresponds to one target array;

[0155] P target arrays are arranged from left to right in the computation array; where P is a positive integer.

[0156] The computation array includes at least one target array arranged from left to right. This can be understood as at least one target array being arranged sequentially in the row direction within the computation array, and when P is greater than or equal to 2, there is no gap between any two adjacent target arrays. Alternatively, the computation array including P target arrays arranged from left to right can be understood as including at least one target array arranged in the row direction within the computation array, and when P is greater than or equal to 2, there is no gap between any two adjacent target arrays.

[0157] Wherein, the number of columns in one of the target arrays is equal to M, and the number of rows in one of the target arrays is equal to N×Q. Then, the number of columns in the calculation array is equal to M×P, and the number of rows in the calculation array is equal to N×Q.

[0158] Furthermore, the initial weight value combination and the at least one replicated weight value combination corresponding to each of the P convolutional kernels are sequentially deployed on each of the P target arrays;

[0159] The initial weight value combination corresponding to the convolution kernel and the at least one duplicate weight value combination are deployed on the target array.

[0160] When deploying P convolutional kernels, each kernel is split and copied to obtain an initial weight value combination and at least one copied weight value combination. These are then deployed sequentially on P target arrays according to the order of the convolutional kernels. One target array deploys the initial weight value combination and copied weight value combination corresponding to one convolutional kernel. The logic for deploying the initial weight value combination and copied weight value combination in each target array is the same; please refer to the description above.

[0161] For example, taking P = 40 as an example, the structure of 40 convolutional kernels is as follows: Figure 11 As shown, Figure 11 In the diagram, "Kernel-1" to "Kernel-40" correspond to 40 convolutional kernels, each of which includes 20 sub-convolutional kernels. Each sub-convolutional kernel includes 3 rows and 3 columns of weight values. Different colors represent different sub-convolutional kernels within a single convolutional kernel.

[0162] According to the above deployment logic Figure 11 The schematic diagram of the target array obtained after deploying 40 convolutional kernels on 40 target arrays is shown below. Figure 12 As shown.

[0163] In some embodiments, the method further includes:

[0164] The input data combination is obtained from the input feature map; wherein the input feature map includes K-dimensional sub-input feature maps, each sub-input feature map includes at least one row and at least one column of input data; the input data combination includes N sub-input data combinations, and one sub-input data combination includes one row and X columns of input data from each dimension of the K-dimensional sub-input feature map;

[0165] Each of the input data in the input data combination is deployed on a column in the input channel array.

[0166] In this embodiment, when performing convolution calculation, an input data combination is obtained from the input feature map. This input data combination is used for convolution calculation, that is, convolution calculation is performed based on the input data combination and the weight values ​​deployed in the above-mentioned calculation array (or target array).

[0167] In this embodiment, the input feature map includes at least one row × at least one column of input data, and the input feature map includes K-dimensional sub-input feature maps, that is, one input feature map corresponds to one sub-convolution kernel.

[0168] For example, when K equals 1, the structural diagram of the input feature map is as follows: Figure 13 The above, Figure 13Input data is represented by numbers, such as "1", "5", "12", "17" to represent different input data.

[0169] As another example, when K equals 20, the structural diagram of the input feature map is as follows: Figure 14 The above, Figure 14 Input data is represented by "In" followed by a number, such as "In 01", "In 11", "In 1k", "In 220", etc., which represent different input data. Here, "k" represents a positive integer greater than 0 and less than K.

[0170] The input data combination obtained from the input feature map includes N sub-input data combinations. Each sub-input data combination includes one row and X columns of input data in each dimension of the sub-input feature map of K dimensions. That is, the N sub-input data combinations include N rows and X columns of input data in each dimension of the sub-input feature map.

[0171] For example, when K equals 1, a sub-input data combination consists of one row and X columns of input data in a sub-input feature map; then N sub-input data combinations consist of 3 rows and X columns of input data in a sub-input feature map. For instance, for a... Figure 5 The target array after the convolution kernel redeployment is shown. The input data combination is obtained. This input data combination includes 3 sub-input data combinations. Each sub-input data combination includes one row and five columns of input data. The combined input data combination includes three rows and five columns of input data.

[0172] When K equals 20, a sub-input data combination includes one row and X columns of input data from each of the 20-dimensional sub-input feature maps. Therefore, N sub-input data combinations include 3 rows and X columns of input data from each of the 20 sub-input feature maps. For example, for a... Figure 6 The target array after the convolutional kernel redeployment is shown. The input data combination is obtained. The input data combination includes 3 sub-input data combinations. Each sub-input data combination includes one row and five columns of input data from each sub-input feature map in 20 dimensions. The combined input data combination includes three rows and five columns of input data from each sub-input feature map in 20 dimensions.

[0173] To enable subsequent convolution calculations, after obtaining the input data from the aforementioned input data combination, all the input data is deployed onto a column of the input channel array. Then, convolution calculations are performed using the input data deployed in the input channel array and the weight values ​​deployed in the computation array (or target array).

[0174] Furthermore, in one column of the input channel array, N combinations of the sub-input data are arranged sequentially from top to bottom;

[0175] The nth sub-input data combination among the N sub-input data combinations includes the first X input data in the nth row of the sub-input feature map for each dimension;

[0176] The nth sub-input data combination includes X first input data combinations arranged in sequence;

[0177] The xth first input data combination in the nth sub-input data combination includes the xth input data in the nth row of the sub-input feature map arranged sequentially in each dimension;

[0178] Where n is a positive integer greater than 0 and less than or equal to N; x is a positive integer greater than 0 and less than or equal to X.

[0179] Specifically, when deploying input data in the input channel matrix, N sub-input data combinations are deployed sequentially in one column of the input channel matrix. During deployment, the N sub-input data combinations are deployed sequentially from top to bottom in one column of the input channel matrix according to the order of the input feature maps to which the input data in the sub-input data combinations belong, or the N sub-input data combinations are deployed sequentially in one column of the input channel matrix according to the row order.

[0180] The nth sub-input data combination deployed on a column of the input channel matrix includes the first X input data in the nth row of the sub-input feature map of each of the K dimensions. For example, when n equals 1 and K equals 20, the first sub-input data combination deployed on a column of the input channel matrix includes the first X input data in the first row of the first sub-input feature map of each of the 20 dimensions. As another example, when n equals 10 and K equals 20, the tenth sub-input data combination deployed on a column of the input channel matrix includes the first X input data in the tenth row of the first sub-input feature map of each of the 20 dimensions.

[0181] The nth sub-input data combination includes X first input data combinations arranged sequentially, and the xth first input data combination in the nth sub-input data combination includes the xth input data in the nth row of the sub-input feature map of each dimension arranged sequentially. That is, each first input data combination is composed of the xth input data in the nth row of the sub-input feature map of each dimension. For example, when K equals 20, the first first input data combination in the first sub-input data combination is composed of the input data in the first row and first column of the 20 sub-dimensional feature maps, the second first input data combination in the first sub-input data combination is composed of the input data in the first row and second column of the 20 sub-dimensional feature maps, and so on, the fourth first input data combination in the third sub-input data combination is composed of the input data in the third row and fourth column of the 20 sub-dimensional feature maps, and so on.

[0182] The xth first input data combination in the nth sub-input data combination includes the xth input data in the nth row of each dimension of the sub-input feature map arranged sequentially. It can be understood that the xth input data in the nth row is arranged sequentially according to the dimensional order of the sub-input feature map to which the input data belongs.

[0183] For example, targeting Figure 5 The weights in the convolution kernel shown are deployed to obtain the result. Figure 8 The target array shown, in Figure 13 The input data combination is obtained, where K equals 1. The obtained input data combination includes three sub-input data combinations. The first sub-input data combination includes the first five input data points of the first row of the input feature map of one sub-dimension. These first five input data points are arranged sequentially according to the row order of the input data, that is, in a column of the input channel array, they are arranged in the order of "1", "2", "3", "10", and "13" respectively, representing the input data. The second sub-input data combination includes the first five input data points of the second row of the input feature map of one sub-dimension. The input data consists of three sub-input data combinations. The first sub-input data combination consists of the first five input data points in the third row of the input feature map of a sub-dimension, which are represented by the numbers "4", "5", "6", "11", and "14" arranged sequentially in a column of the input channel array. The second sub-input data combination consists of the first five input data points in the third row of the input feature map of a sub-dimension, arranged sequentially in a column of the input channel array, as shown below. Figure 15The input channel array shown includes the following input data arranged in a column: “1”, “2”, “3”, “10”, “13”, “4”, “5”, “6”, “11”, “14”, “7”, “8”, “9”, “12”, and “15” respectively represent the input data.

[0184] For example again, regarding Figure 6 The weights in the convolution kernel shown are deployed to obtain the result. Figure 10 The target array shown, in Figure 14 The input data combination is obtained, where K equals 20. The obtained input data combination includes three sub-input data combinations. The first sub-input data combination includes the first five input data in the first row of each sub-dimensional feature map of the 20 sub-dimensions. The first five input data in the first row of each sub-dimensional feature map form five first input data combinations. That is, the first sub-input data combination includes the first input data in the first row of the 20 sub-input feature maps. The order of these first input data combinations in a column of the input channel array is: "In01", "In02", "In03"..."In020", representing the input data respectively. The second sub-input data combination includes the second input data in the first row of the 20 sub-input feature maps. The order of these second input data combinations in a column of the input channel array is: "In11", "In12", "In13", "In14", "In15", "In16", "In17", "In18", "In19 ... The first sub-input data combination consists of the third input data in the first row of the 20-dimensional sub-input feature map, arranged in the following order in a column of the input channel array: "In21", "In22", "In23", ... "In220"; the second sub-input data combination consists of the fourth input data in the first row of the 20-dimensional sub-input feature map, arranged in the following order in a column of the input channel array: "In31", "In32", "In33", ... "In320"; the third sub-input data combination consists of the fifth input data in the first row of the 20-dimensional sub-input feature map, arranged in the following order in a column of the input channel array: "In41", "In42", "In43", ... "In420"; the fourth sub-input data combination consists of the fourth input data in the first row of the 20-dimensional sub-input feature map, arranged in the following order in a column of the input channel array: "In41", "In42", "In43", ... "In420";

[0185] The first sub-input data combination is arranged in the following order in a column of the input channel array: “In01”, “In02”, “In03”...“In020”, “In11”, “In12”, “In13”...“In120”, “In22”, “In23”...“In220”, “In31”, “In32”, “In33”...“In320”, “In41”, “In42”, “In43”...“In420”, which respectively represent the input data;

[0186] Following the above logic, the input data from the second and third sub-input data combinations are sequentially obtained. Then, in a column of the input channel array, the input data from the second sub-input data combination is placed after the first sub-input data combination, and the input data from the third sub-input data combination is placed after the second sub-input data combination. This results in... Figure 16 The input channel array shown.

[0187] It should be noted that, Figure 14 The blue matrix in the image represents the input feature map. Figure 15 and Figure 16 In the colors other than blue, the positions of the same row and the same column in each dimension are represented by the same color.

[0188] In some embodiments, the method further includes:

[0189] According to the matrix calculation method, the output data corresponding to the target array is obtained based on the transpose weight values ​​deployed in the N subarrays of the target array and the input data in a column of the input channel array.

[0190] Specifically, during the calculation, the input data in one column of the input channel array is multiplied by the transpose weight value in the target array in a matrix calculation manner, that is, input channel array * target array, to obtain the output data corresponding to the target array (or convolution kernel).

[0191] Specifically, after the weights in the convolution kernel are deployed in the target array according to the above logic, and after the input data in the input feature map are deployed in a column of the input channel array according to the above logic, the input channel array is placed on the left and the target array is placed on the right in a matrix calculation manner to obtain the output data.

[0192] The output data includes N sub-output data, where one sub-output data corresponds to a subarray of the target array, or one output data corresponds to a row of the convolution kernel.

[0193] For example, according to the matrix calculation method, for Figure 15 The input channel matrix shown is Figure 8 The target matrix shown is multiplied to obtain three sub-output data, denoted by "O1", "O2", and "O3". "O1" corresponds to the sub-output data of the first subarray, calculated as: O1 = 1A + 2B + 3C + 4D + 5E + 6F + 7G + 8H + 9I. "O2" corresponds to the sub-output data of the second subarray, calculated as: O2 = 2A + 3B + 10C + 5D + 6E + 11F + 8G + 9H + 12I. "O3" corresponds to the sub-output data of the second subarray, calculated as: O3 = 3A + 10B + 13C + 6D + 11E + 14F + 9G + 12H + 15I. The matrix calculation process is as follows: Figure 17 As shown.

[0194] Depend on Figure 17 It can be seen that a column of input data in the input channel array is input into the reconstructed computation array, from which the result data of three sliding windows can be obtained. Furthermore, the obtained output data is consistent with... Figure 1 The corresponding output results are consistent.

[0195] The convolution operator reconstruction method provided in this invention splits and copies the weight values ​​in the convolution kernel in row-major logical order, and then deploys them sequentially in the computing array. This enables parallel computation of sliding window input data, allowing parallel processing of data with multiple sliding windows of the convolution kernel size. Compared to traditional parallel processing methods that copy weights, this invention saves row space occupied by the in-memory computing chip's computing array. After the convolution operator is reconstructed, the number of times data is repeatedly read during computation is reduced, thus lowering the chip's storage requirements and power consumption.

[0196] like Figure 18 As shown, this embodiment of the invention also provides a convolution operator deployment device, the device comprising:

[0197] The first processing module 1801 is used to split the weight values ​​in a convolution kernel to obtain convolution kernel splitting information; wherein, a convolution kernel includes K sub-convolution kernels of different dimensions, and each sub-convolution kernel of different dimensions includes weight values ​​in N rows and M columns; the convolution kernel splitting information includes N initial weight value combinations after splitting, each initial weight value combination includes M weight value sub-combinations, and each weight value sub-combination includes the weight values ​​in the nth row and mth column of each sub-convolution kernel of different dimensions;

[0198] The second processing module 1802 is used to copy each of the initial weight value combinations to obtain at least one copied weight value combination corresponding to each of the initial weight value combinations.

[0199] The third processing module 1803 is used to deploy each of the initial weight value combinations and the corresponding at least one replicated weight value combination on a target array in the computing array of the chip;

[0200] One of the target arrays comprises N subarrays;

[0201] An initial weight value combination and a corresponding at least one replicated weight value combination are deployed in one of the subarrays, and each weight value combination in the initial weight value combination and the corresponding at least one replicated weight value combination is deployed sequentially in each column of the subarray, and a weight value combination is deployed in one column of the subarray, and any two adjacent columns of weight value combinations are deployed in a staggered manner;

[0202] Where K is a positive integer, N and M are both positive integers greater than 1, n is a positive integer greater than 0 and less than or equal to N, and m is a positive integer greater than 0 and less than or equal to M.

[0203] Optionally, in one of the weight value sub-combinations, the weight values ​​of the nth row and mth column in the sub-convolution kernel of each dimension are arranged in a row in sequence;

[0204] In one of the initial weight value combinations, the weight values ​​in each of the weight value sub-combinations are arranged sequentially in the same row.

[0205] Optionally, the second processing module includes:

[0206] The first processing unit is used to perform N-1 copies on each of the initial weight value combinations to obtain N-1 copied weight value combinations corresponding to each initial weight value combination.

[0207] Optionally, one of the target arrays comprises N subarrays arranged from top to bottom;

[0208] In this configuration, each of the N initial weight value combinations and its corresponding at least one replicated weight value combination is sequentially deployed in each of the N subarrays; one initial weight value combination and its corresponding at least one replicated weight value combination are deployed in each subarray.

[0209] Optionally, the third processing module includes:

[0210] The second processing unit is configured to transpose a row of weight values ​​in one of the initial weight value combinations, and to transpose a row of weight values ​​in each of the corresponding at least one replicated weight value combinations, to obtain at least two columns of transposed weight values ​​corresponding to one of the initial weight value combinations.

[0211] The third processing unit is used to sequentially deploy each of the at least two columns of transpose weight values ​​in at least two columns of a subarray, wherein one column of the transpose weight values ​​is deployed on one column of the subarray, and any two adjacent columns of transpose weight values ​​are deployed at an offset.

[0212] Optionally, the number of rows of one of the subarrays is Q, where Q = (2M-1) × K, and the number of columns of one of the subarrays is equal to M;

[0213] In one of the subarrays, the first weight value of the transposed weight value of the first target column is deployed in the i-th row of the subarray, and the first weight value of the transposed weight value of the second target column is deployed in the j-th row of the subarray, K = ji, i < j, i is a positive integer greater than 0 and less than Q, and j is a positive integer greater than 0 and less than Q; wherein, the transposed weight values ​​of the first target column and the transposed weight values ​​of the second target column are the transposed weight values ​​of any two adjacent columns in one of the subarrays;

[0214] The first transpose weight value in the first column of the subarray is deployed in the first row and first column of the subarray.

[0215] Optionally, a value of 0 may be deployed at the positions in the subarray where the transpose weight value is not deployed.

[0216] Optionally, the number of convolution kernels is P;

[0217] The computing array includes P target arrays, which are arranged from left to right in the computing array.

[0218] Where P is a positive integer.

[0219] Optionally, the initial weight value combination and the at least one replicated weight value combination corresponding to each of the P convolutional kernels are sequentially deployed on each of the P target arrays;

[0220] The initial weight value combination corresponding to the convolution kernel and the at least one duplicate weight value combination are deployed on the target array.

[0221] Optionally, the device further includes:

[0222] The first acquisition module is used to acquire input data combinations from the input feature map; wherein, the input feature map includes K-dimensional sub-input feature maps, each sub-input feature map includes at least one row and at least one column of input data; the input data combination includes N sub-input data combinations, and one sub-input data combination includes one row and X columns of input data from each dimension of the K-dimensional sub-input feature map;

[0223] The fourth processing module is used to deploy each of the input data in the input data combination onto a column in the input channel array.

[0224] Optionally, in one column of the input channel array, N sub-input data combinations are arranged sequentially from top to bottom;

[0225] The nth sub-input data combination among the N sub-input data combinations includes the first X input data in the nth row of the sub-input feature map for each dimension;

[0226] The nth sub-input data combination includes X first input data combinations arranged in sequence;

[0227] The xth first input data combination in the nth sub-input data combination includes the xth input data in the nth row of the sub-input feature map arranged sequentially in each dimension;

[0228] Where n is a positive integer greater than 0 and less than or equal to N; x is a positive integer greater than 0 and less than or equal to X.

[0229] Optionally, the device further includes:

[0230] The fifth processing module is used to obtain the output data corresponding to the target array according to the transpose weight values ​​deployed in the N subarrays of the target array and the input data in a column of the input channel array, using a matrix calculation method.

[0231] It should be noted that the convolution operator deployment device provided in the embodiments of the present invention is a device capable of executing the above-described convolution operator deployment device method. Therefore, all embodiments of the above-described convolution operator deployment device method are applicable to this device and can achieve the same or similar technical effects.

[0232] like Figure 19 As shown, this embodiment of the invention also provides a convolution operator deployment device, including: a processor 1901; and a memory 1903 connected to the processor 1901 via a bus interface 1902. The memory 1903 is used to store programs and data used by the processor 1901 when performing operations, and the processor 1901 calls and executes the programs and data stored in the memory 1903.

[0233] The transceiver 1904 is connected to the bus interface 1902 and is used to receive and send data under the control of the processor 1901. Specifically, the processor 1901 is used to read the program in the memory 1903 and to execute the following processes:

[0234] The weights in a convolution kernel are split to obtain convolution kernel splitting information; wherein, a convolution kernel includes K sub-convolution kernels of different dimensions, and each sub-convolution kernel of different dimensions includes weights in N rows and M columns; the convolution kernel splitting information includes N initial weight value combinations after splitting, each initial weight value combination includes M weight value sub-combinations, and each weight value sub-combination includes the weights in the nth row and mth column of each sub-convolution kernel of different dimensions;

[0235] Each of the initial weight value combinations is copied to obtain at least one copied weight value combination corresponding to each initial weight value combination;

[0236] Each of the initial weight value combinations and the corresponding at least one replicated weight value combination are deployed on a target array in the chip's computing array;

[0237] One of the target arrays comprises N subarrays;

[0238] An initial weight value combination and a corresponding at least one replicated weight value combination are deployed in one of the subarrays, and each weight value combination in the initial weight value combination and the corresponding at least one replicated weight value combination is deployed sequentially in each column of the subarray, and a weight value combination is deployed in one column of the subarray, and any two adjacent columns of weight value combinations are deployed in a staggered manner;

[0239] Where K is a positive integer, N and M are both positive integers greater than 1, n is a positive integer greater than 0 and less than or equal to N, and m is a positive integer greater than 0 and less than or equal to M.

[0240] Optionally, in one of the weight value sub-combinations, the weight values ​​of the nth row and mth column in the sub-convolution kernel of each dimension are arranged in a row in sequence;

[0241] In one of the initial weight value combinations, the weight values ​​in each of the weight value sub-combinations are arranged sequentially in the same row.

[0242] Optionally, the processor 1901 is used for:

[0243] For each of the initial weight value combinations, N-1 copies are performed to obtain N-1 copied weight value combinations corresponding to each initial weight value combination.

[0244] Optionally, one of the target arrays comprises N subarrays arranged from top to bottom;

[0245] In this configuration, each of the N initial weight value combinations and its corresponding at least one replicated weight value combination is sequentially deployed in each of the N subarrays; one initial weight value combination and its corresponding at least one replicated weight value combination are deployed in each subarray.

[0246] Optionally, the processor 1901 is used for:

[0247] Transpose one row of weight values ​​in one of the initial weight value combinations, and transpose one row of weight values ​​in each of the corresponding at least one replicated weight value combinations to obtain at least two columns of transposed weight values ​​corresponding to one of the initial weight value combinations.

[0248] Each of the at least two columns of transposed weight values ​​is sequentially deployed in at least two columns of a subarray, wherein one column of the transposed weight values ​​is deployed in one column of a subarray, and any two adjacent columns of transposed weight values ​​are deployed in a dislocation manner.

[0249] Optionally, the number of rows of one of the subarrays is Q, where Q = (2M-1) × K, and the number of columns of one of the subarrays is equal to M;

[0250] In one of the subarrays, the first weight value of the transposed weight value of the first target column is deployed in the i-th row of the subarray, and the first weight value of the transposed weight value of the second target column is deployed in the j-th row of the subarray, K = ji, i < j, i is a positive integer greater than 0 and less than Q, and j is a positive integer greater than 0 and less than Q; wherein, the transposed weight values ​​of the first target column and the transposed weight values ​​of the second target column are the transposed weight values ​​of any two adjacent columns in one of the subarrays;

[0251] The first transpose weight value in the first column of the subarray is deployed in the first row and first column of the subarray.

[0252] Optionally, a value of 0 may be deployed at the positions in the subarray where the transpose weight value is not deployed.

[0253] Optionally, the number of convolution kernels is P;

[0254] The computing array includes P target arrays, which are arranged from left to right in the computing array.

[0255] Where P is a positive integer.

[0256] Optionally, the initial weight value combination and the at least one replicated weight value combination corresponding to each of the P convolutional kernels are sequentially deployed on each of the P target arrays;

[0257] The initial weight value combination corresponding to the convolution kernel and the at least one duplicate weight value combination are deployed on the target array.

[0258] Optionally, the processor 1901 is further configured to:

[0259] The input data combination is obtained from the input feature map; wherein the input feature map includes K-dimensional sub-input feature maps, each sub-input feature map includes at least one row and at least one column of input data; the input data combination includes N sub-input data combinations, and one sub-input data combination includes one row and X columns of input data from each dimension of the K-dimensional sub-input feature map;

[0260] Each of the input data in the input data combination is deployed on a column in the input channel array.

[0261] Optionally, in one column of the input channel array, N sub-input data combinations are arranged sequentially from top to bottom;

[0262] The nth sub-input data combination among the N sub-input data combinations includes the first X input data in the nth row of the sub-input feature map for each dimension;

[0263] The nth sub-input data combination includes X first input data combinations arranged in sequence;

[0264] The xth first input data combination in the nth sub-input data combination includes the xth input data in the nth row of the sub-input feature map arranged sequentially in each dimension;

[0265] Where n is a positive integer greater than 0 and less than or equal to N; x is a positive integer greater than 0 and less than or equal to X.

[0266] Optionally, the processor 1901 is used for:

[0267] According to the matrix calculation method, the output data corresponding to the target array is obtained based on the transpose weight values ​​deployed in the N subarrays of the target array and the input data in a column of the input channel array.

[0268] Among them, Figure 19In this context, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 1901) and memory (memory 1903). The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides a user interface 1905. A transceiver 1904 may be multiple elements, including a transmitter and a receiver, providing a unit for communicating with various other devices over a transmission medium. Processor 1901 is responsible for managing the bus architecture and general processing, and memory 1903 may store data used by processor 1901 during operation.

[0269] In addition, specific embodiments of the present invention also provide a readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps in the convolution operator deployment method as described above.

[0270] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0271] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can be physically comprised separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or in the form of hardware plus software functional units.

[0272] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions that cause a computer device (which may be a personal computer, server, or network device, etc.) to execute partial steps of the resource selection method described in the various embodiments of the present invention, or to execute partial steps of the information transmission method described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0273] A specific embodiment of the present invention also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above-described functionality. Figure 4 The various processes of the method embodiments shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0274] The above describes the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also within the scope of protection of the present invention.

Claims

1. A method for deploying convolution operators, characterized in that, The method includes: The weights in a convolution kernel are split to obtain convolution kernel splitting information; wherein, a convolution kernel includes K sub-convolution kernels of different dimensions, and each sub-convolution kernel of different dimensions includes weights in N rows and M columns; the convolution kernel splitting information includes N initial weight value combinations after splitting, each initial weight value combination includes M weight value sub-combinations, and each weight value sub-combination includes the weights in the nth row and mth column of each sub-convolution kernel of different dimensions; Each of the initial weight value combinations is copied to obtain at least one copied weight value combination corresponding to each initial weight value combination; Each of the initial weight value combinations and the corresponding at least one replicated weight value combination are deployed on a target array in the chip's computing array; One of the target arrays comprises N subarrays; An initial weight value combination and a corresponding at least one replicated weight value combination are deployed in one of the subarrays, and each weight value combination in the initial weight value combination and the corresponding at least one replicated weight value combination is deployed sequentially in each column of the subarray, and a weight value combination is deployed in one column of the subarray, and any two adjacent columns of weight value combinations are deployed in a staggered manner; Where K is a positive integer, N and M are both positive integers greater than 1, n is a positive integer greater than 0 and less than or equal to N, and m is a positive integer greater than 0 and less than or equal to M.

2. The method according to claim 1, characterized in that, In one of the weight value sub-combinations, the weight values ​​of the nth row and mth column in the sub-convolution kernel of each dimension are arranged in a row in sequence; In one of the initial weight value combinations, the weight values ​​in each of the weight value sub-combinations are arranged sequentially in the same row.

3. The method according to claim 1, characterized in that, Each of the initial weight value combinations is copied to obtain at least one copied weight value combination corresponding to each initial weight value combination, including: For each of the initial weight value combinations, N-1 copies are performed to obtain N-1 copied weight value combinations corresponding to each initial weight value combination.

4. The method according to claim 1, characterized in that, A target array comprises N subarrays arranged from top to bottom; In this configuration, each of the N initial weight value combinations and its corresponding at least one replicated weight value combination is sequentially deployed in each of the N subarrays; one initial weight value combination and its corresponding at least one replicated weight value combination are deployed in each subarray.

5. The method according to claim 1, characterized in that, Deploying each of the initial weight value combinations and the corresponding at least one replicated weight value combination on a target array within the chip's computing array includes: Transpose one row of weight values ​​in one of the initial weight value combinations, and transpose one row of weight values ​​in each of the corresponding at least one replicated weight value combinations to obtain at least two columns of transposed weight values ​​corresponding to one of the initial weight value combinations. Each of the at least two columns of transposed weight values ​​is sequentially deployed in at least two columns of a subarray, wherein one column of the transposed weight values ​​is deployed in one column of a subarray, and any two adjacent columns of transposed weight values ​​are deployed in a dislocation manner.

6. The method according to claim 5, characterized in that, The number of rows in one of the subarrays is Q, where Q = (2M-1) × K, and the number of columns in one of the subarrays is M. In one of the subarrays, the first weight value of the transposed weight value of the first target column is deployed in the i-th row of the subarray, and the first weight value of the transposed weight value of the second target column is deployed in the j-th row of the subarray, K = ji, i < j, i is a positive integer greater than 0 and less than Q, and j is a positive integer greater than 0 and less than Q; wherein, the transposed weight values ​​of the first target column and the transposed weight values ​​of the second target column are the transposed weight values ​​of any two adjacent columns in one of the subarrays; The first transpose weight value in the first column of the subarray is deployed in the first row and first column of the subarray.

7. The method according to claim 5, characterized in that, The subarray will deploy a value of 0 at the position where the transpose weight value is not deployed.

8. The method according to claim 1, characterized in that, The number of convolutional kernels is P; The computing array includes P target arrays, which are arranged from left to right in the computing array. Where P is a positive integer.

9. The method according to claim 8, characterized in that, The initial weight value combination and the at least one replicated weight value combination corresponding to each of the P convolutional kernels are sequentially deployed on each of the P target arrays; The initial weight value combination and the at least one duplicate weight value combination corresponding to the convolution kernel are deployed on the target array.

10. The method according to claim 6, characterized in that, The method further includes: The input data combination is obtained from the input feature map; wherein the input feature map includes K-dimensional sub-input feature maps, each sub-input feature map includes at least one row and at least one column of input data; the input data combination includes N sub-input data combinations, and one sub-input data combination includes one row and X columns of input data from each dimension of the K-dimensional sub-input feature map; Each of the input data in the input data combination is deployed on a column in the input channel array.

11. The method according to claim 10, characterized in that, In one column of the input channel array, N sub-input data combinations are arranged sequentially from top to bottom; The nth sub-input data combination among the N sub-input data combinations includes the first X input data in the nth row of the sub-input feature map for each dimension; The nth sub-input data combination includes X first input data combinations arranged in sequence; The xth first input data combination in the nth sub-input data combination includes the xth input data in the nth row of the sub-input feature map arranged sequentially in each dimension; Where n is a positive integer greater than 0 and less than or equal to N; x is a positive integer greater than 0 and less than or equal to X.

12. The method according to claim 10, characterized in that, The method further includes: According to the matrix calculation method, the output data corresponding to the target array is obtained based on the transpose weight values ​​deployed in the N subarrays of the target array and the input data in a column of the input channel array.

13. A convolution operator deployment device, characterized in that, The device includes: The first processing module is used to split the weight values ​​in a convolutional kernel to obtain convolutional kernel splitting information; wherein, a convolutional kernel includes K sub-convolutional kernels of different dimensions, and each sub-convolutional kernel of different dimensions includes N rows of weight values ​​and M columns of weight values; the convolutional kernel splitting information includes N initial weight value combinations after splitting, each initial weight value combination includes M weight value sub-combinations, and each weight value sub-combination includes the weight values ​​of the nth row and mth column in each sub-convolutional kernel of different dimensions; The second processing module is used to copy each of the initial weight value combinations to obtain at least one copied weight value combination corresponding to each of the initial weight value combinations. The third processing module is used to deploy each of the initial weight value combinations and the corresponding at least one replicated weight value combination on a target array in the chip's computing array; One of the target arrays comprises N subarrays; An initial weight value combination and a corresponding at least one replicated weight value combination are deployed in a subarray. Each weight value combination in the initial weight value combination and the corresponding at least one replicated weight value combination is deployed sequentially in each column of the subarray. A weight value combination is deployed in one column of the subarray, and any two adjacent columns of weight value combinations are deployed in a staggered manner. Where P is a positive integer, N and M are both positive integers greater than 1, n is a positive integer greater than 0 and less than or equal to N, and m is a positive integer greater than 0 and less than or equal to M.

14. A convolution operator deployment device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps in the convolution operator deployment method as described in any one of claims 1 to 12.

15. A readable storage medium, characterized in that, The readable storage medium stores a program that, when executed by a processor, implements the steps of the convolution operator deployment method as described in any one of claims 1 to 12.

16. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps in the convolution operator deployment method as described in any one of claims 1 to 12.

Images