Method for generating padding value in convolution operation, application method, device, medium, equipment and product

Abstract

The present invention discloses a method for generating padding values for convolution operations, an application method, a device, a medium, a device and a product; the generation method includes: obtaining an input data block and corresponding original attribute information from a first storage space; wherein, the original attribute information includes: the size data of the input data block in the original spatial structure and the spatial coordinates of each data point; obtaining the convolution starting coordinate, and the position offset of each weight in the convolution kernel from the center of the convolution kernel; according to the convolution starting coordinate, the size data and the position offset, obtaining a mask sequence corresponding to each weight; wherein, the mask sequence is used to indicate the padding position of the preset value corresponding to the weight in the convolution operation; according to the mask sequence, obtaining a convolution coverage sequence corresponding to each weight. The present invention can automatically pad preset values to the input data block during convolution operations without pre-storing preset value data, thereby effectively improving the utilization rate of the storage space and reducing the bandwidth of data transfer.

Description

Technical Field

[0001] The present invention relates to the field of computer technologies, and in particular, to a method for generating padding values for convolution operations, an application method, a device, a computer-readable storage medium, an electronic device, and a computer program product. Background Art

[0002] Artificial intelligence processors usually need to perform a large amount of calculations during the training and inference of large models, and the efficiency of data transfer is one of the key factors affecting their performance. Especially during the convolution operation, a large amount of data reuse is involved, and the demand for data transfer bandwidth significantly increases, thus putting pressure on the performance of artificial intelligence processors. Therefore, on the basis of not adding additional hardware, how to effectively reduce the bandwidth required for data transfer in convolution operations is an urgent problem to be solved. Summary of the Invention

[0003] The purpose of the embodiments of the present invention is to provide a method for generating padding values for convolution operations, an application method, a device, a computer-readable storage medium, an electronic device, and a computer program product, which can automatically fill preset values for input data blocks during convolution operations, without pre-storing multiple preset value data in the storage space, thereby effectively improving the utilization rate of the storage space and reducing the bandwidth of data transfer.

[0004] The first aspect of the embodiments of the present invention provides a method for generating padding values for convolution operations, including:

[0005] Obtaining an input data block and corresponding original attribute information from a first storage space; wherein, the original attribute information includes: the size data of the input data block in the original spatial structure and the spatial coordinates of each data point;

[0006] Obtaining the convolution starting point coordinates, and the position offsets of each weight in the convolution kernel from the center of the convolution kernel;

[0007] According to the convolution starting point coordinates, the size data, and the position offsets corresponding to each weight, obtaining a mask sequence corresponding to each weight; wherein, the mask sequence is used to indicate the padding position of the preset value corresponding to the weight in the convolution operation;

[0008] According to the mask sequence, obtaining a convolution coverage sequence corresponding to each weight.

[0009] Optionally, the original spatial structure is an N-dimensional tensor; where N≥1.

[0010] Optionally, the input data block is a one-dimensional array obtained by continuously storing the original spatial structure in row-major order.

[0011] Optionally, acquiring a mask sequence corresponding to each weight according to the convolution starting point coordinates, the size data and the position offset corresponding to each weight includes:

[0012] Starting from the data point corresponding to the coordinates of the convolution starting point, the center of the convolution kernel sequentially traverses the remaining data points in the input data block;

[0013] When the center of the convolution kernel traverses to the current data point, taking the spatial coordinates of the current data point as a reference, combined with the size data and the position offset, it is determined whether each of the weights needs to be masked to the preset value at the current coverage position, so as to obtain a mask sequence corresponding to each of the weights; wherein the mask sequence is used to indicate the filling position of the preset value of the corresponding weight in the convolution operation.

[0014] Optionally, the mask value in the mask sequence is obtained by the following formula:

[0015] ;

[0016] in, The mask sequence The mask value, used to represent the convolution kernel center traversed to the When there are data points, whether the corresponding weight needs to be masked to a preset value at the current coverage position, Indicates that the mask needs to be the default value. Indicates that the mask does not need to be the preset value; The corresponding weight is The position offset from the center of the convolution kernel in the dimensional direction; For the Data points in Coordinate values ​​in the dimensional direction; The original spatial structure is Dimensional data in the dimensional direction; ; N is the number of dimensions of the original spatial structure.

[0017] Optionally, acquiring a convolution cover sequence corresponding to each of the weights according to the mask sequence includes:

[0018] According to the convolution starting point coordinates and the position offset of each weight, the mask starting point position of the corresponding mask sequence in the second storage space is obtained; wherein the second storage space is a temporary storage area where the input data block is stored after being taken out from the first storage space;

[0019] Based on each of the mask starting positions, apply the corresponding mask sequence to the corresponding data range in the second storage space to obtain a convolution coverage sequence corresponding to each weight.

[0020] The second aspect of the embodiments of the present invention provides a method for applying padding values in convolution operations, including:

[0021] Obtain a convolution coverage sequence corresponding to each weight in the convolution kernel; wherein, the convolution coverage sequence is obtained by using the generation method of padding values in convolution operations according to any one of the above first aspects;

[0022] Calculate the product between each weight and the corresponding convolution coverage sequence to obtain a corresponding weight response sequence;

[0023] Perform element summation on all the weight response sequences at corresponding positions to obtain a convolution result sequence.

[0024] The third aspect of the embodiments of the present invention provides a device for generating padding values in convolution operations, including:

[0025] A data acquisition module, configured to acquire an input data block and corresponding original attribute information from a first storage space; wherein, the original attribute information includes: size data of the input data block in the original space structure and spatial coordinates of each data point;

[0026] A convolution positioning module, configured to acquire convolution starting coordinates and the position offset of each weight in the convolution kernel from the center of the convolution kernel;

[0027] A mask generation module, configured to obtain a mask sequence corresponding to each weight according to the convolution starting coordinates, the size data, and the position offset corresponding to each weight; wherein, the mask sequence is used to indicate the padding position of the preset value corresponding to the weight in the convolution operation;

[0028] A mask application module, configured to obtain a convolution coverage sequence corresponding to each weight according to the mask sequence.

[0029] The fourth aspect of the embodiments of the present invention provides a device for applying padding values in convolution operations, including:

[0030] A sequence acquisition module, configured to acquire a convolution coverage sequence corresponding to each weight in the convolution kernel; wherein, the convolution coverage sequence is obtained by using the generation method of padding values in convolution operations according to any one of the above first aspects;

[0031] A weight response module, configured to calculate the product between each weight and the corresponding convolution coverage sequence to obtain a corresponding weight response sequence;

[0032] A result output module for summing up elements of all the weight response sequences at corresponding positions to obtain a convolution result sequence.

[0033] In a fifth aspect embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored computer program. Wherein, when the computer program runs, it controls the device where the computer-readable storage medium is located to execute the method for generating padding values for convolution operations according to any one of the above first aspects, or the method for applying padding values for convolution operations according to the second aspect above.

[0034] In a sixth aspect embodiment of the present invention, a computer program product is provided, including a computer program. The computer program, when executed by a processor, implements the method for generating padding values for convolution operations according to any one of the above first aspects, or the method for applying padding values for convolution operations according to the second aspect above.

[0035] In a seventh aspect embodiment of the present invention, an electronic device is further provided, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the method for generating padding values for convolution operations according to any one of the above first aspects, or the method for applying padding values for convolution operations according to the second aspect above.

[0036] Compared with the prior art, the embodiments of the present invention provide a method for generating padding values for convolution operations, an application method, a device, a computer-readable storage medium, an electronic device, and a computer program product, having the following beneficial effects: In the embodiments of the present invention, the input data block and the corresponding original attribute information are directly obtained from the first storage space, and according to the convolution starting point coordinates, size data, and the position offset corresponding to each weight, the mask sequence corresponding to each weight is obtained. Then, the padding position mask in each mask sequence is set to a preset value to obtain the convolution coverage sequence corresponding to each weight. Therefore, the present invention can automatically fill the input data block with a preset value during convolution operations, without pre-storing multiple preset value data in the storage space, thereby effectively improving the utilization rate of the storage space and reducing the bandwidth of data transfer. Description of the Drawings

[0037] Figure 1 is a flowchart of an embodiment of the method for generating padding values for convolution operations provided by the present invention;

[0038] Figure 2 is a schematic diagram of an embodiment of mask sequence generation and application corresponding to a one-dimensional convolution kernel provided by the present invention;

[0039] Figure 3It is a schematic diagram of an embodiment for generating and applying a mask sequence corresponding to a two-dimensional convolution kernel provided by the present invention;

[0040] Figure 4 It is a schematic flowchart of an embodiment of an application method for padding values in convolution operations provided by the present invention;

[0041] Figure 5 It is a schematic structural diagram of an embodiment of a generating device for padding values in convolution operations provided by the present invention;

[0042] Figure 6 It is a schematic structural diagram of an embodiment of an application device for padding values in convolution operations provided by the present invention;

[0043] Figure 7 It is a schematic structural diagram of an embodiment of an electronic device provided by the present invention;

[0044] Figure 8 It is a schematic structural diagram of an embodiment of an artificial intelligence processor provided by the present invention. Detailed Embodiments

[0045] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art in the technical field of the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0046] The artificial intelligence processor involved in the present invention may be any one of a CPU (Central Processing Unit, central processing unit), a GPU (Graphics Processing Unit, graphics processing unit), a TPU (Tensor Processing Unit, tensor processing unit), an NPU (Neural network Processing Unit, neural network processing unit), a DPU (Deeplearning Processing Unit, deep learning processing unit), an APU (Accelerated Processing Unit, accelerated processing unit), and a GPGPU (General-Purpose computing on Graphics Processing Unit, general-purpose graphics processing unit), which is determined when the present invention is applied to specific products or technologies in the embodiments.

[0047] Currently, there are mainly two methods for implementing convolution operations: the Image to Column method and the hardware implementation method based on systolic arrays. During the implementation of convolution operations, the inventors found that these two methods have the following defects:

[0048] (1) The Image to Column method converts the multi-dimensional convolution operation into a two-dimensional matrix multiplication operation, but still needs to store a large amount of reused data, resulting in a high bandwidth occupancy for data transfer.

[0049] (2) Although the hardware implementation method based on systolic arrays can accelerate convolution operations at the physical level, its implementation control is complex and the cost is high; in addition, the large array area is prone to a high scrap rate, which limits the large-scale application of systolic arrays.

[0050] Therefore, on the basis of not adding additional hardware, how to effectively reduce the bandwidth required for data transfer in convolution operations is an urgent problem to be solved.

[0051] See Figure 1 , which is a schematic flowchart of an embodiment of the method for generating padding values for convolution operations provided by the present invention.

[0052] To solve the above problems, an embodiment of the first aspect of the present invention provides a method for generating padding values for convolution operations, including steps S11 to S14, specifically as follows:

[0053] Step S11: Obtain an input data block and corresponding original attribute information from a first storage space; wherein, the original attribute information includes: size data of the input data block in the original spatial structure and spatial coordinates of each data point.

[0054] Step S12: Obtain the convolution starting point coordinates and the position offsets of each weight in the convolution kernel from the convolution kernel center.

[0055] Step S13: According to the convolution starting point coordinates, the size data, and the position offsets corresponding to each weight, obtain a mask sequence corresponding to each weight; wherein, the mask sequence is used to indicate the padding position of the preset value corresponding to the weight in the convolution operation.

[0056] Step S14: According to the mask sequence, obtain a convolution coverage sequence corresponding to each weight.

[0057] It should be noted that the original spatial structure is the original organizational state of the input data block before storage conversion. For example, the original spatial structure of a grayscale image is a two-dimensional matrix, and the original spatial structure of a color image is a three-dimensional matrix. The convolution coverage sequence is the input sequence after filling with preset values, that is, the data covered by the corresponding weights in the convolution kernel during convolution sliding, including: the original data and / or the filled preset values. In the embodiments of the present invention, the filled preset value can be zero or any other arbitrary value, depending on the specific filling operation used. For example, the preset value filled in the zero-padding operation is 0; the preset value filled in the constant-padding operation is the specified constant value; the preset value filled in the repeating-padding operation is the edge value of the input data block; the preset value filled in the mirror-padding operation is the filled value generated by reflecting the edge value.

[0058] Specifically, during implementation, first obtain the input data block participating in the convolution operation and its original attribute information from the first storage space (such as a mechanical hard disk, a solid-state drive, and memory, etc.); where the original attribute information includes: the size data of the input data block in the original spatial structure (that is, the size scale in each dimension), and the spatial coordinates of each data point in the original spatial structure. Subsequently, obtain the convolution starting coordinate, that is, the starting coordinate of the convolution kernel for the sliding window operation on the input data block. At the same time, the position offset of each weight in the convolution kernel relative to the convolution center is also obtained, which is used to determine the specific coverage position points of the weights during the subsequent convolution sliding window process; for example, the position offset of a 3×3 convolution kernel is: ; where is the position offset of the corresponding weight and the convolution kernel center in the vertical direction is -1, and the position offset in the horizontal direction is 1. Then, according to the convolution starting coordinate, the size data, and the position offset corresponding to each weight, obtain the mask sequence corresponding to each weight; finally, mask the filling position indicated in each mask sequence with the preset value to obtain the convolution coverage sequence corresponding to each weight (that is, the coverage data including the filled preset value); for example, for a convolution kernel, since it contains 9 weights, 9 convolution coverage sequences will ultimately be obtained.

[0059] It is worth noting that the convolution operation is a spatial correlation calculation. Therefore, the original attribute information involved in the embodiments of the present invention is also default stored in the prior art (such as for determining the position of the convolution starting point), without introducing additional memory overhead. In addition, compared with the prior art that needs to pre-store multiple preset value data in the storage space, the embodiments of the present invention can automatically fill the preset value for the input data block during the convolution operation, thereby effectively improving the utilization rate of the storage space and reducing the bandwidth of data transfer.

[0060] In an alternative embodiment, the original spatial structure is an N-dimensional tensor; where N≥1.

[0061] It should be noted that the convolution coverage sequences corresponding to the weights of the convolution kernel are obtained in the embodiments of the present invention. When calculating the convolution result, there is no need to consider the local sliding window operation of the convolution kernel in the original spatial structure. Specifically, only each weight needs to be multiplied by its corresponding convolution coverage sequence to obtain a weight response sequence; then, the element summation of all weight response sequences at the corresponding positions can be performed to obtain the convolution result sequence. There is no need to perform local calculations on the receptive field area of the convolution kernel during each sliding window operation, but the sliding window process of the convolution kernel in the original spatial structure is decoupled into the position calculation of parallel convolution coverage sequences. Based on this, the embodiments of the present invention have strong versatility and adaptability, and can be widely applied to various original spatial structures from one-dimensional to high-dimensional; for example, one-dimensional time series data, two-dimensional grayscale images, three-dimensional color images, and multi-dimensional sensor data, etc. At the same time, whether it is a small-sized convolution kernel or a large-sized convolution kernel, the embodiments of the present invention can be flexibly adapted.

[0062] In an optional embodiment, the input data block is a one-dimensional array obtained by continuously storing the original spatial structure in row-major order.

[0063] It should be noted that the layout of the input data block in the first storage space is a one-dimensional array obtained by continuously storing the data points in the original spatial structure in row-major order, that is, the N-dimensional data is expanded by rows into one-dimensional data and continuously stored in the first storage space. This matches the default sliding of the convolution kernel in the row direction first, so as to ensure the continuity of memory access to the greatest extent and reduce the access latency.

[0064] In an optional embodiment, the obtaining of the mask sequence corresponding to each weight according to the convolution starting coordinate, the size data, and the position offset corresponding to each weight includes:

[0065] Starting from the data point corresponding to the convolution starting coordinate, the center of the convolution kernel traverses the remaining data points in the input data block in sequence;

[0066] When the center of the convolution kernel traverses to the current data point, with the spatial coordinate of the current data point as a reference, combining the size data and the position offset, it is determined whether each weight needs to be masked with the preset value at the current coverage position to obtain the mask sequence corresponding to each weight; wherein, the mask sequence is used to indicate the filling position of the preset value of the corresponding weight in the convolution operation.

[0067] It should be noted that starting from the data point corresponding to the coordinates of the starting point of the convolution, each remaining data point in the input data block is traversed in turn according to the convolution sliding window order. During the traversal of the convolution kernel center, for each weight, the current coverage position can be determined by its position offset. If the coverage position exceeds any boundary in the original spatial structure, it is determined that the coverage position needs to be masked to a preset value. When the convolution kernel center traverses to the last data point of the input data block, the mask sequence corresponding to each weight can be obtained.

[0068] In an optional embodiment, the mask value in the mask sequence is obtained by the following formula:

[0069] ;

[0070] in, The mask sequence The mask value, used to represent the convolution kernel center traversed to the When there are data points, whether the corresponding weight needs to be masked to a preset value at the current coverage position, Indicates that the mask needs to be the default value. Indicates that the mask does not need to be the preset value; The corresponding weight is The position offset from the center of the convolution kernel in the dimensional direction; For the Data points in Coordinate values ​​in the dimensional direction; The original spatial structure is Dimensional data in the dimensional direction; ; N is the number of dimensions of the original spatial structure.

[0071] For example, Figure 2 FIG. 1 is a schematic diagram of an embodiment of the generation and application of a mask sequence corresponding to a one-dimensional convolution kernel provided by the present invention. Figure 2In (a), the preset value for padding is 0. The 1×3 convolutional kernel contains three weights, w0, w1, and w2, and the position offsets (filter_offset(1), filter_offset(2)) are (0, -1), (0, 0), and (0, 1) respectively. Among them, filter_offset(1) is the position offset of the corresponding weight from the center of the convolutional kernel in the vertical direction (d = 1), and filter_offset(2) is the position offset in the horizontal direction (d = 2). Taking the example of obtaining the mask sequence for weight w0, when the center of the convolutional kernel is at the convolutional starting coordinate (0, 0), through the above mask value calculation formula, mask(1) = 1 is obtained, and the mask at the position point (coor(i, 1)+filter_offset(1), coor(i, 2)+filter_offset(2)) needs to be the preset value, that is, (0, -1); when the coordinates covered by the center of the convolutional kernel are (0, 1), through the above mask value calculation formula, mask(1) = 0 is obtained; and so on, until the convolutional center traverses to the last data point of the input data block and then ends. Finally, the mask sequence for w0 is obtained, as shown in Figure 2 Figure (b).

[0072] In an alternative embodiment, the obtaining of the convolutional coverage sequence corresponding to each of the weights according to the mask sequence includes:

[0073] According to the convolutional starting coordinate and the position offset of each of the weights, obtaining the mask starting position of the corresponding mask sequence in the second storage space; wherein, the second storage space is a temporary storage area where the input data block is stored after being taken out from the first storage space;

[0074] Based on each of the mask starting positions, applying the corresponding mask sequence to the corresponding data interval in the second storage space to obtain the convolutional coverage sequence corresponding to each of the weights.

[0075] It should be noted that in the second storage space, the data at the front and back positions of the input data block, such as Figure 2 the positions ① and ② in, and Figure 3 the data at the position in, on the one hand, may come from the data additionally taken out from the first storage space; for example, during the process of batch processing of image data, usually multiple pictures' data are continuously read. Then, for the input data block corresponding to a certain picture, the data at its front and back positions in the second storage space are the data of other pictures read from the first storage space. On the other hand, it may also come from the data generated by other instructions and are successively stored at the front and back positions of the input data block.

[0076] For a clearer description of the technical solutions provided by the embodiments of the present invention, some specific embodiments are provided for reference below:

[0077] As Figure 2 shown, taking the weight w0 in the 1×3 convolution kernel as an example, assuming that the convolution starting coordinates are (0,0) and the position offset of w0 is (0, -1), then the mask starting position corresponding to w0 is (0, -1), that is, the first position before the convolution starting point (marked as position ①); the mask sequence of w0 will start masking from position ① until the subsequent 25th data point (spatial coordinates are (4,3)), so as to obtain the convolution coverage sequence corresponding to w0; this masking process does not occupy additional memory space. From Figure 2 it can be seen that for the weight w0, the 0 value that needs to be filled to the leftmost in the original spatial structure is obtained by masking the rightmost data in the previous row as 0. Similarly, the position offset of the weight w2 is (0, 1), and the corresponding mask starting coordinates are (0, 1). The mask sequence of w2 will start masking from the position after the convolution starting point until it ends at position ②, so as to obtain the convolution coverage sequence corresponding to w2.

[0078] As Figure 3 shown, it is a schematic diagram of an embodiment of the mask sequence generation and application corresponding to the two-dimensional convolution kernel provided by the present invention. Taking the weight q0 in the 3×3 convolution kernel as an example, assuming that the convolution starting coordinates are (0,0) and the position offset of q0 is (-1, -1), then the mask starting position corresponding to q0 is (-1, -1), that is, the first 6 positions before the convolution starting point (marked as position x1); the mask sequence of q0 will start masking from position x1 until the subsequent 25th data point (spatial coordinates are (3,3)), so as to obtain the convolution coverage sequence corresponding to q0.

[0079] Refer to Figure 4 , which is a schematic flowchart of an embodiment of the application method of the padding value in the convolution operation provided by the present invention.

[0080] The second aspect of the embodiments of the present invention provides an application method for the padding value in the convolution operation, including steps S21 to S23, specifically as follows:

[0081] Step S21: Obtain the convolution coverage sequence corresponding to each weight in the convolution kernel; wherein, the convolution coverage sequence is obtained by using the convolution operation padding value generation method described in any one of the above first aspect embodiments;

[0082] Step S22: Calculate the product between each weight and the corresponding convolution coverage sequence to obtain the corresponding weight response sequence;

[0083] Step S23: Perform element summation on all the weight response sequences at the corresponding positions to obtain the convolution result sequence.

[0084] Specifically, in combination with the above embodiments, after obtaining the convolution coverage sequence corresponding to each weight in the convolution kernel through any of the embodiments of the first aspect above, perform element-wise multiplication on each weight and its corresponding convolution coverage sequence to obtain the corresponding weight response sequence. Then, sum the elements of all weight response sequences at the corresponding positions to finally obtain an accurate convolution result sequence, which is equivalent to the convolution operation result of the input data after padding processing and the convolution kernel. The embodiments of the present invention decouple the sliding window process of the convolution kernel in the original space structure into the position calculation of parallel convolution coverage sequences, and can efficiently and accurately complete the convolution operation.

[0085] See Figure 5 , which is a schematic structural diagram of an embodiment of the device for generating padding values for convolution operations provided by the present invention.

[0086] The embodiments of the third aspect of the present invention provide a device for generating padding values for convolution operations, including:

[0087] A data acquisition module 11, configured to acquire an input data block and corresponding original attribute information from a first storage space; wherein, the original attribute information includes: size data of the input data block in the original space structure and the spatial coordinates of each data point;

[0088] A convolution positioning module 12, configured to acquire the convolution starting point coordinates and the position offset of each weight in the convolution kernel from the center of the convolution kernel;

[0089] A mask generation module 13, configured to acquire a mask sequence corresponding to each weight according to the convolution starting point coordinates, the size data, and the position offset corresponding to each weight; wherein, the mask sequence is used to indicate the padding position of the preset value corresponding to the weight in the convolution operation;

[0090] A mask application module 14, configured to acquire a convolution coverage sequence corresponding to each weight according to the mask sequence.

[0091] It should be noted that the device for generating padding values for convolution operations provided by the embodiments of the third aspect of the present invention can implement all the processes of the method for generating padding values for convolution operations described in any of the embodiments of the first aspect above. The functions and the achieved technical effects of each module and unit in the device respectively correspond to the functions and the achieved technical effects of the method for generating padding values for convolution operations described in any of the embodiments of the first aspect above, and will not be elaborated here.

[0092] See Figure 6 , which is a schematic structural diagram of an embodiment of the device for applying padding values for convolution operations provided by the present invention.

[0093] An embodiment of the fourth aspect of the present invention provides an application device for convolution operation padding values, including:

[0094] A sequence acquisition module 21, configured to acquire a convolution coverage sequence corresponding to each weight in the convolution kernel; wherein, the convolution coverage sequence is obtained by using the convolution operation padding value generation method described in any embodiment of the first aspect above;

[0095] A weight response module 22, configured to calculate the product between each weight and the corresponding convolution coverage sequence to obtain a corresponding weight response sequence;

[0096] A result output module 23, configured to perform element summation on all the weight response sequences at corresponding positions to obtain a convolution result sequence.

[0097] It should be noted that the application device for convolution operation padding values provided in the embodiment of the fourth aspect of the present invention can implement all the processes of the convolution operation padding value application method described in the embodiment of the second aspect above. The functions and the achieved technical effects of each module and unit in the device respectively correspond to those of the convolution operation padding value application method described in the embodiment of the second aspect above, and will not be elaborated here.

[0098] An embodiment of the fifth aspect of the present invention further provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program runs, it controls the device where the computer-readable storage medium is located to execute the convolution operation padding value generation method described in any embodiment of the first aspect above, or the convolution operation padding value application method described in the embodiment of the second aspect above.

[0099] An embodiment of the sixth aspect of the present invention provides a computer program product, including a computer program, which implements the convolution operation padding value generation method described in any embodiment of the first aspect above, or the convolution operation padding value application method described in the embodiment of the second aspect above when executed by a processor.

[0100] See Figure 7 , which is a schematic structural diagram of an embodiment of an electronic device provided by the present invention.

[0101] An embodiment of the seventh aspect of the present invention further provides an electronic device, including a processor 31, a memory 32, and a computer program stored in the memory 32 and configured to be executed by the processor 31. When the processor 31 executes the computer program, it implements the convolution operation padding value generation method described in any embodiment of the first aspect above, or the convolution operation padding value application method described in the embodiment of the second aspect above.

[0102] Preferably, the computer program may be divided into one or more modules / units (such as computer program 1, computer program 2, ……), and the one or more modules / units are stored in the memory 32 and executed by the processor 31 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of accomplishing specific functions, and these instruction segments are used to describe the execution process of the computer program in the electronic device.

[0103] The processor 31 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor 31 may also be any conventional processor. The processor 31 is the control center of the electronic device and connects various parts of the electronic device through various interfaces and circuits.

[0104] The memory 32 mainly includes a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function, etc., and the data storage area may store relevant data, etc. In addition, the memory 32 may be a high-speed random access memory, or may also be a non-volatile memory, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., or the memory 32 may also be other volatile solid-state storage devices.

[0105] It should be noted that the above-mentioned electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that Figure 7 The structural block diagram shown is only an example of the structure of the above-mentioned electronic device and does not constitute a limitation on the structure of the above-mentioned electronic device. The above-mentioned electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different components.

[0106] See Figure 8 , which is a schematic structural diagram of an embodiment of the artificial intelligence processor provided by the present invention;

[0107] The artificial intelligence processor provided by the embodiment of the present invention includes: an instruction unit 41, an instruction parsing unit 42, a mask calculation unit 43, a second memory 45, and a result calculation unit 46; wherein, the instruction unit 41 is used to generate and send convolution operation instructions; the instruction parsing unit 42 is used to receive and parse the convolution operation instructions, obtain the control information in the instructions, and send the control information to the mask calculation unit 43; the mask calculation unit 43 is used to execute the generation method of the convolution operation padding value described in any embodiment of the first aspect after receiving the control information; the result calculation unit 46 is used to receive the convolution coverage sequence transmitted by the mask calculation unit 43 and execute the application method of the convolution operation padding value described in the embodiment of the second aspect; the second memory 45 is an intermediate storage device for storing the input data block loaded by the mask calculation unit 43 from the first memory.

[0108] It should be noted that Figure 8 the first memory 44 in can be located in the artificial intelligence processor, or in an external memory or other external devices, and is used to store the input data block.

[0109] The above are only the preferred embodiments of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and deformations can be made, and these improvements and deformations should also be regarded as the protection scope of the present invention.

Claims

1. A method for generating padding values for convolution operations, characterized in that, including: Obtaining an input data block and corresponding original attribute information from a first storage space; wherein, the original attribute information includes: size data of the input data block in an original space structure and spatial coordinates of each data point; Obtaining convolution starting coordinates, and position offsets of each weight in a convolution kernel relative to the center of the convolution kernel; Obtaining a mask sequence corresponding to each weight according to the convolution starting coordinates, the size data, and the position offset corresponding to each weight; wherein, the mask sequence is used to indicate a filling position of a preset value of the corresponding weight in a convolution operation; Obtaining a convolution coverage sequence corresponding to each weight according to the mask sequence; wherein, each convolution coverage sequence is used to represent data covered by the corresponding weight during the sliding of the convolution kernel; The obtaining the convolution coverage sequence corresponding to each weight according to the mask sequence includes: Obtaining a mask starting position of the corresponding mask sequence in a second storage space according to the convolution starting coordinates and the position offset of each weight; wherein, the second storage space is a temporary storage area where the input data block is stored after being taken out from the first storage space; Based on each mask starting position, applying the corresponding mask sequence to a corresponding data interval in the second storage space to obtain a convolution coverage sequence corresponding to each weight.

2. The method for generating padding values for convolution operations according to claim 1, wherein, The original space structure is an N-dimensional tensor; wherein, N≥1.

3. The method for generating the padding value of the convolution operation according to claim 1, wherein The input data block is a one-dimensional array obtained by continuously storing the original space structure in row-major order.

4. The method for generating the padding value of the convolution operation according to claim 1, wherein, The obtaining the mask sequence corresponding to each weight according to the convolution starting coordinates, the size data, and the position offset corresponding to each weight includes: Starting from the data point corresponding to the convolution starting coordinates, sequentially traversing the remaining data points in the input data block with the center of the convolution kernel; When the center of the convolution kernel traverses to the current data point, taking the spatial coordinates of the current data point as a reference, and combining the size data and the position offset, determining whether each weight needs to be masked with the preset value at the current coverage position, so as to obtain a mask sequence corresponding to each weight.

5. The method for generating a padding value for a convolution operation according to claim 4, wherein The mask value in the mask sequence is obtained through the following formula: ； Among them, is the m-th mask value in the mask sequence , used to represent whether the corresponding weight needs to be masked to a preset value at the current covered position when the center of the convolution kernel traverses to the n-th data point . means that it needs to be masked to a preset value, means that it does not need to be masked to a preset value; is the position offset of the corresponding weight from the center of the convolution kernel in the d-th direction; is the coordinate value of the n-th data point in the d-th direction; is the size data of the original spatial structure in the d-th direction; ; N is the number of dimensions of the original spatial structure. ; N is the number of dimensions of the original spatial structure.

6. An application method for padding values in a convolution operation, characterized in that, including: Obtaining a convolution coverage sequence corresponding to each weight in a convolution kernel; wherein, the convolution coverage sequence is obtained by adopting the method for generating filled values in the convolution operation according to any one of claims 1 to 5; Calculating a product between each weight and the corresponding convolution coverage sequence to obtain a corresponding weight response sequence; Performing element summation on all the weight response sequences at corresponding positions to obtain a convolution result sequence.

7. A generating device for padding values of a convolution operation, characterized in that A device for implementing the method for generating filled values in the convolution operation according to any one of claims 1 to 5, the device includes: A data acquisition module, configured to obtain an input data block and corresponding original attribute information from a first storage space; wherein, the original attribute information includes: size data of the input data block in an original space structure and spatial coordinates of each data point; A convolution positioning module for obtaining the convolution starting point coordinates and the position offsets of each weight in the convolution kernel from the center of the convolution kernel; A mask generation module for obtaining a mask sequence corresponding to each weight according to the convolution starting point coordinates, the size data, and the position offset corresponding to each weight; wherein the mask sequence is used to indicate the filling position of the preset value of the corresponding weight in the convolution operation; A mask application module for obtaining a convolution coverage sequence corresponding to each weight according to the mask sequence.

8. An application device for padding values in a convolution operation, characterized in that, Comprising: A sequence acquisition module for obtaining a convolution coverage sequence corresponding to each weight in the convolution kernel; wherein the convolution coverage sequence is obtained by using the method for generating filled values in the convolution operation according to any one of claims 1 to 5; A weight response module for calculating the product between each weight and the corresponding convolution coverage sequence to obtain a corresponding weight response sequence; A result output module for performing element summation on all the weight response sequences at corresponding positions to obtain a convolution result sequence.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program; wherein the computer program, when running, controls the device where the computer-readable storage medium is located to execute the method for generating filled values in the convolution operation according to any one of claims 1 to 5, or the method for applying filled values in the convolution operation according to claim 6.

10. A computer program product, characterized in that, Comprising a computer program which, when executed by a processor, implements the method for generating filled values in the convolution operation according to any one of claims 1 to 5, or the method for applying filled values in the convolution operation according to claim 6.

11. An electronic device, characterized in that, Comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the method for generating filled values in the convolution operation according to any one of claims 1 to 5, or the method for applying filled values in the convolution operation according to claim 6.

Images