Convolution calculation methods, apparatus, computer equipment and storage media

Abstract

This invention discloses a convolution calculation method, apparatus, computer device, and storage medium, applied in the field of artificial intelligence technology, to improve the computational efficiency of convolution calculations in neural networks. The method provided by this invention includes: generating corresponding input data storage rules based on input data and the number of input channels, and storing input data based on the input data storage rules; storing weight data according to the input data storage rules and the correspondence between the input channels and output channels; performing convolution calculations based on the input data and the weight data in the order of row data to obtain the row convolution result for each output channel; and accumulating the row convolution results for each output channel to obtain the convolution calculation result.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a convolution calculation method, apparatus, computer device, and storage medium. Background Technology

[0002] Currently, deep learning technology is developing rapidly and has been widely applied in many fields. Neural network models in deep learning have received much attention.

[0003] The training and derivation of neural network models place high demands on computing power. For some fully connected convolutional neural networks with relatively low computational cost, some computing units stop working during the convolution calculation process because the writing of convolution results causes the computing units to stop working, resulting in low utilization of computing units and poor computing speed. Summary of the Invention

[0004] This invention provides a convolution calculation method, apparatus, computer device, and storage medium to improve the calculation speed of convolution calculation.

[0005] A convolution calculation method, comprising:

[0006] Based on the input data and the number of input channels, generate corresponding input data storage rules, and store the input data based on the input data storage rules;

[0007] According to the input data storage rules, weight data is stored according to the correspondence between the input channels and the output channels;

[0008] Following the order of the row data, convolution calculations are performed based on the input data and the weight data to obtain the row convolution result for each output channel;

[0009] The row convolution results of each output channel are accumulated to obtain the convolution calculation result.

[0010] A convolution computing device, comprising:

[0011] The input data storage module is used to generate corresponding input data storage rules based on the input data and the number of input channels, and to store the input data based on the input data storage rules;

[0012] The weighted data storage module is used to store weighted data according to the input data storage rules and the correspondence between the input channels and the output channels;

[0013] The row convolution result calculation module is used to perform convolution calculation based on the input data and the weight data according to the order of the row data to obtain the row convolution result of each output channel;

[0014] The convolution result generation module is used to accumulate the row convolution results of each output channel to obtain the convolution calculation result.

[0015] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the convolution calculation method described above.

[0016] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described convolution calculation method.

[0017] The convolution calculation method, apparatus, computer device, and storage medium provided by this invention acquire information about the input data and the number of input channels. Based on the row data of the input data, an input data storage rule is generated. According to the convolution calculation rule, the input data of each input channel is stored. Based on the data storage method and the mapping relationship, weight data is stored according to the correspondence between input channels and output channels. Based on the storage method of input data and weight data, and according to the row data order, the row convolution result of each output channel is calculated based on the input data and weight data of each row. Then, the row convolution data of all output channels are accumulated to obtain the convolution calculation result. Convolution calculation is performed according to the row data order of the input data, with the row data order as the first priority and the input channel order as the second priority. The input data is stored to facilitate the rapid calculation of the row convolution result of each output channel according to the calculation cycle characteristics of convolution calculation, reducing the convolution calculation cycle of traversing all rows of data, improving the efficiency of convolution calculation in the convolution calculation cycle, and further improving the convolution calculation speed. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of an application environment for the convolution calculation method in one embodiment of the present invention;

[0020] Figure 2 This diagram illustrates another application environment of the convolution calculation method according to one embodiment of the present invention.

[0021] Figure 3 This is a flowchart of a convolution calculation method according to an embodiment of the present invention;

[0022] Figure 4 This is a flowchart of a convolution calculation method in another embodiment of the present invention;

[0023] Figure 5 This is a flowchart of a convolution calculation method in another embodiment of the present invention;

[0024] Figure 6 This is a flowchart of a convolution calculation method in another embodiment of the present invention;

[0025] Figure 7 This is a flowchart of a convolution calculation method in another embodiment of the present invention;

[0026] Figure 8 This is a schematic diagram of the structure of a convolution calculation device in one embodiment of the present invention;

[0027] Figure 9 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] In this embodiment, the convolution calculation method can be applied to, for example... Figure 2 The neural network processor shown can be a programmable logic device, such as a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a convolutional neural network processor (NPU), etc.

[0030] Among them, such as Figure 2 As shown, the neural network processor includes a data storage module 10 and a computation module 20. The data storage module 10 includes an input data storage module 11 and a weight data storage module 12. The computation module 20 includes at least one computation unit 21. The number of computation units 21 can be set according to the actual situation. The required number of computation modules 20 can be considered comprehensively based on the total amount of computation and the amount of computation that each computation unit 21 can handle. Figure 2 The number of calculation units 21 is listed only for illustrative purposes.

[0031] The input data storage module 11 is used to store the input data that needs to be convolutionally calculated. The input data storage module 11 includes a data interleaving unit 111 and a data storage unit 112. The data interleaving unit 111 is used to change the storage method of the input data to obtain the input data storage rules. The data storage unit 112 is used to store the input data according to the input data storage rules.

[0032] The weight data storage module 12 is used to store the weight data that needs to be calculated for convolution. The weight data storage module 12 includes a weight interleaving unit 121 and a weight storage unit 122. The weight interleaving unit 121 is used to change the storage method of the weight data, and the weight storage unit 122 is used to store the weight data according to the storage method of the weight interleaving unit 121.

[0033] To enhance computing power, the computing module 20 generally includes multiple identical computing units 21, each of which works in parallel and independently.

[0034] Each computation unit 21 includes an input data cache unit 211, a weight data cache unit 212, a convolution computation unit 213, and a result cache unit 214. The input data cache unit 211 stores the input data required for the computation unit 21 to perform convolution computation. The weight data cache unit 212 stores the weight data required for the computation unit 21 to perform convolution computation. The convolution computation unit 213 is responsible for the convolution computation of its own computation unit 21. To enhance computational power, each convolution computation unit 213 generally supports parallel computation of multiple input and output channels. Figure 2 The number of convolution calculation units 213 is listed for illustrative purposes only. The result cache unit 214 is responsible for storing the intermediate results generated by the calculation unit 21 during the convolution calculation process, as well as the convolution result calculated by the calculation unit 21.

[0035] As an alternative implementation, the neural network processor can be used in an electronic device that includes at least a memory, a processor, at least one communication bus, and at least one neural network processor.

[0036] As an optional implementation, the electronic device includes a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. The hardware of the electronic device includes, but is not limited to: microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), and embedded devices.

[0037] The convolution calculation method provided in this invention can be applied to, for example... Figure 1 In this application environment, terminal devices communicate with the server via a network. These terminal devices can be, but are not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be a standalone server or a server cluster consisting of multiple servers.

[0038] Furthermore, embodiments of the present invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0039] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0040] A server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0041] System framework 100 may include terminal devices, a network, and a server. The network serves as the medium for providing a communication link between the terminal devices and the server. The network may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0042] Users can use terminal devices to interact with the server over the network to receive or send messages, etc.

[0043] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Eperts Group Audio Layer III), MP4 players (Moving Picture Eperts Group Audio Layer IV), laptops, and desktop computers, etc.

[0044] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0045] It should be noted that the convolution calculation method provided in this embodiment of the invention is executed by a server, and correspondingly, the convolution calculation device is located in the server.

[0046] It should be noted that, as an optional implementation, the execution subject of the convolution calculation method provided in this embodiment of the invention can be a chip for performing neural network convolution calculations, such as a digital signal processor (DSP), an advanced reduced instruction set computer machine (ASIC), or a field-programmable gate array (FPGA), or a neural network processor chip or other devices with convolution calculation functions.

[0047] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown in this embodiment is merely illustrative. Depending on the implementation needs, there can be any number of terminal devices, networks, and servers. The terminal devices in this embodiment can specifically correspond to application systems in actual production.

[0048] In one embodiment, such as Figure 3 As shown, a convolution calculation method is provided, which is then applied to... Figure 1 Taking the server in the example, the following steps are included:

[0049] S10: Generate corresponding input data storage rules based on the input data and the number of input channels, and store the input data based on the input data storage rules.

[0050] Specifically, the number of rows of input data and the number of input channels are obtained, and corresponding storage space is allocated according to the number of rows of input data and the number of input channels. The order of the number of rows of input data is taken as the first priority, and the order of the input channels is taken as the second priority. Input data storage rules are generated according to the first priority and the second priority, and the input data of each input channel is stored according to the input data storage rules.

[0051] The input data refers to the data that is fed into the convolutional layer for convolution calculations, and the number of input channels refers to the amount of two-dimensional information in the input data. For example, an RGB image as input data has three input channels: r, g, and b.

[0052] The number of rows in the input data refers to the data size in each input channel, i.e., the matrix width of the input data; the number of columns in the input data refers to the matrix height of the input data. In this embodiment, the width and height of the input data are the same, so the explanation is based on the number of rows in the input data.

[0053] For example, the input data has four input channels. The first input channel is represented as Ci_0, the second as Ci_1, the third as Ci_2, and the fourth as Ci_3. The first row of data in the first input channel is represented as Ci_0_0, and the second row of data in the first input channel is represented as Ci_0_1.

[0054] The input data storage rule prioritizes row order as the first priority and input channel order as the second priority. The first row of input data is stored first, and then the first row of input data is stored in the order of the input channels, namely the first input channel, the second input channel, the third input channel, and the fourth input channel, until all rows of input data are stored.

[0055] S20: According to the input data storage rules, weight data is stored according to the correspondence between input channels and output channels.

[0056] Specifically, in this embodiment, in order to reduce the complexity of the parameters, a fully connected layer, i.e., a convolution kernel of 1x1, is used for convolution calculation. A convolution kernel of one pixel size is used, and the number of input channels and output channels are the same.

[0057] Based on the correspondence between input and output channels, the weight data of the calculated output result of the first output channel is stored. Based on the order of the output channels and combined with the storage method of the input data, the weight data of each input channel and output channel are stored alternately.

[0058] For example, there are four input channels. The first output channel is represented as Co_0, the second output channel as Co_1, the third output channel as Co_2, and the fourth output channel as Co_3.

[0059] For example, first store the weight data from the first input channel to the first output channel, then store the weight data from the second input channel to the first output channel, then store the weight data from the first input channel to the second output channel, and finally store the weight data from the second input channel to the second output channel.

[0060] S30 performs convolution calculations based on the input data and weight data in the order of the row data to obtain the row convolution result of each output channel.

[0061] Specifically, following the order of the rows of input data, the row convolution result of the current row of input data for each input channel to each output channel is calculated.

[0062] Using the input data as an image for illustration, because the calculation method of the fully connected layer is adopted, a single convolution calculation through one input channel can only calculate a partial convolution result of one pixel. It is necessary to accumulate the partial convolution results of all input channels to obtain the convolution result of the output channel at one pixel.

[0063] For example, if you need to calculate the convolution result of the first output channel Co_0 at the first pixel, you need to calculate the partial convolution result of each input channel at the first pixel separately, and then accumulate the partial convolution results of each input channel at the first pixel to obtain the convolution result of the first output channel Co_0 at the first pixel.

[0064] In this embodiment, the partial convolution result of the first output channel Co_0 at the first pixel, i.e., the first row of image data, is first calculated. The partial convolution results of all rows are then summed to obtain the row convolution result of the first output channel Co_0 at the first pixel. The calculation process of the row convolution results of other output channels is the same as that of the first output channel, so it will not be described again.

[0065] It should be noted that in this embodiment, in calculating the row convolution result of the first output channel, it is necessary to calculate the convolution result of each row of image data in the first pixel of each input channel separately, and then accumulate the convolution results of all input channels to obtain the row convolution result of the first output channel.

[0066] For example, the convolution result of the first row of image data in the first input channel is Ci_0_0psum. Assuming the weight data of the first input channel Ci_0 and the first output channel Co_0 is k0, and the first input channel in the first row of image data is Ci_0_0, then by performing convolution calculation on Ci_0_0 and the weight data k0, we obtain Ci_0_0psum. We calculate the convolution result of each input channel in the first row of image data, and accumulate them to obtain the row convolution result of the first output channel in the first pixel.

[0067] S40: The row convolution results of each output channel are accumulated and calculated to obtain the convolution calculation result.

[0068] Specifically, based on the number of rows of input data, the row convolution result of each row of input data in each output channel is obtained. For each output channel, the row convolution result of each row of input data in that output channel is obtained, and all row convolution results are accumulated to obtain the initial convolution result of that output channel.

[0069] It should be noted that the initial convolution calculation results of all output channels are obtained by summing the row convolution results of each output channel. The final convolution calculation result is obtained by summing the initial convolution calculation results of each output channel.

[0070] The convolution calculation method provided in this invention obtains the information of the input data and the number of input channels. Based on the row data of the input data, it generates input data storage rules. According to the convolution calculation rules, it stores the input data of each input channel. Based on the data storage method and the mapping relationship, it stores weight data according to the correspondence between input channels and output channels. Based on the storage method of input data and weight data, and according to the order of row data, it calculates the row convolution result of each output channel based on the input data and weight data of each row. Then, it accumulates the row convolution data of all input channels to obtain the convolution calculation result. The convolution calculation is performed according to the row data order of the input data, with the row data order as the first priority and the input channel order as the second priority. This facilitates the rapid calculation of the row convolution result of each output channel according to the calculation cycle characteristics of convolution calculation, reduces the convolution calculation cycle of traversing all row data, improves the efficiency of convolution calculation in the convolution calculation cycle, and further improves the efficiency of convolution calculation.

[0071] Furthermore, as an optional implementation, in step S10, a corresponding input data storage rule is generated based on the input data and the number of input channels, and the input data is stored based on the input data storage rule, including:

[0072] S101, obtain the row data information and input channel information of the input data, take the order of the row data as the first priority, take the order of the input channels as the second priority, and take the first priority and the second priority as the input data storage rules.

[0073] S102, based on the input data storage rules, the input data of each row is stored in the order of the row data. For the same row data, the row input data of each input channel is stored in the order of the input channels.

[0074] Specifically, the number of rows of input data is obtained as row data information, with the order of the row data as the first priority. The number of input channels is obtained as input channel information, where each input channel corresponds to a specific set of input data, and the order of the input channels is the second priority. Input data for each input channel is stored row by row. For the same row of input data, the data is stored sequentially according to the order of the input channels until all rows of input data have been stored.

[0075] The first priority level refers to prioritizing the data according to the order of the rows, storing the input data of the first row, then the second row, and so on, until all rows of input data are stored. It should be noted that the order of the rows can be adjusted based on the actual scenario; the row order in this embodiment is for illustrative purposes only.

[0076] The second priority refers to sorting the data according to the order of the input channels, based on the first priority. The row data of the first input channel is stored first, then the row data of the second input channel, and so on, until all the row data of all input channels are stored. It should be noted that the order of the input channels can be set based on the actual scenario or the requirements of the convolution calculation; the order of the input channels in this embodiment is only for illustrative purposes.

[0077] It should be noted that each convolution calculation occurs in, for example, Figure 2 In the computation module 20 shown, multiple computation units 21 perform parallel computations. Taking one computation unit 21 as an example, the computation result after the computation unit 21 performs convolution computation through the convolution computation unit 213 is not the final output result. Therefore, the computation result output by the convolution computation unit 213 after performing convolution computation needs to be cached in the result cache unit 214 for subsequent accumulation to obtain the row convolution result of the output channel after one row of input data.

[0078] Each convolutional computation unit 213 requires one convolutional computation cycle to perform convolutional computation. A computational unit generally includes multiple convolutional computation units 213. Each convolutional computation unit 213 performs convolutional computation in parallel. However, the computation result can only be stored in the result buffer once in one convolutional computation cycle in order to effectively utilize the computational efficiency of the convolutional computation unit 213 in one convolutional computation cycle.

[0079] Therefore, in this embodiment, by changing the storage method of the input data and storing it in the order of the input channels, the row convolution results of the same output channel are calculated in parallel according to the input data, and the row convolution results of each output channel are quickly obtained. During the convolution calculation process, the data is stored in the order of the input channels, which effectively reduces the time for accessing the memory of the input data and improves the efficiency of the convolution calculation.

[0080] Furthermore, as an optional implementation, in step S20, according to the input data storage rules and the correspondence between input channels and output channels, the weight data is stored including:

[0081] S201, based on the input data storage rules, obtain the input data storage method.

[0082] S202: Based on the input data storage method, a weight data matrix is ​​generated according to the order of the input channels, and the weight data is stored based on the weight data matrix.

[0083] Specifically, the first priority and second priority of the input data are obtained according to the input data storage rules, and the storage method of the weight data is generated according to the first priority and the second priority.

[0084] Based on the storage method of the weight data, a weight data matrix is ​​generated, which includes multiple weight data and the weight data is stored based on the weight data matrix.

[0085] It should be noted that each input channel corresponds to a weight data for each output channel. In order to reduce the time for the convolution calculation unit to obtain the weight data, the storage method of the weight data should correspond one-to-one with the storage method of the input data. Based on the storage method of the input data and the weight data, the corresponding row convolution result is obtained after priority calculation during convolution calculation.

[0086] In this implementation, by changing the storage method of the weight data according to the storage method of the input data, the weight data is stored with the highest priority as the input data. During convolution calculation, the input data and weight data that need to be convolutionally calculated can be accessed quickly, reducing memory access time. Furthermore, by changing the storage method of the weight data according to the characteristics of convolution calculation in the convolution calculation cycle, the order of the convolution results of the output channel is effectively adjusted. This ensures the efficiency of convolution calculation while reducing the time to obtain the convolution results, further improving the efficiency of convolution calculation.

[0087] Furthermore, as an optional implementation, in step S30, convolution calculations are performed based on the input data and weight data according to the order of the row data to obtain the row convolution result for each output channel, including:

[0088] S301 assigns a corresponding convolution calculation cycle to each row of input data. In each convolution calculation cycle, the row convolution result of each output channel is calculated according to the order of the output channels.

[0089] Specifically, for each row of input data, the convolution calculation unit is allocated a corresponding convolution calculation cycle according to the number of input channels, the number of output channels, and the number of rows of input data, so as to obtain the row convolution result of each output channel.

[0090] It should be noted that convolution calculations can be performed in parallel within each convolution calculation cycle. In this embodiment, one convolution calculation cycle is allocated for each row of input data, and within one convolution calculation cycle, the row convolution result of each output channel on each row of input data is output.

[0091] Taking the first row of input data as an example, assuming there are four input channels and four output channels, the row convolution result of each output channel is calculated in parallel within one convolution calculation cycle.

[0092] The input data is stored in the order of row data and input channels. The row convolution result of each output channel in each convolution calculation cycle is calculated in units of row data. This can improve the convolution calculation efficiency of one convolution calculation cycle.

[0093] Furthermore, in each convolution calculation cycle of the input data, the row convolution result of a portion of the corresponding output channel is calculated first, according to the storage order of the input data of the input channel. After the convolution results of all input channels are calculated and accumulated, the row convolution result of the corresponding output channel is obtained.

[0094] For example, taking four input channels and four output channels as an example, the row convolution result of the first row of input data in the first output channel is Co_0_0sum, where the first 0 represents the first output channel and the second 0 represents the row convolution result of the first row of input data.

[0095] The first input channel is denoted as Ci_0. The convolution result obtained after performing convolution calculation on the first row of input data of Ci_0 is Ci_0sum. The convolution result obtained after performing convolution calculation on the second input channel is Ci_1sum. The convolution result obtained after performing convolution calculation on the third input channel is Ci_2sum. The convolution result obtained after performing convolution calculation on the fourth input channel is Ci_3sum.

[0096] In this embodiment, a corresponding convolution calculation cycle is allocated for each row of input data. Taking advantage of the parallel computing characteristics of convolution calculation, the row convolution result corresponding to each output channel is calculated according to each row of input data. Then, the row convolution result corresponding to each output channel obtained after parallel computing is accumulated for all rows, with the output channel as the unit, to obtain the channel convolution result corresponding to that output channel. This effectively improves the efficiency of convolution calculation and thus effectively increases the speed of convolution calculation to obtain the convolution calculation result.

[0097] Furthermore, as an optional implementation, in step S301, the convolution calculation cycle includes multiple calculation sub-cycles, allocating a corresponding convolution calculation cycle for each row of input data. In each convolution calculation cycle, the row convolution result for each output channel is calculated according to the order of the output channels, including:

[0098] S3011, in each computational sub-cycle, calculates the row convolution sub-result of each output channel according to the order of the input channels.

[0099] S3012 performs cumulative calculations on all row convolution sub-results for the same output channel to obtain the row convolution result for each output channel, and caches the row convolution result.

[0100] Specifically, the convolution calculation cycle allocated for each row of input data also includes at least one calculation sub-cycle, and each calculation sub-cycle performs parallel calculations based on the input data of the corresponding input channel.

[0101] In one convolution computation cycle, the row convolution result of the input data for the current row for all output channels can be calculated. In each computation sub-cycle, the convolution sub-result of each output channel is output first, according to the order in which the input data for the current row is stored in the input channels and the order in which the output channels are located.

[0102] For example, if we are calculating the row convolution result corresponding to the first row of input data, the calculation process is as follows:

[0103] In the first calculation sub-cycle, the first row of input data from the first input channel and the first row of input data from the second input channel are used to calculate the convolution result of the first part of the first output channel.

[0104] In the second calculation sub-cycle, the convolution result of the second part of the first output channel is calculated by using the first row of input data from the third input channel and the first row of input data from the fourth input channel.

[0105] The convolution results from the first part and the second part are summed to obtain the row convolution result of the first output channel.

[0106] It should be noted that the first and second computation sub-cycles are computed in parallel. In the first and second computation sub-cycles, the row convolution result of the second output channel is calculated according to the computation process of the first output channel. The computation process of the row convolution result of the other output channels is the same, and will not be repeated here.

[0107] In this embodiment, based on the parallel characteristics of the computation sub-cycle and the storage method of the input data, the convolution sub-results corresponding to each output channel are calculated interleaved according to the order of the input data. This allows the row convolution results of the output channels after accumulating the convolution sub-results to be cached. At the same time, the interleaved calculation ensures the efficiency of the convolution calculation.

[0108] Furthermore, as an optional implementation, in step S40, the row convolution results of each output channel are accumulated to obtain the convolution calculation results, including:

[0109] S401: For the same output channel, obtain the row convolution result of each row of data, and accumulate all the row convolution results to obtain the output channel convolution result.

[0110] S402, obtain the output channel convolution result of each output channel, and accumulate the convolution results of all output channels to obtain the convolution calculation result.

[0111] In this embodiment, for the same output channel, the row convolution result corresponding to each row of input data is obtained, and all row convolution results are accumulated to obtain the output channel convolution result corresponding to the output channel.

[0112] Assuming there are 32 rows of input data, the row convolution results of the first output channel corresponding to rows 1 to 32 of the input data are obtained. The 32 row convolution results are accumulated to obtain the output channel convolution result of the first output channel. The calculation process of other output channels is the same as that of the first output channel, and will not be described in detail here.

[0113] Obtain the convolution result of each output channel, and sum all the convolution results to obtain the final convolution result.

[0114] In this embodiment, the row convolution result of each output channel based on each row of input data is calculated in parallel. Then, the row convolution result of each row of input data is accumulated to obtain the output channel convolution result of each output channel. The output channel convolution results are accumulated according to the number of output channels to obtain the final convolution settlement result. Based on the characteristics of parallel computing and combined with the storage method of input data, the speed of obtaining the output channel convolution result of each output channel is accelerated, thereby improving the efficiency of generating the final convolution settlement result and further improving the efficiency of obtaining the convolution calculation result in a distributed manner.

[0115] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0116] In one embodiment, a convolution calculation device is provided, which corresponds one-to-one with the convolution calculation methods described in the above embodiments. For example... Figure 8 As shown, the convolution calculation device includes an input data storage module 81, a weight data storage module 82, a row convolution result calculation module 83, and a convolution result generation module 84. Detailed descriptions of each functional module are as follows:

[0117] The input data storage module 81 is used to generate corresponding input data storage rules based on the input data and the number of input channels, and to store the input data based on the input data storage rules.

[0118] The weight data storage module 82 is used to store weight data according to the input data storage rules and the correspondence between input channels and output channels.

[0119] The row convolution result calculation module 83 is used to perform convolution calculation based on the input data and weight data according to the order of the row data to obtain the row convolution result of each output channel.

[0120] The convolution result generation module 84 is used to accumulate the row convolution results of each output channel to obtain the convolution calculation result.

[0121] Furthermore, the input data storage module 81 includes:

[0122] The data storage rule generation unit is used to obtain the row data information and input channel information of the input data, take the order of the row data as the first priority, take the order of the input channels as the second priority, and take the first priority and the second priority as the input data storage rules.

[0123] The data storage unit is used to store the input data of each row in the order of the row data according to the input data storage rules. For the same row of data, the row input data of each input channel is stored in the order of the input channels.

[0124] Furthermore, the weighted data storage module 82 includes:

[0125] The storage method acquisition unit is used to acquire the input data storage method based on the input data storage rules.

[0126] The weighted data storage unit is used to generate a weighted data matrix based on the input data storage method and in the order of the input channels, and to store the weighted data based on the weighted data matrix.

[0127] Furthermore, the row convolution result calculation module 83 includes:

[0128] The row convolution calculation unit is used to allocate a corresponding convolution calculation cycle to each row of input data. In each convolution calculation cycle, the row convolution result of each output channel is calculated according to the order of the output channels.

[0129] Furthermore, the row convolution calculation result module 83 also includes:

[0130] The sub-result calculation unit is used to calculate the row convolution sub-result of each output channel in the order of the input channels in each calculation sub-cycle.

[0131] The convolution result caching unit is used to accumulate all the row convolution sub-results for the same output channel to obtain the row convolution result for each output channel, and then cache the row convolution result.

[0132] Furthermore, the convolution result generation module 84 includes:

[0133] The output channel result calculation unit is used to obtain the row convolution result of each row of data for the same output channel, and to accumulate all the row convolution results to obtain the output channel convolution result.

[0134] The convolution calculation result generation unit is used to obtain the output channel convolution result of each output channel and accumulate the convolution results of all output channels to obtain the convolution calculation result.

[0135] The terms "first" and "second" in the above-mentioned modules / units are only used to distinguish different modules / units and are not intended to specify which module / unit has a higher priority or any other limiting meaning. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The module divisions appearing in this application are merely logical divisions; in actual applications, different division methods may be used.

[0136] Specific limitations regarding the convolution calculation device can be found in the limitations of the convolution calculation method above, and will not be repeated here. Each module in the aforementioned convolution calculation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0137] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data involved in the convolution calculation method. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a convolution calculation method.

[0138] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the convolution calculation method described in the above embodiments, for example... Figure 2 The steps S10 to S40 shown, as well as other extensions and related steps of the method, are examples. Alternatively, when the processor executes a computer program, it implements the functions of each module / unit of the convolution calculation device in the above embodiments, for example... Figure 8 The functions of modules 81 to 84 are shown. To avoid repetition, they will not be described again here.

[0139] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the computer device, connecting various parts of the computer device via various interfaces and lines.

[0140] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, video data, etc.).

[0141] The memory can be integrated into the processor or it can be set up separately from the processor.

[0142] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program implements the steps of the convolution calculation method described in the above embodiments, for example... Figure 2 The steps S10 to S40 shown, as well as other extensions and related steps of the method, are examples. Alternatively, when a computer program is executed by a processor, it implements the functions of each module / unit of the convolution calculation device in the above embodiments, for example... Figure 8 The functions of modules 81 to 84 are shown. To avoid repetition, they will not be described again here.

[0143] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0144] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0145] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A convolution calculation method, characterized in that, include: The data interleaving unit generates corresponding input data storage rules based on the input data and the number of input channels, so that the data storage unit stores the input data based on the input data storage rules; According to the input data storage rules, the weight storage unit stores weight data according to the correspondence between the input channels and the output channels; Following the order of the row data, the convolution calculation unit performs convolution calculations based on the input data and the weight data to obtain the row convolution result for each output channel; The convolution calculation unit accumulates the row convolution results of each output channel to obtain the convolution calculation result; The data interleaving unit generates corresponding input data storage rules based on the input data and the number of input channels, so that the data storage unit stores the input data based on the input data storage rules, including: The data interleaving unit acquires the row data information and input channel information of the input data and allocates corresponding storage space. The order of the row data is used as the first priority, the order of the input channels is used as the second priority, and the first priority and the second priority are used as the input data storage rules. Based on the input data storage rules, the data storage unit stores the input data of each row in the order of the row data, wherein, for the same row data, the row input data of each input channel is stored in the order of the input channels.

2. The convolution calculation method according to claim 1, characterized in that, According to the input data storage rules, the weight storage unit stores weight data according to the correspondence between the input channels and the output channels, including: The weighted interleaving unit obtains the input data storage method based on the input data storage rules; The weight storage unit generates a weight data matrix based on the input data storage method and according to the correspondence between the input channel and the output channel, and stores the weight data based on the weight data matrix.

3. The convolution calculation method according to claim 1, characterized in that, The convolution calculation unit performs convolution calculations based on the input data and the weight data according to the row data order, and obtains the row convolution result for each output channel, including: Each row of input data is assigned a corresponding convolution calculation cycle. In each convolution calculation cycle, the convolution calculation unit calculates the row convolution result of each output channel according to the order of the output channels.

4. The convolution calculation method according to claim 3, characterized in that, The convolution calculation cycle includes multiple calculation sub-cycles. A corresponding convolution calculation cycle is allocated to each row of input data. In each convolution calculation cycle, the convolution calculation unit calculates the row convolution result for each output channel according to the order of the output channels, including: In each computation sub-cycle, the convolution computation unit calculates the row convolution sub-result of each output channel according to the order of the input channels; For the same output channel, the convolution calculation unit accumulates all the row convolution sub-results to obtain the row convolution result for each output channel, and caches the row convolution result.

5. The convolution calculation method according to claim 1, characterized in that, The convolution calculation unit accumulates the row convolution results of each output channel to obtain the convolution calculation results, including: For the same output channel, the convolution calculation unit obtains the row convolution result of each row of data, and accumulates all the row convolution results to obtain the output channel convolution result; The convolution calculation unit obtains the output channel convolution result of each output channel and accumulates all the output channel convolution results to obtain the convolution calculation result.

6. A convolution calculation device, characterized in that, include: The input data storage module is used to generate corresponding input data storage rules based on the input data and the number of input channels through the data interleaving unit, and to store the input data based on the input data storage rules through the data storage unit; The weighted data storage module is used to store weighted data according to the input data storage rules, through the weighted storage unit, in accordance with the correspondence between the input channel and the output channel; The row convolution result calculation module is used to perform convolution calculation based on the input data and the weight data by the convolution calculation unit according to the order of the row data to obtain the row convolution result of each output channel; The convolution result generation module is used to accumulate the row convolution results of each output channel through the convolution calculation unit to obtain the convolution calculation result; The input data storage module includes: The data storage rule generation unit is used to obtain the row data information and input channel information of the input data through the data interleaving unit and allocate corresponding storage space, take the order of the row data as the first priority, take the order of the input channels as the second priority, and take the first priority and the second priority as the input data storage rule; A data storage unit is used to store the input data of each row in the order of the row data according to the input data storage rules. For the same row data, the row input data of each input channel is stored in the order of the input channels.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the convolution calculation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the convolution calculation method as described in any one of claims 1 to 5.

Images