A parallel convolutional neural network, convolution operation method, device and equipment

Abstract

The present invention discloses a parallel convolution neural network based on convolution kernel, a convolution operation method, device and computer equipment of parallel convolution, wherein the parallel convolution neural network has a plurality of 2 parallel convolution kernels, and the 2 parallel convolution kernels include: a first convolution factor, whose input end is connected to the first input end, for receiving a first sampling signal; a second convolution factor, whose input end is connected to the second input end, for receiving a second sampling signal; a third convolution factor, whose input end is connected to the first input end and the second input end through an adder, for receiving the superposition result of the first sampling signal and the second sampling signal, and the third convolution factor is obtained by the first convolution factor and the second convolution factor through addition and subtraction operations. The output signal is obtained by performing three multiplication operations on the three convolution factors, which reduces the number of multiplication operations compared to the traditional convolution operation, thereby reducing the amount of operation when performing the convolution operation, improving the operation speed and shortening the operation time.

Description

Technical Field

[0001] The present invention relates to the field of convolutional neural networks, and in particular to a parallel convolutional neural network implemented based on a convolution kernel, a convolution operation method, a device and a computer device for parallel convolution. Background Art

[0002] At present, artificial intelligence usually uses convolutional neural networks (CNNs) to extract features. A large number of convolution operations are involved in the data training of convolutional neural networks and the process of using convolutional neural networks for classification and recognition. Although the complexity of convolution operations is not high, a large number of floating-point addition and multiplication computing resources are consumed during the convolution operation, which increases the operation time, thereby increasing the time for convolutional neural networks to perform data training and extending the overall development cycle of the neural network model. In addition, when using convolutional neural networks for classification and recognition, for some high-real-time usage scenarios, such as face recognition and license plate recognition, the increase in convolution operation time will inevitably lead to an increase in recognition time, affecting the user experience.

[0003] To this end, a parallel convolutional neural network based on convolution kernel, a convolution operation method, device and computer equipment for parallel convolution are provided to reduce the amount of calculation when performing convolution operation, improve the operation speed and shorten the operation time, which has become a technical problem to be solved urgently. Summary of the invention

[0004] Based on the above situation, the main purpose of the present invention is to provide a parallel convolutional neural network based on convolution kernel, a convolution operation method, device and computer equipment for parallel convolution, so as to reduce the amount of calculation when performing convolution operation, improve the operation speed and shorten the operation time.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, an embodiment of the present invention discloses a parallel convolutional neural network based on a convolutional kernel, wherein the parallel convolutional neural network has a plurality of 2 parallel convolutional kernels, wherein the 2 parallel convolutional kernels include: a first input terminal for receiving a first sampling signal, a second input terminal for receiving a second sampling signal, a first output terminal for outputting a first convolutional signal, and a second output terminal for outputting a second convolutional signal; the 2 parallel convolutional kernels also include:

[0007] A first convolution factor, whose input terminal is connected to the first input terminal and is used to receive a first sampled signal;

[0008] A second convolution factor, whose input terminal is connected to the second input terminal and is used to receive a second sampling signal;

[0009] A third convolution factor, whose input end is connected to the first input end and the second input end through an adder, and is used to receive the superposition result of the first sampling signal and the second sampling signal, and the third convolution factor is obtained by adding and subtracting the first convolution factor and the second convolution factor;

[0010] The output terminal of the first convolution factor and the output terminal of the second convolution factor are connected to the first output terminal through an adder;

[0011] An output terminal of the first convolution factor, an output terminal of the second convolution factor, and an output terminal of the third convolution factor are connected to the second output terminal through an adder.

[0012] Optionally, an output signal of the output end of the third convolution factor is subtracted from an output signal of the output end of the first convolution factor and an output signal of the output end of the second convolution factor to obtain a second convolution signal.

[0013] Optionally, the output signal of the output end of the second convolution factor is superimposed with the output signal of the output end of the first convolution factor after a delay link to obtain a first convolution signal, wherein the step value of the second convolution factor is greater than the step value of the first convolution factor.

[0014] Optionally, the convolution coefficients of the two parallel convolution kernels are divided into a first convolution coefficient set and a second convolution coefficient set, the first convolution factor is obtained by performing addition and subtraction operations on the convolution coefficients in the first convolution coefficient set; the second convolution factor is obtained by performing addition and subtraction operations on the convolution coefficients in the second convolution coefficient set;

[0015] The number of convolution coefficients in the first convolution coefficient set is the same as the number of convolution coefficients in the second convolution coefficient set, and the convolution coefficients in the first convolution coefficient set and the convolution coefficients in the second convolution coefficient set are not repeated.

[0016] In a second aspect, an embodiment of the present invention discloses a convolution operation method of parallel convolution, which is applied to a parallel convolutional neural network implemented based on a convolution kernel as in the first aspect, and the convolution operation method includes:

[0017] Step S100, calculating the convolution factors in the independent convolution complex multiplication matrix according to the preset parallelism, where the independent convolution complex multiplication matrix is ​​a diagonal matrix composed of convolution factors of a plurality of 2 parallel convolution kernels;

[0018] Step S200, storing the convolution factor in a register;

[0019] Step S300, performing serial-to-parallel conversion on the sampled signal according to a preset parallelism to obtain a parallel input signal;

[0020] Step S400: extract the convolution factors from the register and perform matrix multiplication and addition operations on the parallel input signals to obtain parallel output signals.

[0021] Optionally, step S100 includes:

[0022] Step S110, dividing the convolution coefficients of the parallel convolutional neural network into n groups according to a preset degree of parallelism;

[0023] Step S120, generating a convolution factor of an independent convolution matrix, wherein the convolution factor includes a first element and a second element, the first element is each group of n groups of convolution coefficients, and the second element is a result of adding and subtracting at least two groups selected from the n groups of convolution coefficients.

[0024] Optionally, the parallel input signals include n groups of input signals, the parallel output signals include n groups of output signals, and the method further includes:

[0025] Pre-processing the n groups of input signals to combine the timing of the n groups of input signals;

[0026] The n groups of output signals are post-processed to integrate the timing of the n groups of output signals to obtain parallel output signals.

[0027] In a third aspect, an embodiment of the present invention discloses a convolution operation device for parallel convolution, which is applied to a parallel convolutional neural network implemented based on a convolution kernel as in the first aspect, and the device includes:

[0028] A factor calculation module is used to calculate the convolution factors in the independent convolution multiplication matrix according to a preset parallelism, where the independent convolution multiplication matrix is ​​a diagonal matrix composed of convolution factors of several 2-parallel convolution kernels;

[0029] A factor storage module, used to store the convolution factors in registers;

[0030] A serial-to-parallel conversion module, used to perform serial-to-parallel conversion on the sampled signal according to a preset parallelism to obtain a parallel input signal;

[0031] The convolution operation module is used to extract the convolution factors from the register and perform matrix multiplication and addition operations on the parallel input signals to obtain parallel output signals.

[0032] In a fourth aspect, an embodiment of the present invention discloses a computer device, characterized in that it includes: a convolution operation method using parallel convolution as described in the second aspect; or, includes the apparatus as described in the third aspect.

[0033] In a fifth aspect, an embodiment of the present invention discloses a computer-readable storage medium having a computer program stored thereon, wherein the computer program stored in the storage medium is used to be executed by a processor to implement the method described in the second aspect.

[0034] Beneficial effects:

[0035] According to the parallel convolutional neural network based on convolutional kernels, the convolution operation method, device and computer equipment of parallel convolution disclosed in the embodiment of the present invention, the parallel convolutional neural network includes a plurality of 2 parallel convolutional kernels, each of the 2 parallel convolutional kernels includes three convolution factors, and the output end of the first convolution factor and the output end of the second convolution factor are connected to the first output end through an adder; the output end of the first convolution factor, the output end of the second convolution factor and the output end of the third convolution factor are connected to the second output end through an adder. In addition, the convolution coefficients in the 2 parallel convolutional kernels are reprocessed to become three convolution factors, and the three convolution factors are pre-calculated and stored in the processor, and then the convolution operation can be performed by directly calling the convolution factors using the input data, so as to obtain the output first convolution signal and second convolution signal by performing three multiplication operations through the three convolution factors. Compared with the traditional convolution operation, the number of multiplication operations is reduced, thereby improving the operation speed, reducing the amount of operations when performing convolution operations, improving the operation speed, and shortening the operation time.

[0036] Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The following will describe a preferred embodiment of a parallel convolutional neural network based on convolution kernel, a convolution operation method, a device and a computer device of the present invention with reference to the accompanying drawings. In the drawings:

[0038] Figure 1 This embodiment discloses a Figure 1 A schematic diagram of the structural framework of a parallel convolutional neural network based on a convolution kernel disclosed in this embodiment;

[0039] Figure 2 A schematic diagram of the structural framework of two parallel convolution kernels in a parallel convolutional neural network implemented based on convolution kernels disclosed in this embodiment;

[0040] Figure 3 It is a schematic diagram of the structural framework of 2 parallel convolution kernels in the prior art;

[0041] Figure 4 A schematic diagram of a method flow of a convolution operation method for parallel convolution disclosed in this embodiment;

[0042] Figure 5 A schematic diagram of the framework structure of a convolution operation device for parallel convolution disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The present invention is described below based on embodiments, but the present invention is not limited to these embodiments. In the following detailed description of the present invention, some specific details are described in detail. In order to avoid confusing the essence of the present invention, known methods, processes, procedures, and components are not described in detail.

[0044] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0045] Unless the context clearly requires otherwise, throughout the specification and claims, the words "include", "comprising" and similar words should be interpreted in an inclusive sense rather than an exclusive or exhaustive sense; that is, in the sense of "including but not limited to".

[0046] In the description of the present invention, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0047] In a convolutional neural network (CNN), the main calculation of the convolution operation includes the element-by-element multiplication of the convolution kernel and the input data, and then adding all the product results to get the convolution output value. This means that multiple multiplication and addition operations are required within each convolution window.

[0048] In convolution operations, it is the multiplication operation, not the addition or subtraction operation, that mainly affects the speed and amount of calculation. Multiplication operations have a higher computational cost than addition operations, especially in hardware implementation (such as on the CPU or GPU of a computer). The amount of computation required for multiplication operations is large because multiplication operations are usually more complex and time-consuming than addition operations. Therefore, in convolution operations, multiplication operations are the main factor affecting the speed and efficiency of calculations. Especially when the size and dimension of the convolution kernel are large (that is, the convolution coefficients are large), the impact of multiplication operations on the speed of calculation is more obvious. Therefore, in this scenario, a new convolutional neural network structure and a new convolution operation method are urgently needed to improve the speed of calculation.

[0049] In order to improve the computational efficiency of convolutional neural networks, reduce the amount of calculations during convolution operations, increase the calculation speed, and shorten the calculation time, this embodiment discloses a parallel convolutional neural network based on convolution kernel implementation, a convolution operation method, device, equipment, and storage medium for parallel convolution.

[0050] Please refer to Figure 1 , Figure 1 FIG. 1 is a schematic diagram of the structural framework of a parallel convolutional neural network based on a convolution kernel disclosed in this embodiment. Figure 1As shown in , the parallel convolutional neural network implemented based on the convolution kernel has multiple 2-parallel convolution kernels. Depending on the degree of parallelism in the parallel convolutional neural network, the parallel convolutional neural network can have different numbers of 2-parallel convolution kernels.

[0051] Among them, please refer to Figure 2 , Figure 2 A schematic diagram of the structural framework of two parallel convolution kernels in a parallel convolutional neural network implemented based on convolution kernels disclosed in this embodiment.

[0052] like Figure 2 As shown in , 2 parallel convolution kernels include: for receiving the first sampling signal x 2k The first input terminal 11 is used to receive the second sampling signal x 2k+1 The second input terminal 12 is used to output the first convolution signal y 2k The first output terminal 21 is used to output the second convolution signal y 2k+1 The second output terminal 22 of . In this embodiment, the 2 parallel convolution kernel includes two signal input terminals and two signal output terminals, wherein the two signal input terminals are respectively used to receive different sampling signals, and the two signal output terminals are used to output the output after the sampling signals received by the two signal input terminals are multiplied. In the specific implementation process, different sampling signals refer to different groups of sampling signals obtained after the initial signal x is grouped according to a preset rule. In 2 parallel, different sampling signals refer to two groups of sampling signals obtained after the initial signal x is divided into two groups, that is, the first sampling signal x 2k and the second sampled signal x 2k+1 .

[0053] like Figure 2 As shown in , the 2 parallel convolution kernels also include a first convolution factor H0, a second convolution factor H1 and a third convolution factor H0+H1. The input end of the first convolution factor H0 is connected to the first input end 11 for receiving the first sampling signal x 2k The input terminal of the second convolution factor H1 is connected to the second input terminal 12 for receiving the second sampling signal x 2k+1 The input end of the third convolution factor H0+H1 is connected to the first input end 11 and the second input end 12 through an adder for receiving the first sampling signal x 2k and the second sampled signal x 2k+1The third convolution factor H0+H1 is obtained by adding and subtracting the first convolution factor H0 and the second convolution factor H1. In this embodiment, the three convolution factors (H0, H1 and H0+H1) in the two parallel convolution kernels are obtained by adding and subtracting the convolution coefficients of the two parallel convolution kernels. That is to say, in this embodiment, this scheme reprocesses several convolution coefficients in the two parallel convolution kernels and converts them into three convolution factors that can perform parallel convolution operations, thereby reducing the multiplication resources required for the operation and improving the operation speed. It should be noted that in this embodiment, although the third convolution factor is expressed as H0+H1, it should be noted that when the third convolution factor H0+H1 is calculated using the first convolution factor H0 and the second convolution factor H1, it is also obtained by performing addition and subtraction operations, that is, there may be both addition and subtraction operations in the calculation process, and it does not necessarily include only addition operations.

[0054] In an optional embodiment, the convolution coefficients of the two parallel convolution kernels are divided into a first convolution coefficient set and a second convolution coefficient set, and the first convolution factor H0 is obtained by performing addition and subtraction operations on the convolution coefficients in the first convolution coefficient set; the second convolution factor H1 is obtained by performing addition and subtraction operations on the convolution coefficients in the second convolution coefficient set; wherein the number of convolution coefficients in the first convolution coefficient set is the same as the number of convolution coefficients in the second convolution coefficient set, and the convolution coefficients in the first convolution coefficient set and the convolution coefficients in the second convolution coefficient set are not repeated. In this embodiment, when calculating the first convolution factor H0 and the second convolution factor H1, the convolution coefficients in the two parallel convolution kernels are grouped to obtain the first convolution coefficient set and the second convolution coefficient set, and then the convolution coefficients in the first convolution coefficient set are added and subtracted to obtain the first convolution factor H0, and the convolution coefficients in the second convolution coefficient set are added and subtracted to obtain the second convolution factor H1. In the process of grouping the convolution coefficients in the 2 parallel convolution kernels, the number of convolution coefficients in the first convolution coefficient set is the same as the number of convolution coefficients in the second convolution coefficient set, and the convolution coefficients in the first convolution coefficient set and the convolution coefficients in the second convolution coefficient set are not repeated. In other words, the convolution coefficients in the 2 parallel convolution kernels are divided into two groups, namely the first convolution coefficient set and the second convolution coefficient set, and the data in the first convolution coefficient set and the second convolution coefficient set are alternated.

[0055] like Figure 2 As shown in FIG. 1 , the output end of the first convolution factor H0 and the output end of the second convolution factor H1 are connected to the first output end 21 through an adder; the output end of the first convolution factor H0, the output end of the second convolution factor H1 and the output end of the third convolution factor H0+H1 are connected to the second output end 22 through an adder. In this embodiment, the first sampling signal x 2kThe output value after multiplication with the first convolution factor H0 is output through the output terminal of the first convolution factor H0. The second sampling signal x 2k+1 The output value after multiplication with the second convolution factor H1 is output through the output terminal of the second convolution factor H1. The two output values ​​are connected to the first output terminal 21 through the adder and output from the first output terminal 21. That is, the first output terminal 21 outputs the first convolution signal y 2k At the same time, the first sampling signal x 2k With the second sampling signal x 2k+1 The output value after multiplication of the sum and the third convolution factor H0+H1 is connected to the second output terminal 22 through an adder with the output value of the output terminal of the first convolution factor H0 and the output value of the output terminal of the second convolution factor H1, and is output from the second output terminal 22, that is, the second output terminal 22 outputs the second convolution signal y 2k+1 .

[0056] In an optional embodiment, the output signal of the output end of the second convolution factor H1 is superimposed with the output signal of the output end of the first convolution factor H0 after passing through the delay link D to obtain the first convolution signal y 2k , wherein the step size of the second convolution factor H1 is greater than the step size of the first convolution factor H0. In this embodiment, the second sampling signal x 2k+1 The output value after multiplication with the second convolution factor H1 passes through the output end of the second convolution factor H1, and then passes through the delay link D and the first sampling signal x 2k The output value after multiplication with the first convolution factor H0 is connected to the first output terminal 21 through an adder. This is because when performing 2 parallel operations, the step value of the second convolution factor H1 is greater than the step value of the first convolution factor H0, that is, the clock cycle of the second convolution factor H1 is faster than the clock cycle of the first convolution factor H0. Therefore, when the data is output, the first convolution signal y 2k When the output signal of the output end of the second convolution factor H1 is delayed, the output signal needs to be delayed to ensure the timing of the data.

[0057] In an optional embodiment, the output signal of the output end of the third convolution factor H0+H1 is subtracted from the output signal of the output end of the first convolution factor H0 and the output signal of the output end of the second convolution factor H1 to obtain the second convolution signal y 2k+1 In this embodiment, the first sampling signal x 2k With the second sampling signal x 2k+1The output value after multiplication of the sum of the first and second convolution factors H0+H1 is connected to the second output terminal 22 after subtraction operation with the output value of the output terminal of the first convolution factor H0 and the output value of the output terminal of the second convolution factor H1 through an adder, and output from the second output terminal 22, that is, the second output terminal 22 outputs the second convolution signal y 2k+1 .

[0058] Take 2 parallel computing as an example, Figure 3 As shown in Figure 3 FIG. 1 is a schematic diagram of the structural framework of two parallel convolution kernels in the prior art. Figure 3 As shown in , in the prior art, when performing 2 parallel convolution operations, 4 multiplier resources are required to implement it, while according to the parallel convolution neural network based on the convolution kernel implementation disclosed in the embodiment of the present invention, when performing 2 parallel operations, only 3 multiplier resources are needed to complete the convolution operation, saving 1 multiplier resource. Similarly, when performing 4 parallel calculations, 2 parallel operations are performed on the 2 parallel data respectively, so only 9 multiplier resources are needed to implement the convolution operation, saving 4 multiplier resources. In other words, in multi-parallel convolution operations, the multiplication resources required are 3 / 4 of the traditional convolution operation algorithm, which can save 1 / 4 of the multiplier resources. When performing higher parallel convolution operations, the number of multiplier resources saved is even greater, thereby greatly improving the efficiency of the convolution operation.

[0059] Please refer to Figure 4 , Figure 4 FIG. 1 is a flow chart of a method for parallel convolution disclosed in this embodiment. The convolution operation method is applied to Figure 1 and Figure 2 In order to facilitate the understanding of those skilled in the art, the parallel convolutional neural network based on the convolution kernel described in the embodiment is described in detail by taking 4 parallel convolution operations as an example. Figure 3 As shown in , the convolution operation method includes:

[0060] Step S100, calculating the independent convolution matrix H according to the preset parallelism n n The convolution factor in, where the independent convolution multiplication matrix H n is a diagonal matrix composed of convolution factors of several 2-parallel convolution kernels. In this embodiment, the preset parallelism n refers to the need to perform n parallel convolution operations, where the value of n can be 2, 4, 8 or other. In addition, the independent convolution complex multiplication matrix H n is a diagonal matrix composed of the convolution factors of several 2 parallel convolution kernels, that is, the independent convolution complex multiplication matrix H nThe number of convolution factors in depends on the preset parallelism n. When a 2-parallel convolution operation is required, the preset parallelism is 2, and the independent convolution matrix H2 is a diagonal matrix composed of the convolution factors of a 2-parallel convolution kernel, where the convolution factors of the independent convolution matrix H2 are 3; when a 4-parallel convolution operation is required, the preset parallelism is 4, and 4 parallel means that the 2-parallel data is subjected to a 2-parallel operation again. At this time, the independent convolution matrix H4 is a diagonal matrix composed of the convolution factors of three 2-parallel convolution kernels, where the number of convolution factors in the independent convolution matrix H4 is 9.

[0061] In an optional embodiment, step S100 further includes:

[0062] Step S110, dividing the convolution coefficients of the parallel convolutional neural network into n groups according to the preset parallelism n. In this embodiment, when calculating the independent convolution complex multiplication matrix H n When the convolution factor in is calculated, the convolution coefficients of the parallel convolutional neural network are first divided into n groups according to the preset parallelism n, that is, the convolution coefficients of the parallel convolutional neural network are serial-to-parallel converted according to the preset parallelism n. After the serial-to-parallel conversion is completed, the convolution coefficients of the parallel convolutional neural network become n groups, and the n groups of data are alternating.

[0063] Step S120, generate an independent convolution matrix H n The convolution factor includes a first element and a second element, the first element is each group of n groups of convolution coefficients, and the second element is a result of adding and subtracting at least two groups selected from the n groups of convolution coefficients.

[0064] In 4 parallel, the convolution coefficients of the parallel convolutional neural network are divided into 4 groups, including H0, H1, H2, and H3. Figure 1 and Figure 2 The structure of the parallel convolutional neural network based on the convolution kernel shown in the figure is used to calculate and generate the convolution factors of the independent convolution matrix H4. Specifically, H0, H1, H2, and H3 are used as the first elements of the independent convolution matrix H4, and then H0 and H2 are grouped as a group, and H1 and H3 are grouped as a group for the first 2-parallel processing, and H0+H2 and H2+H3 are obtained as the second elements of the independent convolution matrix H4, and then the data after 2-parallel processing is processed again in 2-parallel, so that H0+H1, H2+H3 and H0+H1+H2+H3 are obtained as the second elements of the independent convolution matrix H4, and finally 9 convolution factors are obtained, that is, the independent convolution matrix H4 obtained is expressed as:

[0065]

[0066] Step S200, storing the convolution factor in a register. In this embodiment, the convolution factor calculated in step S100 is calculated in advance and stored in a register of the processor for use.

[0067] Step S300: Perform serial-to-parallel conversion on the sampled signal according to the preset parallelism n to obtain a parallel input signal X np In this embodiment, the sampling signal is converted into a parallel signal according to the preset parallel degree n to obtain a parallel input signal X np Among them, the parallel input signal X np The method includes n groups of parallel input signals. When performing serial-to-parallel conversion, the sampling interval can be determined according to the preset parallelism, and sampling can be performed according to the timing of the sampling signal to complete the serial-to-parallel conversion of the sampling signal. For example, when performing a 2-parallel convolution operation, the sampling interval is 1, so that the sampling signal is divided into two groups, and 2 groups of parallel input signals are obtained, and the 2 groups of parallel input signals are alternated. When performing a 4-parallel convolution operation, the sampling interval is 3, so that the sampling signal is divided into four groups, and 4 groups of parallel input signals are obtained, and the 4 groups of parallel input signals are alternated.

[0068] In an optional embodiment, before step S300, the convolution operation method of the parallel convolution further includes performing data preprocessing on the initial signal. The data preprocessing may include digital mixing. Taking the initial signal with a sampling rate of 2.4 GHz as an example, the signal is first digitally mixed. Since the carrier frequency fc=600 MHz, the sampling frequency fs=4 fc / (2n+1) relationship, so frequency conversion without multiplication can be performed. After down-conversion, the I and Q signals are: I0 = d0 (1), I1 = d2 (-1), I2 = d4 (1), I3 = d6 (-1); Q0 = d1 (-1), Q1 = d3 (1), Q2 = d5 (-1), Q3 = d7 (1), the parallel clock is 300MHz. Then the I and Q signals can be convolved separately to achieve FIR filtering.

[0069] Step S400: extract the convolution factors from the register and add them to the parallel input signal X np Perform matrix multiplication and addition operations to obtain the parallel output signal Y np In this embodiment, since the convolution factor is pre-calculated, the pre-calculated convolution factor is extracted from the register, and then the input parallel input signal X is npThe convolution factor is based on Figure 1 and Figure 2 The convolutional neural network structure shown in the figure performs parallel compressed convolution operations to obtain parallel output signals Y np .

[0070] In an alternative embodiment, the parallel input signal X np It includes n groups of input signals and parallel output signal Y np Including n groups of output signals, the convolution operation method also includes:

[0071] Pre-processing the n groups of input signals to combine the timing of the n groups of input signals;

[0072] Post-process the n groups of output signals to integrate the timing of the n groups of output signals to obtain the parallel output signal Y np .

[0073] In this embodiment, since the sampling signal is converted from serial to parallel, and the sampling signal itself has a timing sequence, in order to ensure that the timing sequence of the data is correct during the parallel convolution operation, the input signal can be pre-processed, and the output signal can be post-processed to combine the timing sequence of the input signal and the output signal. In the specific implementation process, the pre-processing matrix can be used to combine the input sequence, and the post-processing matrix can be used to combine the output sequence. Among them, the pre-processing matrix can be, for example:

[0074] , where n represents the number of delayed clock cycles. In 4 parallelism, the value of n is 4.

[0075] The post-processing matrix can be, for example:

[0076] , where n represents the number of delayed clock cycles. In 4 parallelism, the value of n is 4.

[0077] In the specific implementation process, taking 4 parallel operations as an example, the convolution operation process can be expressed as:

[0078] ,

[0079] in, and There are 4 parallel inputs and outputs, is a third-order unit diagonal matrix, , is the independent convolution matrix, is the pre-processing matrix used to combine the input sequence, is the post-processing matrix used to integrate the output sequence.

[0080] The convolution coefficients of the parallel convolutional neural network are divided into 4 groups, and the independent convolution matrix H4 can be calculated. The independent convolution matrix H4 includes 9 convolution factors, and the calculated 9 convolution factors are placed in the processor register for use. In the above operation process, the multiplication operation related to the independent convolution matrix H4 uses DSP resources, and the other operations are extremely simple operations such as adders and D flip-flops.

[0081] In an optional embodiment, the convolution operation method of the parallel convolution can also be used to train a convolutional neural network model. In this embodiment, the data used for model training can be first segmented into a training set and a test set. The training set is used to train the convolutional neural network model, and the test set is used to test the accuracy of the trained convolutional neural network model. The data of the training set and the test set are preprocessed, including batch normalization of the data, and transposition and zero padding operations on some image data. Then, the convolution operation method recorded in the convolution operation method of the above-mentioned parallel convolution is used to perform convolution operation on the training set data. After completing a round of convolution operation, back propagation is performed, and the weight coefficient in the convolutional neural network model is adjusted by the principle of gradient descent to reduce the loss function value. After iterating for many times, when the loss function converges, it is considered that the training at this time has been completed, and the convolutional neural network model obtained by the current training can be used as a classification recognition model.

[0082] In an optional embodiment, the parallel convolution convolution operation method can also be used for classification and recognition, such as face recognition, license plate recognition, etc. Before the data to be recognized enters the fully connected layer, it is still necessary to find features for the preprocessed data, pool the data, reduce the amount of data, and highlight its main features. In this process, the parallel convolution convolution operation method can also be used to perform convolution operations on the data to be recognized. The specific implementation process can refer to the implementation process in the above-mentioned parallel convolution convolution operation method, which will not be repeated here.

[0083] Please refer to Figure 5 , Figure 5 FIG. 1 is a schematic diagram of the framework structure of a convolution operation device for parallel convolution disclosed in an embodiment of the present invention. Figure 5 As shown in , the convolution operation device of the parallel convolution includes:

[0084] The factor calculation module 100 is used to calculate the independent convolution matrix H according to the preset parallelism n. n The convolution factor in, independent convolution multiplication matrix H n It is a diagonal matrix composed of the convolution factors of several 2-parallel convolution kernels.

[0085] The factor storage module 200 is used to store the convolution factors in registers.

[0086] The serial-to-parallel conversion module 300 is used to perform serial-to-parallel conversion on the sampled signal according to a preset parallel degree n to obtain a parallel input signal X np .

[0087] The convolution operation module 400 is used to extract the convolution factors from the register and respectively combine them with the parallel input signal X np Perform matrix multiplication and addition operations to obtain the parallel output signal Y np .

[0088] The embodiment of the present invention also provides a computer device, including: a convolution operation method of parallel convolution disclosed in the above embodiment; or, a device disclosed in the above embodiment. The data operation subject implemented by the convolution operation method of parallel convolution can be a central processing unit (CPU) or an embedded microprocessor (MCU) under the traditional X86 architecture, or a large-scale field programmable logic device (FPGA) and a graphics processing accelerator (GPU) specially used for hardware acceleration.

[0089] In addition, the present invention also provides a computer-readable storage medium, such as a chip, a CD, etc., on which an execution program is stored, and when the execution program is executed, any of the methods described above is implemented.

[0090] It should be noted that the computer-readable storage medium described in the embodiments of the present disclosure is not limited to the above-mentioned embodiments, and may also be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device, or device.

[0091] According to the parallel convolutional neural network based on convolutional kernels, the convolution operation method, device and computer equipment of parallel convolution disclosed in the embodiment of the present invention, the parallel convolutional neural network includes a plurality of 2 parallel convolutional kernels, each of the 2 parallel convolutional kernels includes three convolution factors, and the output end of the first convolution factor and the output end of the second convolution factor are connected to the first output end through an adder; the output end of the first convolution factor, the output end of the second convolution factor and the output end of the third convolution factor are connected to the second output end through an adder. In addition, the convolution coefficients in the 2 parallel convolutional kernels are reprocessed to become three convolution factors, and the three convolution factors are pre-calculated and stored in the processor, and then the convolution operation can be performed by directly calling the convolution factors using the input data, so as to obtain the output first convolution signal and second convolution signal by performing three multiplication operations through the three convolution factors. Compared with the traditional convolution operation, the number of multiplication operations is reduced, thereby improving the operation speed, reducing the amount of operations when performing convolution operations, improving the operation speed, and shortening the operation time.

[0092] It will be appreciated by those skilled in the art that, under the premise of no conflict, the above-mentioned preferred solutions can be freely combined and superimposed. Among them, the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings, for example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. The numbering of each step in this article is only for the convenience of description and reference, and is not used to limit the order of execution. The specific execution order is determined by the technology itself, and those skilled in the art can determine various allowable and reasonable orders based on the technology itself.

[0093] Those skilled in the art will appreciate that, without conflict, the above-mentioned preferred solutions can be freely combined and superimposed.

[0094] It should be understood that the above-mentioned embodiments are merely illustrative and not restrictive. Without departing from the basic principles of the present invention, various obvious or equivalent modifications or substitutions that can be made by those skilled in the art to the above-mentioned details will all be included in the scope of the claims of the present invention.

Claims

1. A classification and recognition method based on a parallel convolutional neural network implemented by a convolution kernel, wherein the parallel convolutional neural network is used to classify and recognize image data, and the parallel convolutional neural network has multiple 2 parallel convolutional kernels, and the 2 parallel convolutional kernels include: For receiving the first image data (x 2k ) is used to receive the second image data (x 2k+1 ) is used to output the first convolution signal (y 2k ) and a first output terminal (21) for outputting a second convolution signal (y 2k+1 ) a second output terminal (22); characterized in that the 2 parallel convolution kernels further include: A first convolution factor (H0), whose input terminal is connected to the first input terminal (11), is used to receive the first image data (x 2k ); A second convolution factor (H1), whose input terminal is connected to the second input terminal (12), is used to receive the second image data (x 2k+1 ); The third convolution factor (H0+H1) has an input end connected to the first input end (11) and the second input end (12) through an adder, and is used to receive the first image data (x 2k ) and the second image data (x 2k+1 ), the third convolution factor (H0+H1) is obtained by adding and subtracting the first convolution factor (H0) and the second convolution factor (H1); The output end of the first convolution factor (H0) and the output end of the second convolution factor (H1) are connected to the first output end (21) via an adder; The output end of the first convolution factor (H0), the output end of the second convolution factor (H1) and the output end of the third convolution factor (H0+H1) are connected to the second output end (22) through an adder; The parallel convolutional neural network is trained with a training set, and the trained parallel convolutional neural network model is used as a classification and recognition model to classify and recognize image data.

2. The classification and recognition method of a parallel convolutional neural network based on a convolution kernel as claimed in claim 1, characterized in that: The output signal of the output end of the third convolution factor (H0+H1) is subtracted from the output signal of the output end of the first convolution factor (H0) and the output signal of the output end of the second convolution factor (H1) to obtain a second convolution signal (y 2k+1 ).

3. The classification and recognition method of the parallel convolutional neural network based on the convolution kernel as claimed in claim 1, characterized in that: The output signal of the output end of the second convolution factor (H1) is added to the output signal of the output end of the first convolution factor (H0) after passing through the delay link (D) to obtain the first convolution signal (y 2k ), wherein the step value of the second convolution factor (H1) is greater than the step value of the first convolution factor (H0).

4. The classification and recognition method of a parallel convolutional neural network based on a convolution kernel as claimed in claim 1, characterized in that: The convolution coefficients of the two parallel convolution kernels are divided into a first convolution coefficient set and a second convolution coefficient set, wherein the first convolution factor (H0) is obtained by performing addition and subtraction operations on the convolution coefficients in the first convolution coefficient set; and the second convolution factor (H1) is obtained by performing addition and subtraction operations on the convolution coefficients in the second convolution coefficient set; The number of convolution coefficients in the first convolution coefficient set is the same as the number of convolution coefficients in the second convolution coefficient set, and the convolution coefficients in the first convolution coefficient set and the convolution coefficients in the second convolution coefficient set are not repeated.

5. A convolution operation method of parallel convolution, applied to the classification and recognition method of the parallel convolutional neural network based on the convolution kernel as described in any one of claims 1 to 4, to classify and recognize image data, characterized in that: The convolution operation method comprises: Step S100, calculating the independent convolution matrix (H n ) in the convolution factor, the independent convolution complex multiplication matrix (H n ) is a diagonal matrix consisting of the convolution factors of several 2-parallel convolution kernels; Step S200, storing the convolution factor in a register; Step S300, performing serial-to-parallel conversion on the image data according to the preset parallelism (n) to obtain a parallel input signal (X np ); Step S400, extracting the convolution factors from the register and respectively adding them to the parallel input signal (X np ) to perform matrix multiplication and addition operations to obtain parallel output signals (Y np ).

6. The convolution operation method according to claim 5, characterized in that: The step S100 includes: Step S110, dividing the convolution coefficients of the parallel convolutional neural network into n groups according to a preset parallelism (n); Step S120, generating an independent convolution matrix (H n ), wherein the convolution factor includes a first element and a second element, the first element is each group of n groups of convolution coefficients, and the second element is a result of adding and subtracting at least two groups selected from the n groups of convolution coefficients.

7. The convolution operation method according to claim 5, characterized in that: The parallel input signal (X np ) includes n sets of input signals, the parallel output signal (Y np ) includes n groups of output signals, the method further comprising: Pre-processing the n groups of input signals to combine the timing of the n groups of input signals; Post-process the n groups of output signals to integrate the timing of the n groups of output signals to obtain the parallel output signals (Y np ).

8. A convolution operation device for parallel convolution, applied to the classification and recognition method of a parallel convolutional neural network based on a convolution kernel as described in any one of claims 1 to 4, to classify and recognize image data, characterized in that: The device comprises: The factor calculation module (100) is used to calculate the independent convolution multiplication matrix (H) according to the preset parallelism (n). n ) in the convolution factor, the independent convolution complex multiplication matrix (H n ) is a diagonal matrix consisting of the convolution factors of several 2-parallel convolution kernels; A factor storage module (200), used for storing the convolution factor in a register; A serial-to-parallel conversion module (300) is used to perform serial-to-parallel conversion on the image data according to the preset parallelism (n) to obtain a parallel input signal (X np ); A convolution operation module (400) is used to extract the convolution factor from the register and respectively combine it with the parallel input signal (X np ) to perform matrix multiplication and addition operations to obtain parallel output signals (Y np ).

9. A computer device, characterized in that: Comprising the device as claimed in claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program stored in the storage medium is used to be executed by a processor to implement the method according to any one of claims 5 to 7.

Images