Fusion module and multi-scale feature fusion convolutional neural network and image recognition method

Abstract

The present invention relates to a fusion module and a multi-scale feature fusion convolutional neural network and an image recognition method, comprising a previous layer, a fusion layer and a cascade layer, wherein the previous layer operates an input image through the fusion layer, and then merges and outputs the result of the operation through the cascade layer, and the fusion layer is formed by adding a bottleneck structure and a hole convolution structure to the original module. The present invention not only has a low error rate, but also effectively reduces the number of network parameters, which is conducive to improving the generalization ability of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a fusion module, a multi-scale feature fusion convolutional neural network and an image recognition method. Background Art

[0002] As one of the research hotspots in the field of machine learning, deep learning has made significant breakthroughs in recent years. As a representative of deep learning, convolutional neural networks are the most widely used, and their typical applications include image recognition, scene segmentation, and object detection.

[0003] Convolutional neural networks have achieved remarkable results in different applications and tasks, but such excellent performance is accompanied by significant computational costs. Compared with traditional algorithms in this field, they have a larger amount of computation and parameters. As convolutional neural network models continue to approach the accuracy limit of computer vision tasks, the structure of most deep convolutional neural networks has become increasingly complex. Calculating convolutional layers and fully connected layers requires a large number of floating-point matrix operations, and the computational overhead is extremely expensive. From the perspective of implementation in reality, models that are too large and too deep are not conducive to use on mobile devices and embedded chips that have strict restrictions on real-time performance, storage space, and energy consumption. In order to facilitate application, it is necessary to design a more lightweight network model. Summary of the invention

[0004] To this end, the technical problem to be solved by the present invention is to overcome the problems in the prior art that the convolution kernel of the convolutional neural network is too single, lacks diversity, and has a complex network structure and redundant parameters, thereby providing a fusion module and a multi-scale feature fusion convolutional neural network and image recognition method with low error rate, which effectively reduces the number of network parameters and improves the generalization ability of the model.

[0005] To solve the above technical problems, a fusion module of the present invention includes an upper layer, a fusion layer and a cascade layer. The upper layer operates the input image through the fusion layer, and then merges and outputs the result of the operation through the cascade layer. The fusion layer is composed of a bottleneck structure and a hole convolution structure added to the original module.

[0006] In one embodiment of the present invention, convolution kernels of different scales are used in the fusion layer.

[0007] In one embodiment of the present invention, a 1×1 convolutional layer and a 3×3 convolutional layer are provided in the bottleneck structure.

[0008] In one embodiment of the present invention, the area size calculation formula of the dilated convolution is: F(r)=(2 r+1 -1)×(2 r+1 -1), where the hyperparameter r means that r-1 spaces are filled between each pixel.

[0009] In one embodiment of the present invention, the fusion layer has five convolution branch channels.

[0010] In one embodiment of the present invention, the calculation method of the parameter F(i,n) of the fusion module is: Where k×k is the current convolution kernel size, C in is the number of input channels, C out is the number of output channels.

[0011] In one embodiment of the present invention, the calculation method of the calculation amount Flops(i,n) of the fusion module is: Where k×k is the current convolution kernel size, C in is the number of input channels, C out is the number of output channels.

[0012] The present invention also discloses a multi-scale feature fusion convolutional neural network, comprising: an input layer, a first convolutional layer, a maximum pooling layer, a plurality of fusion modules described in any of the above, a classification output layer and an output layer, and the previous layer and the next layer of each fusion module use a 1×1 convolution operation.

[0013] In one embodiment of the present invention, the previous layer and the next layer of each fusion module use a 1×1 convolution operation including: a second convolution layer, a first fusion module, a third convolution layer, a second fusion module, a fourth convolution layer, a third fusion module, a fifth convolution layer, a fourth fusion module, a sixth convolution layer, a seventh convolution layer, a first fusion module, a second fusion module, a third fusion module, and a fourth fusion module, and the input layer, the first convolution layer, the maximum pooling layer, the second convolution layer, the first fusion module, the third convolution layer, the second fusion module, the fourth convolution layer, the third fusion module, the fifth convolution layer, the fourth fusion module, the sixth convolution layer, the seventh convolution layer, the classification output layer and the output layer are connected in order according to the topological structure.

[0014] The present invention also discloses an image recognition method of a multi-scale feature fusion convolutional neural network, comprising: step S1: performing a convolution pooling operation on an input image of the multi-scale feature fusion convolutional neural network; step S2: using a 1×1 convolution operation on both the front and rear layers of each fusion module; step S3: classifying and outputting the output end of the multi-scale feature fusion convolutional neural network through a global average pooling layer or a convolution layer.

[0015] The above technical solution of the present invention has the following advantages compared with the prior art:

[0016] The fusion module, multi-scale feature fusion convolutional neural network and image recognition method described in the present invention aim at the shortcomings of too many parameters and complex network structure of traditional convolutional neural network. By analyzing and comparing the existing convolutional neural network models, a multi-scale feature fusion convolutional neural network MS-FNet is designed. The network introduces a fusion module and extracts features of different scales through convolution kernels of different sizes, which not only increases the width of the network, but also improves the robustness, and solves the problem that the convolution kernel used to extract features in general convolutional neural networks is too single and lacks diversity and comprehensiveness; in addition, the convolution layer is used at the end of the network instead of the traditional fully connected layer, which effectively controls the number of network model parameters. Experimental results show that compared with common convolutional neural network models, MS-FNet has a more reasonable network structure and a faster convergence speed. When the number of parameters is much smaller than that of other convolutional neural network models, the error rate is lower and it has better generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

[0018] Figure 1 is a schematic diagram of a fusion module of the present invention;

[0019] Figure 2 is a schematic diagram of the original module of the present invention;

[0020] Figure 3a and Figure 3b They are schematic diagrams of the standard convolution and bottleneck structures of the present invention respectively;

[0021] Figure 4 Schematic diagram of a multi-scale feature fusion convolutional neural network of the present invention;

[0022] Figure 5 It is the parameter setting of MS-FNet of the present invention on the CIFAR-10 dataset;

[0023] Figure 6 This is a comparison of the CIFAR-10 experimental results of the present invention;

[0024] Figure 7 This is a comparison of the CIFAR-100 experimental results of the present invention;

[0025] Figure 8 This is the comparison result of the present invention using dilated convolution and not using dilated convolution;

[0026] Fig. 9 This is a comparison of the experimental results of the MNIST data set of the present invention;

[0027] Fig.10It is a schematic diagram of the comparison of accuracy curves on the MNIST data set of the present invention;

[0028] Fig.11 It is a schematic diagram of the comparison of the cross entropy loss function curves on the CIFAR-10 dataset of the present invention.

[0029] Explanation of the reference numerals in the specification: 10 - previous layer, 20 - fusion layer, 21 - bottleneck structure, 22 - dilated convolution structure, 30 - cascade layer. DETAILED DESCRIPTION

[0030] Embodiment 1

[0031] like Figure 1 As shown, this embodiment provides a fusion module, including an upper layer 10, a fusion layer 20 and a cascade layer 30. The upper layer 10 operates the input image through the fusion layer 20, and then merges and outputs the result of the operation through the cascade layer 30. The fusion layer 20 is composed of a bottleneck structure 21 and a dilated convolution (dilated conv) structure 22 added to the original module.

[0032] The fusion module described in this embodiment includes an upper layer 10, a fusion layer 20 and a cascade layer 30. The upper layer 10 inputs the image through the fusion layer operation 20, and then merges the result of the operation through the cascade layer 30 for output. The fusion layer 20 is composed of a bottleneck structure 21 and a hole convolution structure 22 added to the original module. Since the fusion layer 20 can use features of different pixel sizes to represent different extracted details, and then fuse these areas to represent the fusion of different details, it is equivalent to performing convolution at multiple scales at the same time. Therefore, the extracted features are also of different scales. These features of different scales are richer, which is conducive to the output of the final classification result; at the same time, the difference in receptive fields between different feature channels can be achieved, so that the output features are no longer evenly distributed, but the features with strong correlation are clustered together. Therefore, the output feature redundant information is less, which solves the problem that the convolution kernel used to extract features in general convolutional neural networks is too single and lacks diversity and comprehensiveness, which is conducive to improving the feature extraction ability of the network model.

[0033] In the fusion layer 20, by using convolution kernels of different scales, it is possible to achieve different receptive fields between different feature channels, so that the output features are no longer evenly distributed, but the features with strong correlation are clustered together. Therefore, the output features have less redundant information, thereby achieving the purpose of improving the feature extraction capability of the network model.

[0034] Specifically, Figure 2As shown in the figure, the structural features of the original module are as follows: first, by using convolution decomposition, since convolution kernels of different scales are used, the same input can be convolved using multiple convolution kernels of different scales (such as 1×1, 3×3), and then the results of the operation are merged and output. Since the original module encapsulates convolution kernels of different sizes to capture spatial information of different scales, the obtained features can be fused to obtain richer features than a single convolution kernel; secondly, in the original module, the traditional convolution layer is replaced by a small block, which not only improves the learning ability, but also improves the utilization efficiency of parameters; thirdly, a 1×1 point-by-point convolution kernel is used for calculation. Since it is a sparse structure that efficiently expresses features, it improves the accuracy without overfitting, thereby reducing the parameters of the network, and can establish a deeper network structure under limited computing resources, so that the depth and width of the network can be efficiently expanded.

[0035] The principle of the original module is to use the dense part to perfectly fit the sparse part, use feature extractors of different pixels to represent different areas, and then combine these areas. The extracted feature details are richer, and the classification results of the network model will be more accurate.

[0036] like Figure 3a and Figure 3b As shown, compared with the standard convolutional layer Figure 3a In comparison, assuming that the number of input and output feature maps N is 128, the standard convolution layer parameter amount P = 128*3*3*128 = 147456, while the convolution layer parameter amount P of the bottleneck structure 21 is 128*1*1*32+32*3*3*32+32*1*1*128 = 17408. It can be seen from the calculation that the parameter amount of the bottleneck structure 21 is about one eighth of the parameter amount of the standard convolution layer. In addition, since the bottleneck structure 21 is deeper than the standard convolution structure, the nonlinear expression ability of the network is also stronger. Therefore, the bottleneck structure 21 not only reduces the network parameters, but also increases the network depth, thereby improving the accuracy of the network.

[0037] The bottleneck structure 21 is provided with a 1×1 convolution layer and a 3×3 convolution layer. This structure can not only reduce the amount of network parameters, but also improve the computational efficiency of the network.

[0038] In traditional convolutional neural networks, feature maps are generally downsampled after the convolution layer, which will lead to information loss. In order to reduce the loss, a hollow convolution structure22 can be added to the network structure. This expands the receptive field and improves network performance without increasing model parameters and computational complexity.

[0039] The atrous convolution structure 22 is actually a special way of data sampling. It changes from dense sampling of data to sparse sampling. It does not change the learned parameters, but changes the sampling method of the input data. Therefore, it can be used seamlessly in the network model without changing the structure of the network model and without adding additional parameters and calculations.

[0040] The area size calculation formula of the dilated convolution structure 22 is: F(r) = (2 r+1 -1)×(2 r+1 -1), where the hyperparameter r means that r-1 spaces are filled between each pixel.

[0041] Assuming that the convolution kernel size is k×k and the dilated convolution step size is r, it is equivalent to obtaining the k2 values ​​used to calculate the convolution from positions separated by r-1 in the feature map, so that the receptive field changes from k×k to k+(r-1)×(k-1).

[0042] The fusion layer 20 has five convolution branch channels. Since features of different pixel sizes are used to represent different extracted details, and then these areas are fused to represent the fusion of different details, which is equivalent to performing convolution at multiple scales at the same time, the extracted features are also of different scales. Therefore, features of different scales are richer, which is conducive to the output of the final classification results.

[0043] In this application, the calculation method of the parameter quantity F(i,n) of the fusion module is: Where k×k is the current convolution kernel size, C in is the number of input channels, C out is the number of output channels.

[0044] The calculation method of the calculation amount Flops(i,n) of the fusion module is: Where k×k is the current convolution kernel size, C in is the number of input channels, C out is the number of output channels.

[0045] Embodiment 2

[0046] like Figure 4 As shown, this embodiment provides a multi-scale feature fusion convolutional neural network (MS-FNet for short), including: an input layer, a first convolution layer, a maximum pooling layer, multiple fusion modules described in Example 1, a classification output layer and an output layer, and the previous layer and the next layer of each fusion module use a 1×1 convolution operation.

[0047] The multi-scale feature fusion convolutional neural network described in this embodiment includes: an input layer, a first convolutional layer, a maximum pooling layer, multiple fusion modules described in Example 1, a classification output layer and an output layer, and the previous layer and the next layer of each fusion module use a 1×1 convolution operation, thereby reducing or increasing the dimension of the feature map, and at the same time, a nonlinear layer is added with a very small amount of calculation, which enhances the spatial information exchange of the feature map after the cascade operation and realizes feature fusion between feature channels with different receptive fields; in addition, the output end of the network uses a convolutional layer instead of a fully connected layer for classification output, converts the feature map into the original input size, ensures that the output layer nodes have a larger receptive field in the original input space, and realizes pixel-level classification through upsampling.

[0048] In this embodiment, in order to increase the width of the network and improve the robustness, the number of the fusion modules is four. Specifically, the previous layer and the next layer of each fusion module use a 1×1 convolution operation including: a second convolution layer, a first fusion module, a third convolution layer, a second fusion module, a fourth convolution layer, a third fusion module, a fifth convolution layer, a fourth fusion module, a sixth convolution layer, and a seventh convolution layer, and the input layer, the first convolution layer, the maximum pooling layer, the second convolution layer, the first fusion module, the third convolution layer, the second fusion module, the fourth convolution layer, the third fusion module, the fifth convolution layer, the fourth fusion module, the sixth convolution layer, the seventh convolution layer, the classification output layer, and the output layer are connected in order according to the topological structure, so that the difference in receptive fields between different feature channels can be achieved, so that the output features are no longer evenly distributed, while reducing the computational cost and achieving higher accuracy.

[0049] In the multi-scale feature fusion convolutional neural network, a convolution pooling operation is first performed on the network input, which is beneficial to combining the extracted features, thereby increasing the output channel of the next layer; then a 1×1 convolution operation is used before and after each fusion module, which can reduce or increase the dimension of the feature map. At the same time, a nonlinear layer is added with a very small amount of calculation, which enhances the spatial information exchange of the feature map after the cascade operation, realizes the feature fusion between feature channels of different receptive fields, and finally uses a convolution layer for classification output.

[0050] Since the distribution of input data changes after convolution operation, batch normalization (Batch Normalization) is performed on the input of the previous layer before and after each activation function operation to process the data distribution. The mean of the data after the BN layer becomes 0 and the standard deviation becomes 1, which reduces the correlation between features and normalizes the output of the network to a normal distribution. Therefore, it can speed up the training of deep convolutional networks, solve the problem of internal covariate shift caused by changes in data distribution during network training, and alleviate the phenomenon of network overfitting. In addition, all convolution operations are activated using the ReLU activation function, which can enhance feature sparsity and linear separability.

[0051] The main indicator for evaluating the time performance of an algorithm is the time complexity of the algorithm. Assuming that the dimension of the input feature graph of the multi-scale feature fusion convolutional neural network is n, the overall time complexity of MS-FNet is:

[0052]

[0053] Among them, N represents the depth of the network, i represents the i-th convolutional layer, M represents the output feature map size, K represents the convolution kernel size, and N i-1 Represents the dimension of the feature map of the previous layer, N i Indicates the dimension of the feature map of the next layer.

[0054] Embodiment 3

[0055] The present embodiment provides an image recognition method of a multi-scale feature fusion convolutional neural network, comprising: step S1: performing a convolution pooling operation on an input image of the multi-scale feature fusion convolutional neural network; step S2: using a 1×1 convolution operation on both the front and rear layers of each fusion module; step S3: classifying and outputting the output end of the multi-scale feature fusion convolutional neural network through a global average pooling layer or a convolution layer.

[0056] In the image recognition method of the multi-scale feature fusion convolutional neural network described in this embodiment, in the step S1, a convolution pooling operation is performed on the input image of the multi-scale feature fusion convolutional neural network, which is conducive to combining the extracted features, thereby increasing the output channel of the next layer; in the step S2, a 1×1 convolution operation is used for the previous layer and the next layer of each fusion module, and the feature map can be reduced or increased in dimension. At the same time, a nonlinear layer is added with a very small amount of calculation, which enhances the spatial information exchange of the feature map after the cascade operation and realizes the feature fusion between feature channels with different receptive fields; in the step S3, the output end of the multi-scale feature fusion convolutional neural network is classified and output through a global average pooling layer or a convolution layer, and the feature map is converted to the original input size to ensure that the output layer node has a larger receptive field in the original input space, and pixel-level classification is realized through upsampling.

[0057] The test results and analysis of this application are as follows:

[0058] The experimental platform of the present invention is a GTX1080Ti GPU, and the deep learning framework used is TensorFlow. Experiments are carried out on three data sets: MNIST, CIFAR-10, and CIFAR-100. The parameter quantity and error rate of the network are used to verify the performance of the network model MS-FNet.

[0059] CIFAR-10 Dataset:

[0060] The CIFAR-10 dataset contains 60,000 RGB three-channel images of size 32×32, of which 50,000 images are used for training and 10,000 images are used for testing. The dataset consists of 10 classes, each with 5,000 training images and 1,000 test images. Before training the model, the samples are preprocessed, each image is randomly flipped, random brightness changes and contrast changes are set, and the images are cropped to a size of 28×28, so that more samples are produced, the utilization rate of sample images is improved, and the sample images have both random noise and data augmentation.

[0061] like Figure 5As shown in the figure, taking the CIFAR-10 dataset as an example, the parameter settings of the network model MS-FNet are introduced in detail. The first column Layer name indicates the operation of each layer of the network model, the second column Filter size / stride indicates the convolution kernel size and stride used in the current layer, the third column Output size is the feature map size and number of channels output by the current layer, the fourth column Filter size / Channel numbers (Fusion module) is the convolution kernel size and number of feature map channels output by each branch of the Fusion module, the fifth column Parameter is the parameter amount generated by each layer operation, and the last line is the sum of the parameter amounts of all layers. Finally, the total parameter amount of MS-FNet is calculated to be about 0.37M.

[0062] like Figure 6 As shown in the figure, the parameter amount and error rate comparison results of the network model MS-FNet and other different network models on the CIFAR-10 dataset are shown. From the data comparison in the table, it can be seen that the parameter amount of the network model MS-FNet on the CIFAR-10 dataset is only 0.37M, which is 1 / 7 of FitNet, 1 / 23 of SqueezeNet, and 1 / 100 of FractalNet. From the analysis of the parameter amount, the network model MS-FNet is far lower than the traditional convolutional neural network model. At the same time, the error rate is only 6.19%, which is relatively lower than other network models. Although the error rates of WideResNet and FractalNet are lower, their parameter amounts are much larger than the network model MS-FNet. This shows that MS-FNet greatly reduces the parameter amount of the network while ensuring a low error rate, and has better classification accuracy and generalization ability.

[0063] CIFAR-100 Dataset:

[0064] The CIFAR-100 dataset is similar to the CIFAR-10 dataset, and also contains 60,000 RGB three-channel images of size 32×32. Unlike CIFAR-10, the CIFAR-100 dataset contains 100 classes, which are expanded from 20 superclasses. Each class contains 500 training images and 100 test images, for a total of 600 images.

[0065] like Figure 7 As shown in Figure 2, the network structure parameters are set on the CIFAR-100 dataset. Figure 5 Consistent. Figure 7 The experimental results of MS-FNet and other network models on the CIFAR-100 dataset are compared. Figure 7It can be seen that the network MS-FNet greatly reduces the number of network parameters while ensuring that the error rate does not increase.

[0066] MNIST Dataset:

[0067] The MNIST dataset is a subset of the handwriting database NIST. The dataset contains handwritten digits 0 to 9. Each type of digit has 6,000 training images and 1,000 test images. Each image is a 28×28 single-channel image, totaling 70,000 images. Since the MNIST dataset is relatively simple, the number of convolutional layers and Fusiom modules is reduced in the network parameter setting of MS-FNet, and the number of feature map channels is reduced to a single channel.

[0068] from Figure 8 It can be seen that under the same experimental environment and number of iterations, the training time of using dilated convolution is reduced by 13% on average compared to not using dilated convolution. With the increase in the number of training iterations, the training time of the network models using and not using dilated convolution increases, while the error rate decreases. However, the error rate of the network model using dilated convolution is slightly higher than that of the network model not using dilated convolution. Therefore, this group of experiments proves that adding dilated convolution appropriately to the network model can speed up the training of the model, make the network model converge faster, and thus improve network performance.

[0069] Fig. 9 The experimental results of MS-FNet and other networks on the MNIST dataset are compared. Fig. 9 It can be seen that the error rate of the network model MS-FNet is only 0.42%, which is lower than that of other network models, indicating that the network model MS-FNet has better classification ability.

[0070] Network performance analysis:

[0071] In order to intuitively analyze the network performance of the network model, the accuracy curve and loss function curve of MS-FNet are listed respectively.

[0072] like Fig.10 As shown in the figure, the accuracy curves of MS-FNet on the MNIST dataset are shown under three different processing methods: fully connected layer, global average pooling layer and convolutional layer. Fig.10 The horizontal axis is the number of iterations of network model training, which was iterated 1500 times on the MNIST dataset; the vertical axis is the accuracy value of the network model. Fig.10It can be seen that as the number of iterations increases, the accuracy of the three processing methods is on the rise, but the accuracy curve using the convolution layer curve rises the fastest, and the final accuracy value is also the highest. The global average pooling layer curve rises slightly slower than the convolution layer curve, but the final accuracy value is close to the convolution layer curve. The fully connected layer curve rises the slowest, and the final accuracy value is also the lowest. Therefore, through the analysis of the accuracy curve, it can be seen that using the convolution layer instead of the fully connected layer can speed up the network training speed and improve the network accuracy accordingly.

[0073] The loss function curve is one of the performance indicators for evaluating convolutional neural networks. It can reflect the convergence of network model training. Generally, the convergence speed and convergence value of the model are used as the standard for evaluating model performance. Fig.11 This is a comparison of the cross entropy loss function curves of the network model MS-FNet on the CIFAR-10 dataset. At the end of the network, three different processing methods are used: fully connected layer, global average pooling layer, and convolution layer. The convergence ability of the network model is analyzed through the loss function curves of these three methods. Fig.11 It can be seen that the horizontal axis of the loss function curve is the number of iterations of network model training, which was 5000 times on the CIFAR-10 dataset; the vertical axis is the error loss value of different network models during the training process. Fig.11 It can be seen that as the number of iterations increases, the loss function curves of the three processing methods tend to converge, but the fully connected layer curve converges to a higher position and decreases the slowest. The global average pooling layer curve decreases faster than the fully connected layer curve and the convergence value is also smaller. The convolution layer curve decreases the fastest and converges to a smaller value than the global average pooling layer curve. Comprehensive comparative analysis shows that using convolution layers instead of fully connected layers can make the network model converge faster and achieve better results.

[0074] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0075] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0076] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0077] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0078] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. An image recognition system based on multi-scale feature fusion convolutional neural network, characterized in that: include: The input layer, the first convolution layer, the maximum pooling layer, multiple fusion modules, the classification output layer and the output layer, and the previous layer and the next layer of each fusion module use 1×1 convolution operation, wherein the fusion module includes the previous layer, the fusion layer and the cascade layer, the previous layer operates the input image through the fusion layer, and then merges and outputs the result of the operation through the cascade layer, the fusion layer is composed of adding the bottleneck structure and the hole convolution structure to the original module, and the parameter quantity of the fusion module is F(i,n) The calculation method is: ,in k × k is the current convolution kernel size, C in is the number of input channels, C out is the number of output channels. The bottleneck structure is provided with a 1×1 convolution layer and a 3×3 convolution layer. The area size calculation formula of the hole convolution is: , where the hyperparameters r Indicates that each pixel is filled r -1 space.

2. The image recognition system based on multi-scale feature fusion convolutional neural network according to claim 1, characterized in that: The previous layer and the next layer of each fusion module use 1×1 convolution operations, including: the second convolution layer, the first fusion module, the third convolution layer, the second fusion module, the fourth convolution layer, the third fusion module, the fifth convolution layer, the fourth fusion module, the sixth convolution layer, the seventh convolution layer, the first fusion module, the second fusion module, the third fusion module, and the fourth fusion module, and the input layer, the first convolution layer, the maximum pooling layer, the second convolution layer, the first fusion module, the third convolution layer, the second fusion module, the fourth convolution layer, the third fusion module, the fifth convolution layer, the fourth fusion module, the sixth convolution layer, the seventh convolution layer, the classification output layer and the output layer are connected in order according to the topological structure.

3. The image recognition system based on multi-scale feature fusion convolutional neural network according to claim 1, characterized in that: In the fusion layer, convolution kernels of different scales are used.

4. The image recognition system based on multi-scale feature fusion convolutional neural network according to claim 1, characterized in that: The fusion layer has five convolutional branch channels.

5. The image recognition system based on multi-scale feature fusion convolutional neural network according to claim 1, characterized in that: The computational effort of the fusion module Flops(i,n) The calculation method is: ,in k × k is the current convolution kernel size, C in is the number of input channels, C out is the number of output channels.

6. An image recognition method based on multi-scale feature fusion convolutional neural network, characterized in that: include: Step S1: performing convolution pooling operation on the input image of the multi-scale feature fusion convolutional neural network; Step S2: A 1×1 convolution operation is used for the previous layer and the next layer of each fusion module, wherein the fusion module includes the previous layer, the fusion layer and the cascade layer. The previous layer operates the input image through the fusion layer, and then merges and outputs the result of the operation through the cascade layer. The fusion layer is composed of adding a bottleneck structure and a hole convolution structure to the original module. The parameter quantity of the fusion module is F(i,n) The calculation method is: ,in k × k is the current convolution kernel size, C in is the number of input channels, C out is the number of output channels. The bottleneck structure is provided with a 1×1 convolution layer and a 3×3 convolution layer. The area size calculation formula of the hole convolution is: , where the hyperparameters r Indicates that each pixel is filled r -1 space; Step S3: The output end of the multi-scale feature fusion convolutional neural network is classified and output through a global average pooling layer or a convolution layer.

Images