A convolutional neural network construction method for improving network performance

Abstract

The application discloses a convolutional neural network construction method for improving network performance, and belongs to the technical field of artificial intelligence. The method comprises the following steps: according to different low-level computer vision tasks, selecting corresponding image data sets and dividing the image data sets into a training set and a test set; pre-processing pictures in the training set; constructing a generalization convolutional neural network for improving network performance; training the generalization convolutional neural network based on the pre-processed training set pictures to obtain a trained generalization convolutional neural network, and realizing different low-level computer vision tasks of the pictures. The method enables the convolutional neural network to have a multi-scale receptive field to capture and understand features at different levels. In addition, the method promotes the transmission of feature information between network architectures by constructing a multi-branch dense interactive connection mode, improves the training speed of the network, greatly increases the utilization rate of the computing resources of the network, and improves the performance and generalization ability of the network.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and relates to a convolutional neural network construction method for improving network performance. BACKGROUND

[0002] With the continuous development of deep learning, a large number of advanced models have emerged, and the convolutional neural network model based on deep learning technology has been widely applied in many fields due to its strong feature learning and representation ability. At present, relevant scholars are always committed to further tapping the potential of convolutional neural network and improving the comprehensive performance of convolutional neural network, and the network performance improvement through network construction is a very common technical means.

[0003] At present, there are various ways to further enhance the performance of neural network in network construction. For example, increasing network depth, increasing network width, increasing convolution kernel size, setting jump connection, optimizing loss function and the like. These methods can help the network capture more complex features and expression ability. However, at present, it is still difficult for convolutional neural network to achieve the balance between receptive field and feature capture effect, and the balance between network depth and information transmission efficiency, so that the convolutional neural network still has great space for improvement in performance and generalization ability.

[0004] The convolutional neural network model has very wide application prospects in the field of computer vision, but the performance of the convolutional neural network is highly related to the following two factors:

[0005] (1) The balance problem between receptive field and feature capture effect: the receptive field in the convolutional neural network has a great influence on the network performance. Small receptive field can capture and understand more refined features, but it will have errors when dealing with high-level abstract problems. Large receptive field can capture a wide range of context information, but the precision and efficiency are not outstanding when dealing with local features.

[0006] (2) The balance problem between network depth and information transmission efficiency: deep convolutional neural network with more network layers can effectively improve the feature expression ability and generalization ability of the network, and thus improve the network performance, but at the same time, the feature information needs to go through more convolution or pooling operations to reach the later layers, which reduces the information transmission efficiency, makes the network difficult to train, and reduces the resource utilization rate. SUMMARY

[0007] In order to solve the above problems, the technical scheme adopted by the application is:

[0008] A convolutional neural network construction method for improving network performance, comprising the following steps:

[0009] According to different low-level computer vision tasks, corresponding image data sets are selected, and are divided into training sets and test sets;

[0010] The pictures in the training set are preprocessed;

[0011] A general-purpose generalization convolutional neural network for improving network performance is constructed;

[0012] Based on the preprocessed training set pictures, the general-purpose generalization convolutional neural network is trained to obtain a trained general-purpose generalization convolutional neural network, and different low-level computer vision tasks of pictures are realized.

[0013] Further, the different low-level computer vision tasks include but are not limited to image enhancement, image denoising, image defogging and image super-resolution.

[0014] Further, the general-purpose generalization convolutional neural network for improving network performance includes a cascaded multi-scale hollow convolutional feature pre-extraction module and a three-branch dense interactive connection module.

[0015] Further, the multi-scale hollow convolutional feature pre-extraction module contains two 3*3 hollow convolutional layers with a hollow rate of 3, two 3*3 hollow convolutional layers with a hollow rate of 2, and two 3*3 ordinary convolutional layers, wherein each convolutional layer is connected to a batch normalization layer and a ReLU activation function; then the input feature map of the first hollow convolutional layer is introduced into the output part of the last ordinary convolutional layer through a residual connection.

[0016] Further, the three-branch dense interactive connection module includes a first branch, a second branch and a third branch; the first branch includes a 3*3 hollow convolutional layer with a hollow rate of 3, the second branch is a 3*3 hollow convolutional layer with a hollow rate of 2, and the third branch includes a 3*3 ordinary convolutional layer.

[0017] The first branch, the second branch and the third branch are connected through dense connection, that is, the output feature map of any 3*3 convolutional layer in any branch will interact with the other two branches through channel merging; a 1*1 convolutional layer needs to be embedded between any two adjacent 3*3 convolutional layers in the same branch.

[0018] Further, the process of training the convolutional neural network based on the preprocessed training set pictures to obtain a trained general-purpose generalization convolutional neural network to realize different low-level computer vision tasks of pictures is as follows:

[0019] The preprocessed training set image data is input into the multi-scale hollow convolutional feature pre-extraction module for image feature pre-extraction to obtain pre-extracted feature maps;

[0020] The pre-extracted feature map is input into the three-branch dense interactive connection module, comprehensive extraction and representation of features are carried out, and efficient transmission of feature information is carried out, so as to obtain a generalization convolutional neural network output image of the network;

[0021] The loss function is used to compare the difference between the actual output image of the generalization convolutional neural network and the true value image in the training set, the parameters of the convolutional neural network model are optimized by minimizing the loss function, so as to achieve the purpose of training the convolutional neural network model;

[0022] The trained generalization convolutional network model is tested by using test set image data, and when the test index meets the identified standard, the current generalization neural network model is determined to be applied in practice, if the identified standard is not reached, the generalization convolutional neural network model training needs to be continued until the identified standard is reached.

[0023] Further, the identified standard is that when the value of the loss function tends to be stable or is near the convergence threshold, or the loss function converges on the training set while converging on the test set.

[0024] Further, the loss function is selected according to different low-level computer vision tasks.

[0025] The present application provides a convolutional neural network construction method for improving network performance, so that the network has a multi-scale receptive field to capture and understand different levels of features for problem (1), in addition, the method also promotes the transmission of feature information between network architectures by constructing a multi-branch dense interactive connection, so as to improve the training speed of the network, greatly increase the utilization rate of the network's computing resources, and further help to improve the performance and generalization ability of the network for problem (2).

[0026] The method has the following advantages:

[0027] By constructing a multi-scale hollow convolution feature pre-extraction module, the features of the original image can be deeply pre-extracted, and the combination of convolution with different hollow rates and ordinary convolution can help the network to extract the features of the image under different scale receptive fields, and achieve the balance between receptive field and feature capture effect.

[0028] By constructing the three-branch dense interactive connection module, the pre-extracted feature map obtained by using the multi-scale hollow convolution feature pre-extraction module can be further extracted and characterized, and different forms of convolution used in different branches further improve the diversity of the receptive field in the network, so that the features under different scale receptive fields can be further fully expressed; in addition, the three-branch dense interactive connection mode effectively balances the problem between network depth and information transmission efficiency, and the features between different branches are interacted by means of dense interactive connection, which not only can effectively improve the feature expression ability and generalization ability of the network, but also can significantly improve the information transmission efficiency and speed up the network training.

[0029] The convolutional neural network proposed in the application is a general model, which can be applied to various different low-level computer vision tasks, including but not limited to image enhancement, image denoising, image defogging, image super-resolution, etc. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0031] Figure 1 Flow chart for training and testing of the convolutional neural network of the present application;

[0032] Figure 2 Overall block diagram of the multi-scale hollow convolution designed in the present application;

[0033] Figure 3 Process diagram of the multi-scale hollow convolution feature pre-extraction module designed in the present application;

[0034] Figure 4 (a) is a hollow convolution diagram with a hollow rate of 3; (b) is a hollow convolution diagram with a hollow rate of 2; (c) is a normal convolution diagram;

[0035] Figure 5 Three-branch dense interactive connection module designed in the present application. DETAILED DESCRIPTION

[0036] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict, and the present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0039] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following figures denote similar items; therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0040] In the description of this invention, it should be understood that the orientation or positional relationship indicated by directional terms such as "front, back, up, down, left, right", "horizontal, vertical, horizontal" and "top, bottom" is generally based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing this invention and simplifying the description. Unless otherwise stated, these directional terms do not indicate or imply that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the scope of protection of this invention. The directional terms "inner" and "outer" refer to the inner and outer contours relative to the outline of each component itself.

[0041] For ease of description, spatial relative terms such as "above," "over," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation besides the orientation of the device as described in the figures. For example, if the device in the figures is inverted, a device described as "above" or "above" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.

[0042] Furthermore, it should be noted that the use of terms such as "first" and "second" to define components is merely for the purpose of distinguishing the corresponding components. Unless otherwise stated, the above terms have no special meaning and therefore should not be construed as limiting the scope of protection of this invention.

[0043] A method for constructing convolutional neural networks to improve network performance includes the following steps:

[0044] S1: Select the corresponding image datasets according to different low-level computer vision tasks, and divide them into training sets and test sets;

[0045] S2: Preprocess the images in the training set;

[0046] S3: Construct a generalized convolutional neural network to improve network performance;

[0047] S4: Based on the preprocessed training set images, the generalized convolutional neural network is trained to obtain a trained generalized convolutional neural network, which can then be used to perform different low-level computer vision tasks on images.

[0048] The various low-level computer vision tasks include, but are not limited to: image enhancement, image denoising, image dehazing, and image super-resolution.

[0049] Steps S1 / S2 / S3 / S4 are executed sequentially;

[0050] Furthermore, depending on the specific low-level computer vision task to be solved, an appropriate image dataset is selected and divided into training and testing sets. That is, based on the different low-level computer vision tasks to be solved, a publicly available online dataset corresponding to that task is selected, and then the images within the selected dataset are randomly divided. According to common partitioning rules in the field of artificial intelligence, the ratio of training to testing sets can generally be set to 8:2, 7:3, 8.5:1.5, etc., and can also be adjusted according to the actual situation.

[0051] The preprocessing method uses random pruning to allow the network to learn better; however, the test set does not require this operation.

[0052] The generalized convolutional neural network used to improve network performance includes a cascaded multi-scale dilated convolutional feature pre-extraction module and a three-branch dense interactive connection module.

[0053] The multi-scale dilated convolutional feature pre-extraction module includes two 3x3 dilated convolutional layers with a dilation rate of 3, two 3x3 dilated convolutional layers with a dilation rate of 2, and two 3x3 ordinary convolutional layers. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function. Then, the input feature map of the first dilated convolutional layer is introduced into the output part of the last ordinary convolutional layer through residual connections. This allows the neural network to better learn the complex features in the input image, effectively improves the training efficiency of the network, and solves the gradient vanishing and gradient exploding problems that may occur during network training.

[0054] The three-branch dense interactive connection module includes a first branch, a second branch, and a third branch; the first branch includes a 3*3 dilated convolutional layer with a dilation rate of 3, the second branch is a 3*3 dilated convolutional layer with a dilation rate of 2, and the third branch includes a 3*3 ordinary convolutional layer.

[0055] The first, second, and third branches are interconnected using dense connections. This means that the output feature map of any 3x3 convolutional layer in any branch interacts with the other two branches through channel merging. Within the same branch, a 1x1 convolutional layer is embedded between any two adjacent 3x3 convolutional layers. Since the essence of dense connections is channel merging of different output feature maps, the number of channels after merging can be too high (each merged feature map has 9 channels), affecting computational efficiency. To address this, this module embeds a 1x1 convolutional layer between any two adjacent 3x3 convolutional layers within the same branch, reducing the number of channels from 9 to 3. Finally, dense connections are used to merge the three branches into one, and a 1x1 convolutional layer further reduces the number of channels from 9 to 3.

[0056] Furthermore: the process of training the generalized convolutional neural network based on the preprocessed training set images to obtain a trained generalized convolutional neural network, and implementing different low-level computer vision tasks for images, is as follows:

[0057] The preprocessed training set image data is input into the multi-scale dilated convolution feature pre-extraction module to perform image feature pre-extraction and obtain the pre-extracted feature map.

[0058] The pre-extracted feature map is input into the three-branch dense interactive connection module to perform comprehensive feature extraction and representation, as well as efficient transmission of feature information, to obtain the actual output image of the network's convolutional neurons.

[0059] The loss function is used to compare the difference (i.e. model error) between the actual output image of the generalized convolutional neural network and the ground truth image in the training set. The parameters of the generalized convolutional neural network model are optimized by minimizing the loss function, thereby achieving the purpose of training the generalized convolutional neural network model.

[0060] The trained convolutional neural network model is tested using test set image data. Once the test metrics meet the established standards, the model is deemed suitable for practical application. If the standards are not met, training of a generalized convolutional neural network model continues until the standards are met. The loss function is determined based on the specific low-level computer vision task.

[0061] The criteria for determining convergence are: when the value of the loss function tends to stabilize or is near the convergence threshold, or when the loss function converges on both the training and test sets. The convergence threshold can be set to a small number, such as 0.001 or 0.004, where small errors (such as 3% or 5%) are permissible.

[0062] During training, the loss function value for each round needs to be recorded and the weights saved for loading during testing. The training time can be determined by whether the loss function for each round meets the criteria. If it does, training can be stopped, the weights loaded, and testing can be performed. If it does not meet the criteria, training can continue.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1.A method for constructing a convolutional neural network for improving network performance, characterized in that: The method comprises the following steps: According to different low-level computer vision tasks, select the corresponding image dataset, and divide it into training set and test set; Preprocess the pictures in the training set; Construct a generalization convolutional neural network for improving network performance; Based on the preprocessed training set pictures, the generalization convolutional neural network is trained to obtain a trained generalization convolutional neural network, and different low-level computer vision tasks of pictures are realized; The generalization convolutional neural network for improving network performance includes a cascaded multi-scale hollow convolution feature pre-extraction module and a three-branch dense interactive connection module. The multi-scale dilated convolution feature pre-extraction module includes two 3 3 dilated convolution layers, two 3 3 dilated convolution layers, and two 3 3 ordinary convolution layers, wherein each convolution layer is connected with a batch normalization layer and a ReLU activation function; then the input feature map of the first dilated convolution layer is introduced into the output part of the last ordinary convolution layer through a residual connection. 2.The method for constructing a convolutional neural network for improving network performance according to claim 1, wherein: The different low-level computer vision tasks include but are not limited to image enhancement, image denoising, image dehazing and image super-resolution. 3.The method for constructing a convolutional neural network for improving network performance according to claim 1, wherein: The three-branch dense interactive connection module comprises a first branch, a second branch and a third branch; the first branch comprises a 3 3 hollow convolutional layer, the second branch is a 3 3 hollow convolutional layer, and the third branch comprises a 3 3 ordinary convolutional layer; The first branch, the second branch and the third branch are interactively connected through dense connection, that is, for any one 3 3 convolutional layer in any one branch, the output feature map will be interacted with the other two branches through channel merging; any two adjacent 3 3 convolutional layers in the same branch need to embed a 1 1 convolutional layer. 4.The method for constructing a convolutional neural network for improving network performance according to claim 1, wherein: The process of training the convolutional neural network based on the preprocessed training set pictures to obtain a trained generalization convolutional neural network to realize different low-level computer vision tasks of pictures is as follows: The preprocessed training set image data is input into the multi-scale hollow convolution feature pre-extraction module for image feature pre-extraction to obtain pre-extracted feature maps; The pre-extracted feature maps are input into the three-branch dense interactive connection module for comprehensive extraction and representation of features, as well as efficient transmission of feature information, to obtain the actual output image of the generalization convolutional neural network; The loss function is used to compare the difference between the actual output image of the generalization convolutional neural network and the true value image in the training set, and the parameters of the convolutional neural network model are optimized by minimizing the loss function, so as to achieve the purpose of training the convolutional neural network model; The trained generalization convolutional network model is tested by using test set image data, and when the test index meets the identified standard, the current generalization neural network model is determined to be applied in practice. If the identified standard is not met, the generalization convolutional neural network model needs to be continuously trained until the standard is met. 5.The method for constructing a convolutional neural network for improving network performance according to claim 4, wherein: The loss function is determined according to different low-level computer vision tasks.

Images