A general deep convolutional neural network construction method for improving generalization ability
By constructing a deep convolutional neural network containing an encoder and a decoder, and employing downsampling and upsampling strategies in the feature extraction and reconstruction modules, as well as a skip connection mechanism, the problem of insufficient generalization ability of deep convolutional neural networks is solved, enabling effective application in low-level computer vision tasks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DALIAN MARITIME UNIVERSITY
- Filing Date
- 2023-06-14
- Publication Date
- 2026-05-29
Smart Images
Figure CN116644786B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence and relates to a method for constructing a general-purpose deep convolutional neural network to improve generalization ability. Background Technology
[0002] Deep Convolutional Neural Network (DCNN) is an artificial neural network that can be used for a variety of computer vision tasks. Its design is inspired by the biological visual processing system. Through the complex construction operations between multiple convolutional layers, activation functions, pooling layers and fully connected layers, it can gradually extract feature information from images and perform high-level abstraction and classification.
[0003] As network depth increases, deep convolutional neural networks (CNNs) can gradually extract increasingly abstract features, thus better handling complex computer vision tasks. However, CNNs still have shortcomings in generalization ability. This is because increasing the number of network layers reduces the efficiency of effective feature transfer within the network and easily leads to feature information loss. Furthermore, increasing the number of network layers can cause gradient vanishing or exploding phenomena during training. Therefore, at present, it is difficult to find a general-purpose deep convolutional neural network model applicable to various low-level computer vision tasks.
[0004] Currently, deep convolutional neural networks (CNNs) exhibit superior performance in various computer vision tasks. However, existing deep CNNs still have shortcomings in generalization ability, mainly in the following two aspects:
[0005] (1) As the number of network layers increases, the transmission efficiency of effective features in the image in the network decreases, and feature information is easily lost.
[0006] (2) As the number of network layers increases, the network is prone to gradient vanishing or exploding during training, making it difficult for the training process to converge or to train the correct parameters.
[0007] All of the above factors can affect the generalization ability of deep convolutional neural networks, making it difficult to apply the same network to different low-level vision tasks. Summary of the Invention
[0008] To solve the above problems, the present invention provides the following technical solution:
[0009] A general method for constructing deep convolutional neural networks to improve generalization ability includes the following steps:
[0010] Based on different visual processing tasks, select the corresponding datasets and divide the datasets into training sets and test sets;
[0011] Preprocess the training and test sets;
[0012] A general-purpose deep convolutional neural network with improved generalization ability was constructed.
[0013] The preprocessed training data is input into the constructed general-purpose deep convolutional neural network for multi-scale extraction and reconstruction to obtain the actual output image; the general-purpose deep convolutional neural network is then trained.
[0014] The preprocessed test set data is input into a trained general-purpose deep convolutional neural network model to achieve different low-level computer vision tasks for image processing.
[0015] Furthermore: A general-purpose deep convolutional neural network includes an encoder part and a decoder part;
[0016] The encoder section includes four feature extraction modules; the decoder section includes four feature reconstruction modules.
[0017] The feature extraction modules employ a value extraction downsampling strategy.
[0018] The feature reconstruction modules employ an interpolation upsampling strategy.
[0019] A skip connection mechanism is used to connect the symmetrical feature extraction module and the feature reconstruction module.
[0020] Furthermore, the feature extraction module includes a 7×7 convolutional layer with a normalized transition layer and two 3×3 convolutional layers with ReLU activation functions, and introduces the input features of the first convolutional layer into the output part of the last convolutional layer through residuals.
[0021] Furthermore, the feature reconstruction module includes three 3×3 convolutional layers, with each convolutional layer followed by a normalization layer and a ReLU activation function. The input features of the first 3×3 convolutional layer are introduced into the output part of the second 3×3 convolutional layer through the residuals, and the output features of the first 3×3 convolutional layer are introduced into the output part of the third 3×3 convolutional layer.
[0022] Furthermore: the sampling downsampling strategy extracts feature values at fixed intervals to form a new feature map, thereby reducing the feature map size while increasing the number of feature map channels;
[0023] During downsampling, the formula for feature map size change is:
[0024] (C,H,W)→(r 2*C,H / r,W / r)
[0025] Where C is the number of feature maps, H and W are the height and width of the feature maps, respectively, and r represents the fixed sampling interval, i.e., the downsampling factor.
[0026] Furthermore: the interpolation upsampling strategy inserts feature values at fixed intervals to form a new feature map, thereby increasing the feature map size while decreasing the number of feature map channels;
[0027] The interpolation upsampling strategy, after conventional upsampling of the input feature map, first pads the edges of the output feature map with zeros, then performs feature map shifting, averaging and fusion, and finally removes the edge padding. While the feature map size increases, the number of feature map channels decreases. The formula for feature map size change during upsampling is:
[0028] (C,H,W)→(C / r 2 ,H*r,W*r)
[0029] Where C is the number of feature maps, H and W are the height and width of the feature maps, respectively, and r represents the upsampling factor.
[0030] Furthermore, it also includes introducing a channel attention mechanism into the skip connection mechanism, including the following steps:
[0031] 1) Perform global average pooling on the input feature map to obtain a feature vector of size 1×1×C;
[0032] 2) Input the obtained feature vector into two fully connected layers. The first fully connected layer doubles the number of channels in the feature vector, and the second fully connected layer reduces it by half, thus restoring the original number of channels.
[0033] 3) Finally, the input feature map is processed by the Sigmoid function and multiplied channel by channel with the original input feature map.
[0034] 4) Add the obtained output to the output of the referenced feature reconstruction module.
[0035] Furthermore, the various low-level computer vision tasks include, but are not limited to, image enhancement, image denoising, image dehazing, and image super-resolution.
[0036] The present invention provides a general deep convolutional neural network construction method for improving generalization ability, which has the following advantages:
[0037] By constructing a general-purpose deep convolutional neural network structure, it can be applied to a variety of low-level computer vision tasks;
[0038] The constructed encoder can efficiently extract features and effectively suppress information loss in deep convolutional neural networks, while preventing gradient vanishing or exploding that may occur during network training.
[0039] The constructed decoder can preserve input features to the greatest extent, avoid information loss during feature reconstruction, and increase the nonlinear relationship between layers to further prevent gradient vanishing or exploding.
[0040] The proposed sampling downsampling strategy can retain feature information while performing feature map downsampling, thus avoiding the loss of feature information.
[0041] The proposed interpolation upsampling strategy can ensure that upsampling is achieved while preventing positional deviation of feature information, thus facilitating image detail recovery. Attached Figure Description
[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0043] Figure 1 This is a flowchart of the network training and testing process of the present invention;
[0044] Figure 2 This is a diagram of the encoder-decoder structure constructed in this invention;
[0045] Figure 3 This is a structural diagram of the feature extraction module and feature reconstruction module constructed in this invention;
[0046] Figure 4 This is a structural diagram of the sampling downsampling module and the interpolation upsampling module designed in this invention;
[0047] Figure 5 This is a structural diagram of the channel attention mechanism referenced in this invention. Detailed Implementation
[0048] It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.
[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0050] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.
[0051] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values of the components and steps described in these embodiments do not limit the scope of the invention. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following figures denote similar items; therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0052] In the description of this invention, it should be understood that the orientation or positional relationship indicated by directional terms such as "front, back, up, down, left, right", "horizontal, vertical, horizontal" and "top, bottom" is generally based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing this invention and simplifying the description. Unless otherwise stated, these directional terms do not indicate or imply that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the scope of protection of this invention. The directional terms "inner" and "outer" refer to the inner and outer contours relative to the outline of each component itself.
[0053] For ease of description, spatial relative terms such as "above," "over," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation besides the orientation of the device as described in the figures. For example, if the device in the figures is inverted, a device described as "above" or "above" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.
[0054] Furthermore, it should be noted that the use of terms such as "first" and "second" to define components is merely for the purpose of distinguishing the corresponding components. Unless otherwise stated, the above terms have no special meaning and therefore should not be construed as limiting the scope of protection of this invention.
[0055] A general method for constructing deep convolutional neural networks to improve generalization ability includes the following steps:
[0056] S1: Select the corresponding dataset according to different vision processing tasks, and divide the dataset into training set and test set;
[0057] The various low-level computer vision tasks include, but are not limited to, image enhancement, image denoising, image dehazing, and image super-resolution.
[0058] S2: Preprocess the training and test sets;
[0059] S3: Constructing a general-purpose deep convolutional neural network to improve generalization ability;
[0060] S4: Input the preprocessed training set data into the constructed general-purpose deep convolutional neural network to perform multi-scale extraction and reconstruction to obtain the actual output image; train the constructed general-purpose deep convolutional neural network;
[0061] S5: The preprocessed test set data is input into the trained general-purpose deep convolutional neural network model to achieve different low-level computer vision tasks for image processing.
[0062] The steps S1 / S2 / S3 / S4 / S5 are executed sequentially;
[0063] Based on different visual processing tasks, select the corresponding dataset and divide the dataset into training set and test set; adjust the ratio of training set to test set according to the actual situation, usually set to 7:3, 8:2, etc.
[0064] There are publicly available datasets for image enhancement, image denoising, image dehazing, and image super-resolution, which are randomly divided according to proportions.
[0065] Furthermore, the preprocessing process for the training and test sets is as follows:
[0066] In the training set, images are first scaled to 1.5 times the specified size, then randomly cropped according to the specified size, and finally randomly flipped. In the test set, images are only scaled to the specified size. It should be noted that the specified size depends on the hardware computing resources; the images in the training and test sets are input and label pairs, and the processing of each pair of images should be identical.
[0067] Furthermore: A general-purpose deep convolutional neural network includes an encoder part and a decoder part;
[0068] The encoder section includes four feature extraction modules; the decoder section includes four feature reconstruction modules.
[0069] The feature extraction modules employ a value extraction downsampling strategy.
[0070] The feature reconstruction modules employ an interpolation upsampling strategy.
[0071] A skip connection mechanism is used to connect the symmetrical feature extraction module and the feature reconstruction module.
[0072] The feature extraction module includes a 7×7 convolutional layer with layer normalization and two 3×3 convolutional layers with ReLU activation functions. The input features of the first convolutional layer are introduced into the output of the last convolutional layer through residuals.
[0073] The feature reconstruction module includes three 3×3 convolutional layers, with each convolutional layer followed by a normalization layer and a ReLU activation function. The input features of the first 3×3 convolutional layer are introduced into the output part of the second 3×3 convolutional layer through residuals, and the output features of the first 3×3 convolutional layer are introduced into the output part of the third 3×3 convolutional layer.
[0074] The sampling downsampling strategy extracts feature values at fixed intervals to form a new feature map, thereby reducing the feature map size while increasing the number of feature map channels.
[0075] During downsampling, the formula for feature map size change is:
[0076] (C,H,W)→(r 2 *C,H / r,W / r)
[0077] Where C is the number of feature maps, H and W are the height and width of the feature maps, respectively, and r represents the fixed sampling interval, i.e., the downsampling factor.
[0078] The interpolation upsampling strategy inserts feature values at fixed intervals to form a new feature map, thereby increasing the feature map size while decreasing the number of feature map channels.
[0079] The interpolation upsampling strategy described above is an improvement on the sampling downsampling strategy. After conventional upsampling of the input feature map, zero-padding is first applied to the edges of the output feature map, followed by feature map shifting, mean fusion, and finally, the edge padding is removed. As the feature map size increases, the number of feature map channels decreases. The formula for feature map size change during upsampling is:
[0080] (C,H,W)→(C / r 2 ,H*r,W*r)
[0081] Where C is the number of feature maps, H and W are the height and width of the feature maps, respectively, and r represents the upsampling factor.
[0082] Furthermore, it also includes introducing a channel attention mechanism into the skip connection mechanism, including the following steps:
[0083] 1) Perform global average pooling on the input feature map to obtain a feature vector of size 1×1×C;
[0084] 2) Input the obtained feature vector into two fully connected layers. The first fully connected layer doubles the number of channels in the feature vector, and the second fully connected layer reduces it by half, thus restoring the original number of channels.
[0085] 3) Finally, the input feature map is processed by the Sigmoid function and multiplied channel by channel with the original input feature map.
[0086] 4) Add the obtained output to the output of the referenced feature reconstruction module.
[0087] Furthermore, a general-purpose deep convolutional neural network is trained, including comparing the error between the network output and the label using a loss function, optimizing the network model parameters by using the Adam algorithm for gradient descent, training the network, and thus realizing the network's function, iterating multiple times and saving the weights.
[0088] Since this invention is aimed at general low-level computer vision tasks, the specific loss function can be determined according to different low-level computer vision tasks.
[0089] The loss function is usually chosen from one or more of the following: L2 loss, SSIM loss, and perceptual loss.
[0090] The preprocessed test set data is input into a trained general-purpose deep convolutional neural network model to perform different low-level computer vision tasks on images. This includes inputting the preprocessed test set data into the general-purpose deep convolutional neural network model, performing index testing on the obtained output to detect the capability of the general-purpose deep convolutional neural network model. If the capability of the general-purpose deep convolutional neural network model does not meet the certification standard, training continues. If the capability of the general-purpose deep convolutional neural network model meets the certification standard, it can perform different low-level computer vision tasks on images.
[0091] The metrics tests include network loss and task evaluation metrics (generally PSNR, SSIM, MSE, etc.) used to determine the completion status of low-level vision tasks.
[0092] The criteria for determining whether a general-purpose deep convolutional neural network has converged are as follows: specifically, observe whether the loss on the test set stops decreasing or the metric stops increasing.
[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for constructing a general-purpose deep convolutional neural network to improve generalization ability, characterized in that: Includes the following steps: Based on different visual processing tasks, select the corresponding datasets and divide the datasets into training sets and test sets; Preprocess the training and test sets; A general-purpose deep convolutional neural network with improved generalization ability was constructed. The preprocessed training data is input into the constructed general-purpose deep convolutional neural network for multi-scale extraction and reconstruction to obtain the actual output image; the general-purpose deep convolutional neural network is then trained. The preprocessed test set data is input into a trained general-purpose deep convolutional neural network model to achieve different low-level computer vision tasks for image processing; A general-purpose deep convolutional neural network consists of an encoder and a decoder. The encoder section includes four feature extraction modules; The decoder section includes four feature reconstruction modules; The feature extraction modules employ a value extraction downsampling strategy; this strategy extracts feature values at fixed intervals to form a new feature map, thereby reducing the feature map size while increasing the number of feature map channels. During downsampling, the formula for feature map size change is: Where C is the number of feature maps, H and W are the height and width of the feature maps, respectively, and r represents the fixed sampling interval, i.e. the downsampling factor; The feature reconstruction modules employ an interpolation upsampling strategy; this strategy inserts feature values at fixed intervals to form a new feature map, thereby increasing the feature map size while decreasing the number of feature map channels. The interpolation upsampling strategy, after conventional upsampling of the input feature map, first pads the edges of the output feature map with zeros, then performs feature map shifting, averaging and fusion, and finally removes the edge padding; while the feature map size increases, the number of feature map channels decreases. The formula for feature map size change during upsampling is: Where C is the number of feature maps, H and W are the height and width of the feature maps, respectively, and r represents the upsampling factor; A skip connection mechanism is used to connect the symmetrical feature extraction module and the feature reconstruction module.
2. The method for constructing a general-purpose deep convolutional neural network to improve generalization ability according to claim 1, characterized in that: The feature extraction module includes a 7×7 convolutional layer with a normalized transition layer and two 3×3 convolutional layers with ReLU activation functions. The input features of the first convolutional layer are introduced into the output of the last convolutional layer through residuals.
3. The method for constructing a general-purpose deep convolutional neural network to improve generalization ability according to claim 1, characterized in that: The feature reconstruction module includes three 3×3 convolutional layers, with each convolutional layer followed by a normalization layer and a ReLU activation function. The input features of the first 3×3 convolutional layer are introduced into the output part of the second 3×3 convolutional layer through residuals, and the output features of the first 3×3 convolutional layer are introduced into the output part of the third 3×3 convolutional layer.
4. The method for constructing a general-purpose deep convolutional neural network to improve generalization ability according to claim 1, characterized in that: Also includes: Introducing a channel attention mechanism into the skip connection mechanism includes the following steps: 1) Perform global average pooling on the input feature map to obtain a feature vector of size 1×1×C; 2) Input the obtained feature vector into two fully connected layers. The first fully connected layer doubles the number of channels in the feature vector, and the second fully connected layer reduces it by half, thus restoring the original number of channels. 3) Finally, the input feature map is processed by the Sigmoid function and multiplied channel by channel with the original input feature map; 4) Add the obtained output to the output of the referenced feature reconstruction module.
5. The method for constructing a general-purpose deep convolutional neural network to improve generalization ability according to claim 1, characterized in that: The various low-level computer vision tasks include, but are not limited to, image enhancement, image denoising, image dehazing, and image super-resolution.