Image enhancement method, device and storage medium

Abstract

The present invention provides an image enhancement method, device and storage medium, belonging to the field of image processing, the method comprising: performing image scaling on each original insect image to obtain scaled insect images; dividing all scaled insect images according to a preset ratio to obtain an insect training set and an insect test set; training a training model according to the insect training set to obtain a trained model; testing the trained model according to the insect test set to obtain an image enhancement model; performing image enhancement on the insect image to be enhanced by the image enhancement model to obtain an image enhancement result. The present invention solves the problem of few public large-scale partial insect data sets and a large workload of manually collecting data, effectively generates insect scarce target class images, improves the accuracy of insect scarce target classification, saves a lot of manpower and material resources, and reduces manual intervention.

Description

Technical Field

[0001] The present invention mainly relates to the field of image processing technology, and in particular to an image enhancement method, device and storage medium. Background Art

[0002] The diversity of biological species in nature is of great significance to maintaining ecological balance and promoting the harmonious development of man and nature. Insects are the largest group of organisms on the earth, and insects are a rare target that is closely related to human production and life. Insects are rare targets in nature and often have complex color and texture features. The error rate of manual recognition methods is high and the efficiency is low. Therefore, the automatic recognition method for insect rare target images has very important application value in many scenarios, such as the protection of insect rare target populations, crop pest control, border quarantine, etc.

[0003] Building a generative model requires prior knowledge of the corresponding data and a large number of parameters. The accuracy of the prior knowledge directly affects the quality of the model, and a large number of parameters leads to a huge amount of calculation. The GAN originally proposed, the network structure is mainly based on a multi-layer perceptron with fully connected layers. Its network includes a generator network and a discriminator network. The generator generates images, and the discriminator supervises the quality of the generated images, and the purpose of image generation is achieved through adversarial learning. However, it is difficult to adjust the parameters, difficult to train, and the generation effect is not good, especially for those data sets with more complex details. Later, someone proposed a new training architecture based on the original GAN ​​- the deep convolutional generative adversarial network DCGAN. This model introduces convolutional neural networks into the generator and discriminator, and uses convolutional layers instead of fully connected layers. Although DCGAN has good feature extraction capabilities, there are problems such as increased errors and gradient disappearance during training. Previous studies on automatic identification of scarce insect targets were mostly based on specimen photo data of scarce insect targets. The types and number of photos of scarce insect targets involved were relatively small, and most of them used manually designed methods to extract classification features of scarce insect targets, which wasted a lot of manpower and material resources and reduced the accuracy of image recognition. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide an image enhancement method, device and storage medium in view of the deficiencies in the prior art.

[0005] The technical solution of the present invention to solve the above technical problem is as follows: an image enhancement method, comprising the following steps:

[0006] S1: importing a plurality of original insect images, and scaling each of the original insect images to obtain scaled insect images corresponding to each of the original insect images;

[0007] S2: Divide all scaled insect images according to a preset ratio to obtain an insect training set and an insect test set;

[0008] S3: constructing a training model, and training the training model according to the insect training set to obtain a trained model;

[0009] S4: Testing the trained model according to the insect test set to obtain an image enhancement model;

[0010] S5: importing the insect image to be enhanced, and performing image enhancement on the insect image to be enhanced by using the image enhancement model to obtain an image enhancement result.

[0011] Another technical solution of the present invention to solve the above technical problem is as follows: an image enhancement device, comprising:

[0012] An image scaling module, used for importing a plurality of original insect images, and performing image scaling on each of the original insect images respectively, to obtain scaled insect images corresponding to each of the original insect images;

[0013] An image segmentation module is used to segment all scaled insect images according to a preset ratio to obtain an insect training set and an insect test set;

[0014] A model training module, used for constructing a training model, training the training model according to the insect training set, and obtaining a trained model;

[0015] A model testing module, used to test the trained model according to the insect test set to obtain an image enhancement model;

[0016] The image enhancement result obtaining module is used to import the insect image to be enhanced, perform image enhancement on the insect image to be enhanced through the image enhancement model, and obtain the image enhancement result.

[0017] Another technical solution of the present invention to solve the above technical problem is as follows: an image enhancement device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the image enhancement method described above is implemented.

[0018] Another technical solution of the present invention to solve the above technical problem is as follows: a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image enhancement method as described above is implemented.

[0019] The beneficial effects of the present invention are as follows: scaled insect images are obtained by scaling each original insect image respectively, an insect training set and an insect test set are obtained by dividing all scaled insect images according to a preset ratio, a trained model is obtained by training the training model according to the insect training set, an image enhancement model is obtained by testing the trained model according to the insect test set, and an image enhancement result is obtained by performing image enhancement on the insect image to be enhanced through the image enhancement model, which solves the problems of few public large-scale partial insect data sets and a large workload of manual data collection, effectively generates insect scarce target class images, improves the accuracy of insect scarce target classification, saves a lot of manpower and material resources, and reduces manual intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a flow chart of an image enhancement method provided by an embodiment of the present invention;

[0021] Figure 2 A module block diagram of an image enhancement device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0023] Figure 1 A flowchart of an image enhancement method provided by an embodiment of the present invention.

[0024] like Figure 1 As shown, an image enhancement method comprises the following steps:

[0025] S1: importing a plurality of original insect images, and scaling each of the original insect images to obtain scaled insect images corresponding to each of the original insect images;

[0026] S2: Divide all scaled insect images according to a preset ratio to obtain an insect training set and an insect test set;

[0027] S3: constructing a training model, and training the training model according to the insect training set to obtain a trained model;

[0028] S4: Testing the trained model according to the insect test set to obtain an image enhancement model;

[0029] S5: importing the insect image to be enhanced, and performing image enhancement on the insect image to be enhanced by using the image enhancement model to obtain an image enhancement result.

[0030] It should be understood that the preset ratio may be 1:9.

[0031] It should be understood that the data set is collected and preprocessed, and the data set (i.e., the multiple scaled insect images) is divided into a training set (i.e., the insect training set) and a test set (i.e., the insect test set) in a certain ratio, and then the image format is processed to adapt it to the input of the generation network (i.e., the training model).

[0032] Specifically, a high-quality image dataset is crucial for the success of image generation. To train the generative model, we use the Butterfly Image Classification 50species dataset provided on Kaggle, taking the butterfly dataset as an example. It is an open source dataset of butterfly classes with 50 classes, including 4955 training samples (i.e., the insect training set) and 250 test samples (i.e., the insect test set). The data is saved in the image format of 224×224×3. In order to adapt to the generator parameter setting, we uniformly scale these images to 96×96 resolution for training the generative network (i.e., the training model).

[0033] In the above embodiment, the scaled insect image is obtained by scaling each original insect image respectively, and all the scaled insect images are divided according to a preset ratio to obtain an insect training set and an insect test set. The training model is trained according to the insect training set to obtain a trained model, and the image enhancement model is tested according to the insect test set to obtain the image enhancement model. The image enhancement result is obtained by performing image enhancement on the insect image to be enhanced through the image enhancement model, which solves the problems of small number of public large-scale partial insect data sets and large workload of manual data collection, effectively generates scarce target class images of insects, improves the accuracy of classification of scarce insect targets, saves a lot of manpower and material resources, and reduces manual intervention.

[0034] Optionally, as an embodiment of the present invention, the insect training set includes a plurality of insect training images, the training model includes a generator and a discriminator, and the process of step S3 includes:

[0035] Generate a feature map for each of the insect training images by the generator to obtain a target feature map corresponding to each of the insect training images;

[0036] The discriminator performs discriminant analysis on each of the target feature images to obtain a discrimination probability corresponding to each of the insect training images;

[0037] Determine whether all the identification probabilities are greater than or equal to the first preset discrimination threshold and less than or equal to the second preset discrimination threshold. If so, use the training model as the trained model and execute step S4; if not, update the parameters of the training model according to all the identification probabilities to obtain the updated training model, and return to step S2.

[0038] Preferably, the first preset discrimination threshold may be 0.4, and the second preset discrimination threshold may be 0.6.

[0039] It should be understood that the network structure (ie, the training model) is optimized to achieve the purpose of improving the sample generation quality and convergence speed.

[0040] It should be understood that the processed data set (i.e., the insect training set) is used as the input of the improved generative adversarial neural network (i.e., the training model), the optimizer uses Adam, the learning rates of the network generator and discriminator are both 0.002, the momentum is 0.5, and the experiment generates images at different training stages, i.e., the number of iterations are 100, 500, 1000, and 5000 respectively.

[0041] In the above embodiment, the trained model is obtained by training the training model according to the insect training set, which is of great significance for maintaining ecological balance in the field of biodiversity in nature, improving the quality of sample generation and convergence speed, and promoting the harmonious development of man and nature. It also has very important application value in the protection of scarce insect populations, control of crop diseases and pests, border quarantine, etc.

[0042] Optionally, as an embodiment of the present invention, the generator includes a first fully connected layer, a first deconvolution layer, a ReLU activation function layer, a destrided convolution layer and a Tanh activation function layer;

[0043] The process of generating a feature map for each of the insect training images by the generator to obtain a target feature map corresponding to each of the insect training images includes:

[0044] Performing dimension-upgrading processing on each of the insect training images through the first fully connected layer to obtain dimension-upgraded insect feature images corresponding to each of the insect training images;

[0045] Performing a first feature extraction on each of the dimensionally upgraded insect feature images through the first deconvolution layer to obtain an insect feature image after the first feature extraction corresponding to each of the insect training images;

[0046] Performing a first mapping process on each of the insect feature images after the first feature extraction through the ReLU activation function layer to obtain an insect feature image after the first mapping corresponding to each of the insect training images;

[0047] Re-feature extraction is performed on each of the insect feature images after the first mapping through the de-strided convolution layer to obtain insect feature images after re-feature extraction corresponding to each of the insect training images;

[0048] The Tanh activation function layer is used to re-map each of the insect feature images after the feature extraction to obtain a target feature map corresponding to each of the insect training images.

[0049] It should be understood that the step size of the de-strided convolution layer is 2, and the first deconvolution layer includes 3 deconvolution layers.

[0050] It should be understood that the analytical formula of the ReLU activation function layer is:

[0051] ReLU(x)=max(0,x),

[0052] When the input x<0, the output is 0, and when x>0, the output is x. This activation function will not saturate, can combat the gradient vanishing problem, uses a simple threshold, and has high computational efficiency, so it can make the network converge faster.

[0053] It should be understood that the analytical formula of the Tanh activation function layer is:

[0054]

[0055] Tanh is also a very common activation function. Compared with sigmoid, its output mean is 0, which makes it converge faster than sigmoid and reduces the number of iterations.

[0056] It should be understood that all pooling layers are cancelled, and upsampling is achieved through deconvolution (i.e., the first deconvolution layer) in the generation network (i.e., the generator); secondly, except for the output layer of the generation network and the input layer of the identification network, other layers use Batch Normalization to deal with training problems caused by poor initialization; ReLU (i.e., the ReLU activation function layer) is used as the activation function in the generation network, and the last layer uses the Tanh activation function (i.e., the Tanh activation function layer).

[0057] Specifically, the learning rate of the generator is 0.002 and the momentum is 0.5. First, a 100-dimensional random variable that obeys a (0,1) uniform distribution is used as the input of the generator, and then converted into a 4×4×512 feature map (i.e., the insect feature image after dimensionality increase) through a fully connected layer (i.e., the first fully connected layer), and then through three deconvolution layers and an activation function (i.e., the ReLU activation function layer) to generate a 32×32×64 image for ReLU (i.e., the insect feature image after the first mapping), and finally through a layer of deconvolution with a step size of 2 (i.e., the deconvolution layer) and a Tanh activation function (i.e., the Ta nh activation function layer) generates a 64×64×3 image (i.e., the target feature map), where H represents the height of the input image, W represents the width of the input image, and C represents the number of channels of the image after the discriminator convolutional neural network filters it. The generated network is a 5-layer structure, and the inverse VGG network mode is adopted in the middle layer. The feature map is mainly processed by a convolutional network with multiple layers of small convolutional kernels. At the same time, strided deconvolution (with a step size of 2) (i.e., the inverse strided convolution layer) is used instead of de-pooling for image upsampling, which reduces the loss of information and makes the parameter information transmission between different layers more effective. Compared with the 4-layer structure of the generated network in DCGAN, the improved DCGAN network adds a deconvolution layer, which doubles the size of the generated image and improves the resolution of the generated image. In the improved DCGAN generated network, the deconvolution kernel size is 5×5, the step size is 2, and the number of convolution kernels is gradually reduced from 1024, and the number of convolution kernels in each layer is 1 / 2 of the previous layer.

[0058] In the above embodiment, the target feature map is obtained by generating the feature map of each insect training image through the generator, which reduces the information loss, makes the parameter information transmission between different layers more effective, improves the resolution of the generated image, and enables the network to converge more quickly.

[0059] Optionally, as an embodiment of the present invention, the discriminator includes a second deconvolution layer, a Leaky ReLU activation function layer, a residual block network, a second fully connected layer and a Sigmoid activation function layer;

[0060] The process of performing discriminant analysis on each of the target feature images by the discriminator to obtain the identification probability corresponding to each of the insect training images includes:

[0061] Performing a first feature extraction on each of the target feature maps through the second deconvolution layer to obtain a target feature map after the first feature extraction corresponding to each of the insect training images;

[0062] Mapping the target feature maps after the first feature extraction is performed respectively through the Leaky ReLU activation function layer to obtain mapped target feature maps corresponding to the insect training images;

[0063] Re-feature extraction is performed on each of the mapped target feature maps through the residual block network to obtain a target feature map after re-feature extraction corresponding to each of the insect training images;

[0064] Performing dimensionality reduction processing on each of the target feature maps after feature extraction again through the second fully connected layer to obtain a target feature map after dimensionality reduction corresponding to each of the insect training images;

[0065] The sigmoid activation function layer is used to perform image identification on each of the target feature maps after dimensionality reduction, so as to obtain an identification probability corresponding to each of the insect training images.

[0066] It should be understood that the residual block network includes 3 residual blocks.

[0067] It should be understood that the analytical formula of the Leaky ReLU activation function layer is:

[0068] LeakyReLU(x)=max(0.01α,x),

[0069] In this activation function, when x<0, it obtains a positive gradient of 0.01. It is computationally efficient, converges quickly, and does not saturate in the positive interval, which alleviates the dead ReLU problem to a certain extent.

[0070] It should be understood that the analytical formula of the Sigmoid activation function layer is:

[0071]

[0072] This activation function is the most widely used type of activation function. It has an exponential function shape. It is physically closest to biological neurons and is continuous, smooth, strictly monotonic, and symmetric around (0,0.5). It is a very good threshold function.

[0073] It should be understood that convolution with a stride is used instead of pooling in the identification network (i.e., the discriminator), and Leaky ReLU (i.e., the Leaky ReLU activation function layer) is used as the activation function in the identification network (i.e., the discriminator).

[0074] Specifically, the introduction of the residual block Conv Block and Identity Block in the residual network ResNet structure not only deepens the network depth but also solves the problem of gradient disappearance, so that the error rate gradually decreases and the accuracy gradually increases, while also improving the generalization ability of the model.

[0075] Specifically, for the discriminator network (i.e., the discriminator), its network structure has reached dozens of layers, the optimizer uses Adam, the network learning rate is 0.002, and the momentum is 0.5. First, the generated image of size 64×64×3 (i.e., the target feature map) is taken as input, and after one deconvolution (i.e., the second deconvolution layer), the activation function is Leaky ReLU (i.e., the Leaky ReLU activation function layer), and a 32×32×64 feature map (i.e., the mapped target feature map) is extracted. Next, the three residual block structures introduced by us are sequentially generated into a 4×4×512 feature map (i.e., the target feature map after feature extraction again), and then through a fully connected layer (i.e., the second fully connected layer), the final probability (i.e., the identification probability) is output through the Sigmoid activation function (i.e., the Sigmoid activation function layer), where H represents the height of the input image, W represents the width of the input image, and C represents the number of channels of the image after filtering by the discriminator convolutional neural network. Secondly, the original pooling layer commonly used in convolutional neural networks is replaced with a convolution layer with a larger convolution kernel with a stride of 2 to achieve the purpose of downsampling. This method can retain more information than direct mean pooling or maximum pooling. At the same time, in order to better adapt to the loss function defined by the Wasserstein distance, we add a batch normalization layer after the strided convolution layer of each convolution group to solve the problem of poor initialization of the discriminator feature extraction network parameters. We introduce the residual block ConvBlock and Identity Block in the residual network (ResNet) structure, which not only deepens the network depth but also solves the problem of gradient disappearance, so that the error rate gradually decreases and the accuracy gradually increases, while also improving the generalization ability of the model.

[0076] In the above embodiment, the identification probability is obtained by the discriminant analysis of each target feature map by the discriminator, which not only deepens the network depth but also solves the problem of gradient vanishing, so that the error rate gradually decreases and the accuracy gradually increases. At the same time, the generalization ability of the model is improved, and more information can be retained, solving the problem of poor initialization of the discriminator feature extraction network parameters.

[0077] Optionally, as an embodiment of the present invention, the process of updating the parameters of the training model according to all identification probabilities to obtain an updated training model includes:

[0078] Importing the real image probabilities corresponding to the respective insect training images, calculating the loss values ​​for all the identification probabilities and all the real image probabilities, and obtaining the loss values;

[0079] The parameters of the training model are updated according to the loss value to obtain an updated training model.

[0080] It should be understood that the true image probability is an imported known value.

[0081] In the above embodiment, the parameters of the training model are updated according to all the identification probabilities to obtain an updated training model, which solves the problems of small number of public large-scale partial insect data sets and large workload of manual data collection, effectively generates scarce target class images of insects, improves the accuracy of classification of scarce insect targets, saves a lot of manpower and material resources, and reduces human intervention.

[0082] Optionally, as an embodiment of the present invention, the process of calculating the loss value for all identification probabilities and all true image probabilities to obtain the loss value includes:

[0083] The loss value is calculated by using the first formula for all identification probabilities and all true image probabilities, and the loss value is obtained. The first formula is:

[0084]

[0085] Among them, max D V(D,G) is the loss value, m is the number of insect training images, and x (i) is the probability of the real image corresponding to the i-th insect training image, G(Z (i) ) is the target feature map corresponding to the i-th insect training image, D(G(Z (i) )) is the identification probability corresponding to the i-th insect training image.

[0086] It should be understood that the improved DCGAN network synthesizes the generated image (i.e., the target feature map) through the generative network (i.e., the generator) to make the discriminative network (i.e., the discriminator) make incorrect judgments as the ultimate goal. The discriminative network (i.e., the discriminator) hopes to maximize the distinction between the generated image (i.e., the target feature map) and the real image, and finally reach a Nash equilibrium through the game between the two.

[0087] Specifically, the real data x follows the distribution pr(x), that is, x∈pr(x). Assuming z follows the random noise distribution pz(z), the loss function of the image generation network in the improved DCGAN network is as follows:

[0088]

[0089] Among them, G(z) represents the image obtained by the generative network, and D(G(z)) represents the identification probability of the generated image. The loss function of the discriminant network in the improved DCGAN network includes the discriminant error of the real image and the discriminant error of the generated image, and the specific form is as follows:

[0090]

[0091] G hopes that the generated images are indistinguishable from the real ones, so it hopes that D(G(z)) is larger, so it uses the clever expression of 1-D(G(z)), so the loss value should be as small as possible. D hopes that its identification ability is super strong, so D(G(z)) should be very small, and D(x) should be very large, one large and one small, changing unpredictably, so it uses the clever expression of 1-D(G(z)), so the loss value should be as large as possible.

[0092] In the above embodiment, the loss value is calculated by the first formula for all identification probabilities and all true image probabilities, which solves the problems of insufficient samples of scarce target image data of insects and class imbalance, helps to increase the quantity and quality of data sets, and also improves the recognition accuracy of different models.

[0093] Optionally, as another embodiment of the present invention, the present invention integrates the features learned by the upper network through skip connections, enhances the diversity and quality of the generated insect scarce target images, enhances the stability of the generative adversarial network training, and alleviates the gradient disappearance. Batch normalization is then used to adapt to the learning of different batches to make the training process more stable. Generate higher quality and more available insect scarce target samples to provide to the deep learning classification network training, and improve the accuracy of insect scarce target image recognition.

[0094] Optionally, as another embodiment of the present invention, the present invention mainly adopts two methods, subjective evaluation and objective evaluation, when evaluating the model effect and image quality. In terms of subjective vision, it can be found through experimental results that with the continuous iteration and update of the network, the quality of the generated image has been significantly improved. For objective evaluation and analysis, we use the FID indicator to judge the quality of the model. A lower FID means higher quality and diversity of the picture. Because FID is more sensitive to model collapse and has better robustness to noise.

[0095] It should be understood that FID is more sensitive to model collapse and more robust to noise. When there is only one type of image, the distance score will be quite high, so FID can be used to describe the diversity of the network.

[0096] Specifically, for objective evaluation and analysis, the FID indicator is used to judge the quality of the model. A lower FID means higher image quality and diversity. Since FID is more sensitive to model collapse and has better robustness to noise. x represents the distribution of the real sample set, g represents the distribution of the generator output, μ represents the mean of the distribution, Σ represents the variance of the distribution, and Tr represents the trace of the matrix, where μx and μg represent the feature mean of the real image and the generated image after the same Inception Network extracts the intermediate layer, and Σx and Σg represent the variance of the extracted intermediate layer feature values. Then FID is calculated as follows:

[0097]

[0098] The present invention experiments on several classic GAN models and our improved model on the data set, and trains and iterates 5000 epochs to obtain the generator model without mode collapse. The samples generated by this model are used as the distribution sampling of g, and the FID distance with the real data set is calculated as the FID score of the generative model. It can be analyzed from the FID data under different models that the traditional GAN ​​network is prone to mode collapse in this generation task and the generated images are relatively simple, while the improved DCGAN model performs better, effectively reducing the probability of mode collapse and better ensuring the diversity of image generation.

[0099] Optionally, as another embodiment of the present invention, the overall training process of the present invention is divided into two stages:

[0100] In the first stage, the parameters of the generating network are fixed, and the parameters of the discriminating network are trained and updated. After random noise is input into the generating network G, the generated image is obtained, and the label of the generated image is set to 0. After that, the parameters of the generating network are fixed, and no data transmission is performed. The generated image with label 0 and the real image with label 1 are used as the training data set of the discriminative network D for network training. The discriminative network D is equivalent to a binary classifier. The parameters of the discriminative network D are updated according to the error between the probability value output by the discriminative network D and the actual labels 0 and 1.

[0101] In the second stage, the parameters of the discriminant network are fixed, and the parameters of the generative network are updated during training. The discriminant network D, which has been trained in the previous stage, directly migrates the parameters obtained in the previous stage to the network D trained in this stage through parameter sharing. The noise is generated through the generative network G to obtain a generated image. The label is no longer marked as 0, but as 1. It is considered that the generated sample is a real image. The purpose is to encourage the generative network G to fit the random noise in the direction close to the real image. The generated image and the corresponding label are used as the training set to train the discriminant network D. The parameters of D are fixed, and only the parameters of the generative network G are updated to improve the image generation ability of the generative network G, so that it can generate generated images that are difficult for the discriminant network to distinguish between true and false, so as to achieve the purpose of making the fake look real. These two stages are repeated until the discriminant network D cannot distinguish between the generated image and the real image. At this time, the identification probability value output by the discriminant network is close to 0.5, and the training ends.

[0102] Optionally, as another embodiment of the present invention, by improving the DCGAN generative data enhancement, and then using the VGG16 and ResNet50 network models for training respectively, the accuracy of image classification after each training is completed is recorded.

[0103] Optionally, as another embodiment of the present invention, the beneficial effects of the present invention are:

[0104] 1. By improving the amplification of the DCGAN generative data enhancement method, the images generated by the generator are screened to achieve data enhancement.

[0105] 2. VGG16 and ResNet50 network models were used for training respectively, and the accuracy of image classification was recorded after each training. Through multiple comparisons of the original data on two different models, the accuracy of insect image recognition for scarce target insects was improved.

[0106] 3. Based on the basic structure of DCGAN, an improved model Re-DCGAN is proposed for data image generation in combination with ResNet network structure and VGG model. The model is used for data enhancement to solve the problems of insufficient samples and class imbalance of insect image data of scarce target insects, which helps to increase the quantity and quality of data sets, and also improves the recognition accuracy of different models.

[0107] Optionally, as another embodiment of the present invention, the present invention proposes an improved model Re-DCGAN for data image generation based on the basic structure of DCGAN, combined with ResNet network structure, VGG model, etc. The model is used for data enhancement, which solves the problems of insufficient samples of scarce target image data of insects and class imbalance, helps to increase the quantity and quality of data sets, and also improves the recognition accuracy of different models.

[0108] Optionally, as another embodiment of the present invention, the present invention takes the scarce data of insects as an example. Aiming at the problem that there are few public large-scale insect data sets and the workload of manually collecting data is large, it is a feasible way to enhance the data set by using the sample generation method. As one of the important methods of sample generation, the generative adversarial network can generate samples according to the data distribution in any given data set, but it has problems such as blurred texture of generated samples, unstable training process and mode collapse in the actual training process. In view of the above problems, on the basis of the deep convolutional generative adversarial network, combined with the residual network, a sample generation method based on the deep residual convolutional generative adversarial network is designed. The method uses the residual network and the convolutional network to construct the generative model and the discriminative model respectively, wherein the deep residual network can restore rich image textures and effectively alleviate the occurrence of instability in adversarial network training and mode collapse. We take butterflies as an example, train the Butterfly Image Classification 50 species dataset on kaggle and test the performance of the algorithm through indicators such as loss function curve and FID score, then apply the network model to the scarce target dataset of insects to achieve data enhancement, and finally use the two deep convolutional neural network models VGG16 and ResNet50 to train and identify the samples. The results show that the proposed method has improved the accuracy of classification and recognition on the scarce target class dataset of insects compared with the network trained on the dataset without data enhancement. The present invention is of great significance in maintaining ecological balance and promoting the harmonious development of man and nature in the field of biodiversity in nature, and has very important application value in the protection of scarce insect populations, control of crop pests and diseases, border quarantine, etc.

[0109] Figure 2 A module block diagram of an image enhancement device provided by an embodiment of the present invention.

[0110] Optionally, as another embodiment of the present invention, Figure 2 As shown, an image enhancement device comprises:

[0111] An image scaling module, used for importing a plurality of original insect images, and performing image scaling on each of the original insect images respectively, to obtain scaled insect images corresponding to each of the original insect images;

[0112] An image segmentation module is used to segment all scaled insect images according to a preset ratio to obtain an insect training set and an insect test set;

[0113] A model training module, used for constructing a training model, training the training model according to the insect training set, and obtaining a trained model;

[0114] A model testing module, used to test the trained model according to the insect test set to obtain an image enhancement model;

[0115] The image enhancement result obtaining module is used to import the insect image to be enhanced, perform image enhancement on the insect image to be enhanced through the image enhancement model, and obtain the image enhancement result.

[0116] Optionally, as an embodiment of the present invention, the insect training set includes a plurality of insect training images, the training model includes a generator and a discriminator, and the model training module is specifically used for:

[0117] Generate a feature map for each of the insect training images by the generator to obtain a target feature map corresponding to each of the insect training images;

[0118] The discriminator performs discriminant analysis on each of the target feature images to obtain a discrimination probability corresponding to each of the insect training images;

[0119] Determine whether all the identification probabilities are greater than or equal to the first preset discrimination threshold and less than or equal to the second preset discrimination threshold. If so, use the training model as the trained model and enter the model testing module; if not, update the parameters of the training model according to all the identification probabilities, obtain the updated training model, and return to the image segmentation module.

[0120] Optionally, another embodiment of the present invention provides an image enhancement device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image enhancement method described above is implemented. The device may be a computer or other device.

[0121] Optionally, another embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image enhancement method as described above is implemented.

[0122] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0123] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0124] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0125] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0126] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0127] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0128] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image enhancement method, It is characterized in that The steps include: S1: importing a plurality of original insect images, and scaling each of the original insect images to obtain scaled insect images corresponding to each of the original insect images; S2: Divide all scaled insect images according to a preset ratio to obtain an insect training set and an insect test set; S3: constructing a training model, and training the training model according to the insect training set to obtain a trained model; S4: Testing the trained model according to the insect test set to obtain an image enhancement model; S5: importing the insect image to be enhanced, and performing image enhancement on the insect image to be enhanced by using the image enhancement model to obtain an image enhancement result; The insect training set includes a plurality of insect training images, the training model includes a generator and a discriminator, and the process of step S3 includes: Generate a feature map for each of the insect training images by the generator to obtain a target feature map corresponding to each of the insect training images; The discriminator performs discriminant analysis on each of the target feature images to obtain a discrimination probability corresponding to each of the insect training images; Determine whether all the identification probabilities are greater than or equal to the first preset discrimination threshold and less than or equal to the second preset discrimination threshold. If so, use the training model as the trained model and execute step S4; if not, update the parameters of the training model according to all the identification probabilities to obtain the updated training model, and return to step S2; The generator includes a first fully connected layer, a first deconvolution layer, a ReLU activation function layer, a destrided convolution layer and a Tanh activation function layer; The process of generating a feature map for each of the insect training images by the generator to obtain a target feature map corresponding to each of the insect training images includes: Performing dimension-upgrading processing on each of the insect training images through the first fully connected layer to obtain dimension-upgraded insect feature images corresponding to each of the insect training images; Performing a first feature extraction on each of the dimensionally upgraded insect feature images through the first deconvolution layer to obtain an insect feature image after the first feature extraction corresponding to each of the insect training images; Performing a first mapping process on each of the insect feature images after the first feature extraction through the ReLU activation function layer to obtain an insect feature image after the first mapping corresponding to each of the insect training images; Re-feature extraction is performed on each of the insect feature images after the first mapping through the de-strided convolution layer to obtain insect feature images after re-feature extraction corresponding to each of the insect training images; Re-mapping each of the insect feature images after the feature extraction is performed again through the Tanh activation function layer to obtain a target feature map corresponding to each of the insect training images; The discriminator includes a second deconvolution layer, a Leaky ReLU activation function layer, a residual block network, a second fully connected layer and a Sigmoid activation function layer; The process of performing discriminant analysis on each of the target feature images by the discriminator to obtain the identification probability corresponding to each of the insect training images includes: Performing a first feature extraction on each of the target feature maps through the second deconvolution layer to obtain a target feature map after the first feature extraction corresponding to each of the insect training images; Mapping the target feature maps after the first feature extraction is performed respectively through the Leaky ReLU activation function layer to obtain mapped target feature maps corresponding to the insect training images; Re-feature extraction is performed on each of the mapped target feature maps through the residual block network to obtain a target feature map after re-feature extraction corresponding to each of the insect training images; Performing dimensionality reduction processing on each of the target feature maps after feature extraction again through the second fully connected layer to obtain a target feature map after dimensionality reduction corresponding to each of the insect training images; The sigmoid activation function layer is used to perform image identification on each of the target feature maps after dimensionality reduction, so as to obtain an identification probability corresponding to each of the insect training images.

2. The image enhancement method according to claim 1, It is characterized in that The process of updating the parameters of the training model according to all identification probabilities to obtain an updated training model includes: Importing the real image probabilities corresponding to the respective insect training images, calculating the loss values ​​for all the identification probabilities and all the real image probabilities, and obtaining the loss values; The parameters of the training model are updated according to the loss value to obtain an updated training model.

3. The image enhancement method according to claim 2, It is characterized in that The process of calculating the loss value for all identification probabilities and all true image probabilities to obtain the loss value includes: The loss value is calculated by using the first formula for all identification probabilities and all true image probabilities, and the loss value is obtained. The first formula is: ， in, is the loss value, The number of training images for insects, For the The probability of the real image corresponding to the insect training image, For the The target feature map corresponding to the insect training image, For the The identification probability corresponding to each insect training image.

4. An image enhancement device, It is characterized in that include: An image scaling module, used for importing a plurality of original insect images, and performing image scaling on each of the original insect images respectively, to obtain scaled insect images corresponding to each of the original insect images; An image segmentation module is used to segment all scaled insect images according to a preset ratio to obtain an insect training set and an insect test set; A model training module, used for constructing a training model, training the training model according to the insect training set, and obtaining a trained model; A model testing module, used to test the trained model according to the insect test set to obtain an image enhancement model; An image enhancement result obtaining module is used to import an insect image to be enhanced, perform image enhancement on the insect image to be enhanced through the image enhancement model, and obtain an image enhancement result; The insect training set includes a plurality of insect training images, the training model includes a generator and a discriminator, and the model training module is specifically used for: Generate a feature map for each of the insect training images by the generator to obtain a target feature map corresponding to each of the insect training images; The discriminator performs discriminant analysis on each of the target feature images to obtain a discrimination probability corresponding to each of the insect training images; Determine whether all the identification probabilities are greater than or equal to a first preset discrimination threshold and less than or equal to a second preset discrimination threshold. If so, use the training model as a trained model and enter the model testing module; If not, update the parameters of the training model according to all identification probabilities to obtain an updated training model, and return to the image segmentation module; The generator includes a first fully connected layer, a first deconvolution layer, a ReLU activation function layer, a destrided convolution layer and a Tanh activation function layer; In the model training module, the process of generating a feature map for each of the insect training images by the generator to obtain a target feature map corresponding to each of the insect training images includes: Performing dimension-upgrading processing on each of the insect training images through the first fully connected layer to obtain dimension-upgraded insect feature images corresponding to each of the insect training images; Performing a first feature extraction on each of the dimensionally upgraded insect feature images through the first deconvolution layer to obtain an insect feature image after the first feature extraction corresponding to each of the insect training images; Performing a first mapping process on each of the insect feature images after the first feature extraction through the ReLU activation function layer to obtain an insect feature image after the first mapping corresponding to each of the insect training images; Re-feature extraction is performed on each of the insect feature images after the first mapping through the de-strided convolution layer to obtain insect feature images after re-feature extraction corresponding to each of the insect training images; Re-mapping each of the insect feature images after the feature extraction is performed again through the Tanh activation function layer to obtain a target feature map corresponding to each of the insect training images; The discriminator includes a second deconvolution layer, a Leaky ReLU activation function layer, a residual block network, a second fully connected layer and a Sigmoid activation function layer; In the model training module, the process of performing discriminant analysis on each of the target feature graphs by the discriminator to obtain the identification probability corresponding to each of the insect training images includes: Performing a first feature extraction on each of the target feature maps through the second deconvolution layer to obtain a target feature map after the first feature extraction corresponding to each of the insect training images; Mapping the target feature maps after the first feature extraction is performed respectively through the Leaky ReLU activation function layer to obtain mapped target feature maps corresponding to the insect training images; Re-feature extraction is performed on each of the mapped target feature maps through the residual block network to obtain a target feature map after re-feature extraction corresponding to each of the insect training images; Performing dimensionality reduction processing on each of the target feature maps after feature extraction again through the second fully connected layer to obtain a target feature map after dimensionality reduction corresponding to each of the insect training images; The sigmoid activation function layer is used to perform image identification on each of the target feature maps after dimensionality reduction, so as to obtain an identification probability corresponding to each of the insect training images.

5. An image enhancement system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the image enhancement method according to any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium storing a computer program, It is characterized in that When the computer program is executed by a processor, the image enhancement method according to any one of claims 1 to 3 is implemented.

Images