Super-resolution reconstruction and semi-supervised identification classification method for low-resolution rice image

Abstract

The invention relates to a super-resolution reconstruction and semi-supervised identification classification method for a low-resolution rice image, and the method comprises the steps: training a generator, and obtaining a super-resolution reconstruction image through the generator; judging the quality of the super-resolution reconstructed image through a first discriminator and a second discriminator; judging the category of the qualified image through a classifier, and endowing the qualified image with label information; and optimizing the generator, the first discriminator, the second discriminator and the classifier through a small-batch stochastic gradient descent training algorithm until a Nash equilibrium state is reached. The method has the advantages that a binary topological structure of a traditional GAN is broken, super-resolution reconstruction and classification of fuzzy rice images are achieved, a generative adversarial network with a quaternary structure is provided, complex distribution of source domain data can be learned, fine structures and features in the data can be captured, and the super-resolution reconstruction and classification of the fuzzy rice images can be achieved. Therefore, the generated image is excellent in detail, texture and overall visual effect.

Description

Technical Field

[0001] The invention relates to the technical field of image reconstruction, and more specifically, to a super-resolution reconstruction and semi-supervised recognition and classification method of a low-resolution rice image. Background Art

[0002] The existing super-resolution image reconstruction technologies are as follows: 1) Interpolation-based image reconstruction methods are the most primitive and intuitive methods, which can be divided into linear interpolation algorithms and nonlinear interpolation algorithms. Interpolation methods use interpolation formulas to enhance the grayscale information between pixels, and are mainly used to solve the problem of blurring after the image is enlarged. However, since the interpolation method directly uses the value of the nearest pixel, it will cause the interpolation result to be rough, with jagged and blocky effects, especially when the image is enlarged. 2) Reconstruction-based super-resolution image reconstruction methods are widely used in the field of image processing, and are mainly divided into frequency domain methods and spatial domain methods. They extract the required image feature information from multiple low-resolution images and unknown high-resolution images, and reconstruct high-definition images after estimating the high-resolution image feature information. However, since reconstruction-based methods usually rely on prior information or global constraints, although they can maintain the edge information of the image, they are often difficult to effectively restore the high-frequency detail information, resulting in insufficient texture details of the reconstructed image, which will introduce artifacts or distortion, especially when processing complex textures or high-frequency signals. 3) The image super-resolution reconstruction method based on optimization is currently the mainstream. It uses certain logic (such as neural networks) to focus on learning the feature distribution of a given image, establishes a sequence prior relationship between low-resolution images and high-resolution images, and then realizes image reconstruction through image generation algorithms. However, the current optimization process is a black box operation with poor interpretability and human interactivity. The reconstructed image feature distribution is highly random and cannot accurately meet human needs. Summary of the invention

[0003] The purpose of the present invention is to address the deficiencies of the prior art and to propose a super-resolution reconstruction and semi-supervised recognition and classification method for low-resolution rice images.

[0004] First, a super-resolution reconstruction and semi-supervised recognition and classification method for low-resolution rice images is provided, including:

[0005] S1. Training a generator, and obtaining a super-resolution reconstructed image through the generator;

[0006] S2, judging the quality of the super-resolution reconstructed image by a first discriminator and a second discriminator, and sending a qualified image to a classifier;

[0007] S3, judging the category of qualified images through a classifier, and assigning label information to the qualified images;

[0008] S4, optimizing the generator, the first discriminator, the second discriminator and the classifier by a small batch stochastic gradient descent training algorithm;

[0009] S5. Under the constraint of the global optimization loss function, the optimization process of the generator, the first discriminator, the second discriminator, and the classifier continues until the Nash equilibrium state is reached.

[0010] Preferably, S1 comprises:

[0011] S101, obtaining an original high-definition image training set;

[0012] S102, inputting an original training image in an original high-definition image training set into a generator, and the generator learning a feature distribution of the original training image; the original training image is an unlabeled high-definition rice image;

[0013] S103, inputting a low-resolution rice image into a generator, and the generator outputting a super-resolution reconstructed image according to the feature distribution of the high-definition rice image.

[0014] Preferably, S2 comprises:

[0015] S201, inputting the super-resolution reconstructed image to the first discriminator, the first discriminator judging whether the super-resolution reconstructed image conforms to the high-definition feature distribution, and sending the super-resolution reconstructed image to the second discriminator if it conforms;

[0016] S202: The second discriminator determines whether the super-resolution reconstructed image conforms to the feature distribution of the original training image, and if so, sends the super-resolution reconstructed image to the classifier.

[0017] Preferably, in S2, the images that do not meet the judgment conditions of the first discriminator and the second discriminator are returned to the generator, and the generator continues to learn the original training images on the existing basis.

[0018] Preferably, in S3, the classifier includes a Softmax function; the Softmax function is used to generate a feature distribution vector, the first K dimensions of the feature distribution vector correspond to K image categories, and the last two dimensions of the feature distribution vector correspond to the probabilities of the image belonging to a high-definition image and an original image, respectively.

[0019] Preferably, S3 further includes: placing qualified images containing label information into the original high-definition image training set.

[0020] Preferably, in S4, the parameters of the first discriminator and the second discriminator, the classifier and the generator are updated, wherein the first discriminator, the second discriminator and the generator are updated by increasing the stochastic gradient, and the classifier is updated by decreasing the stochastic gradient.

[0021] In a second aspect, a super-resolution reconstruction and semi-supervised recognition and classification system for low-resolution rice images is provided, which is used to execute any of the methods described in the first aspect, including:

[0022] A training module, used for training the generator and obtaining a super-resolution reconstructed image through the generator;

[0023] A first judging module, configured to judge the quality of the super-resolution reconstructed image by using a first discriminator and a second discriminator, and send a qualified image to a classifier;

[0024] The second judgment module is used to judge the category of the qualified image through a classifier and assign label information to the qualified image;

[0025] An optimization module, used for optimizing the generator, the first discriminator, the second discriminator and the classifier by a small batch stochastic gradient descent training algorithm;

[0026] The constraint module is used to continuously optimize the generator, the first discriminator, the second discriminator, and the classifier under the constraint of the global optimization loss function until the Nash equilibrium state is reached.

[0027] According to a third aspect, a computer storage medium is provided, wherein a computer program is stored in the computer storage medium; when the computer program is executed on a computer, the computer executes any method described in the first aspect.

[0028] In a fourth aspect, an electronic device is provided, including:

[0029] Memory, used to store computer programs;

[0030] A processor is used to execute the computer program to implement any method as described in the first aspect.

[0031] The beneficial effects of the present invention are as follows: the present invention breaks the binary topological structure of the traditional GAN, realizes super-resolution reconstruction and classification of blurred rice images, and proposes a generative adversarial network (GAN) model with a quaternary structure (generator, first discriminator, second discriminator, classifier), which can learn the complex distribution of source domain data and capture the subtle structure and features in the data. Therefore, the images generated by the GAN model have excellent performance in details, texture and overall visual effects, and can introduce conditional information through the designed generator, discriminator and classifier, thereby improving the interpretability and human interactivity, thereby controlling the style, content or category of the generated image, and adjusting the image generated by the neural network at the pixel level at any time. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic diagram of the composition of the generative adversarial network model provided for this application;

[0033] Figure 2 The model structure diagram of the generator R provided for this application;

[0034] Figure 3 The model structure of the discriminators D1 and D2 provided in this application;

[0035] Figure 4 Schematic diagram of the plasma rice growth image set collected for the verification experiment of this application;

[0036] Figure 5 This is a schematic diagram of the changing trend of the training accuracy and verification accuracy of the Q-GAN model and the DCGAN model with the training cycle in the verification experiment of this application;

[0037] Figure 6 This is a schematic diagram of the changing trend of the training accuracy and verification accuracy of the Q-GAN model and the Info-GAN model with the training cycle in the verification experiment of this application;

[0038] Figure 7 This is a schematic diagram of the changing trend of the training accuracy and verification accuracy of the Q-GAN model and the CGAN model with the training cycle in the verification experiment of this application;

[0039] Figure 8 This is a schematic diagram of the changing trend of the training accuracy and verification accuracy of the Q-GAN model and the ACGAN model with the training cycle in the verification experiment of this application;

[0040] Fig. 9 This is a schematic diagram of the changing trend of the training accuracy and verification accuracy of the Q-GAN model and the WGAN model with the training cycle in the verification experiment of this application;

[0041] Fig.10 This is a schematic diagram of the changing trend of the training accuracy and verification accuracy of the Q-GAN model and the WGAN_GP model with the training cycle in the verification experiment of this application;

[0042] Fig.11 This is a schematic diagram of the changing trend of the training accuracy and verification accuracy of the Q-GAN model and the EBGAN model with the training cycle in the verification experiment of this application;

[0043] Fig.12 This is a schematic diagram of the changing trend of the training accuracy and verification accuracy of the Q-GAN model and the BEGAN model with the training cycle in the verification experiment of this application;

[0044] Fig.13This is a schematic diagram of the confusion matrix of 38 groups of rice growth image recognition in the verification experiment of this application;

[0045] Fig.14 This is a schematic diagram of the recognition rate of 38 groups of rice growth images in the verification experiment of this application;

[0046] Fig.15 A schematic diagram of the accuracy of all neural network models in the test phase in the verification experiment of this application;

[0047] Fig.16 This is a schematic diagram of the recall rates of all neural network models in the test phase in the verification experiment of this application;

[0048] Fig.17 This is a schematic diagram of the F1 scores of all neural network models in the test phase in the verification experiment of this application;

[0049] Fig.18 This is a schematic diagram of the test accuracy of all neural network models in the test phase in the verification experiment of this application;

[0050] Fig.19 This is the effect of super-resolution reconstruction of blurred rice images in the test phase of all models in the verification experiment of this application. DETAILED DESCRIPTION

[0051] The present invention is further described below in conjunction with embodiments. The description of the following embodiments is only used to help understand the present invention. It should be noted that for ordinary persons in the art, without departing from the principle of the present invention, the present invention can also be modified in some ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0052] Embodiment 1:

[0053] The interference of various environmental factors in the natural environment, such as displacement and jitter of the target, underexposure, etc., will cause some of the collected rice images to be blurred, such as radial blur, Gaussian blur, dynamic blur, etc. This paper provides a method for using GAN to achieve super-resolution image reconstruction and recognition and classification of such low-resolution rice images.

[0054] However, conventional GAN ​​cannot achieve both super-resolution reconstruction and semi-supervised recognition and classification of images at the same time, because it is limited by its own binary structure, that is, one generator and one discriminator are not enough to achieve the above functions at the same time. On the one hand, a single discriminator needs to judge whether the generated image meets the feature distribution of the original standard image, and on the other hand, it also needs to judge whether the generated image meets the high-resolution feature distribution. Only one discriminator cannot meet the above requirements. Moreover, the learning paradigm of traditional GAN ​​does not involve sample label information, so it is impossible to perform supervised classification and recognition of images.

[0055] In order to break the binary topological structure of the traditional GAN ​​and realize super-resolution reconstruction and classification of blurred rice images, Example 1 of the present application provides a super-resolution reconstruction and semi-supervised recognition and classification method for low-resolution rice images, and proposes a generative adversarial network model Q-GAN with a quaternary structure (such as Figure 1 As shown in the figure, it can learn the complex distribution of source domain data and capture the subtle structures and features in the data. Therefore, the images it generates have excellent details, textures and overall visual effects. It can also introduce conditional information through the designed generator R, discriminator and classifier C, thereby controlling the style, content or category of the generated images and making pixel-level adjustments to the images generated by the neural network at any time.

[0056] Specifically, the super-resolution reconstruction and semi-supervised recognition and classification methods of low-resolution rice images include:

[0057] S1. Train a generator R, and obtain a super-resolution reconstructed image through the generator R.

[0058] S1 includes:

[0059] S101. Obtain an original high-definition image training set.

[0060] S102, inputting an original training image in an original high-definition image training set into the generator R, and the generator R learns the feature distribution of the original training image; the original training image is an unlabeled high-definition rice image.

[0061] In S102, the generator R can learn the feature distribution of the original high-definition images in the training set and generate new rice pictures based on the learned graphic features.

[0062] S103, inputting a low-resolution rice image to a generator R, and the generator R outputting a super-resolution reconstructed image according to the feature distribution of the high-definition rice image.

[0063] Specifically, the training process of Q-GAN starts with inputting an unlabeled high-definition rice image. The generator R accepts it and extracts the visual features it carries, and then reconstructs a high-definition image based on the learning situation. Therefore, the generator R has two tasks, one is to learn the feature distribution of the original high-definition image in the training set, and the other is to super-resolution reconstruct the image. Figure 1 The Gaussian filter in will only appear in the model verification and testing phase. It reduces the resolution of the high-definition image and transmits it to the generator R to test the image generation ability of the generator R. The model structure diagram of the generator R is as follows Figure 2 shown.

[0064] S2. The quality of the super-resolution reconstructed image is judged by the first discriminator D1 and the second discriminator D2, and a qualified image is sent to the classifier C.

[0065] S2 includes:

[0066] S201, inputting the super-resolution reconstructed image to the first discriminator D1, the first discriminator D1 determines whether the super-resolution reconstructed image conforms to the high-definition feature distribution, and if so, sends the super-resolution reconstructed image to the second discriminator D2;

[0067] S202 , the second discriminator D2 determines whether the super-resolution reconstructed image conforms to the feature distribution of the original training image, and sends the super-resolution reconstructed image to the classifier C if it conforms.

[0068] Specifically, the image generated by S1 is first sent to the first discriminator D1 for detection. The first discriminator D1 is responsible for judging whether the generated image conforms to the high-definition feature distribution. Only high-definition images can be allowed to pass through the first discriminator D1 and continue to be sent to the second discriminator D2, otherwise they will be rejected. The second discriminator D2 is responsible for judging whether the image is an original image (i.e., an image from the training set) or an imitation image generated by the generator R. Only the original image or the imitation image that highly fits the original feature distribution can be allowed to pass. Therefore, only when the image generation ability of the generator R is very outstanding, that is, it successfully fits the feature distribution of the original training image, can it pass the detection of D1 and D2 at the same time. When the rejected image is returned to the generator R, the generator R must continue to learn the original image data on the existing basis and improve its image generation ability to meet the standards required by the first discriminator D1 and the second discriminator D2 at the same time. At the same time, the first discriminator D1 and the second discriminator D2 are also constantly improving their identification capabilities to avoid being deceived by the imitation images generated by the generator R. The model structure of the first discriminator D1 and the second discriminator D2 is as follows Figure 3 shown.

[0069] S3. Determine the category of the qualified image through classifier C (assuming there are K categories of images in total) and assign label information to the qualified image.

[0070] In S3, the classifier C includes a Softmax function; the Softmax function generates a series of feature distribution vectors, whose first K dimensions correspond to K image categories, and the last two dimensions, namely (K + 1) and (K + 2), correspond to the probability that the image belongs to a high-definition image and the probability that it belongs to an original image. After the category judgment of an image is completed, the image assigned to the correct category label will be put into the training set, thereby gradually expanding the size of the data set, which is also an advantage of the Q-GAN model.

[0071] S3 also includes: placing qualified images containing label information into the original high-definition image training set.

[0072] S4. Optimize the generator R, the first discriminator D1, the second discriminator D2 and the classifier C through a small batch stochastic gradient descent training algorithm.

[0073] In S4, in order to achieve the model convergence of Q-GAN, the present invention proposes a training algorithm MSGDA, which includes three stages, respectively updating the parameters of the first discriminator D1, the second discriminator D2, the classifier C and the generator R, wherein the first discriminator D1, the second discriminator D2 and the generator R are updated by increasing their stochastic gradients, while the classifier C is updated by reducing its own stochastic gradient. This process does not wait until the first discriminator D1 and the second discriminator D2 are trained before starting to train the classifier C and the generator R, because the training of the first discriminator D1 and the second discriminator D2 also requires the output values ​​of the classifier C and the generator R in the previous round of error feedback as input. Assume that there are n images in total Taken from the original image set, the present invention assumes that each data mini-batch contains:

[0074] (1) An input random noise signal ;

[0075] (2) Accordingly, there will be Super-resolution reconstructed image key-value pairs ;

[0076] (3) The key-value pairs predicted by the classifier ;

[0077] (4) Key-value pairs ;

[0078] (5) Key-value pairs .

[0079] During training, Q-GAN expects and The jointly defined joint distribution fits the feature distribution of the original training set images Therefore, the converged Q-GAN can predict the label information y of the unlabeled blurred image, and can also generate a new image x with y as a priori condition. Therefore, the present invention proposes an iterative calculation process to perform segmented optimization on the four key parts of Q-GAN. Specifically, the small batch stochastic gradient descent algorithm of Q-GAN is as follows:

[0080] The adaptive hyperparameter k represents the number of iterations applied to the two discriminators. The value of k is set to 2. , and represents a trainable parameter. The gradient-based updates involved can be implemented using any standard gradient learning rule.

[0081] Perform the following loop calculations in the maximum number of training iterations:

[0082] Perform the following loop calculation in k iteration steps:

[0083] From the noise prior distribution Sampling A random noise signal ,

[0084] The sampling batch size is Key-value pairs ,

[0085] The sampling batch size is Key-value pairs ,

[0086] The sampling batch size is Annotated data ,

[0087] The sampling batch size is Annotated data ,

[0088] By promoting and The stochastic gradient of is updated:

[0089] ,

[0090] End the loop calculation in k iteration steps

[0091] calculate and Unbiased estimators and ,

[0092] By reducing Stochastic gradient updates it:

[0093] ,

[0094] By promoting The stochastic gradient of is updated:

[0095] ,

[0096] End the loop calculation at the maximum number of training iterations.

[0097] S5. Under the constraint of the global optimization loss function, the optimization process of the generator R, the first discriminator D1, the second discriminator D2, and the classifier C continues until the Nash equilibrium is reached. That is, the two discriminators D1 and D2 cannot distinguish whether the image is high-definition or blurred, whether it is an original image or a fake image, that is, the probability of judging it as each type of image is 50%.

[0098] In S5, the global loss function of Q-GAN is based on the generator, two discriminators and one classifier. In line with the structure, the function is also a state that is expected to be achieved by the four-element competitive optimization, that is, the generator R expects that the images it generates will have the greatest possibility of passing the tests of the first discriminator D1 and the second discriminator D2, while the first discriminator D1 and the second discriminator D2 expect to have the greatest possibility of discovering blurred and imitated images. These two are similar to the idea of ​​the global loss function in the standard GAN. Finally, the classifier C expects to have the greatest possibility of correctly predicting the image category. This is the main idea of ​​the global loss function of Q-GAN.

[0099] Assume that the feature distribution of the original image is , the distribution of generator R is , then the first discriminator D1 and the second discriminator D2 obey the following formula:

[0100]

[0101] Let the original sample be x and obey , R(x) is distributed according to the conditional labels Generate an image label y corresponding to x. According to the chain rule, the label distribution can be further expanded to:

[0102]

[0103] Similarly, according to ,Depend on and The generated key-value pair (x, y) can also be generated from R(z), where z is random noise, and we can get ,therefore Specifically, for , the present invention assumes that x is obtained by transforming the latent variable of random noise z and has been assigned an image label y, that is, , Then, the key-value pair (x, y) is sent to D1 and D2 for image quality detection. At this time, the quaternary optimization objective of Q-GAN is defined by the following formula:

[0104]

[0105]

[0106] in, is a constant that determines the weight of image generation and recognition capabilities in Q-GAN. This paper takes a value of 0.5 because the present invention pursues both the super-resolution image reconstruction capability of the model and the image recognition capability. is a standard cross entropy loss function. When the above formula reaches its global optimum, Used to ensure critical convergence conditions ( ) can be satisfied forever. Because when the above formula reaches its global optimum, it means that when C or R tends to fit the original feature distribution, the other one moves away from the distribution. However, unlike the binary structure of traditional GAN, the above critical condition cannot always guarantee can be realized, therefore, this application introduces the cross entropy loss function in C, that is, , which is also equivalent to and The KL divergence between the global optimal function and its It can be defined as the following formula:

[0107]

[0108]

[0109] In this formula It is defined by the following formula:

[0110]

[0111]

[0112] When R, C and the state of any discriminator are fixed, the distribution of the other discriminator obeys the following formula:

[0113]

[0114] Among them, the symbol Possible values ​​are 1 and 2.

[0115] Embodiment 2:

[0116] Based on Example 1, Example 2 of the present application provides a more specific method for super-resolution reconstruction and semi-supervised recognition and classification of low-resolution rice images, including:

[0117] S1. Train a generator R, and obtain a super-resolution reconstructed image through the generator R.

[0118] S2. The quality of the super-resolution reconstructed image is judged by the first discriminator D1 and the second discriminator D2, and a qualified image is sent to the classifier C.

[0119] S3. Determine the category of the qualified image through classifier C and assign label information to the qualified image.

[0120] S4. Optimize the generator R, the first discriminator D1, the second discriminator D2 and the classifier C through a small batch stochastic gradient descent training algorithm.

[0121] S5. Under the constraint of the global optimization loss function, the optimization process of the generator R, the first discriminator D1, the second discriminator D2, and the classifier C continues until the Nash equilibrium state is reached.

[0122] In order to verify the performance of the Q-GAN proposed in the present invention, 24754 plasma rice growth images in the tillering period were taken at a rice planting base as experimental data, covering 38 different plasma treatment schemes, so the growth conditions of these 38 groups of rice are different. All images can be divided into two roughly equal parts, one is the horizontally shot image, and the other is the vertically shot image. The former can roughly reflect indicators such as the coverage area of ​​the rice plant, and the latter can reflect the plant extension area and leaf surface area. The image format is JPEG, and each image is a 24-bit color bitmap. Image samples are shown in the figure. Figure 4As shown. It can be observed that the growth morphology of rice treated with different plasma schemes is also different. The growth morphology of rice refers to the phenotypic appearance of rice, which is obtained and reflected through images, such as plant height, color, coverage, leaf width, etc. Different rice varieties, planting methods, management strategies, etc. will affect the growth of rice, and the difference in growth conditions can be reflected through the growth morphology of rice. For example, in the 38 groups of plasma rice images constructed in this article, different plasma treatment strategies will have different promoting effects on the growth of rice. Therefore, by distinguishing the most vigorous experimental group through growth morphology, the most suitable plasma treatment strategy for promoting rice growth can be found quickly and accurately. This patent hopes that the neural network model can correctly extract and distinguish the morphological characteristics of rice plants in different experimental groups from the image, and correctly distinguish the category of rice according to the differences in rice phenotypes between the experimental groups. Next, each experimental group is named according to the parameters involved in the plasma treatment of rice, and Table 1 summarizes these parameters according to the treatment order.

[0123] Table 1 Parameters involved in plasma treatment of rice

[0124]

[0125] According to the parameter values ​​and treatment sequence in Table 1, the present invention uses a series of abbreviations to name each experimental group. For example, the abbreviation "3-1-BW-PW-N" means that the plasma treatment object of this group is rice sprouts, the plasma generation gear is 3, the treatment time is 1 hour, the state of planting after treatment is moist, and plasma activated water is given for irrigation later, and no fertilizer is applied. Table 2 gives the names of all experimental groups and the number of images in each group.

[0126] Table 2 Names and sample numbers of plasma rice experimental groups

[0127]

[0128] For the evaluation of fuzzy rice image classification and recognition, it is incomplete to use only classification accuracy. Therefore, this experiment also uses precision, recall and F1 score as the evaluation criteria for model image classification. The above four indicators are defined as shown in the following four formulas:

[0129]

[0130]

[0131]

[0132]

[0133] Where N represents the total number of test sample types. , , and They represent the number of images in the true positive sample group (TP), false positive sample group (FP), false negative sample group (FN) and true negative sample group (TN).

[0134] For the evaluation of the effect of super-resolution image reconstruction, this paper uses Peak Signal-to-Noise Ratio (PNSR) and Structural Similarity (SSIM) as evaluation indicators, which are obtained by the following two formulas:

[0135]

[0136]

[0137] Among them, X and Y represent two images, and represent the means of X and Y respectively, and represents the variance of X and Y, represents the X and Y offsets, C1 and C2 are constants used to maintain stability, and L is the dynamic range of pixel values.

[0138] In order to verify the performance of Q-GAN, the experiment compared it with 8 other neural network models also implemented based on GAN, namely the standard deep convolution GAN (DCGAN), Info-GAN, conditional GAN ​​(CGAN), Auxiliary Classifier GAN (ACGAN), Wasserstein GAN (WGAN), WGAN with gradient penalty (Wasserstein GAN - Gradient Penalty, WGAN_GP), Energy-Based GAN (EBGAN) and Boundary Equilibrium GAN (BEGAN).

[0139] First, Q-GAN and the other eight models were trained on a set of high-definition rice images to observe their ability to classify and recognize rice images. Before training, 80% of the rice images in each category were randomly selected as training samples, and the remaining 20% ​​were used for model validation. For the 80% training samples, 70% of them had no category labels to achieve semi-supervised learning. Figures 5 to 12 The figure shows the changing trends of the training accuracy and validation accuracy of the 9 models as the training cycle increases throughout the training.

[0140] Table 3 quantifies the performance indicators of the algorithms in the above training process. It can be seen that Q-GAN has certain advantages, which proves the effectiveness of its quaternary optimization mode. Although the training accuracy of Q-GAN is not as high as that of EBGAN, WGAN_GP and BEGAN, Q-GAN exceeds all models in terms of verification accuracy. Specifically, when all models converged, Q-GAN achieved a verification accuracy of 94.09%, the highest among all models, followed by EBGAN with 93.96% and WGAN_GP with 91.97%. The verification accuracies of BEGAN and WGAN were 91.89% and 91.52%, respectively, while those of CGAN, ACGAN and Info-GAN were 90.57%, 88.18% and 80.98%, respectively. DCGAN obtained the lowest verification accuracy of 78.05%. This experiment proves that Q-GAN has a good classification and recognition effect on rice images during the training stage.

[0141] Table 3 Comparison of image classification performance between Q-GAN model and other algorithms in the model training stage

[0142]

[0143] After that, all the models that have converged are tested for recognition and classification of blurred rice images. In order to simulate the rice images that are blurred during the actual acquisition process, all high-definition images in the test data set are blurred by a 7×7 Gaussian filter (with a standard deviation of σ of 1.6), and then downsampled by a bicubic interpolation algorithm (with a coefficient p of 3) to generate low-resolution blurred images. This patent hopes that the model can correctly identify the experimental group numbers corresponding to these blurred images based on plant morphology. Fig.13The confusion matrix in the figure shows the recognition rate of Q-GAN for 38 groups of rice growth images. The ordinate represents the true category (Expected category) and the abscissa represents the actual predicted category (Predicted category). The confusion matrix proves from a macro perspective that Q-GAN has a high recognition rate for the 38 groups of blurred rice images through color distribution.

[0144] Fig.14 The specific recognition rate of Q-GAN for each group of images is described. Among them, the experimental groups with recognition rates exceeding 90% are marked in red, while the experimental groups with recognition rates below 80% are marked in blue. The rice samples in the red experimental groups with higher recognition rates all have more conspicuous phenotypic appearances. For example, the rice in the NNSD-PW-N group grows very short, and the rice in the 3.5-4-SD-PW-CF group grows vigorously, which makes them easy to distinguish from other experimental groups. On the contrary, since the rice growth status of the blue experimental groups (such as 3-1-BW-PW-N, 3-1-SW-PW-N, and 3-2-SD-PW-N) is at an average level, they are more similar to each other, and it is relatively difficult to distinguish them, and the recognition rate is also relatively low. It is also worth noting that the red group with a high recognition rate does not mean that the rice in this group grows well, and the rice in the blue group with a low recognition rate does not necessarily grow worse than the rice in the red group, because not only the rice with vigorous growth is easy to be recognized by the model, but also the rice with very short plants is easy to be recognized by the model. Based on this, it can be inferred that the growth of rice in the blue group is between the best and the worst. In order to obtain the best plasma treatment scheme to promote rice growth, it is necessary to find it in the red group. There are both the best and the worst experimental groups in the red group, so it is necessary to exclude the worst growing group from the red group. At this time, it is only necessary to combine the actual rice growth image to easily filter out the experimental groups with poor growth in the red group. It is not difficult to find that the treatment schemes of the experimental groups 3.5-4-SD-PW-N and 3.5-4-SD-PW-N are most conducive to promoting rice growth, which provides an accurate basis for quickly obtaining the best plasma treatment strategy for promoting rice growth.

[0145] Figures 15 to 18 The accuracy, recall rate, F1 score and test precision of 9 neural network models in the test phase are shown. Among them, Q-GAN has the highest accuracy, F1 score and test accuracy of all models, which are 94.97%, 93.16% and 93.08% respectively. The recall rate ranks second, which is 91.42%, second only to BEGAN, which ranks first, with 91.57%. The above series of experimental results prove the advantages of Q-GAN in the recognition of fuzzy rice images in the test phase.

[0146] In addition, this application continues to evaluate the super-resolution reconstruction performance of Q-GAN for blurred rice images, with the evaluation indicators being PSNR and SSIM, as well as the time consumed to reconstruct each image. As can be seen from Table 4, Q-GAN achieves better image reconstruction results than all other models. Specifically, Q-GAN's PSNR and SSIM are better than the second-ranked WGAN_GP, 34.132 vs. 33.814, and 0.8968 vs. 0.8907, respectively. On the other hand, the time required for WGAN_GP to reconstruct each image is longer than that of Q-GAN, requiring 16.49 milliseconds, while the latter only requires 11.62 milliseconds. The above reconstruction results of blurred images reflect that Q-GAN has obvious advantages among many models.

[0147] Table 4 Performance comparison of Q-GAN model and other algorithms in super-resolution image reconstruction

[0148]

[0149] In addition, the image reconstruction performance of the models can also be intuitively reflected in the images they generate. Fig.19 The results of super-resolution reconstruction of blurred rice images by all models during the test phase are shown. Fig.19 (a) is the original blurred image, and all models need to generate its high-resolution image based on the feature distribution of this group of images; Fig.19 In (b), the high-resolution enhanced image generated by Q-GAN shows good results overall, but jagged textures appear at the edges of the rice plants; Fig.19 In (c), the image reconstructed by DCGAN has too many overlapping areas, and the entire image is still blurry; Fig.19 In (d), the image reconstructed by Info-GAN still has some overlapping areas, but its overall resolution is significantly improved compared to DCGAN; Fig.19 In (e), the image reconstructed by CGAN is blurry and distorted; Fig.19 In (f), the image reconstructed by ACGAN shows ablation phenomenon at the edge of rice. Fig.19 In (g), the image reconstructed by WGAN has blurry artifacts; Fig.19 In (h), the image reconstructed by WGAN_GP shows some jagged and interlaced traces; similar to the image generated by CGAN, EBGAN does not preserve texture details well, and EBGAN generates obvious halo areas along the edges of rice. In contrast, Q-GAN can generate clear rice plant boundaries relatively well and recover more fine-grained features than other models.

[0150] In summary, in the classification and recognition experiment of blurred rice images, the verification accuracy of the model Q-GAN in this application is the highest among all models, which is 94.09%. In the test phase, the accuracy, F1 score and test accuracy of Q-GAN are the highest among all models, which are 94.97%, 93.16% and 93.08% respectively. The recall rate ranks second at 91.72%. In the super-resolution image reconstruction experiment, Q-GAN achieved the highest PSNR and SSIM, which are 0.34132 and 0.8968 respectively. Taking into account various evaluation indicators, Q-GAN surpasses all other models participating in the comparative experiment. The experimental results prove that the expanded quaternary optimization model in Q-GAN is feasible and effective in semi-supervised recognition and classification of blurred rice images and super-resolution image reconstruction.

[0151] It should be noted that the parts in this embodiment that are the same or similar to those in Embodiment 1 can be referenced to each other and will not be described in detail in this application.

[0152] Embodiment 3:

[0153] Based on Examples 1 and 2, Example 3 of the present application provides a super-resolution reconstruction and semi-supervised recognition and classification system for low-resolution rice images, including:

[0154] A training module, used for training the generator R and obtaining a super-resolution reconstructed image through the generator R;

[0155] A first judging module, used for judging the quality of the super-resolution reconstructed image through a first discriminator D1 and a second discriminator D2, and sending a qualified image to a classifier C;

[0156] The second judgment module is used to judge the category of the qualified image through the classifier C and assign label information to the qualified image;

[0157] An optimization module, used to optimize the generator R, the first discriminator D1, the second discriminator D2 and the classifier C through a small batch stochastic gradient descent training algorithm;

[0158] The constraint module is used to continuously optimize the generator R, the first discriminator D1, the second discriminator D2, and the classifier C under the constraint of the global optimization loss function until the Nash equilibrium state is reached.

[0159] It should be noted that the system provided in this embodiment is a system corresponding to the method provided in Embodiments 1 and 2. Therefore, the parts in this embodiment that are the same or similar to Embodiments 1 and 2 can be referenced to each other and will not be repeated in this application.

Claims

1. A super-resolution reconstruction and semi-supervised recognition and classification method for low-resolution rice images, characterized in that: include: S1. Training a generator, and obtaining a super-resolution reconstructed image through the generator; S2, judging the quality of the super-resolution reconstructed image by a first discriminator and a second discriminator, and sending a qualified image to a classifier; S3, judging the category of qualified images through a classifier, and assigning label information to the qualified images; S4, optimizing the generator, the first discriminator, the second discriminator and the classifier by a small batch stochastic gradient descent training algorithm; S5. Under the constraint of the global optimization loss function, the optimization process of the generator, the first discriminator, the second discriminator, and the classifier continues until the Nash equilibrium state is reached.

2. The method for super-resolution reconstruction and semi-supervised recognition and classification of low-resolution rice images according to claim 1, characterized in that: S1 includes: S101, obtaining an original high-definition image training set; S102, inputting an original training image in an original high-definition image training set into a generator, and the generator learns a feature distribution of the original training image; The original training image is an unlabeled high-definition rice image; S103, inputting a low-resolution rice image into a generator, and the generator outputting a super-resolution reconstructed image according to the feature distribution of the high-definition rice image.

3. The method for super-resolution reconstruction and semi-supervised recognition and classification of low-resolution rice images according to claim 2, characterized in that S2 include: S201, inputting the super-resolution reconstructed image to the first discriminator, the first discriminator judging whether the super-resolution reconstructed image conforms to the high-definition feature distribution, and sending the super-resolution reconstructed image to the second discriminator if it conforms; S202: The second discriminator determines whether the super-resolution reconstructed image conforms to the feature distribution of the original training image, and if so, sends the super-resolution reconstructed image to the classifier.

4. The method for super-resolution reconstruction and semi-supervised recognition and classification of low-resolution rice images according to claim 3, characterized in that: In S2, the images that do not meet the judgment conditions of the first discriminator and the second discriminator are returned to the generator, and the generator continues to learn the original training images on the existing basis.

5. The method for super-resolution reconstruction and semi-supervised recognition and classification of low-resolution rice images according to claim 4, characterized in that: In S3, the classifier includes a Softmax function; the Softmax function is used to generate a feature distribution vector, the first K dimensions of the feature distribution vector correspond to K image categories, and the last two dimensions of the feature distribution vector correspond to the probabilities of the image belonging to a high-definition image and an original image, respectively.

6. The method for super-resolution reconstruction and semi-supervised recognition and classification of low-resolution rice images according to claim 5, characterized in that: S3 also includes: placing qualified images containing label information into the original high-definition image training set.

7. The method for super-resolution reconstruction and semi-supervised recognition and classification of low-resolution rice images according to claim 6, characterized in that: In S4, the parameters of the first discriminator and the second discriminator, the classifier and the generator are updated, wherein the first discriminator, the second discriminator and the generator are updated by increasing the stochastic gradient, and the classifier is updated by decreasing the stochastic gradient.

8. Super-resolution reconstruction and semi-supervised recognition and classification system for low-resolution rice images, characterized in that: Used to perform the method according to any one of claims 1 to 7, comprising: A training module, used for training the generator and obtaining a super-resolution reconstructed image through the generator; A first judging module, configured to judge the quality of the super-resolution reconstructed image by using a first discriminator and a second discriminator, and send a qualified image to a classifier; The second judgment module is used to judge the category of the qualified image through a classifier and assign label information to the qualified image; An optimization module, used for optimizing the generator, the first discriminator, the second discriminator and the classifier by a small batch stochastic gradient descent training algorithm; The constraint module is used to continuously optimize the generator, the first discriminator, the second discriminator, and the classifier under the constraint of the global optimization loss function until the Nash equilibrium state is reached.

9. A computer storage medium, characterized in that: The computer storage medium stores a computer program; when the computer program is executed on a computer, the computer executes any one of the methods described in claims 1 to 7.

10. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the method according to any one of claims 1 to 7.

Images