Germ-remaining rice small target image feature enhancement method in visual blurring state, program, equipment and storage medium

Abstract

The invention belongs to the technical field of germ-remaining rice processing quality detection, and particularly relates to a germ-remaining rice small target image feature enhancement method, a program, equipment and a storage medium in a vision blurring state. According to the method, the overall structure of the generative adversarial network of the DF-SRGAN is provided, the features of the image are extracted through a parallel residual fusion module in a generator of the network, context information is learned through an inverse convolution fusion module, local details of the image are enhanced through a residual feedback enhancement module, and the robustness of the image is improved. And then a super-resolution image is obtained after up-sampling and convolution operation. According to the method, the loss functions of the generative network and the adversarial network are improved, the trained DF-SRGAN network can generate a high-quality germ-remaining rice image, and high-quality image data support is provided for subsequent research. According to the method, a large amount of high-quality germ-remaining rice image data can be generated at the same time, and the problem that the image cannot be used due to poor quality of a small target image is avoided.

Description

Technical Field

[0001] The present invention belongs to the technical field of processing quality detection of germ-retained rice, and in particular relates to a method, program, device and storage medium for enhancing features of a small target image of germ-retained rice in a visually blurred state. Background Art

[0002] During the processing of embryo-retained rice, the image information of embryo-retained rice is usually affected by low light conditions or vibrations due to the complex actual processing environment. In addition, due to the small size of embryo-retained rice, a single grain of rice occupies less than 0.12% of the area of ​​the entire image, and the germ only accounts for 3% to 5% of the volume of the embryo-retained rice. Especially under low-resolution conditions, the image quality and visualization of embryo-retained rice are poor, and it is difficult to accurately identify small defects or impurities on the rice grains, which can easily cause a decrease in detection accuracy and increase the incidence of missed detection and false detection. These factors make it difficult for traditional visual inspection methods to extract effective distinguishing features, resulting in information loss during the inspection process, which in turn affects subsequent image processing and quality assessment.

[0003] With the continuous development of computer vision technology, super-resolution-based technologies have been widely used in many industries, including film and television production, security monitoring, medical image analysis, etc., and have proposed effective solutions to problems such as blurred images and difficulty in identifying small targets under low resolution. The image feature enhancement based on super-resolution method for embryo-retained rice can effectively identify the processing quality of rice grains. Compared with traditional image enhancement methods, it has the advantages of high image quality, strong reliability, and wide application range. It fully overcomes the difficulties of traditional detection technology and provides high-quality images for subsequent embryo-retained rice detection. Summary of the invention

[0004] The purpose of the present invention is to provide a method, program, device and storage medium for feature enhancement of small target images of embryo-retained rice under visual blur state, and to enhance the features of small target images of embryo-retained rice by a deep learning-based method, so as to provide high-quality embryo-retained rice image data support for subsequent research.

[0005] A method for enhancing the image features of a small target of rice with embryos in a visually blurred state comprises the following steps:

[0006] Step 1: Obtain embryo-retained rice image data and construct a training set;

[0007] Step 2: Build the DF-SRGAN model and train it using the training set;

[0008] The DF-SRGAN model includes a parallel residual module, a deconvolution fusion module and a residual feedback enhancement module;

[0009] The parallel residual fusion module is used to extract features from the input image and reconstruct the complex details of the small target image with the embryo retained; the deconvolution fusion module is used to make the reconstructed small target image with the embryo retained visually closer to the real scene; the residual feedback enhancement module is used to enhance the local details of the image and suppress noise and artifacts. The image output by the residual feedback enhancement module is subjected to upsampling and convolution operations to obtain a super-resolution image;

[0010] Step 3: Input the original embryo-retained rice image into the trained DF-SRGAN model to obtain the embryo-retained rice super-resolution image after image feature enhancement.

[0011] Furthermore, when constructing the embryo-retained rice image training set, the embryo-retained rice image data is expanded using geometric transformation to increase the diversity of the images, so that the rice grains are presented in different positions, reducing the model's dependence on position information.

[0012] Furthermore, the parallel residual fusion module extracts features of different dimensions and levels through parallel residual paths and multi-level feature fusion, thereby enhancing the ability to express complex structures of small target images with preserved embryos.

[0013] Furthermore, the deconvolution fusion module enhances important semantic information in the small target image with embryos retained during the process of enlarging the image, reduces artifacts and avoids the loss of detail features of the small target image with embryos retained through deep feature fusion, so that the details of the generated small target image with embryos retained are more realistic and natural, ensuring visual realism.

[0014] Furthermore, the residual feedback enhancement module fuses information from different receptive fields and fully learns feature information at different levels, alleviating the gradient vanishing problem and ensuring that input features are not lost. At the same time, it enhances the network's learning of high-level features, enhances the clarity of small target images with embryos left in the image, and suppresses background noise interference.

[0015] Furthermore, when training the DF-SRGAN model, the high-resolution image with embryos left and the super-resolution image with embryos left output by the DF-SRGAN model are input into the discriminator for target discrimination, and the similarity between the output of the DF-SRGAN model and the real image is evaluated; the loss function is:

[0016]

[0017] Among them, α and β are balancing factors; represents the adversarial loss function, represents the content loss function;

[0018]

[0019] Among them, I HRI represents the reference image, i.e., the real high-resolution image of embryo-retained rice; LR Represents the low-resolution image of embryo-retained rice as input to the DF-SRGAN model; is the function representation of the generator, which is the input low-resolution image I LR Convert to high quality super-resolution images and as closely as possible to the reference image I HR Similarity; θ G Represents the set of trainable parameters of the generator; is the function representation of the discriminator, which is used to evaluate the images generated by the generator, θ D Represents the set of trainable parameters of the discriminator; W and H represent the width and height of the image, respectively. The reference image is equal to the super-resolution image in size; Represents the deep feature map output by convolution in the feature extraction network VGG; the image loss is the target reconstructed image G θG (I LR ) and the target reference image I HR The Euclidean distance between .

[0020] Furthermore, the discriminator adopts a discriminator network structure based on cross-connected convolution fusion, introduces jump connections between multiple convolutional layers, performs weighted fusion on features from different layers, effectively integrates detail feature information of shallow and deep layers, realizes cross-layer feature fusion, and uses a fully connected layer at the last stage to output the distinguishability of the entire image, providing more accurate feedback for the optimization of the generator, thereby improving the quality of super-resolution reconstructed images;

[0021] The loss function of the discriminator is:

[0022] l D = -log(D(I HR )-range(-0.05,0.05))-log(range(0.95,1.05)-D(I LR ))

[0023] Where D(·) is the output probability function of the discriminator, which is used to determine whether the input image is a real image or a generated image. By introducing perturbations range(-0.05, 0.05) and range(0.95, 1.05), the robustness and generalization ability of the discriminator are enhanced.

[0024] A computer device / equipment / system comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-mentioned method for enhancing the image features of a small target of rice with embryos left in a visually blurred state.

[0025] A computer-readable storage medium stores a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned method for enhancing the image features of a small target with embryos retained in a visually blurred state.

[0026] A computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned method for enhancing the image features of a small target with a germ-free rice grain under a visually blurred state.

[0027] The beneficial effects of the present invention are:

[0028] The present invention designs a super-resolution reconstruction algorithm for small target images with embryos left, and improves the loss functions of the generative network and the adversarial network. The trained DF-SRGAN network can generate high-quality embryo-left images, providing high-quality image data support for subsequent embryo-left segmentation research for dense adhesion and dense superposition. The present invention can simultaneously generate a large amount of high-quality embryo-left image data, avoiding the problem of unusable images due to poor quality of small target images, and the present invention is also applicable to other small target image enhancements, with a wide range of applications and strong usability. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is the overall architecture diagram of the present invention.

[0030] Figure 2 This is the network structure diagram of the super-resolution reconstruction algorithm DF-SRGAN.

[0031] Figure 3 This is the structure diagram of the parallel residual fusion block.

[0032] Figure 4 This is the network structure diagram of the residual feedback enhancement block.

[0033] Figure 5 This is the structure diagram of the cross-connected convolutional neural network discriminator.

[0034] Figure 6 Comparison diagram of the original image and the image after feature enhancement, where (a) is a low-resolution image; (b) is a high-resolution image; and (c) is a super-resolution image. DETAILED DESCRIPTION

[0035] The present invention is further described below in conjunction with the accompanying drawings.

[0036] The present invention uses a deep learning-based method to enhance the features of small target images of rice with embryos, providing high-quality data support for subsequent research. The present invention first collects images of rice with embryos in real processing scenes and produces a training data set for rice with embryos; secondly, a super-resolution reconstruction algorithm for small target images of rice with embryos is designed, and the loss function is optimized to train the super-resolution reconstruction algorithm DF-SRGAN; finally, the trained network model is used to reconstruct the small target images of rice with embryos and perform quality assessment.

[0037] Combination Figure 1 First, use industrial cameras to collect image data of rice with embryos in real processing scenarios, and expand the data through geometric transformation; secondly, build a super-resolution reconstruction algorithm DF-SRGAN according to demand, design parallel residual fusion module, inverse convolution fusion module, residual feedback enhancement module and discriminator based on cross-connected convolution fusion, optimize the loss function of the generation network and loss network, and train the built network model; finally, use the trained network model to reconstruct the small target image of rice with embryos and perform quality assessment. It includes the following steps:

[0038] Step 1: Create a training dataset for embryo-free rice. Collect image data of embryo-free rice in actual processing scenarios and expand the data samples.

[0039] Step 2: Design a parallel residual fusion module in the network generator to extract image features and reconstruct complex details of high-resolution images of small targets with preserved embryos.

[0040] Step 3: Build a deconvolution fusion module to make the reconstructed small target image with preserved embryos visually closer to the real scene.

[0041] Step 4: Since noise and artifacts will be generated when reconstructing the small target image with embryos retained, the local details of the image are enhanced through the residual feedback enhancement module to further improve the image quality.

[0042] Step 5: After upsampling and convolution operations, a super-resolution image is obtained and input into the discriminator together with the high-resolution image for target discrimination to evaluate the similarity between the generated image and the real image.

[0043] Step 6: Apply different loss functions in the generative network and adversarial network, and train the constructed network model on the self-made 8,000-image rice dataset.

[0044] Step 7: After the network training is completed, the network is used to evaluate the quality of the embryo-retained rice image. The test perceptual quality index LPIPS can reach 0.362. The algorithm has a significant advantage in processing the details of small objects in the embryo-retained rice.

[0045] 1. The following method is used to expand the embryo rice image:

[0046] The embryo rice image data is expanded by using geometric transformations such as flipping, cropping, and rotation to increase the diversity of the images, presenting rice grains in different positions, reducing the model's dependence on position information, improving the model's generalization ability and robustness, and thus reducing overfitting.

[0047] 2. Use the following method to extract features from the image:

[0048] like Figure 3 As shown in the figure, a parallel residual fusion module is designed to extract image features. The main branch of each parallel block consists of three 3×3 convolutional layers and two ReLU functions, and the secondary branch consists of a 3×3 convolutional layer. Through residual connection and pixel fusion, two 1×1 convolutional layers are used to increase the nonlinear characteristics of the network, and finally the output is sent to the next residual block after the ReLU function.

[0049] 3. Enhance image details using the following methods:

[0050] The deconvolution fusion module is designed to improve the resolution of small target images and enhance the authenticity and detail of the images. Figure 4 As shown in the figure, a residual feedback enhancement module is designed to enhance the local details of the image. The output features of the residual module are optimized by two 3×3 and two 1×1 convolutional layers with 64 channels and ReLU functions, and the number of channels of the features is halved through a 1×1 convolutional layer to achieve inverse convolution splicing to improve the network's expression ability. Finally, by combining the previous and next convolutional layers, the contextual semantic information is enhanced, making the reconstructed small target image realistic in overall and local details.

[0051] 4. Reconstruct the embryo-retained rice image using the following network:

[0052] The network used is a generative adversarial network based on DF-SRGAN. The low-resolution image is pre-extracted through a convolution layer, and then passes through a parallel residual fusion module composed of 8 feature parallel blocks. The receptive field of the feature map is expanded through a dilated convolution, and the local feature details are enhanced by an inverse convolution fusion module. Finally, two sub-pixel convolutions are used to achieve a 4-fold upsampling, and multiple convolution operations are used to output a small target reconstructed image with a preserved seed.

[0053] 5. The loss function of network training uses the following method:

[0054] The complete weighted perceptual loss formula for the generative network is:

[0055]

[0056] Where α and β are balance factors, which are 1.0 and 10e-3 respectively; represents the adversarial loss function, Represents the content loss function. The specific function calculation formula is:

[0057]

[0058] Among them, I HR I represents the reference image, i.e., the real high-resolution image of embryo-retained rice; LR Represents the low-resolution image of embryo-retained rice as input to the DF-SRGAN model; is the function representation of the generator, which is the input low-resolution image I LR Convert to high quality super-resolution images and as closely as possible to the reference image I HR Similar, θ G Represents the set of trainable parameters of the generator; is the function representation of the discriminator, which is used to evaluate the images generated by the generator, θ D Represents the set of trainable parameters of the discriminator; W and H represent the width and height of the image, respectively. The reference image is equal to the super-resolution image in size; Represents the deep feature map of the fifth group of convolution outputs after the fourth pooling layer in the feature extraction network VGG; the image loss is the target reconstructed image G θG (I LR ) and the target reference image I HR The Euclidean distance between .

[0059] The loss function formula of the adversarial network is:

[0060] l D = -log(D(I HR )-range(-0.05,0.05))-log(range(0.95,1.05)-D(I LR ))

[0061] Where D(·) is the output probability function of the discriminator, which is used to determine whether the input image is a real image or a generated image. By introducing perturbations (range(-0.05, 0.05) and range(0.95, 1.05)), the robustness and generalization ability of the discriminator are enhanced.

[0062] 6. Image quality assessment using the following methods:

[0063] The main objective evaluation method used in the image quality evaluation method is the perceptual model, which can help people judge the type of image. The principle provided is to extract information from the image and the reconstructed image, activate the patch using BaseNet according to the trained model, scale the weight value and calculate the distance and the sum. The lower the value, the better the enhancement effect. The calculation formula is:

[0064]

[0065] Where x is the reference image, x0 is the noisy distorted image, dis(x,x0) is the distance between x0 and x, l is the feature extraction layer, and w l is the scaling activation function.

[0066] The main principle of the present invention is to train the super-resolution reconstruction network through a large amount of actual processed embryo-retained rice image data, and reconstruct the embryo-retained rice image using the trained network model. First, the embryo-retained rice training data set is collected and produced, and the data is expanded by geometric transformation; secondly, the overall structure of the generative adversarial network of DF-SRGAN is proposed, and a parallel residual fusion module is proposed in the generator of the network to extract the features of the image, and then the context information is learned through the deconvolution fusion module, and the residual feedback enhancement module is proposed to enhance the local details of the image, and then the super-resolution image is obtained after upsampling and convolution operations; then, the high-resolution and super-resolution target images are input into the discriminator based on the cross-connected convolutional neural network for target discrimination, and the similarity between the generated image and the real image is evaluated; finally, the loss function in the generative network and the adversarial network is optimized, and the DF-SRGAN network is trained using the homemade embryo-retained rice image data. After the training is completed, the network can be used to reconstruct the embryo-retained rice image to obtain a high-quality embryo-retained rice image.

[0067] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for enhancing the image features of a small target of rice germs under a visually blurred state, characterized in that: The following steps are involved: Step 1: Obtain embryo-retained rice image data and construct a training set; Step 2: Build the DF-SRGAN model and train it with the training set; The DF-SRGAN model includes a parallel residual module, a deconvolution fusion module and a residual feedback enhancement module; The parallel residual fusion module is used to extract features from the input image and reconstruct the complex details of the small target image with the embryo retained; the deconvolution fusion module is used to make the reconstructed small target image with the embryo retained visually closer to the real scene; The residual feedback enhancement module is used to enhance local details of the image and suppress noise and artifacts. The image output by the residual feedback enhancement module is subjected to upsampling and convolution operations to obtain a super-resolution image; Step 3: Input the original embryo-retained rice image into the trained DF-SRGAN model to obtain the embryo-retained rice super-resolution image after image feature enhancement.

2. The method for enhancing the image features of small target rice with embryos left in a visually blurred state according to claim 1, characterized in that: When constructing the embryo-retained rice image training set, the embryo-retained rice image data was expanded using geometric transformation to increase the diversity of the images, present the rice grains in different positions, and reduce the model's dependence on position information.

3. The method for enhancing the image features of small target rice with embryos in a visually blurred state according to claim 1, characterized in that: The parallel residual fusion module extracts features of different dimensions and levels through parallel residual paths and multi-level feature fusion, thereby enhancing the ability to express complex structures of small target images with preserved embryos.

4. The method for enhancing the image features of small target rice with embryos left in a visually blurred state according to claim 1, characterized in that: The deconvolution fusion module enhances important semantic information in the small target image with embryos retained during the process of image enlargement, reduces artifacts and avoids loss of detail features of the small target image with embryos retained through deep feature fusion, so that the details of the generated small target image with embryos retained are more realistic and natural, ensuring visual realism.

5. The method for enhancing the image features of small target rice with embryos left in a visually blurred state according to claim 1, characterized in that: The residual feedback enhancement module fuses information from different receptive fields and fully learns feature information at different levels, alleviating the gradient vanishing problem and ensuring that input features are not lost. At the same time, it enhances the network's learning of high-level features, enhances the clarity of the small target image with embryo retained, and suppresses background noise interference.

6. The method for enhancing the image features of small target rice with embryos left in a visually blurred state according to claim 1, characterized in that: When training the DF-SRGAN model, the high-resolution image with embryos left and the super-resolution image with embryos left output by the DF-SRGAN model are input into the discriminator for target discrimination, and the similarity between the output of the DF-SRGAN model and the real image is evaluated; the loss function is: Among them, α and β are balancing factors; represents the adversarial loss function, represents the content loss function; Among them, I HR I represents the reference image, i.e., the real high-resolution image of embryo-retained rice; LR Represents the low-resolution image of embryo-retained rice as input to the DF-SRGAN model; is the function representation of the generator, which is the input low-resolution image I LR Convert to high quality super-resolution images and as closely as possible to the reference image I HR Similarity; θ G Represents the set of trainable parameters of the generator; is the function representation of the discriminator, which is used to evaluate the images generated by the generator, θ D Represents the set of trainable parameters of the discriminator; W and H represent the width and height of the image, respectively. The reference image is equal to the super-resolution image in size; Represents the deep feature map output by convolution in the feature extraction network VGG; the image loss is the target reconstructed image G θG (I LR ) and the target reference image I HR The Euclidean distance between .

7. The method for enhancing the image features of small target rice with embryos left in a visually blurred state according to claim 6, characterized in that: The discriminator adopts a discriminator network structure based on cross-connected convolution fusion, introduces jump connections between multiple convolutional layers, performs weighted fusion on features from different layers, effectively integrates detail feature information of shallow and deep layers, realizes cross-layer feature fusion, and uses a fully connected layer at the last stage to output the distinguishability of the entire image, providing more accurate feedback for the optimization of the generator, thereby improving the quality of super-resolution reconstructed images; The loss function of the discriminator is: l D =-log(D(I HR )-range(-0.05,0.05))-log(range(0.95,1.05)-D(I LR )) Where D(·) is the output probability function of the discriminator, which is used to determine whether the input image is a real image or a generated image. By introducing perturbations range(-0.05, 0.05) and range(0.95, 1.05), the robustness and generalization ability of the discriminator are enhanced.

8. A computer device / equipment / system comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images