Generative adversarial network-based pear flower target detection and segmentation method

Abstract

The invention discloses a generative adversarial network-based pear flower target detection and segmentation method, which comprises the steps of 1, collecting and marking pear flower image data and then establishing a real image data set, 2, constructing a generative adversarial network model, 3, training the generative adversarial network model, and 4, extracting a real image data set. 4, extracting pear flower image features generated by the generative adversarial network model and establishing a general training set, 5, establishing and training a target detection model and a segmentation model, and 6, inputting an image to obtain a detection and segmentation result of a pear flower target. According to the method, the adversarial learning mechanism of the generative adversarial network is utilized, the pear flower image of the detection frame and the segmentation mask closer to the real label can be generated, and the learning ability of the model to the pear flower features is enhanced, so that the detection and segmentation precision of the pear flower image is improved, and the robustness is improved; the method can effectively cope with the influence of factors such as complex backgrounds, illumination changes and shielding, and provides a new technical means for agricultural automation and botany research.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a pear blossom target detection and segmentation method based on a generative adversarial network. Background Art

[0002] As an important ornamental plant, the target detection and segmentation of pear blossoms are of great significance in the fields of agricultural automation and botanical research. Traditional pear blossom target detection and segmentation methods mainly rely on manually designed feature extraction algorithms, such as color threshold segmentation and edge detection. However, these methods are easily affected by factors such as illumination changes and occlusion in complex backgrounds, resulting in low detection and segmentation accuracy.

[0003] In recent years, deep learning technology has made remarkable progress in the field of image processing, especially generative adversarial networks (GANs), which have shown powerful capabilities in image generation and image restoration. However, there are relatively few studies on the application of generative adversarial networks in pear blossom target detection and segmentation. Therefore, the present invention proposes a pear blossom target detection and segmentation method based on generative adversarial networks to solve the problems existing in the prior art. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to propose a pear blossom target detection and segmentation method based on a generative adversarial network. The pear blossom target detection and segmentation method based on a generative adversarial network utilizes the adversarial learning mechanism of the generative adversarial network to generate pear blossom images with detection boxes and segmentation masks that are closer to the real annotations, thereby enhancing the model's learning ability for pear blossom features, thereby improving the detection and segmentation accuracy of pear blossom images, improving robustness, and being able to effectively cope with the influence of complex backgrounds, lighting changes, occlusions and other factors, providing new technical means for the fields of agricultural automation, botanical research and so on.

[0005] To achieve the purpose of the present invention, the present invention is implemented by the following technical scheme: a pear flower target detection and segmentation method based on a generative adversarial network, comprising the following steps:

[0006] Step 1: Collect image data containing pear blossoms, and mark the pear blossom targets in the images to obtain a real image dataset;

[0007] Step 2: Build a generative adversarial network model including a generator network and a discriminator network, where the generator network adopts an encoder-decoder structure and the discriminator network adopts a convolutional neural network structure;

[0008] Step 3: Import the real image dataset as a training set into the generative adversarial network model to train the model;

[0009] Step 4: Generate annotated pear blossom images through the trained generative adversarial network model, and then use the convolutional neural network to extract annotation features from them to establish a general training set;

[0010] Step 5: Establish a target detection model and a segmentation model, and use a common training set to train the target detection model and the segmentation model;

[0011] Step 6: Input the pear blossom image to be detected into the trained target detection model and segmentation model, generate the target detection frame and segmentation mask, and post-process the generated detection frame and segmentation mask to obtain the pear blossom detection and segmentation results.

[0012] Further improvements are: the image data in step one is obtained based on network big data and field shooting methods; the image annotation includes a target bounding box and a pixel-level segmentation mask, wherein the target bounding box is the position and contour information of the pear blossom.

[0013] A further improvement is that in step 2, the generator network is used to generate realistic pear blossom annotated images, wherein the encoder is used to extract image features, and the decoder is used to generate target detection boxes and segmentation masks; the discriminator network is used to distinguish the differences between the detection boxes and segmentation masks generated by the generator network and the real annotations.

[0014] The further improvement is that the model training in step 3 is specifically

[0015] S1, first input the real image dataset into the generator network to generate predicted target detection boxes and segmentation masks;

[0016] S2, then input the generated detection box, segmentation mask and true annotation into the discriminator network to obtain the loss function;

[0017] S3. Finally, alternately optimize and train the generator network and the discriminator network until the generative adversarial network model converges.

[0018] A further improvement is that: in S3, the adversarial loss function is specifically used to optimize the generator network so that the trained generative adversarial network model generates high-quality images.

[0019] Further improvements are: in the step 5, the target detection model adopts Faster R-CNN and YOLO; the segmentation model adopts U-Net image segmentation model.

[0020] A further improvement is that the post-processing in step six specifically includes using a non-maximum suppression algorithm to remove redundant detection frames, and using morphological operations to optimize the segmentation mask.

[0021] The beneficial effects of the present invention are as follows: the present invention utilizes the adversarial learning mechanism of the generative adversarial network to generate a pear blossom image with a detection frame and a segmentation mask that are closer to the real annotation, thereby enhancing the model's learning ability for pear blossom features, thereby improving the detection and segmentation accuracy of the pear blossom image;

[0022] Generative adversarial networks can learn more robust feature representations and effectively cope with the influence of complex backgrounds, lighting changes, occlusions, and other factors;

[0023] The method of the invention can realize end-to-end pear flower target detection and segmentation without manual intervention and with a high degree of automation, thus providing a new technical means for the fields of agricultural automation, botanical research, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0025] In order to deepen the understanding of the present invention, the present invention will be further described in detail below in conjunction with examples. The examples are only used to explain the present invention and do not constitute a limitation on the protection scope of the present invention.

[0026] As an important economic crop, pear blossoms play an important role in agricultural production. Accurate target detection and segmentation of pear blossoms are of great significance for agricultural management and scientific research. Traditional image processing methods have poor detection and segmentation effects on pear blossoms under complex backgrounds and are easily affected by lighting, shadows, and background interference. In recent years, deep learning technology has made significant progress in the field of image recognition, among which generative adversarial networks (GANs) have attracted much attention due to their powerful generation and feature extraction capabilities. However, there are still some challenges in the application of existing GANs in pear blossom target detection and segmentation, such as low quality of generated images and insufficient detection accuracy.

[0027] Example 1

[0028] Based on the above problems, this embodiment provides a pear flower target detection and segmentation method based on a generative adversarial network. Figure 1 As shown, the following steps are included:

[0029] Step 1: Based on the network big data and field shooting methods, 1,000 images containing pear blossoms were obtained, and the pear blossom targets in the images were labeled using the Labelme tool to obtain a real image dataset;

[0030] Image annotation includes the target bounding box and pixel-level segmentation mask, where the target bounding box is the location and contour information of the pear blossom.

[0031] Step 2: Build a generative adversarial network model including a generator network and a discriminator network, where the generator network adopts an encoder-decoder structure and the discriminator network adopts a convolutional neural network structure;

[0032] The generator network is used to generate realistic pear blossom annotated images. The encoder uses a pre-trained ResNet-50 network to extract image features, and the decoder uses a deconvolution layer and an upsampling layer to generate target detection boxes and segmentation masks to obtain pear blossom annotated images.

[0033] The discriminator network specifically adopts the PatchGAN structure, uses convolutional layers and LeakyReLU activation functions, and is mainly used to distinguish the differences between the detection boxes and segmentation masks generated by the generator network and the actual annotations.

[0034] Step 3: Import the real image dataset as a training set into the generative adversarial network model to train the model;

[0035] Specifically include:

[0036] S1, first input the real image dataset into the generator network to generate predicted target detection boxes and segmentation masks;

[0037] S2, then input the generated detection box, segmentation mask and true annotation into the discriminator network to obtain the loss function;

[0038] S3. First use the adversarial loss function to optimize the generator network. Use the Adam optimizer, set the learning rate to 0.0002, and the batch size to 8. Alternately optimize and train the generator network and the discriminator network for 100 epochs in total until the generative adversarial network model converges, so that the trained generative adversarial network model can generate high-quality images.

[0039] Step 4: Generate annotated pear blossom images through the trained generative adversarial network model, and then use the convolutional neural network to extract annotation features from them to establish a general training set for pear blossom target detection and segmentation tasks;

[0040] Step 5: Establish the Faster R-CNN target detection model and the U-Net image segmentation model, and use the common training set to train the target detection model and the segmentation model;

[0041] Step 6. Input the pear blossom image to be detected into the trained target detection model and segmentation model to generate the target detection frame and segmentation mask, and post-process the generated detection frame and segmentation mask, including using the non-maximum suppression algorithm to remove redundant detection frames and using morphological operations to optimize the segmentation mask, and finally obtain the pear blossom detection and segmentation results.

[0042] Example 2

[0043] This embodiment provides a pear flower target detection and segmentation method based on a generative adversarial network. Figure 1 As shown, the following steps are included:

[0044] Step 1: Based on the network big data and field shooting methods, 1,000 images containing pear blossoms were obtained, and the pear blossom targets in the images were labeled using the Labelme tool to obtain a real image dataset;

[0045] Image annotation includes the target bounding box and pixel-level segmentation mask, where the target bounding box is the location and contour information of the pear blossom.

[0046] Step 2: Build a generative adversarial network model including a generator network and a discriminator network, where the generator network adopts an encoder-decoder structure and the discriminator network adopts a convolutional neural network structure;

[0047] The generator network is used to generate realistic pear blossom annotated images. The encoder uses a pre-trained ResNet-50 network to extract image features, and the decoder uses a deconvolution layer and an upsampling layer to generate target detection boxes and segmentation masks to obtain pear blossom annotated images.

[0048] The discriminator network specifically adopts the PatchGAN structure, uses convolutional layers and LeakyReLU activation functions, and is mainly used to distinguish the differences between the detection boxes and segmentation masks generated by the generator network and the actual annotations.

[0049] Step 3: Import the real image dataset as a training set into the generative adversarial network model to train the model;

[0050] Specifically include:

[0051] S1, first input the real image dataset into the generator network to generate predicted target detection boxes and segmentation masks;

[0052] S2, then input the generated detection box, segmentation mask and true annotation into the discriminator network to obtain the loss function;

[0053] S3. First use the adversarial loss function to optimize the generator network. Use the Adam optimizer, set the learning rate to 0.0002, and the batch size to 8. Alternately optimize and train the generator network and the discriminator network for 100 epochs in total until the generative adversarial network model converges, so that the trained generative adversarial network model can generate high-quality images.

[0054] Step 4: Generate annotated pear blossom images through the trained generative adversarial network model, and then use the convolutional neural network to extract annotation features from them to establish a general training set for pear blossom target detection and segmentation tasks;

[0055] Step 5: Establish a YOLO target detection model and a U-Net image segmentation model, and use the common training set to train the target detection model and the segmentation model;

[0056] Step 6. Input the pear blossom image to be detected into the trained target detection model and segmentation model to generate the target detection frame and segmentation mask, and post-process the generated detection frame and segmentation mask, including using the non-maximum suppression algorithm to remove redundant detection frames and using morphological operations to optimize the segmentation mask, and finally obtain the pear blossom detection and segmentation results.

[0057] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A pear blossom target detection and segmentation method based on a generative adversarial network, characterized in that: The following steps are involved: Step 1: Collect image data containing pear blossoms, and mark the pear blossom targets in the images to obtain a real image dataset; Step 2: Build a generative adversarial network model including a generator network and a discriminator network, where the generator network adopts an encoder-decoder structure and the discriminator network adopts a convolutional neural network structure; Step 3: Import the real image dataset as a training set into the generative adversarial network model to train the model; Step 4: Generate annotated pear blossom images through the trained generative adversarial network model, and then use the convolutional neural network to extract annotation features from them to establish a general training set; Step 5: Establish a target detection model and a segmentation model, and use a common training set to train the target detection model and the segmentation model; Step 6: Input the pear blossom image to be detected into the trained target detection model and segmentation model, generate the target detection frame and segmentation mask, and post-process the generated detection frame and segmentation mask to obtain the pear blossom detection and segmentation results.

2. A pear flower target detection and segmentation method based on a generative adversarial network according to claim 1, characterized in that: The image data in step 1 is obtained based on network big data and field shooting methods; the image annotation includes a target bounding box and a pixel-level segmentation mask, wherein the target bounding box is the position and contour information of the pear blossom.

3. The pear flower target detection and segmentation method based on a generative adversarial network according to claim 1, characterized in that: In the step 2, the generator network is used to generate a realistic pear blossom annotated image, wherein the encoder is used to extract image features, and the decoder is used to generate target detection frames and segmentation masks; the discriminator network is used to distinguish the differences between the detection frames and segmentation masks generated by the generator network and the real annotations.

4. The pear flower target detection and segmentation method based on a generative adversarial network according to claim 1, characterized in that: The model training in step 3 is specifically as follows: S1, first input the real image dataset into the generator network to generate predicted target detection boxes and segmentation masks; S2, then input the generated detection box, segmentation mask and true annotation into the discriminator network to obtain the loss function; S3. Finally, alternately optimize and train the generator network and the discriminator network until the generative adversarial network model converges.

5. The pear flower target detection and segmentation method based on a generative adversarial network according to claim 4, characterized in that: In S3, the adversarial loss function is specifically used to optimize the generator network so that the trained generative adversarial network model generates high-quality images.

6. The pear blossom target detection and segmentation method based on a generative adversarial network according to claim 1, characterized in that: In the step 5, the target detection model adopts Faster R-CNN and YOLO; the segmentation model adopts U-Net image segmentation model.

7. The pear blossom target detection and segmentation method based on a generative adversarial network according to claim 1, characterized in that: The post-processing in step six specifically includes using a non-maximum suppression algorithm to remove redundant detection frames, and using morphological operations to optimize the segmentation mask.

Images